[LinuxPPS] Ultimate time server

[Cirilo Bernardo]

On Wed, Dec 29, 2010 at 12:24 AM, Paul Lavender <paul at lavender-fam.net> wrote:
> I have a (very) ongoing project to make a 'state of the art' time
> server. Currently my pps (from gps) server does everything - firewall,
> file server, email server etc. etc. etc.
>
> I intend to use one one of the used Rubidium clocks from ebay, to
> synchronise a dedicated time server, possibly a low powered ITX board.

I think a good small rubidium clock costs ~$200 at the moment new.  I
would want more information on a second hand clock before I bought
one; like vacuum tubes, these components degrade substantially with
normal use.  Once warmed up the Rb clock gives you a very good PPS
reference; that alone plus another (possibly very crude) reference
clock like GPS (backed up by NTP over the internet or a wireless modem
in case you lose GPS) can already give you a pretty good timeserver.
Personally there is only one experiment I had been involved in which I
needed anything better.

The only other tip I would have here is do not use anything which
connects to the USB bus; the USB specification is not appropriate for
precision timekeeping.

>
> Your comments would be appreciated, particularly with regard to the time
> server. I have seen frequent discussion regarding stability of the on
> board crystal.

Now first of all the question is *which* crystal on the board is
responsible for the onboard timekeeping? Unlike ancient beasts such as
the Apple2e, modern computers often have much more than two crystals.
Which one controls the timing? Unfortunately that's not so obvious
either. In many cases of computers I have worked with, a low(ish)
frequency clock is used to control a frequency multiplier which is
then divided back down to give the timekeeping frequency.  Sometimes
you may still see the good ol' 32.7680KHz crystal and stabilizing that
can help.  However, to get any benefit from stabilizing the
temperature of an onboard component, you really need to understand the
design of the computer and how the operating system interacts with it.
For example, if the 3.7680KHz crystal is only driving the "real time
clock" then you gain nothing because that clock is only read on
start-up and intermittently as commanded (it is too slow to read to
serve as a useful reference when the computer is running). If the
system time is derived from an internal CPU counter and the clock
driving that counter is also internal to the CPU then you have no
choice but to temperature stabilize the CPU and also keep in mind that
internally the temperature may still fluctuate significantly.

The less power consumed by your computer, the easier to regulate
temperature if you should go for one of the temperature stabilization
schemes suggested by others. (I'm also skeptical about the controller
mentioned as having a +-1C control - that is very poor T control even
for circuitry from 40 years ago.) Rather than an ITX form factor, you
may consider a PC104 or even a non-standard form computer.  In the ITX
family of course the Pico-ITX is very small.  Anyway, you can probably
get away with even something like an ARM8 or ARM9 series running at a
mere 100MHz. Microsecond precision should be attainable even on such a
modest machine. For anything better than that you really need to build
your own specialized hardware, and in most cases that doesn't matter
because the other computers on the network will have much more jitter
than that anyway. You also want to operate that CPU at a fixed clock
rate; a varying clock rate introduces all sorts of problems. I
remember ~2 years ago the Linux kernel had a number of issues with
high resolution timers and CPU speed and I have no idea if all issues
have been sorted out; it was partly to do with varying timer designs
within various CPUs - they didn't all behave the same.

 Now without some specialized hardware, you may also need to limit the
incoming packet requests to prevent a flood of requests from upsetting
the system.

 Now I'm not trying to scare you - just saying that improving
precision is not trivial.

> Has anybody replaced the crystal with a higher stability
> device, such as a temperature compensated crystal, or even better
> something derived from the Rubidium maser?
>

The Rubidium clock is not a maser nor am I aware of any designs that
use a maser.  The newer (and compact and cheaper) Rb clocks stabilize
(aka 'discipline') a crystal-based oscillator.  If the output of that
oscillator is a very high frequency, for example 100MHz, then it is
relatively simple to build a very high resolution clock which uses the
oscillator output (rather than the PPS output) to drive it.  Such a
high resolution clock in turn can be used to discipline the computer
clock, or if you design your system with low enough latency you can
simply read the high res clock, calculate the current time, and send
that information down the line.

(Just a note: when the clock frequency is even higher - for example
1GHz or more, then timing delays along wire traces becomes important,
so pushing your timing towards nanosecond precision is non-trivial.)