[9fans] timesync doubletime

[9fans] timesync doubletime

[erik quanstrom]

the rtc clock in one of my machines runs a bit slow -- maybe 2 minutes a month.
i ran timesync -h /srv/sources and now the system clock runs ~28 sec/min too fast.
killing timesync had no effect.

; cat /dev/rtc > /dev/time ; sleep 60 ; date ; date `{cat /dev/rtc}
Mon Aug  7 19:07:11 CDT 2006
Mon Aug  7 19:06:44 CDT 2006
; ; cat /dev/rtc > /dev/time ; sleep 60 ; date ; date `{cat /dev/rtc}
Mon Aug  7 19:09:34 CDT 2006
Mon Aug  7 19:09:06 CDT 2006
; ; ; cat /dev/rtc > /dev/time ; sleep 60 ; date ; date `{cat /dev/rtc}
Mon Aug  7 19:10:19 CDT 2006
Mon Aug  7 19:09:51 CDT 2006

- erik

Re: [9fans] timesync doubletime

[Kris Maglione]

On Mon, Aug 07, 2006 at 07:12:41PM -0500, erik quanstrom wrote:
>; cat /dev/rtc > /dev/time ; sleep 60 ; date ; date `{cat /dev/rtc}
>Mon Aug  7 19:07:11 CDT 2006
>Mon Aug  7 19:06:44 CDT 2006
>; ; cat /dev/rtc > /dev/time ; sleep 60 ; date ; date `{cat /dev/rtc}
>Mon Aug  7 19:09:34 CDT 2006
>Mon Aug  7 19:09:06 CDT 2006
>; ; ; cat /dev/rtc > /dev/time ; sleep 60 ; date ; date `{cat /dev/rtc}
>Mon Aug  7 19:10:19 CDT 2006
>Mon Aug  7 19:09:51 CDT 2006

That output isn't very useful. This would be more useful:
fn dates { date; date `{cat /dev/rtc} }
dates; cat /dev/rtc>/dev/time; dates; sleep 60; dates; sleep 60; dates

I suspect that the issue is that your clock is being synchronized to an ntp 
server which has a different time from your real time clock.

-- 
Kris Maglione

If it would be cheaper to buy a new unit, the company
will insist upon repairing the old one.

Re: [9fans] timesync doubletime

[erik quanstrom]

since /dev/rtc is verified working and since i am not running timesync at all,
it seems to me that the problem must be in the kernel and sleep (thus the
kernel's fastticks) must be running too fast.

in 60s sleep (same pace as date output), the rtc advances only 32 or 33sec
(6:44-6:11, 09:06-08:34, 09:51-09:19).

doing things the other way gives 60 rtc sec = 110 system sec.

; fn rtcmin { s = `{cat /dev/rtc} ; end = `{echo $s + 60 | hoc} ; while (! ~ `{cat /dev/rtc} $end) usleep 4}
; date ; rtcmin ; date
Mon Aug  7 20:33:25 CDT 2006
Mon Aug  7 20:35:15 CDT 2006

- erik

On Mon Aug  7 19:43:27 CDT 2006, bsdaemon@comcast.net wrote:
> On Mon, Aug 07, 2006 at 07:12:41PM -0500, erik quanstrom wrote:
> >; cat /dev/rtc > /dev/time ; sleep 60 ; date ; date `{cat /dev/rtc}
> >Mon Aug  7 19:07:11 CDT 2006
> >Mon Aug  7 19:06:44 CDT 2006
> >; ; cat /dev/rtc > /dev/time ; sleep 60 ; date ; date `{cat /dev/rtc}
> >Mon Aug  7 19:09:34 CDT 2006
> >Mon Aug  7 19:09:06 CDT 2006
> >; ; ; cat /dev/rtc > /dev/time ; sleep 60 ; date ; date `{cat /dev/rtc}
> >Mon Aug  7 19:10:19 CDT 2006
> >Mon Aug  7 19:09:51 CDT 2006
> 
> That output isn't very useful. This would be more useful:
> fn dates { date; date `{cat /dev/rtc} }
> dates; cat /dev/rtc>/dev/time; dates; sleep 60; dates; sleep 60; dates
> 
> I suspect that the issue is that your clock is being synchronized to an ntp 
> server which has a different time from your real time clock.
> 
> -- 
> Kris Maglione
> 
> If it would be cheaper to buy a new unit, the company
> will insist upon repairing the old one.

Re: Re: [9fans] timesync doubletime

[andrey mirtchovski]

i've seen this behaviour before. start 'stats' and see whether it
updates unreasonably fast/slow. reboot fixed the problem in my case
and i've only seen it a couple of times circa 2003 or so...

> it seems to me that the problem must be in the kernel and sleep (thus the
> kernel's fastticks) must be running too fast.

Re: Re: Re: [9fans] timesync doubletime

[andrey mirtchovski]

> were you using a multiprocessor?

no... unfortunately rebooting was one of the first things i tried.
it's been too long ago for me to remember any useful information about
it too, sorry. not useful, but at least you know you're not the only
one.

Re: [9fans] timesync doubletime

[geoff]

We've recently seen some slowness here that affects the clock.  So far
I think it's only been seen on multiprocessor (multicore) AMD64
systems with Realtek 8169 Gb Ethernet.  Setting *ncpu=1 fixes the
problem at the cost of using only one processor.  When we tried to add
debugging to the kernel to track this down, the problem vanished.

Re: Re: [9fans] timesync doubletime

[erik quanstrom]

i'm sure rebooting would solve the problem.  that seems like the easy way out.
there seems to be a real bug here, and i'd like to figure out what's going on.

were you using a multiprocessor?

- erik

On Mon Aug  7 20:47:05 CDT 2006, mirtchovski@gmail.com wrote:
> i've seen this behaviour before. start 'stats' and see whether it
> updates unreasonably fast/slow. reboot fixed the problem in my case
> and i've only seen it a couple of times circa 2003 or so...
> 
> > it seems to me that the problem must be in the kernel and sleep (thus the
> > kernel's fastticks) must be running too fast.

Re: [9fans] timesync doubletime

[Kris Maglione]

On Mon, Aug 07, 2006 at 10:40:16PM -0500, erik quanstrom wrote:
>this kernel has been running correctly for weeks.  only when i tried running
>"aux/timesync -f /srv/sources". (i accidently had two running at once at one point.)
>did things go haywire. killing the timesync procs has not helped.
It seems likely to me that if you had two running at once on different 
servers, one set the time while one was calculating the new calibration for 
the clock. Have you tried running timesync since the problem occured?

-- 
Kris Maglione

A hug is the perfect gift - one size fits all, and
nobody minds if you exchange it.

Re: Re: Re: [9fans] timesync doubletime

[erik quanstrom]

right.  that's very useful information.  that makes it much more likely a 
(non-mp-specific) bug.  the timing ratio seems auspicious, too.

- erik

On Mon Aug  7 20:59:39 CDT 2006, mirtchovski@gmail.com wrote:
> > were you using a multiprocessor?
> 
> no... unfortunately rebooting was one of the first things i tried.
> it's been too long ago for me to remember any useful information about
> it too, sorry. not useful, but at least you know you're not the only
> one.

Re: [9fans] timesync doubletime

[erik quanstrom]

this kernel has been running correctly for weeks.  only when i tried running
"aux/timesync -f /srv/sources". (i accidently had two running at once at one point.)
did things go haywire. killing the timesync procs has not helped.

so, without the benefit of timesync,  have not seen any problems. i may
be having a different or only sightly related problem.  especially since i have
neither a realtech, gbe, amd nor a multicore processor.  just a "blue-light
special" va linux dual coppermine.

- erik

On Mon Aug  7 21:13:17 CDT 2006, geoff@plan9.bell-labs.com wrote:
> We've recently seen some slowness here that affects the clock.  So far
> I think it's only been seen on multiprocessor (multicore) AMD64
> systems with Realtek 8169 Gb Ethernet.  Setting *ncpu=1 fixes the
> problem at the cost of using only one processor.  When we tried to add
> debugging to the kernel to track this down, the problem vanished.

Re: Re: Re: [9fans] timesync doubletime

[Steve Simon]

On Mon Aug  7 20:59:39 CDT 2006, mirtchovski@gmail.com wrote:
> it's been too long ago for me to remember any useful information about

There was some weirdness a few years ago with laptops whos clock would run very fast if the machine was spended and then resumed, maybe that is what you mean?

My fs/auth/cpu server is a dual P3 machine which timesync's to an ntp server,
so far my clock seems stable (Vote one for SMP is not the problem).

-Steve

Re: Re: Re: Re: [9fans] timesync doubletime

[andrey mirtchovski]

> There was some weirdness a few years ago with laptops whos clock would run very fast if the machine was spended and then resumed, maybe that is what you mean?

definitely not a laptop. it was one of the desktops at the University
of Calgary i used as a cpu server...

Re: Re: Re: Re: Re: [9fans] timesync doubletime

[andrey mirtchovski]

i do wish gmail would stop adding Re: in front of my replies' Subject
once it has done so.

Re: [9fans] timesync doubletime

[Russ Cox]

> the rtc clock in one of my machines runs a bit slow -- maybe 2 minutes a month.
> i ran timesync -h /srv/sources and now the system clock runs ~28 sec/min too fast.
> killing timesync had no effect.

Killing timesync probably did have an effect.

As Kris pointed out, if you were a few minutes behind sources
then maybe timesync was running time fast to catch up.  Killing it
doesn't help -- now time will run fast forever, when if you'd left it,
it probably would have slowed to regular speed once it caught up.
The speed in use at time of kill is what the kernel will use until
told otherwise.

Also, you might get more precise results by using NTP instead
of relying on a long-distance TCP connection to sources.

Russ

Re: Re: Re: [9fans] timesync doubletime

[John Floren]

On 8/7/06, Steve Simon <steve@quintile.net> wrote:
> There was some weirdness a few years ago with laptops whos clock would run
> very fast if the machine was spended and then resumed, maybe that is what you
> mean?
This still happens on my laptop. Now, the install cd was about a year
old, so I don't know if the problem still exists.


John
-- 
"The first thing we do, let's kill all the lawyers" -- Shakespeare, Henry VI

Re: [9fans] timesync doubletime

[erik quanstrom]

what you're saying makes sense.  the problem is that wasn't quite what
happened.

when i started timesync, the clock was running about 10 minutes fast.
when i killed timesync, the clock was about 8 hours fast.

i think the problem is that the network connection was interrupted.
but i can't figure out why that would cause this problem. 
if that were to happen, sample(fstime) will return 0, i think, and then
try to write n0 to /dev/bintime.  the kernel takes that as a delta of 0.
things should have been okay.  maybe i'm reading the code incorrectly.

a local ntp server would be a good idea, but when your network connection
is a modem, i don't think it makes much difference.  ;-) also, the accuracy
i was looking for was "getting to work on time" not intercepting the iss. ;-)

- erik

On Tue Aug  8 09:54:32 CDT 2006, rsc@swtch.com wrote:
> > the rtc clock in one of my machines runs a bit slow -- maybe 2 minutes a month.
> > i ran timesync -h /srv/sources and now the system clock runs ~28 sec/min too fast.
> > killing timesync had no effect.
> 
> Killing timesync probably did have an effect.
> 
> As Kris pointed out, if you were a few minutes behind sources
> then maybe timesync was running time fast to catch up.  Killing it
> doesn't help -- now time will run fast forever, when if you'd left it,
> it probably would have slowed to regular speed once it caught up.
> The speed in use at time of kill is what the kernel will use until
> told otherwise.
> 
> Also, you might get more precise results by using NTP instead
> of relying on a long-distance TCP connection to sources.
> 
> Russ