Re: [9fans] how to fix: 'arena arenas00 creation time after last write time'

Re: [9fans] how to fix: 'arena arenas00 creation time after last write time'

[Corey]

On Monday 27 July 2009 17:35:31 erik quanstrom wrote:
> > "I suspect what's happening is the motherboard clock is set in
> > the future, you are formatting venti based on that time, and then later
> > firing up timesync which interprets the RTC as local time. If your RTC is
> > set to UTC and you're in the western hemisphere, the RTC clock will be
> > ahead of your local time."
> >
> > ... it's kindofa bummer that apparently I can't setup fossil+venti during
> > the plan 9 install process without hosing up venti when I change my
> > timezone appropriately to get the correct time on the machine...
> >
> > So I guess I either:
> >
> > * keep the machine on UTC, which allows me to install fossil+venti
> >   during installation
>
> ; man timesync
>
>      TIMESYNC(8)                                           TIMESYNC(8)
>
>  [...]
>
>           -L   used with -r to indicate the real time clock is in
>                local time rather than GMT.  This is useful on PCs that
>                also run the Windows OS.
>

I must be stuck in some sort of logic error, so please correct me where I'm
wrong:

* the Plan 9 installer assumes EDT (UTC -4)

* however I live in MST (UTC -7)

* thus no matter what, when I set my timezone after the install, the time
  goes _backward_ X number of hours (EDT is -4, MST is -7; so time goes
  back by 3 hours) - (this will occur regardless of whether or not -L is used
  with aux/timesync in termrc) - and so I get those venti errors, due to the
  fact that previous writes now appear to the system to have occurred in
  the future.


Thanks for baring with me, and I'm probably just being obtuse here - but why
doesn't the Plan 9 installer ask and set the appropriate time as one of the
first necessary steps of the installation routine?

Re: [9fans] how to fix: 'arena arenas00 creation time after last write time'

[erik quanstrom]

> >           -L   used with -r to indicate the real time clock is in
> >                local time rather than GMT.  This is useful on PCs that
> >                also run the Windows OS.
> >
>
> I must be stuck in some sort of logic error, so please correct me where I'm
> wrong:
>
> * the Plan 9 installer assumes EDT (UTC -4)
>
> * however I live in MST (UTC -7)
>
> * thus no matter what, when I set my timezone after the install, the time
>   goes _backward_ X number of hours (EDT is -4, MST is -7; so time goes

this is incorrect.  time stays the same.  you can verify with date -n.
plan 9 keeps time in utc.  in fact the timezone is just an environment
variable.  so two different programs on the same machine can be set
to different timezones.

your problem is with the rtc.  it sounds like between your install and your
reboot, the rtc was accidentally changed from utc to local time.  this
may be a bug in plan 9's installer, but i suspect it was something else
you ran.

- erik

Re: [9fans] how to fix: 'arena arenas00 creation time after last write time'

[Russ Cox]

I don't think we need to debate the exact semantics of
time on Plan 9 in order to figure this out.

It's easy to believe that the installer and the main distribution
end up with different time settings, no matter what the exact
details are.  The right fix is probably to comment out the print
in venti and move on, which you have already done.
If you are feeling particularly motivated, you could contribute
back the patch commenting out the print so that other people
don't have the same problem.

Russ

Re: [9fans] how to fix: 'arena arenas00 creation time after last write time'

[Dave Eckhardt]

> The right fix is probably to comment out the print in venti
> and move on, which you have already done.

I'd suggest: if the time is off by more than 24 hours, warn once.
That will mean timezone problems will be silently ignored, but
something "really odd" will still make it presence known.

> If you are feeling particularly motivated, you could contribute
> back the patch commenting out the print so that other people
> don't have the same problem.

Hear, hear.

Dave Eckhardt

Re: [9fans] how to fix: 'arena arenas00 creation time after last write time'

[erik quanstrom]

On Tue Jul 28 18:16:08 EDT 2009, davide+p9@cs.cmu.edu wrote:
> > The right fix is probably to comment out the print in venti
> > and move on, which you have already done.
>
> I'd suggest: if the time is off by more than 24 hours, warn once.
> That will mean timezone problems will be silently ignored, but
> something "really odd" will still make it presence known.

ignoring little bugs is the path to ruin.

- erik

Re: [9fans] how to fix: 'arena arenas00 creation time after last write time'

[Corey]

On Tuesday 28 July 2009 15:16:23 erik quanstrom wrote:
> On Tue Jul 28 18:16:08 EDT 2009, davide+p9@cs.cmu.edu wrote:
> > > The right fix is probably to comment out the print in venti
> > > and move on, which you have already done.
> >
> > I'd suggest: if the time is off by more than 24 hours, warn once.
> > That will mean timezone problems will be silently ignored, but
> > something "really odd" will still make it presence known.
>
> ignoring little bugs is the path to ruin.
>

Agreed.

But irregardless, the problem still would not be fully addressed by
simply suppressing the output in question, as the second half of
the symptoms include the behavior where venti insists on re-indexing
with every reboot.

Re: [9fans] how to fix: 'arena arenas00 creation time after last write time'

[Russ Cox]

On Tue, Jul 28, 2009 at 3:16 PM, erik quanstrom<quanstro@quanstro.net> wrote:
> ignoring little bugs is the path to ruin.

That's why the print should just go away entirely.
The code assumes that the time from one boot
to the next only ever increases, which has been
demonstrated not to be true.  Maybe during one
boot your clock is screwed up for one reason or
another and you don't notice for a few days.
Then when you fix it, venti prints messages ad
infinitum?  That's just broken behavior.

It's not a question of time zones.  Time zones don't matter.
It's just that the clock was wrong before and later is
correct--there are many reasons this might happen--
and venti shouldn't care.

The time stamps, like the ones in a file system, are
informational.  It's okay if they're wrong.  There's no
need to print.

Or should we continue to discuss tangents?

Russ

Re: [9fans] how to fix: 'arena arenas00 creation time after last write time'

[Corey]

On Tuesday 28 July 2009 18:42:05 Russ Cox wrote:
> On Tue, Jul 28, 2009 at 3:16 PM, erik quanstrom<quanstro@quanstro.net>
wrote:
> > ignoring little bugs is the path to ruin.
>
> That's why the print should just go away entirely.
> The code assumes that the time from one boot
> to the next only ever increases, which has been
> demonstrated not to be true.  Maybe during one
> boot your clock is screwed up for one reason or
> another and you don't notice for a few days.
> Then when you fix it, venti prints messages ad
> infinitum?  That's just broken behavior.
>
> It's not a question of time zones.  Time zones don't matter.
> It's just that the clock was wrong before and later is
> correct--there are many reasons this might happen--
> and venti shouldn't care.
>
> The time stamps, like the ones in a file system, are
> informational.  It's okay if they're wrong.  There's no
> need to print.
>

The problem isn't confined to unnecessary warning messages
being printed.

What about the 'arena arenas00: header is out-of-date' error,
and the subsequent re-indexing (on every reboot) which occurs
as a result of the condition?

I'm new here... so I don't know...

What I do know is that I'm having real and present issues with
venti, that are in some direct or indirect way related to at least
one or more of the following actors:

bios/rtc clock

/env/timezone

/dev/rtc

aux/timesync

Re: [9fans] how to fix: 'arena arenas00 creation time after last write time'

[Corey]

On Tuesday 28 July 2009 21:24:01 Corey wrote:
> On Tuesday 28 July 2009 18:42:05 Russ Cox wrote:
<snip>
> > It's not a question of time zones.  Time zones don't matter.
> > It's just that the clock was wrong before and later is
> > correct--there are many reasons this might happen--
> > and venti shouldn't care.
> >
> > The time stamps, like the ones in a file system, are
> > informational.  It's okay if they're wrong.  There's no
> > need to print.
>
> The problem isn't confined to unnecessary warning messages
> being printed.
>
> What about the 'arena arenas00: header is out-of-date' error,
> and the subsequent re-indexing (on every reboot) which occurs
> as a result of the condition?
<snip>

I think now that the constant indexing on reboot I'm seeing is an
entirely different problem.

It seems very similar to this:

http://www.mail-archive.com/9fans@cse.psu.edu/msg16585.html

http://www.mail-archive.com/9fans@cse.psu.edu/msg16590.html

My standalone terminal is always doing the index, the problem seemed
to have just suddenly showed up for no reason - the system hasn't crashed,
I'm not doing anything 'weird', and I always run fshalt before shutting
down. And this persists across fresh installs.

It looks like I ran into two different problems at around the same
time.

Re: [9fans] how to fix: 'arena arenas00 creation time after last write time'

[Steve Simon]

> My standalone terminal is always doing the index, the problem seemed
> to have just suddenly showed up for no reason - the system hasn't crashed,
> I'm not doing anything 'weird', and I always run fshalt before shutting
> down. And this persists across fresh installs.

Not sure what you mean by "doing the index".

When you install you will have an empty venti and a full fossil.
on the next reboot the whole of fossil will be dumped to venti.
This first dump can take a long time (some hours). After this dumps
only take (typicially) a few secconds as only changed blocks are written.

Is it possible that you did not let this intiial dump finish when
you first rebooted your machine. In that case each time you reboot
it will be trying to continue this dump (not sure if it starts again from
the begining or just from where it left off).

Perhaps this could be the "doing the index"? if so, try just leaving it
rattling overnight and then fshalt the next morning and it may come back
happy the next day.

Beware: shutting down during a dump is a bit unfriendly even if you did
an fshalt, it may be worth running a "venti check" check on fossil -
see fossilcons(8) - this is another very slow process I'am afraid.

-Steve

Re: [9fans] how to fix: 'arena arenas00 creation time after last write time'

[Russ Cox]

> The problem isn't confined to unnecessary warning messages
> being printed.
>
> What about the 'arena arenas00: header is out-of-date' error,
> and the subsequent re-indexing (on every reboot) which occurs
> as a result of the condition?

Despite the mention of "date" in the message,
the logic behind that print is not time-sensitive.
It's a different issue.

How many clumps does it say it is indexing?
How long does that take?  It shouldn't be too long.

Venti writes data blocks (clumps) to an append-only log.
Then it does two more bookkeeping steps so it
can find the data later.  The first step is to update
the block count in the arena header.  The second step
is to record an index entry for the block in the index disk
and then update the indexed block count in the arena header.
The data is committed once it is written to the log:
the bookkeeping is just an optimization.  The fact that
you are getting the "out-of-date" print means that the
first step has not happened for at least one block:
venti looked ahead and found more data blocks than
the header led it to expect.  That's fine, it just means
that venti was behind in its bookkeeping when you
shut it down.  (fshalt makes sure that the data is on
disk; it doesn't wait around for the bookkeeping.)
The next message you saw--indexing %d clumps--means
that the second step hadn't happened for a perhaps
larger number of blocks.  Indexing here means loading
information about the blocks into venti's in-memory
cache, so that lookups for the blocks will succeed.
If you leave the machine up long enough, venti will update
the on-disk index slowly, in the background.

If the pause at boot time due to this bookkeeping is
bothering you, you can flush the current venti state
to disk by running
    hget http://localhost:8000/flushdcache
    hget http://localhost:8000/flushicache    # can take a while
    hget http://localhost:8000/flushdcache
but replace localhost:8000 with your venti server's
HTTP address (maybe that's the right one; I don't know).

The fact that this is happening repeatedly suggests
that fossil is trying to do a dump to venti but you are not
leaving the machine up long enough that it can complete,
so every time you reboot, it starts writing blocks to venti
again, causing new indexing work that doesn't finish before
you reboot again.

If you want to end this cycle, the thing to do is let fossil
finish its archive and let venti finish indexing it.
If you run
    hget http://localhost:8000/storage
one if the numbers is the total number of clumps written to
the entire venti server.  If you wait a minute and run it again
and the number is growing, then fossil is writing to venti.
Let it run until the number stops growing, and then wait for
the venti flush above.

> What I do know is that I'm having real and present issues with
> venti, that are in some direct or indirect way related to at least
> one or more of the following actors:
>
> bios/rtc clock
> /env/timezone
> /dev/rtc
> aux/timesync

Again, if you've commented out the "creation time" print,
then your remaining issues are not clock-related.
I think you're just rebooting more often than usual.

Russ

Re: [9fans] how to fix: 'arena arenas00 creation time after last write time'

[Corey]

On Tuesday 28 July 2009 06:22:19 erik quanstrom wrote:
> > >           -L   used with -r to indicate the real time clock is in
> > >                local time rather than GMT.  This is useful on PCs that
> > >                also run the Windows OS.
> >
> > I must be stuck in some sort of logic error, so please correct me where
> > I'm wrong:
> >
> > * the Plan 9 installer assumes EDT (UTC -4)
> >
> > * however I live in MST (UTC -7)
> >
> > * thus no matter what, when I set my timezone after the install, the time
> >   goes _backward_ X number of hours (EDT is -4, MST is -7; so time goes
>
> this is incorrect.  time stays the same.  you can verify with date -n.
> plan 9 keeps time in utc.  in fact the timezone is just an environment
> variable.  so two different programs on the same machine can be set
> to different timezones.
>

This is useful info, thanks.

But if this is in fact certainly true in all cases, then I do not understand
why I can so easily affect venti in negative ways simply by changing the
timezone on the terminal. (I have verified with certainty that the bios 
rtc is not somehow becoming reset or changed at any point)


> your problem is with the rtc.  it sounds like between your install and your
> reboot, the rtc was accidentally changed from utc to local time.  this
> may be a bug in plan 9's installer, but i suspect it was something else
> you ran.
>

I'm guessing I'd probably wear out my welcome if I continue to press the
issue.  (c8=


But I'm a stubborn punk, so I'll here's another attempt:

The steps are short and simple, and the results are confirmed reproducible.

#0 - start with a fully 0'd hd (i.e.: no 'sticky' plan 9 partitions still
        lingering around with data; I do this using dd or shred)

#1 - boot into bios, set the cmos clock to UTC

#2 - reboot and confirm bios clock remains accurate after above change

#3 - reboot and begin Plan 9 install onto the hardware using latest Plan 9 cd

#4 - choose 'fossil+venti' for configfs, and go on to complete install using
       appropriate defaults

#5 - on first reboot after install, before first login, go into bios and confirm
       that bios clock is still in fact set accurately to UTC

#6 - reboot, (first-time login), as glenda: 
         - remove -L switch from $TIMESYNCARGS in /rc/bin/termrc
         - as an adm user: cp /adm/timezone/US_Arizona /adm/timezone/local
         - fshalt

#7 - reboot, (second-time login), as glenda: 
         - venti starts up.... aaaannd.... et voila:
           "arenas00: indexing 1169 clumps..."

From that point and onward, your venti will insist on performing the index
with every reboot and _additionally_, you will be plagued with those
previously described 'creation time after last write time' errors - which
you need to quickly cat /dev/kprint or they'll often eat up your console
( and suppressing them in arena.c obviously does nothing to prevent the
chronic (and lengthy) re-indexing of venti on every reboot ).

I'm not trying to prove that I'm not mistaken somewhere, I'm just explaining
what I'm seeing, what appears to be preventing me from setting the correct
timezone on my terminal, and the precise steps I use to repeatedly 
reproduce the symptoms involved. This is in an effort to better understand 
the problem, but more pragmatically - I just want to know how to set my 
freakin' timezone accurately without hosing venti...  <grin>


As a corollary - what makes this problem even more vexing, is that somehow
venti just kindof "sticks around", even after deleting the partitions, and
re-creating and reformatting them via the Plan 9 installer... the problem
above will continue to plague unless you fully zero out the plan 9 partitions
involved before starting the install.

 
Kind regards, your time and assistance is appreciated -

Corey

Re: [9fans] how to fix: 'arena arenas00 creation time after last write time'

[erik quanstrom]

> #6 - reboot, (first-time login), as glenda:
>          - remove -L switch from $TIMESYNCARGS in /rc/bin/termrc

if the time was already correct modulo timezone, why did you do this?
also, after allowing the machine to run for some time, does timesync
cause the system time (date -n) to jump?  i.e. is the nvram time wrong.

you don't say, but i'm quite sure that on reboot you'll find
that date -n will have jumped by 7 hrs + whatever time
it took to do the reboot.  bios time will not be affected.

>          - as an adm user: cp /adm/timezone/US_Arizona /adm/timezone/local

you should note that this has absolutely zero to do with
timesync, the bios clock or tea in china.  this changes the
human readable time only.  fossil and venti just don't care.

just as a note, you can make timezones appear instantly by
	; timezone = `{cat /adm/timezone/local}
	; date
and you can always get gmt time by
	; timezone=() date

- erik

Re: [9fans] how to fix: 'arena arenas00 creation time after last write time'

[Corey]

On Tuesday 28 July 2009 16:50:15 erik quanstrom wrote:
> > #6 - reboot, (first-time login), as glenda:
> >          - remove -L switch from $TIMESYNCARGS in /rc/bin/termrc
>
> if the time was already correct modulo timezone, why did you do this?
>

My (ill-founded?) reasoning went like this:

* -L is for if "the real time clock is in local time rather than GMT"

* my bios rtc is not set in local time, it is in UTC time

* therefore, I do not want the -L switch passed to aux/timesync


> also, after allowing the machine to run for some time, does timesync
> cause the system time (date -n) to jump?  i.e. is the nvram time wrong.
>

I do not know. I'll try this though.

Approximately how long is 'some time'? (so I know how long to wait)


> you don't say, but i'm quite sure that on reboot you'll find
> that date -n will have jumped by 7 hrs + whatever time
> it took to do the reboot.  bios time will not be affected.
>

I'll look into this also, thanks for the suggestion.


> >          - as an adm user: cp /adm/timezone/US_Arizona
> > /adm/timezone/local
>
> you should note that this has absolutely zero to do with
> timesync, the bios clock or tea in china.  this changes the
> human readable time only.  fossil and venti just don't care.
>

Please see below. My experience is indicating a different reality,
or I'm still not interpreting my experience correctly.

Clearly no one's under any obligation to help me see the light, but
I do value the time and effort spent attempting to do so!

And I realize that it's entirely possible I'm just incurably chrono-impaired.


> just as a note, you can make timezones appear instantly by
> 	; timezone = `{cat /adm/timezone/local}
> 	; date
>

Yes I was aware of this, although I did it more naively (cp
 /adm/timezone/local /env/timezone).

My point though is that yes, it makes the timezone change take
effect immediately - and it likewise causes the 'creation time after
last write time' /dev/kprint's to appear instantly as well...

And yet, timezone does not affect venti?


> and you can always get gmt time by
> 	; timezone=() date
>

Good to know - thank you.

Re: [9fans] how to fix: 'arena arenas00 creation time after last write time'

[erik quanstrom]

> Approximately how long is 'some time'? (so I know how long to wait)

5-10 minutes.  /sys/log/timesync should be informative.

> My point though is that yes, it makes the timezone change take
> effect immediately - and it likewise causes the 'creation time after
> last write time' /dev/kprint's to appear instantly as well...

impossible.  venti doesn't even share a namespace with you.

if you want to prove me wrong, modify /sys/src/cmd/vent/srv/utils.c:/^now
like so (untested)

u32int
now(void)
{
	uint t;
	static uint t0, jump;

	t = time(nil);
	if(t < t0){
		if(jump == 0)
			fprint(2, "now earlier than before: %ud %ud\n", t0, t);
		jump = 1;
		return t0;
	}
	jump = 0;
	t0 = t;
	return t;
}

- erik

Re: [9fans] how to fix: 'arena arenas00 creation time after last write time'

[Corey]

On Monday 27 July 2009 15:52:28 erik quanstrom wrote:
> > ls -l /386/bin/venti
> >
> > reboot
> >
> >
> > ... but the changes I applied to arena.c do show up; which
> > leads me to believe my changes aren't being used by the
> > system.
>
> no you're not missing anything except for the sleezy trick.
> venti is built into the kernel.  you will also need to rebuild
> your kernel.
>

Cool - that did the trick.


So, I edited the offending line in arena.c to output the arena->ctime and
arena->wtime, and it appears to resemble the situation Lyndon explained:

"I suspect what's happening is the motherboard clock is set in
the future, you are formatting venti based on that time, and then later
firing up timesync which interprets the RTC as local time. If your RTC is
set to UTC and you're in the western hemisphere, the RTC clock will be
ahead of your local time."


... it's kindofa bummer that apparently I can't setup fossil+venti during
the plan 9 install process without hosing up venti when I change my
timezone appropriately to get the correct time on the machine...

So I guess I either:

* keep the machine on UTC, which allows me to install fossil+venti
  during installation

... or:

* set up venti manually after the install, and after I've set the timezone


I"m surprised this issue doesn't seem to show up more often. I've only seen
two different instances of this brought up on 9fans.

This person asked what seems a reasonable question, but was not answered:

http://article.gmane.org/gmane.os.plan9.general/39933

He points out:

"[...] perhaps the option of changing the timezone needs to be included in
the install script if venti is to be installed."


Cheers, and thanks for the assistance guys!

Re: [9fans] how to fix: 'arena arenas00 creation time after last write time'

[erik quanstrom]

> "I suspect what's happening is the motherboard clock is set in
> the future, you are formatting venti based on that time, and then later
> firing up timesync which interprets the RTC as local time. If your RTC is
> set to UTC and you're in the western hemisphere, the RTC clock will be
> ahead of your local time."
>
>
> ... it's kindofa bummer that apparently I can't setup fossil+venti during
> the plan 9 install process without hosing up venti when I change my
> timezone appropriately to get the correct time on the machine...
>
> So I guess I either:
>
> * keep the machine on UTC, which allows me to install fossil+venti
>   during installation
>

; man timesync

     TIMESYNC(8)                                           TIMESYNC(8)

 [...]

          -L   used with -r to indicate the real time clock is in
               local time rather than GMT.  This is useful on PCs that
               also run the Windows OS.

- erik

Re: [9fans] how to fix: 'arena arenas00 creation time after last write time'

[Steve Simon]

> * set up venti manually after the install, and after I've set the timezone

Sorry if I'am being a pedant, but its not the timezone - that
is an offset that effects how dates and times are printed, plan9
like Unix always uses UTC for all time/date stamps. The problem is
the time in the RTC chip was wrong.

there is a recuring problem that Windows sets the RTC to localtime
whereas Unix expects it to be UTC. There is an option to timesync
to inform it if you want to continue using localtime on your RTC
(because you want to dual boot with windows).

-Steve

[9fans] how to fix: 'arena arenas00 creation time after last write time'

[Corey]

The following is being printed to the console non-stop:

err 2: arena arenas00 creation time after last write time


... and I saw this during bootup:

arena arenas00: header is out-of-date


Apparently my clock/date was set a day ahead when I installed the terminal.

How do I correct the situation?

I searched around, but only found one other post regarding this problem,
but I did not see a solution in that thread:

http://www.mail-archive.com/9fans@cse.psu.edu/msg08167.html

Re: [9fans] how to fix: 'arena arenas00 creation time after last write time'

[Russ Cox]

On Sun, Jul 26, 2009 at 6:03 PM, Corey<corey@bitworthy.net> wrote:
> The following is being printed to the console non-stop:
> err 2: arena arenas00 creation time after last write time
> arena arenas00: header is out-of-date
> Apparently my clock/date was set a day ahead when I installed the terminal.
>
> How do I correct the situation?

You could comment out the print in the venti server,
or make it only print once and then shut up.

Russ

Re: [9fans] how to fix: 'arena arenas00 creation time after last write time'

[Corey]

On Monday 27 July 2009 09:28:39 Russ Cox wrote:
> On Sun, Jul 26, 2009 at 6:03 PM, Corey<corey@bitworthy.net> wrote:
> > The following is being printed to the console non-stop:
> > err 2: arena arenas00 creation time after last write time
> > arena arenas00: header is out-of-date
> > Apparently my clock/date was set a day ahead when I installed the
> > terminal.
> >
> > How do I correct the situation?
>
> You could comment out the print in the venti server,
> or make it only print once and then shut up.
>

Thankyou for your response;  just to clarify: you're saying that I need to
edit the venti source and recompile in order to fix the problem described?

One thing that's weird and vexing though, is that this problem
seems to have come out of nowhere (I had successfully installed and
reinstalled on this machine about a dozen times over the past few days,
each time using the same process/steps) - and now it doesn't go away,
even on a completely fresh install, even after I wiped the drives completely.

Now, every time I install plan 9 on this machine, even when I don't change
the clock or timezone in any way, it ends up with this issue where venti
complains about 'arenas00: header is out-of-date', then indexes clumps,
then starts rolling out those 'creation time after last write time' messages.
I have confirmed and verified this continues even after multiple attempts
at a 100% prestine re-install.

Any ideas what could be the cause?  I wouldn't care quite so much if it was
just some sort of fluke that was remedied after another clean install - but
it's persisting even after I totally wipe the drive (using dd and shred from
a linux recovery disk), delete the partition table, and start from scratch.


Any advice/suggestions is appreciated.

Re: [9fans] how to fix: 'arena arenas00 creation time after last write time'

[erik quanstrom]

> each time using the same process/steps) - and now it doesn't go away,
> even on a completely fresh install, even after I wiped the drives completely.

what's your disk wiping procdure?

> Now, every time I install plan 9 on this machine, even when I don't change
> the clock or timezone in any way, it ends up with this issue where venti
> complains about 'arenas00: header is out-of-date', then indexes clumps,
> then starts rolling out those 'creation time after last write time' messages.
> I have confirmed and verified this continues even after multiple attempts
> at a 100% prestine re-install.
>
> Any ideas what could be the cause?  I wouldn't care quite so much if it was
> just some sort of fluke that was remedied after another clean install - but
> it's persisting even after I totally wipe the drive (using dd and shred from
> a linux recovery disk), delete the partition table, and start from scratch.

motherboard battery dead?  as long as you're editing venti
source, you could print out both dates.  that would give a
better picture of what's really going on.

- erik

Re: [9fans] how to fix: 'arena arenas00 creation time after last write time'

[Corey]

On Monday 27 July 2009 13:38:58 erik quanstrom wrote:
> > each time using the same process/steps) - and now it doesn't go away,
> > even on a completely fresh install, even after I wiped the drives
> > completely.
>
> what's your disk wiping procdure?
>

I reboot with a gentoo linux rescue cd.

I run the following.

shred -zvun 1 /dev/hda1 # wipe data, zero out (takes about 90 minutes)

dd if=/dev/zero of=/dev/hda bs=512 count=1 # clear mbr

Then I delete the org plan 9 partition (/dev/hda1; i.e. /dev/sdC0) using fdisk

Finally, I reboot with the same plan 9 install cd I've been using, and
go through the standard install using fossil+venti


Then, after the install:

Log into system for first time as glenda; everything apparently ok at
this point.

But, when I fshalt, then reboot - at that point, the second login into the
machine, is when I start seeing the venti described venti errors.

I have repeated the above steps three times now, and have confidently
determined that the condition is persistent, and repeatable.

I'm also fairly certain that I'm not doing anything 'wierd', and there has
been no change in hardware or behavior of the machine throughout the
entire time I've been experimenting.



> > Now, every time I install plan 9 on this machine, even when I don't
> > change the clock or timezone in any way, it ends up with this issue where
> > venti complains about 'arenas00: header is out-of-date', then indexes
> > clumps, then starts rolling out those 'creation time after last write
> > time' messages. I have confirmed and verified this continues even after
> > multiple attempts at a 100% prestine re-install.
> >
> > Any ideas what could be the cause?  I wouldn't care quite so much if it
> > was just some sort of fluke that was remedied after another clean install
> > - but it's persisting even after I totally wipe the drive (using dd and
> > shred from a linux recovery disk), delete the partition table, and start
> > from scratch.
>
> motherboard battery dead?
>

Nope, I checked that after the second time I hit the issue all over again; and
verified the cmos time is correct.


> as long as you're editing venti
> source, you could print out both dates.  that would give a
> better picture of what's really going on.
>

Ok, I'll try this next.


Cheers

Re: [9fans] how to fix: 'arena arenas00 creation time after last write time'

[Corey]

On Monday 27 July 2009 13:38:58 erik quanstrom wrote:
> as long as you're editing venti
> source, you could print out both dates.  that would give a
> better picture of what's really going on.
>

I think I must be missing something, this is what I did:


cd /sys/src/cmd/venti/srv

acme arena.c

cd ..

mk all &&  mk install

ls -l /386/bin/venti

reboot


... but the changes I applied to arena.c do show up; which
leads me to believe my changes aren't being used by the
system.

When I edit source under /sys/src/, is there something else I
need to do besides running mk as above?


Thanks

Re: [9fans] how to fix: 'arena arenas00 creation time after last write time'

[erik quanstrom]

>
> ls -l /386/bin/venti
>
> reboot
>
>
> ... but the changes I applied to arena.c do show up; which
> leads me to believe my changes aren't being used by the
> system.

no you're not missing anything except for the sleezy trick.
venti is built into the kernel.  you will also need to rebuild
your kernel.

it really pays to add a new kernel in a standalone system
in a seperate 9load menu option, in case your new kernel has
trouble for one reason or another.

- erik

Re: [9fans] how to fix: 'arena arenas00 creation time after last write time'

[Lyndon Nerenberg]

What does the BIOS setup screen say about the motherboard clock's idea of
the time? I suspect what's happening is the motherboard clock is set in
the future, you are formatting venti based on that time, and then later
firing up timesync which interprets the RTC as local time. If your RTC is
set to UTC and you're in the western hemisphere, the RTC clock will be
ahead of your local time.

The venti message spits out two ctime values IIRC. One is the timestamp
from when venti created the store, the other the block write time. date(1)
will decipher the values, and tell the true story.

Re: [9fans] how to fix: 'arena arenas00 creation time after last write time'

[Corey]

On Tuesday 28 July 2009 17:42:23 Corey wrote:
> My experience is indicating a different reality,
> or I'm still not interpreting my experience correctly.
>

It was definitely the latter.


On Tuesday 28 July 2009 16:50:15 erik quanstrom wrote:
> you should note that this has absolutely zero to do with
> timesync, the bios clock or tea in china.  this changes the
> human readable time only.  fossil and venti just don't care.
>
On Wednesday 29 July 2009 07:48:54 Russ Cox wrote:
> The fact that this is happening repeatedly suggests
> that fossil is trying to do a dump to venti but you are not
> leaving the machine up long enough that it can complete,
> so every time you reboot, it starts writing blocks to venti
> again, causing new indexing work that doesn't finish before
> you reboot again.
>
On Wednesday 29 July 2009 07:02:01 Steve Simon wrote:
> Is it possible that you did not let this intiial dump finish when
> you first rebooted your machine. In that case each time you reboot
> it will be trying to continue this dump
>

Erik, Russ and Steve - thanks a ton for sticking with me on that;
and for explaining things so well. Very much appreciated.

Quick summary of what went wrong for me there:

It's exactly as Russ and Steve described - at some point I began
rebooting the terminal before fossil had finished dumping to venti.
The reason I became so confused was because prior to that, I had
been leaving my terminal up for long periods between reboots. It
wasn't until I first attempted to change the timezone that I _also_
began rebooting my terminal soon after the first-time login after
an install had completed. I was ignorant of how fossil+venti
interacted, and I had not yet experienced the results of rebooting
before the dump had completed - so I fixated on the timezone as
the only thing that seemed to be connected to the "problem" I was
seeing - and the 'creation time after last write time' error from venti
was an additional red-herring.

Thanks again for helping me along guys, and to Russ for providing
such an in depth look into what I was missing regarding fossil+venti;
way cool!


Respectfully,

Corey




On Sunday 26 July 2009 18:03:53 Corey wrote:
> The following is being printed to the console non-stop:
>
> err 2: arena arenas00 creation time after last write time
>
>
> ... and I saw this during bootup:
>
> arena arenas00: header is out-of-date
>
>
> Apparently my clock/date was set a day ahead when I installed the terminal.
>
> How do I correct the situation?
>
> I searched around, but only found one other post regarding this problem,
> but I did not see a solution in that thread:
>
> http://www.mail-archive.com/9fans@cse.psu.edu/msg08167.html
>