[9fans] rio and acme scrolling

[9fans] rio and acme scrolling

[Micah Stetson]

Lately, I've been having a problem with scrolling under
acme and rio.  Basically, unless I am VERY quick with my
click on the scroll bar, the window scrolls much more
than a single click should.  The delay that used to occur
between the initial one-click scroll and the hold-the-button
continuous scrolling is no longer there.  However, I
don't see any recent changes (looking in sources' dump)
that look to me like they would affect that.

Also, the symptoms are kind of weird.  When I boot my
laptop off of my newly assembled (yay) Fossil/Venti/Auth
server, which is up to date as of today, I get the effect
consistently.  I think I remember seeing it happen booting
from the hard disk on my laptop a few weeks ago, but I
can't be sure.  Still, I booted on my laptop this afternoon
and couldn't reproduce the problem.  Even when I would
cpu to the new server and run the rio or acme binary from
there, it didn't happen.  So I updated the laptop from
sources (it hadn't been since late February) and continued
to puzzle about it.  I did not reboot, and unsurprisingly,
the behavior didn't change.  But after the update, I left
the machine sitting for an hour or two while I worked on
other things.  When I came back, I got the weird scrolling.
But I still haven't rebooted.  Now this puzzles me more
than everything else put together.  What happened while
I wasn't looking?

Thanks,

Micah

Re: [9fans] rio and acme scrolling

[Russ Cox]

When you get the weird scrolling, try:

date; sleep 10; date

and see if the clock is running at the right speed.
Look at a real clock too.  Sleep 10 is what will
be too fast if the clock is bogus.  The date may
actually be accurate, thanks to timesync.

Russ

Re: [9fans] rio and acme scrolling

[Micah Stetson]

On Fri, Mar 26, 2004 at 08:51:26PM -0500, Russ Cox wrote:
> date; sleep 10; date

On Fri, Mar 26, 2004 at 10:14:49PM -0500, jmk@plan9.bell-labs.com wrote:
> We discovered that when the laptop has the lid closed, on re-opening
> some of the clocks have been altered by the BIOS/APM. The real
 ...
> As Presotto said, time is, indeed, slippin' slippin' slippin' into
> the future.

Thanks, guys, that's exactly it.  When I boot the system
from the hard disk, it doesn't have the problem.  But if
I simply close the lid and reopen it, windows scroll like
lightning and sleep 10 sleeps for about 3 seconds.  The
thought of the clock running out of control had flitted
into my mind yesterday, but I didn't actually consider
it, that should teach me.

Micah

Re: [9fans] rio and acme scrolling

[David Presotto]

[-- Attachment #1: Type: text/plain, Size: 783 bytes --]

The clock (timer 2) speeds up by a factor of 5 or so.
I can only assume that the firmware uses it for something
to do with detecting the lid opening again.

Timesync doesn't change the local hardware, it just
tweaks the kernel's idea of what time it is and of
what the clock frequency is.  It would cope with
the clock change except that I made timesync ignore any huge
swings (anything more than 25%) because I had assumed that
would only be the result of a samplng mistake.

I changed the limits and timesync will now put up with such a
change, though it will take some minutes to latch onto
the exact new value (needs enough of a time base to
compare to).

Tell me if this is a problem.  Short of actually
implementing ACPI, I think this is as good as we can
do.

[-- Attachment #2: Type: message/rfc822, Size: 2874 bytes --]

From: Micah Stetson <micah@cnm-vra.com>
To: 9fans@cse.psu.edu
Subject: Re: [9fans] rio and acme scrolling
Date: Fri, 26 Mar 2004 21:07:54 -0800
Message-ID: <20040327050754.GA20193@epaphras.cnm-vra.com>

On Fri, Mar 26, 2004 at 08:51:26PM -0500, Russ Cox wrote:
> date; sleep 10; date

On Fri, Mar 26, 2004 at 10:14:49PM -0500, jmk@plan9.bell-labs.com wrote:
> We discovered that when the laptop has the lid closed, on re-opening
> some of the clocks have been altered by the BIOS/APM. The real
 ...
> As Presotto said, time is, indeed, slippin' slippin' slippin' into
> the future.

Thanks, guys, that's exactly it.  When I boot the system
from the hard disk, it doesn't have the problem.  But if
I simply close the lid and reopen it, windows scroll like
lightning and sleep 10 sleeps for about 3 seconds.  The
thought of the clock running out of control had flitted
into my mind yesterday, but I didn't actually consider
it, that should teach me.

Micah

Re: [9fans] rio and acme scrolling

[Sape Mullender]

> The clock (timer 2) speeds up by a factor of 5 or so.
> I can only assume that the firmware uses it for something
> to do with detecting the lid opening again.
> 
> Timesync doesn't change the local hardware, it just
> tweaks the kernel's idea of what time it is and of
> what the clock frequency is.  It would cope with
> the clock change except that I made timesync ignore any huge
> swings (anything more than 25%) because I had assumed that
> would only be the result of a samplng mistake.
> 
> I changed the limits and timesync will now put up with such a
> change, though it will take some minutes to latch onto
> the exact new value (needs enough of a time base to
> compare to).
> 
> Tell me if this is a problem.  Short of actually
> implementing ACPI, I think this is as good as we can
> do.

Does this work if timesync doesn't have an external time source to synchronize to?

	Sape

Re: [9fans] rio and acme scrolling

[David Presotto]

[-- Attachment #1: Type: text/plain, Size: 149 bytes --]

You always have the local real time clock.  It doesn't tick very
often but is fairly accurate.

	aux/timesync -r

This is the default in termrc.

[-- Attachment #2: Type: message/rfc822, Size: 2845 bytes --]

From: Sape Mullender <sape@huygens.org>
To: 9fans@cse.psu.edu
Subject: Re: [9fans] rio and acme scrolling
Date: Sat, 27 Mar 2004 10:07:12 -0500
Message-ID: <609fa6e57786ca89ae54d85144ba7630@plan9.bell-labs.com>

> The clock (timer 2) speeds up by a factor of 5 or so.
> I can only assume that the firmware uses it for something
> to do with detecting the lid opening again.
> 
> Timesync doesn't change the local hardware, it just
> tweaks the kernel's idea of what time it is and of
> what the clock frequency is.  It would cope with
> the clock change except that I made timesync ignore any huge
> swings (anything more than 25%) because I had assumed that
> would only be the result of a samplng mistake.
> 
> I changed the limits and timesync will now put up with such a
> change, though it will take some minutes to latch onto
> the exact new value (needs enough of a time base to
> compare to).
> 
> Tell me if this is a problem.  Short of actually
> implementing ACPI, I think this is as good as we can
> do.

Does this work if timesync doesn't have an external time source to synchronize to?

	Sape

Re: [9fans] rio and acme scrolling

[Derek Fawcus]

On Sat, Mar 27, 2004 at 10:12:47AM -0500, David Presotto wrote:
> You always have the local real time clock.  It doesn't tick very
> often but is fairly accurate.

One can always reprogram it to tick faster (I've previously used 1024 Hz)

In fact one can even use it as the systems clock - though I've only ever
come across one OS that did.

DF

Re: [9fans] rio and acme scrolling

[Richard Miller]

> The clock (timer 2) speeds up by a factor of 5 or so.
> I can only assume that the firmware uses it for something
> to do with detecting the lid opening again.

This seems to be a particular problem with thinkpads - and it's
not just closing the lid: screen blanking will reset the clock
period as well.

Fixing it with timesync is not good enough, because bad things
can still happen while the clock is racing until timesync notices.
In particular, the timeout waiting for a spun-down hard drive
to come up is much too short, resulting in spurious "disk errors" -
I've lost my fossil fs twice because of this.  (Thanks to venti,
this was an irritation rather than a disaster.)

Here's the workaround I use to detect the loss of the clock period
and repair it as soon as it happens.  You'll still lose the odd
millisecond but it's better than an ongoing 5x clock overrun:

term% diff /sys/src/9/pc/i8253.c i8253.c
268c269
< 	else
---
> 	else {
269a271,278
> 		if (x > 3*MaxPeriod) {
> 			outb(Tmode, Load2|Square);
> 			outb(T2cntr, 0);		/* low byte */
> 			outb(T2cntr, 0);		/* high byte */
> 			y = 0xFFFF;
> 			x = i8253.period;
> 		}
> 	}

Alternatively, why not use the TSC when there is one?  I note that
this is the default for the MP architecture, but only for CPU server
kernels -- does anyone know why?

Here's a one-line fix which also seems to work on my thinkpad:

term% diff /sys/src/9/pc/devarch.c devarch.c
692a693
> 		archgeneric.fastclock = tscticks;

-- Richard

Re: [9fans] rio and acme scrolling

[Sape Mullender]

> Alternatively, why not use the TSC when there is one?  I note that
> this is the default for the MP architecture, but only for CPU server
> kernels -- does anyone know why?

TSC is a problem on laptops too.  The TSC frequency changes as the
power management changes the CPU speed.  This was why we didn't
use it.

	Sape

Re: [9fans] rio and acme scrolling

[David Presotto]

[-- Attachment #1: Type: text/plain, Size: 446 bytes --]

Too many of the laptops systems have clock scaling for power
conservarion.  I used to use the TSC everywhere I could but
it's even more of a problem with laptops.  People kept complaining
that when plugging/unplugging their machines into power supplies
or when their batteries ran down, time would speed up or slow
down.  I use it on my own laptop.

I agree that timesync doesn't notice anywhere near fast enough.
I'll add ytour hack too.

[-- Attachment #2: Type: message/rfc822, Size: 3264 bytes --]

From: Richard Miller <rm@hamnavoe.com>
To: 9fans@cse.psu.edu
Subject: Re: [9fans] rio and acme scrolling
Date: Sat, 27 Mar 2004 15:22:57 0000
Message-ID: <78e57cf1aaa73ca03a1e3a5df2385bd8@hamnavoe.com>

> The clock (timer 2) speeds up by a factor of 5 or so.
> I can only assume that the firmware uses it for something
> to do with detecting the lid opening again.

This seems to be a particular problem with thinkpads - and it's
not just closing the lid: screen blanking will reset the clock
period as well.

Fixing it with timesync is not good enough, because bad things
can still happen while the clock is racing until timesync notices.
In particular, the timeout waiting for a spun-down hard drive
to come up is much too short, resulting in spurious "disk errors" -
I've lost my fossil fs twice because of this.  (Thanks to venti,
this was an irritation rather than a disaster.)

Here's the workaround I use to detect the loss of the clock period
and repair it as soon as it happens.  You'll still lose the odd
millisecond but it's better than an ongoing 5x clock overrun:

term% diff /sys/src/9/pc/i8253.c i8253.c
268c269
< 	else
---
> 	else {
269a271,278
> 		if (x > 3*MaxPeriod) {
> 			outb(Tmode, Load2|Square);
> 			outb(T2cntr, 0);		/* low byte */
> 			outb(T2cntr, 0);		/* high byte */
> 			y = 0xFFFF;
> 			x = i8253.period;
> 		}
> 	}

Alternatively, why not use the TSC when there is one?  I note that
this is the default for the MP architecture, but only for CPU server
kernels -- does anyone know why?

Here's a one-line fix which also seems to work on my thinkpad:

term% diff /sys/src/9/pc/devarch.c devarch.c
692a693
> 		archgeneric.fastclock = tscticks;

-- Richard

Re: [9fans] rio and acme scrolling

[Russ Cox]

>Alternatively, why not use the TSC when there is one?  I note that
>this is the default for the MP architecture, but only for CPU server
>kernels -- does anyone know why?
>
>Here's a one-line fix which also seems to work on my thinkpad:
>
>term% diff /sys/src/9/pc/devarch.c devarch.c
>692a693
>  
>
>>		archgeneric.fastclock = tscticks;
>>    
>>

The TSC is not reliable once the BIOS starts playing tricks
with CPU speeds.  Try unplugging your Thinkpad and see
if your TSC keeps running at the same speed.

Russ

Re: [9fans] rio and acme scrolling

[Richard Miller]

> Try unplugging your Thinkpad and see
> if your TSC keeps running at the same speed.

I have, and it does.  But there must be other circumstances where it
wouldn't (it is supposed to have "SpeedStep™ Technology", after all).
I may have thought about this and then forgotten again, after I
settled on the other fix.

> What's really nuts is that after rebooting with your fix and closing/opening
> the lid a few times now my t23 runs much colder and the fan rarely spins up.

No idea why this should be - maybe some other APM side effect in the BIOS?
But I daren't close the lid on my t21, because the vga doesn't recover properly
(the left third of the screen image is replicated three times).  The t23 must
use a different vga chip.

-- Richard

Re: [9fans] rio and acme scrolling

[andrey mirtchovski]

[snip]
> term% diff /sys/src/9/pc/i8253.c i8253.c
[snip]

to confirm it works on my T23, thanks!

Re: [9fans] rio and acme scrolling

[Micah Stetson]

> This seems to be a particular problem with thinkpads - and it's
> not just closing the lid: screen blanking will reset the clock
> period as well.

Yeah, mine's a T22, and before that I had a 600.  Thinkpads
seem to be very common on this list.  Is it because they
have three buttons on the builtin pointer thing?  That
was one of the selling points for me.

Micah

thinkpads (was Re: [9fans] rio and acme scrolling)

[Axel Belinfante]

Micah wrote:
> Yeah, mine's a T22, and before that I had a 600.  Thinkpads
> seem to be very common on this list.  Is it because they
> have three buttons on the builtin pointer thing?  That
> was one of the selling points for me.

(I don't have one, unfortunately)
it would be the sellng point for me
(the other would be my (correct or not?) impression of robustness)

Axel.

Re: [9fans] rio and acme scrolling

[matt]

> Is it because they have three buttons on the builtin pointer thing?

of the 21 laptops listed in the supported hardware list, 10 of them are IBM Thinkpads

Of the remainder, many are hard to find and when you do find them, you discover they are early pentium 1s.

The only non IBM one's from the list I've regularly found on ebay.co.uk were the Dell Inspiron 7000 and the Toshiba Portege 3440CT, which are at the expensive end of the spectrum.

m

Re: [9fans] rio and acme scrolling

[splite]

On Sun, Mar 28, 2004 at 12:07:46PM +0100, matt@proweb.co.uk wrote:
> The only non IBM one's from the list I've regularly found on ebay.co.uk
> were the Dell Inspiron 7000 and the Toshiba Portege 3440CT, which are
> at the expensive end of the spectrum.

The cost for the Inspiron is in the shipping; the thing's a tank.

I just did a net install on an Inspiron 3500 and it's working fine, modulo
sound.  Same Pentium II, XGA TFT, and 440BX chipset as the 7000, but about
2/3 the height and a bit lighter.  Sadly, it also has the same detestable
two-button trackpad.

Would be nice if one of the useless "billg ego keys" could be used as a
third mouse button instead of shift+button 2.

Re: [9fans] rio and acme scrolling

[Micah Stetson]

> The clock (timer 2) speeds up by a factor of 5 or so.

I think this might also explain another problem I've had.
Sometimes, when I've booted from the fileserver using
IL, my laptop will work fine for a while and then decide
that the fileserver connection has hung up.  Running
netstat on the server says it's still established.  This
has happened 2 or 3 times in as many days, and, if I
remember correctly, always after a screen blank.  I
haven't noticed it using TCP, but I've only booted with
TCP once or twice.  I'm thinking the factor five speedup
is causing a race between the 30 second keep-alive timer
and the 6-second delay before a query is sent on an
inactive connnection.  When the timer wins, the connection
is torn down.

I've also seen a disk error or two recently from fossil
on this machine, as someone else talked about in this
thread.

It's so nice to have something to blame it all on.

Micah

Re: [9fans] rio and acme scrolling

[Micah Stetson]

On Sat, Mar 27, 2004 at 12:27:13PM -0800, Micah Stetson wrote:
> TCP once or twice.  I'm thinking the factor five speedup
> is causing a race between the 30 second keep-alive timer
> and the 6-second delay before a query is sent on an
> inactive connnection.  When the timer wins, the connection
> is torn down.

Well, if that was happening, it wasn't the only problem.
I just had the fileserver connection hang up without APM
being involved at all (as far as I can see).  I ran sleep
from my hard disk and it took the appropriate amount of
time.

Back to puzzling,

Micah

Re: [9fans] rio and acme scrolling

[jmk]

I remarked on this yesterday at work when it happened on my laptop.
We discovered that when the laptop has the lid closed, on re-opening
some of the clocks have been altered by the BIOS/APM. The real
solution would be to make Plan 9 deal with these events properly,
but in the meantime we're testing something else.

As Presotto said, time is, indeed, slippin' slippin' slippin' into
the future.

Re: [9fans] rio and acme scrolling

[a]

// As Presotto said, time is, indeed, slippin' slippin' slippin'
// into the future.

huh. that was presotto? coulda' *swore* that was someone else...
steve something-or-other, maybe...
ア

Re: [9fans] rio and acme scrolling

[Dave Lukes]

On Sat, 2004-03-27 at 10:21, a@9srv.net wrote:
> // As Presotto said, time is, indeed, slippin' slippin' slippin'
> // into the future.
> 
> huh. that was presotto? coulda' *swore* that was someone else...
> steve something-or-other, maybe...

Not quite ...  That was "Time _keeps on_ slippin', slippin' ..."
(my emphasis)

Back to your usual programming:-)

	Dave.

Re: [9fans] rio and acme scrolling

[blstuart]

rsc@swtch.com (Russ Cox) wrote:
>When you get the weird scrolling, try:
>
>date; sleep 10; date
>
>and see if the clock is running at the right speed.
>Look at a real clock too.  Sleep 10 is what will
>be too fast if the clock is bogus.  The date may
>actually be accurate, thanks to timesync.

I've seen the same sort of thing with the new
build of acme on Linux.  The really wierd thing
is that I've seen a similar thing in Inferno
running hosted in Linux.  Occationally, on
a single click, it takes off as if I were holding
it, and only stops if I move the mouse out of
the scroll bar.

Brian

Re: [9fans] rio and acme scrolling

[Russ Cox]

blstuart@bellsouth.net wrote:

>rsc@swtch.com (Russ Cox) wrote:
>  
>
>>When you get the weird scrolling, try:
>>
>>date; sleep 10; date
>>
>>and see if the clock is running at the right speed.
>>Look at a real clock too.  Sleep 10 is what will
>>be too fast if the clock is bogus.  The date may
>>actually be accurate, thanks to timesync.
>>    
>>
>
>I've seen the same sort of thing with the new
>build of acme on Linux.  The really wierd thing
>is that I've seen a similar thing in Inferno
>running hosted in Linux.  Occationally, on
>a single click, it takes off as if I were holding
>it, and only stops if I move the mouse out of
>the scroll bar.
>  
>

I've seen this under FreeBSD under VMware.  I think
it's an XFree86 bug.  When I stopped using VMware
and FreeBSD, the problem went away, but I'm sure
I'm also running a different version of XFree86.

Re: [9fans] rio and acme scrolling

[matt]

I have had this in native plan9 in acme more than once

m

Re: [9fans] rio and acme scrolling

[Axel Belinfante]

I have slightly faster than expected scrolling in ports acme.
On the other hand, button2 in acme scroll bar sometimes is slow,
and a bit jerky (and 'elavator' overshoots?)
as if it has trouble keeping up with the mouse movement.

Re: [9fans] rio and acme scrolling

[Axel Belinfante]

> I have slightly faster than expected scrolling in ports acme.
> On the other hand, button2 in acme scroll bar sometimes is slow,
> and a bit jerky (and 'elavator' overshoots?)
> as if it has trouble keeping up with the mouse movement.

To be more precise:
I run acme 'on' a sunos vnc server at work, vnc client in plan 9 at home.
(cable modem connection)
I click button 1 or button 3 _once_ in a scroll bar: it scrolls _twice_.
Of course, it also happens that it scrolls only once.

Axel.

[9fans] hget

[rog]

i just noticed that hget always seeks to the start of its output
fd, which meant that the script i was trying to run:

	{for(i in $urls) hget $i} > allurls

can't work. (and it does seem reasonable to be able to pipe hget
through other commands, etc).

a simple fix won't work, due to the "partial content" error,
and the fact that hget restarts after error.

does anyone know how common these conditions are?

i often do:

	<hget someurl

in acme (highlight, then middle-button click, to substitute the
contents of the url); i realise now that this won't always work.

should hget get to a temporary file when writing to stdout,
or avoid retry, or... ?

has anyone got strong views on this?

Re: [9fans] hget

[Russ Cox]

rog@vitanuova.com wrote:

>i often do:
>
>	<hget someurl
>
>in acme (highlight, then middle-button click, to substitute the
>contents of the url); i realise now that this won't always work.
>  
>

it won't work if the connection gets halfway done, hangs up,
and the remote server doesn't support content-range requests.
but how often are you in this situation?

Re: [9fans] hget

[rog]

> it won't work if the connection gets halfway done, hangs up,
> and the remote server doesn't support content-range requests.
> but how often are you in this situation?

actually, it seems (from the hget code) that the server can return
error 206 (partial content) which causes a seek on the output, even if
the content-range request is supported.  i've no idea how common this
is though.

i was thinking of changing hget so that if it's writing to stdout, it
would give up with an error rather than trying to seek back and try
again.

the would at least mean you could use it in a pipeline without
worrying about occasionally duplicated portions of output.

an option similar to -o but without the modification time semantics
could yield the current behaviour (ideally, i'd make the modification
time stuff a separate option (-u?)  and use it in addition to -o to
get the current -o semantics; but that would break some things).

Re: [9fans] hget

[Russ Cox]

rog@vitanuova.com wrote:

>>it won't work if the connection gets halfway done, hangs up,
>>and the remote server doesn't support content-range requests.
>>but how often are you in this situation?
>>    
>>
>
>actually, it seems (from the hget code) that the server can return
>error 206 (partial content) which causes a seek on the output, even if
>the content-range request is supported.  i've no idea how common this
>is though.
>
>i was thinking of changing hget so that if it's writing to stdout, it
>  
>

down that path lies madness...

>would give up with an error rather than trying to seek back and try
>again.
>  
>

how about hget -1 meaning "try once, then give up."?

Re: [9fans] hget

[rog]

> how about hget -1 meaning "try once, then give up."?

no, 'cos it's fine to try again, just as long as you haven't already
written anything.  (and i'll bet that's by far the most common failure
mode).

> down that path lies madness...

hmm. maybe.

however we're already admitting a qualititive difference in types of
file by using the -o option as it is.  (oh no, danger!  danger!  file
seek thread trying to re-establish; apologies!)

at least when you're creating a file, you're reasonably sure you can
seek on it ok (although of course that's not the case for >{}).

i think the default case should be not to seek, because that
will produce less surprises.

Re: [9fans] hget

[Russ Cox]

>however we're already admitting a qualititive difference in types of
>file by using the -o option as it is.  (oh no, danger!  danger!  file
>seek thread trying to re-establish; apologies!)
>
>at least when you're creating a file, you're reasonably sure you can
>seek on it ok (although of course that's not the case for >{}).
>
>i think the default case should be not to seek, because that
>will produce less surprises.
>  
>

right enough.   then how about hget -1 meaning "one shot at writing data"
(or if that doesn't describe it, "what rog thinks it should do"  ;-) )
my point is mainly that i'd rather see an option than some magic
guessing that will confuse people once they see the difference.
i do hget >file all the time, when i want to make sure i get a new
copy (whereas hget -o file will not replace the file if the server
or some broken intermediate cache thinks things are up-to-date).

Re: [9fans] hget

[rog]

[-- Attachment #1: Type: text/plain, Size: 847 bytes --]

> whereas hget -o file will not replace the file if the server
> or some broken intermediate cache thinks things are up-to-date

that's why i thought it might be reasonable to divorce the "look at
modification time of file" semantics of -o from the "can seek on
output" semantics of same.

i've never used -o as it currently is because i don't trust
computers to keep or record time properly, either here or out there
(i've the same problem with my thinkpad time speeding up as everyone
else).

i'd expect

	hget | wc

to produce the same results as

	hget > file
	wc < file

>  then how about hget -1 meaning "one shot at writing data"

i think the option should the other way around,
e.g. hget -m meaning "multiple shots at writing data.

then the above correspondence still holds.
(and -o still does the same thing)

[-- Attachment #2: Type: message/rfc822, Size: 2891 bytes --]

From: Russ Cox <rsc@swtch.com>
To: 9fans@cse.psu.edu
Subject: Re: [9fans] hget
Date: Mon, 29 Mar 2004 15:42:01 -0500
Message-ID: <40688A19.3000602@swtch.com>


>however we're already admitting a qualititive difference in types of
>file by using the -o option as it is.  (oh no, danger!  danger!  file
>seek thread trying to re-establish; apologies!)
>
>at least when you're creating a file, you're reasonably sure you can
>seek on it ok (although of course that's not the case for >{}).
>
>i think the default case should be not to seek, because that
>will produce less surprises.
>  
>

right enough.   then how about hget -1 meaning "one shot at writing data"
(or if that doesn't describe it, "what rog thinks it should do"  ;-) )
my point is mainly that i'd rather see an option than some magic
guessing that will confuse people once they see the difference.
i do hget >file all the time, when i want to make sure i get a new
copy (whereas hget -o file will not replace the file if the server
or some broken intermediate cache thinks things are up-to-date).

Re: [9fans] hget

[Charles Forsyth]

if you're anxious, perhaps it might be reasonable to provide an
option (say -e if it's not used) that tells hget not to produce
output until it is error-free (or at an end, -e works either way).

i actually have got another idea but it's more involved,
and probably not worthwhile for this case.

Re: [9fans] hget

[rog]

[-- Attachment #1: Type: text/plain, Size: 280 bytes --]

i think output duplication is more dangerous than output truncation.
the latter can be determined from the error status anyway, which the
former cannot.

hence my preference for an option that will explicitly allow seeking
on output, rather than one which will suppress it.

[-- Attachment #2: Type: message/rfc822, Size: 1943 bytes --]

From: Charles Forsyth <forsyth@terzarima.net>
To: 9fans@cse.psu.edu
Subject: Re: [9fans] hget
Date: Mon, 29 Mar 2004 23:23:16 +0100
Message-ID: <7a4a127e36d445afb8476a59d48819d6@terzarima.net>

if you're anxious, perhaps it might be reasonable to provide an
option (say -e if it's not used) that tells hget not to produce
output until it is error-free (or at an end, -e works either way).

i actually have got another idea but it's more involved,
and probably not worthwhile for this case.

Re: [9fans] hget

[Charles Forsyth]

>>i think output duplication is more dangerous than output truncation.

i see: you can have too much of a good thing?

Re: [9fans] hget

[rog]

> i see: you can have too much of a good thing?

unfortunately most of what can be obtained using
hget is not good at all...

Re: [9fans] hget

[boyd, rounin]

> > i see: you can have too much of a good thing?
> 
> unfortunately most of what can be obtained using
> hget is not good at all...

truncation is better than duplication?  explain that to me ...

Re: [9fans] hget

[rog]

> truncation is better than duplication?  explain that to me ...

a file is usually treated as a stream of bytes.

particular files will obey particular conventions.  for instance, a
body of data will follow a header.

conventions for truncation are well defined, easily observed, and
usually checked for (premature eof).  if however an arbitrary part of
the file can be repeated, there's no way of knowing how it can happen,
no way of knowing when it has happened, and nobody checks for it.

hence my statement.

Re: [9fans] hget

[boyd, rounin]

> conventions for truncation are well defined, easily observed, and
> usually checked for (premature eof).

so /dev/null actually gives you information?

Re: [9fans] hget

[rog]

> so /dev/null actually gives you information?

of course.

Re: [9fans] hget

[boyd, rounin]

> > so /dev/null actually gives you information?
> 
> of course.

i see you're not much of a fan of Shannon:

    http://www.lucent.com/minds/infotheory/who.html

Re: [9fans] hget

[Steve Simon]

> > > so /dev/null actually gives you information?
> > 
> > of course.
> 
> i see you're not much of a fan of Shannon:
> 
>     http://www.lucent.com/minds/infotheory/who.html

But it _does_ give you information.

The quantity of information is equal to log() of the
improbability of that information arriving.

Thus you expected to read a file you got EOF, you get information,
"this file is zero length".

-Steve

Re: [9fans] hget

[Charles Forsyth]

>>conventions for truncation are well defined, easily observed, and

unfortunately, it is not in fact guaranteed in practice that the http message will have
a header that includes a byte count (at all) or an accurate one (if that).
hence some of the heuristics.

Re: [9fans] hget

[rog]

> unfortunately, it is not in fact guaranteed in practice that the http
> message will have a header that includes a byte count (at all) or an
> accurate one (if that).  hence some of the heuristics.

agreed, but i'd still maintain that it's not good for it to
potentially produce the same data multiple times on an output stream.

in fact it seems to me it should never do this, and doesn't need to:
there's a known point up to which it has good data.  if it has to
refetch data before this point it can just discard it.

getting partial parts of a file there's always a danger that you're a
getting a partial part of a different file, but that case is wrong for
the current hget too (if the new file is shorter you'll get a part of
the older one tacked on to the end, as it doesn't truncate the file
when it seeks to 0).

so maybe that's the right solution: avoid all seeks always; they're never
necessary.

no extra options, and the code should be straightforward.

Re: [9fans] hget

[David Tolpin]

> should hget get to a temporary file when writing to stdout,
> or avoid retry, or... ?
>
> has anyone got strong views on this?
>

hcat ?
A separate utility that does just what hget is mostly used for?

Re: [9fans] hget

[David Tolpin]

> i just noticed that hget always seeks to the start of its output
> fd, which meant that the script i was trying to run:
>
> 	{for(i in $urls) hget $i} > allurls
>
> can't work. (and it does seem reasonable to be able to pipe hget
> through other commands, etc).

{for(i in $urls) hget $i} | cat > allurls 

should work, shouldn't it? seek on a pipe should do nothing?

Re: [9fans] hget

[rog]

> {for(i in $urls) hget $i} | cat > allurls 
>
> should work, shouldn't it? seek on a pipe should do nothing?

yes.

except that in the case of a retry after premature eof,
you'll get duplicated output, something it's probably
better to avoid if poss.