Re: [9fans] seeking for the truth

Re: [9fans] seeking for the truth

[David Presotto]

I can't find anything that currently uses on the length of
stream files, just debugging stuff I have.  I'ld be willing to
get rid of the length, though its helped me many times
debugging things especially with IP.

Re: [9fans] seeking for the truth

[David Presotto]

[-- Attachment #1: Type: text/plain, Size: 61 bytes --]

The problem is that the file isn't always there, as in pipes.

[-- Attachment #2: Type: message/rfc822, Size: 1919 bytes --]

From: rog@vitanuova.com
To: 9fans@cse.psu.edu
Subject: Re: [9fans] seeking for the truth
Date: Thu, 12 Feb 2004 18:30:14 0000
Message-ID: <c901327c41d26f44589e429d49ed219a@vitanuova.com>

> I'ld be willing to
> get rid of the length, though its helped me many times
> debugging things especially with IP.

wouldn't it be more appropriate to have it available
in another file (e.g. status)?

Re: [9fans] seeking for the truth

[rog]

> I'ld be willing to
> get rid of the length, though its helped me many times
> debugging things especially with IP.

wouldn't it be more appropriate to have it available
in another file (e.g. status)?

Re: [9fans] seeking for the truth

[boyd, rounin]

> wouldn't it be more appropriate to have it available
> in another file (e.g. status)?

no, we already have the length and i think it should
indicate the amount of readable data (like a real
file) on pipes and network connections and other
'synthetic' files, if that information is available.

Re: [9fans] seeking for the truth

[Scott Schwartz]

On Thu, Feb 12, 2004 at 01:19:51PM -0500, David Presotto wrote:
> I can't find anything that currently uses on the length of
> stream files, just debugging stuff I have.  I'ld be willing to
> get rid of the length, though its helped me many times
> debugging things especially with IP.

I'm not sure I like getting rid of it;  I feel like buffers ought to
have a cheap way to ask how full they are.

Re: [9fans] seeking for the truth

[rog]

> I'm not sure I like getting rid of it;  I feel like buffers ought to
> have a cheap way to ask how full they are.

stat is no cheaper than read.

and since we have fd2path, it's not too difficult to work out where to
find the information (for /net at any rate), even if you only have a
file descriptor.

Re: [9fans] seeking for the truth

[boyd, rounin]

> > I'm not sure I like getting rid of it;  I feel like buffers ought to
> > have a cheap way to ask how full they are.
>
> stat is no cheaper than read.

ls -l and others use stat, not read.

Re: [9fans] seeking for the truth

[boyd, rounin]

> thoughts?

seeking on pipes doesn't scale.

adding more junk to support special cases is a bad idea.

Re: [9fans] seeking for the truth

[David Presotto]

OK.  Pipes, uarts, and tcp data files now stat as 0 length.  I'll probably
change the stats files for uarts and tcp to show how much is really there
for debuggering.

Re: [9fans] seeking for the truth

[rog]

> > simpler might be: seekable = seek(fd, 0, 2) > 0;
>
> now you're talking.

pity it doesn't work on /net/tcp/0/data.

but i always thought that using the file size for "amount of unbuffered data"
was a nasty thing to do... does anything actually use that property?

Re: [9fans] seeking for the truth

[David Presotto]

[-- Attachment #1: Type: text/plain, Size: 119 bytes --]

It's true of qio files in general, including pipes.

I've used it often to know how much I can read without blocking.

[-- Attachment #2: Type: message/rfc822, Size: 1977 bytes --]

From: rog@vitanuova.com
To: 9fans@cse.psu.edu
Subject: Re: [9fans] seeking for the truth
Date: Thu, 12 Feb 2004 18:10:18 0000
Message-ID: <3b828656ea8bbad209d8a8a91f883860@vitanuova.com>

> > simpler might be: seekable = seek(fd, 0, 2) > 0;
>
> now you're talking.

pity it doesn't work on /net/tcp/0/data.

but i always thought that using the file size for "amount of unbuffered data"
was a nasty thing to do... does anything actually use that property?

Re: [9fans] seeking for the truth

[Charles Forsyth]

i ought to have pointed out that in some applications the
pointer needs to be restored after the seek(,2) test; i knew that
at the time, but it made the general case a bit less pretty!
tail does more messing about than i remembered.

Re: [9fans] seeking for the truth

[rob pike, esq.]

> simpler might be: seekable = seek(fd, 0, 2) > 0;

now you're talking.

-rob

Re: [9fans] seeking for the truth

[Dave Lukes]

> unseekable synthetic files typically are length 0 (because they have
> no particular length), or tiny (pipes have a hack that stat shows what's in them)

The man is, as usual, correct:
I had my mind too far into a un*x model of files/devices/pipes only.
My apologies.

> so change the test to declare seekable only if the file is big enough
> to warrant it.
:-))

>   one hack deserves another.  then we can get back
> to trying to get a version of ghostscript that doesn't blow up on
> recent pdf/ps.

Amen.
	Dave.

Re: [9fans] seeking for the truth

[Fco.J.Ballesteros]

[-- Attachment #1: Type: text/plain, Size: 275 bytes --]

I'd say this is a fundamental problem.
seek has sense for files that are always there.
But, as you said, many of your files are made
on demand, and the seek concept does not
apply well for them.

Isn't just enough to try to keep the seeking programs
to a bare minimum?

[-- Attachment #2: Type: message/rfc822, Size: 2423 bytes --]

From: Charles Forsyth <forsyth@terzarima.net>
To: 9fans@cse.psu.edu
Subject: Re: [9fans] seeking for the truth
Date: Thu, 12 Feb 2004 17:17:59 0000
Message-ID: <c2f71f0dc611079cbba4d0909fc1afff@terzarima.net>

i'm not enthusiastic about having to change nearly every file server.
status files will be seekable but most synthetic ones won't.
some synthetic files will never be seekable in tail's sense because
they have no end, but they do interpret offsets, so are they QTSEEK or not?

isn't tail's seek an optimisation to avoid reading the file to the end?
unseekable synthetic files typically are length 0 (because they have
no particular length), or tiny (pipes have a hack that stat shows what's in them)
so change the test to declare seekable only if the file is big enough
to warrant it.  one hack deserves another.  then we can get back
to trying to get a version of ghostscript that doesn't blow up on
recent pdf/ps.

Re: [9fans] seeking for the truth

[Charles Forsyth]

>>so change the test to declare seekable only if the file is big enough
>>to warrant it.  one hack deserves another.  then we can get back

simpler might be: seekable = seek(fd, 0, 2) > 0;

Re: [9fans] seeking for the truth

[Charles Forsyth]

i'm not enthusiastic about having to change nearly every file server.
status files will be seekable but most synthetic ones won't.
some synthetic files will never be seekable in tail's sense because
they have no end, but they do interpret offsets, so are they QTSEEK or not?

isn't tail's seek an optimisation to avoid reading the file to the end?
unseekable synthetic files typically are length 0 (because they have
no particular length), or tiny (pipes have a hack that stat shows what's in them)
so change the test to declare seekable only if the file is big enough
to warrant it.  one hack deserves another.  then we can get back
to trying to get a version of ghostscript that doesn't blow up on
recent pdf/ps.

Re: [9fans] seeking for the truth

[rog]

presotto:
> except diff doesn't seek the files it is comparing, only its temps...

actually, diff only makes temp files if it thinks it has to,
which is why diff <{gunzip<x.gz} <{gunzip<y.gz} didn't work
(actually i think the checks are a bit more rigorous now).

> Maybe, need a few days to think about it.  I'm vaguely disturbed by it
> but don't know why.  Perhaps because 9p2000 has no concept of seek and
> having a bit that talks about it seems odd.  Then again that's true of the
> exec permission bit also...

could call it "QTIGNORESOFFSETS" but i couldn't think of anything
in that line that was snappy enough.

rob:
> i can't get excited about this.  you used the word 'hack'
> yourself to describe the result.

it's a hack now. it wouldn't be if there was some system support
for it.

Re: [9fans] seeking for the truth

[Fco.J.Ballesteros]

> here's a fix:
>
> 	|cat|tail -r

But this shows a non-uniform behaviour.
I'd expect
	|tail -r
to be exactly like
	|cat|tail -r
but for the buffering involved.

I'm not sure about the real fix. My first attempt would
be to ban out seek; although I know that's not feasible in Plan 9...

Re: [9fans] seeking for the truth

[Rob Pike]

dave's right. there's no seek concept in 9P, only offsets.

i can't get excited about this.  you used the word 'hack'
yourself to describe the result.

here's a fix:

	|cat|tail -r

-rob

Re: [9fans] seeking for the truth

[Dave Lukes]

Definite plus vote from me.
Nugatory seeks have annoyed me ever since 6th Ed.

Also,
if anything out there _is_ seek()ing without checking the result,
then either it's broke anyway,
or it knows it can safely ignore the result.
So, off the top of my head, I don't see compatibility as an issue.

Obviously this would need some more looking at,
and a pile of "grep seek ..."s, but it sounds like a plan to me ...

	Dave.

On Thu, 2004-02-12 at 16:16, rog@vitanuova.com wrote:
> i was caught up short just now when i tried to reverse
> some lines in acme by selecting them and executing "|tail -r".
>
> it didn't work.
> the reason?
>
> /sys/src/cmd/tail.c:94: 	seekable = seek(file,0L,0) == 0;
>
> the seek() is succeeding, even though standard input
> isn't actually seekable.
>
> the trouble is, it can't do anything else.
>
> we don't see this problem much because of the special case hack for
> "#|" in sseek(), but the tail code is a common hack, and an insidious
> one.  there are loads of special files around that don't allow
> seeking, and many programs that require files that are seekable (diff
> being the age-old example).
>
> surely a better solution is possible?  in general, a fileserver knows
> whether its files are seekable or not, because it determines how to
> interpret the seek offset.
>
> why not add a new bit to the file attributes, say DMNOSEEK (and
> associated QTNOSEEK)?  if set, it would indicate that file offsets
> when reading and writing the file will be ignored.  0x2 seems to be
> available.
>
> then the above line from tail.c could work, and finally it would be
> possible to write programs that know definitively whether they are
> able to seek on a particular file or not, rather than failing
> strangely sometime later.
>
> the amount of code involved would be tiny, and as far as i can see
> it would be completely backwardly compatible.
>
> thoughts?
>

Re: [9fans] seeking for the truth

[David Presotto]

> seeking, and many programs that require files that are seekable (diff
> being the age-old example).

except diff doesn't seek the files it is comparing, only its temps...

> surely a better solution is possible?  in general, a fileserver knows
> whether its files are seekable or not, because it determines how to
> interpret the seek offset.

Fixing all the programs that seek without checking might also be nice.

> why not add a new bit to the file attributes, say DMNOSEEK (and
> associated QTNOSEEK)?  if set, it would indicate that file offsets
> when reading and writing the file will be ignored.  0x2 seems to be
> available.

Maybe, need a few days to think about it.  I'm vaguely disturbed by it
but don't know why.  Perhaps because 9p2000 has no concept of seek and
having a bit that talks about it seems odd.  Then again that's true of the
exec permission bit also...

[9fans] seeking for the truth

[rog]

i was caught up short just now when i tried to reverse
some lines in acme by selecting them and executing "|tail -r".

it didn't work.
the reason?

/sys/src/cmd/tail.c:94: 	seekable = seek(file,0L,0) == 0;

the seek() is succeeding, even though standard input
isn't actually seekable.

the trouble is, it can't do anything else.

we don't see this problem much because of the special case hack for
"#|" in sseek(), but the tail code is a common hack, and an insidious
one.  there are loads of special files around that don't allow
seeking, and many programs that require files that are seekable (diff
being the age-old example).

surely a better solution is possible?  in general, a fileserver knows
whether its files are seekable or not, because it determines how to
interpret the seek offset.

why not add a new bit to the file attributes, say DMNOSEEK (and
associated QTNOSEEK)?  if set, it would indicate that file offsets
when reading and writing the file will be ignored.  0x2 seems to be
available.

then the above line from tail.c could work, and finally it would be
possible to write programs that know definitively whether they are
able to seek on a particular file or not, rather than failing
strangely sometime later.

the amount of code involved would be tiny, and as far as i can see
it would be completely backwardly compatible.

thoughts?