[9fans] sam language question

[9fans] sam language question

[Rudolf Sykora]

Hello,

I have a file in which I wanted to insert an empty line just before
any line on which a string, say '0.00' is present.

My attempts to do it this way:
,x/0.00/+-i/\n/
or similarly were not succesful.

I am not sure where the problem is exactly. (Only have some feeling
about it, but couldn't find anything authorative...)

Can somebody help?

Thanks
Ruda

[9fans] corrupted update on 9vx

[EBo]

OK.  I've been trying to track down the odd 9vx crash and discovered some
other oddities and request some guidance in what might be causing the pull to
become corrupted and installing entire blocks of nil's in updated files.

Following details outlined in
http://fixunix.com/plan9/501128-9fans-pull-9vx.html, I first tried to update
the original plan9 distro, then moved to a new 9atom.iso.  To get the pull to
work I had to update the network and client db:

# cp -a plan9/dist/replica/client/plan9.* root9atom/dist/replica/client/
# cp -a plan9/dist/replica/network root9atom/dist/replica/network

Then I ran /usr/glenda/bin/rc/pull.  At one point it seemed to work until I
decided to recompile a couple of the commands in /sys/src/cmd and discovered
that several of the source files had blocks of nil's in them.  I wanted to see
if the problem was systematic or more random, so I saved that root tree and
reran the test.  What I found is that the corruption happens very often and at
seemingly random files and locations.  I also wanted to check to see if these
differences were consistent (like 4K blocks, etc).  I found that the number of
corrupted blocks, and their respective lengths, vary.  I also found that they
are not guaranteed to be multiples of 512.

These tests were so far run on my AMD x86_64.  I'll try rerunning these tests
on an Intel based laptop to see if this is some weird x86_64'ism.  I will also
try the Tvx code compiled for i486 on both machines

Does anyone have an idea what could be causing replica to behave so?  Has
anyone ever seen this behavior before?

  Thanks and best regards,


  EBo --

Re: [9fans] corrupted update on 9vx

[Steve Simon]

replica uses fcp which works multithreaded for speed.

Perhaps the implementation of fswrite() #Z in 9vx uses a seek()
followed by a write() rather than pwrite() which is not thread safe.

Just a guess.

-Steve

Re: [9fans] corrupted update on 9vx

[erik quanstrom]

On Thu Apr 22 14:45:38 EDT 2010, steve@quintile.net wrote:
> replica uses fcp which works multithreaded for speed.
>
> Perhaps the implementation of fswrite() #Z in 9vx uses a seek()
> followed by a write() rather than pwrite() which is not thread safe.

replica actually uses replica/applylog.c:^/worker internally, not
fcp.  but essentially, it's the same thing.  it copies NBUF=8192
bytes at a time with 16 parallel processes.  #Z uses pwrite on non-sockets.

in theory, since NBUF < SSIZE_MAX, pwrite should be atomic
to normal files.

you're not using a fuse-based fs, are you?

possible ways to debug:
1.  reduce NBUF to PIPE_BUF=4096.  guard against someone treating
the fd as a pipe.
2.  put a qlock in the UnixFd structure in devfs-posix.c.  lock for all io.

- erik

Re: [9fans] corrupted update on 9vx

[EBo]

Sorry for the delay

> On Thu Apr 22 14:45:38 EDT 2010, steve@quintile.net wrote:
> > replica uses fcp which works multithreaded for speed.
> >
> > Perhaps the implementation of fswrite() #Z in 9vx uses a seek()
> > followed by a write() rather than pwrite() which is not thread safe.
>
> replica actually uses replica/applylog.c:^/worker internally, not
> fcp.  but essentially, it's the same thing.  it copies NBUF=8192
> bytes at a time with 16 parallel processes.  #Z uses pwrite on non-sockets.
>
> in theory, since NBUF < SSIZE_MAX, pwrite should be atomic
> to normal files.

Thanks for the pointer.  I'll try playing around with NBUF and also see if I
can play with the number of processes (thwack it to 1 for testing and see if
that helps).

> you're not using a fuse-based fs, are you?

no.  I did have fuse installed, but it was only used by usermode-utilities,
and I have since removed both that and fuse.

> possible ways to debug:
> 1.  reduce NBUF to PIPE_BUF=4096.  guard against someone treating
> the fd as a pipe.
> 2.  put a qlock in the UnixFd structure in devfs-posix.c.  lock for all io.

If I recall correctly, qlock has not been ported to 9vx.  Ahhh... I see that
it has.  This saves me the hassles of porting it myself ;-)

Laters,

  EBo --

Re: [9fans] sam language question

[Rob Pike]

,x/0\.00/-a/\n/

-rob

Re: [9fans] sam language question

[Rudolf Sykora]

On 22 April 2010 20:21, Rob Pike <robpike@gmail.com> wrote:
> ,x/0\.00/-a/\n/
>
> -rob
>
>
ok. this works. But what is wrong with mine?!
thanks!
Ruda

Re: [9fans] sam language question

[yy]

2010/4/22 Rudolf Sykora <rudolf.sykora@gmail.com>:
> Hello,
>
> I have a file in which I wanted to insert an empty line just before
> any line on which a string, say '0.00' is present.
>

Edit ,x/^.*0\.00/i/\n/

--
- yiyus || JGL . 4l77.com

Re: [9fans] sam language question

[Rudolf Sykora]

On 22 April 2010 20:24, yy <yiyu.jgl@gmail.com> wrote:
> Edit ,x/^.*0\.00/i/\n/

The same answer as to Rob Pike. Your solution is ok, thanks. But I
still can't see what's wrong with mine... :(
R

Re: [9fans] sam language question

[yy]

2010/4/22 Rudolf Sykora <rudolf.sykora@gmail.com>:
> On 22 April 2010 20:24, yy <yiyu.jgl@gmail.com> wrote:
>> Edit ,x/^.*0\.00/i/\n/
>
> The same answer as to Rob Pike. Your solution is ok, thanks. But I
> still can't see what's wrong with mine... :(
> R

You would need to group your last commands (think carefuly about it):

,x/0.00/{+-i/\n/
}

but then you would have to type a bit more.

--
- yiyus || JGL . 4l77.com

Re: [9fans] sam language question

[Rudolf Sykora]

On 22 April 2010 21:28, yy <yiyu.jgl@gmail.com> wrote:
> You would need to group your last commands (think carefuly about it):
>
> ,x/0.00/{+-i/\n/
> }
>
> but then you would have to type a bit more.

I don't know how it could happen, by the problem I had with
,x/0.00/+-i/\n/
is that it should have been
,x/0\.00/+-i/\n/
Now it does the job. (i.e. just a missing \)

Finally, to add just one line in case there are more '0.00's on a line:
,x g/0\.00/-a/\n/
is fine as well as
,x g/0\.00/+-i/\n/

Thanks and sorry for the noise.
Ruda

Re: [9fans] sam language question

[roger peppe]

another way (and how i would usually do it):

,x g/0\.00/i/\n/


On 22 April 2010 18:32, Rudolf Sykora <rudolf.sykora@gmail.com> wrote:
> Hello,
>
> I have a file in which I wanted to insert an empty line just before
> any line on which a string, say '0.00' is present.
>
> My attempts to do it this way:
> ,x/0.00/+-i/\n/
> or similarly were not succesful.
>
> I am not sure where the problem is exactly. (Only have some feeling
> about it, but couldn't find anything authorative...)
>
> Can somebody help?
>
> Thanks
> Ruda
>
>

Re: [9fans] sam language question

[Rudolf Sykora]

On 23 April 2010 00:04, roger peppe <rogpeppe@gmail.com> wrote:
> another way (and how i would usually do it):
>
> ,x g/0\.00/i/\n/
>

Oh yes, that's probably the best.+- is really unnecessary in this case.
Thanks
R