[9fans] awk help; not plan9 matter

[9fans] awk help; not plan9 matter

[Rudolf Sykora]

Hello,

simple task.
I want to change the 2nd field on each line of a file, but preserve
the spacing of the lines.

If I do
  awk '{$2="hell"; print}' file
the field gets changed, but all the spacing of the lines is gone; i.e.
any space is now just ' ' like this:
1      3 4           8
changes to
1 hell 4 8
while I need
1      hell 4           8.

Any help?
Thanks
Ruda

Re: [9fans] awk help; not plan9 matter

[matt]

sed 's/^([^ ]+ +)([^ ]+)/\1HELL/'

>Hello,
>
>simple task.
>I want to change the 2nd field on each line of a file, but preserve
>the spacing of the lines.
>
>If I do
>  awk '{$2="hell"; print}' file
>the field gets changed, but all the spacing of the lines is gone; i.e.
>any space is now just ' ' like this:
>1      3 4           8
>changes to
>1 hell 4 8
>while I need
>1      hell 4           8.
>
>Any help?
>Thanks
>Ruda
>
>
>

Re: [9fans] awk help; not plan9 matter

[Rudolf Sykora]

2009/9/17 matt <maht-9fans@maht0x0r.net>:
> sed 's/^([^ ]+ +)([^ ]+)/\1HELL/'

Well, actually, my problem is a part of a more complicated script; I
don't know in advance neither the column nor what to put there.
I currently have this
awk '
	/TH/ {$'$pos'='$new'}
	{print}
' inpch
where 'TH' plays a role of a guard, and $pos and $new are the field to
be changed and with what, respectively.
I am afraid this rules out sed.

Thanks
Ruda

Re: [9fans] awk help; not plan9 matter

[matt]

awk '{a=$2; sub(a, "hell"); print}' file

also works if it contains no regex special chars

this seems to do the trick otherwise

awk ' { p=substr($0, index($0, " ")); split(p, a, "[^ ]"); sub(/ +[^ ]+/, "", p); print $1 a[1] "hell" p} '



Rudolf Sykora wrote:

>Hello,
>
>simple task.
>I want to change the 2nd field on each line of a file, but preserve
>the spacing of the lines.
>
>If I do
>  awk '{$2="hell"; print}' file
>the field gets changed, but all the spacing of the lines is gone; i.e.
>any space is now just ' ' like this:
>1      3 4           8
>changes to
>1 hell 4 8
>while I need
>1      hell 4           8.
>
>Any help?
>Thanks
>Ruda
>
>
>

Re: [9fans] awk help; not plan9 matter

[Rudolf Sykora]

2009/9/17 matt <maht-9fans@maht0x0r.net>:
> awk '{a=$2; sub(a, "hell"); print}' file

The trouble with this is that the same string can appear more than
once (before, after the field, ...), so the simple substitution isn't
enough.
Ruda

Re: [9fans] awk help; not plan9 matter

[Rudolf Sykora]

Ok. Thanks again. I think I've found what I need.
(on lines of 'inpch' that contain 'TH' I change the field at position
$pos to be $new, preserving spacing made of 'space's and/or tabs).
If anyone comes up with sth. simpler...

Ruda

(The script is for 'rc', btw., so that the concatanation on the 3rd line work.)

	9 awk '
		/TH/ {n = split($0, spaces, /[^ 	]+/);
			$'$pos'='$new';
			line = "";
			for(i = 1; i < n; i++)
				line = line spaces[i] $(i);
			line = line spaces[n];
			print line;
			next;}
		{print}
		' inpch

Re: [9fans] awk help; not plan9 matter

[yy]

if you want to preserve white-space, you better forget about fields
and work with indexes on the string, match is your friend:

% echo '1      3 4           8' | awk '{match($0, /[ \t]*[^ \t]+[
\t]+/);a=RLENGTH+1;match(substr($0, a), /[ \t]/);print
substr($0,0,a-1) "hell" substr($0,RSTART+a)}'
1      hell 4           8

this is indeed a bit OT here, maybe next time you prefer trying in
#awk at freenode, where this kind of problems are welcomed

--
- yiyus || JGL . 4l77.com

Re: [9fans] awk help; not plan9 matter

[Rudolf Sykora]

2009/9/17 yy <yiyu.jgl@gmail.com>:
> if you want to preserve white-space, you better forget about fields
> and work with indexes on the string, match is your friend:
>
> % echo '1      3 4           8' | awk '{match($0, /[ \t]*[^ \t]+[
> \t]+/);a=RLENGTH+1;match(substr($0, a), /[ \t]/);print
> substr($0,0,a-1) "hell" substr($0,RSTART+a)}'
> 1      hell 4           8

Well, it's just so difficult for me to read this. :)
Also, i don't see how to easily modify it to flexibly work for any
(not known beforehand) column, as I need.
Why do you think this is better than my final solution?

> this is indeed a bit OT here, maybe next time you prefer trying in
> #awk at freenode, where this kind of problems are welcomed

I didn't know there is a separate channel for awk...

Thanks
Ruda

Re: [9fans] awk help; not plan9 matter

[Lyndon Nerenberg - VE6BBM/VE7TFX]

> The trouble with this is that the same string can appear more than
> once (before, after the field, ...), so the simple substitution isn't
> enough.

It's sounding like awk is the wrong tool. It should be trivial to code
up a short piece of C to do the job.

Re: [9fans] awk help; not plan9 matter

[erik quanstrom]

i don't know why this can't be done with sed.  if the
task is to just change the second field without messing
with whitespace, why doesn't this work

; cat x
1     3 4	8
1 2	3 4
; sed 's:^([^ 	][ 	]*)([^ 	]):\1hell:g' < x
1     hell 4	8
1 hell	3 4

- erik

Re: [9fans] awk help; not plan9 matter

[roger peppe]

2009/9/17 erik quanstrom <quanstro@quanstro.net>:
> i don't know why this can't be done with sed.  if the
> task is to just change the second field without messing
> with whitespace, why doesn't this work

indeed. i did the same thing (see previous post, except i've
just noticed that i forgot the ^ at the start of the regex).

BTW, no need for the g flag, i think.

Re: [9fans] awk help; not plan9 matter

[Rudolf Sykora]

2009/9/17 erik quanstrom <quanstro@quanstro.net>:
> i don't know why this can't be done with sed.  if the
> task is to just change the second field without messing
> with whitespace, why doesn't this work

As I said in my second post, neither the field (the problem with sed)
nor the string to be used as a replacement (no problem) is not known
in advance...
Apparently nobody reads but the 1st post... :)
Thanks!
Ruda

Re: [9fans] awk help; not plan9 matter

[erik quanstrom]

> As I said in my second post, neither the field (the problem with sed)
> nor the string to be used as a replacement (no problem) is not known
> in advance...
> Apparently nobody reads but the 1st post... :)

why would it be hard to build up the regular expression
with a script?

something like this (untested, getting lazy)

# usage: buildre n replacement
fn buildre {
	re = '^'
	for(i in `{seq 1 $1})
		re = $re ^ '([^ 	][ 	]*)'
	re = $re ^ ([^ 	]):\' ^ $2 ^ ':'
}

- erik

Re: [9fans] awk help; not plan9 matter

[erik quanstrom]

> # usage: buildre n replacement
> fn buildre {
> 	re = '^'
> 	for(i in `{seq 1 $1})
> 		re = $re ^ '([^ 	][ 	]*)'
> 	re = $re ^ ([^ 	]):\' ^ $2 ^ ':'
> }

sorry.  quote snafu.

fn buildre {
	re = 's:^'
	for(i in `{seq 1 $1})
		re = $re ^ '([^ 	][ 	]*)'
	re = $re ^ '([^ 	]):\' ^ $1 ^ $2 ^ :
}

- erik

Re: [9fans] awk help; not plan9 matter

[Rudolf Sykora]

2009/9/17 erik quanstrom <quanstro@quanstro.net>:
> fn buildre {
>        re = 's:^'
>        for(i in `{seq 1 $1})
>                re = $re ^ '([^         ][      ]*)'
>        re = $re ^ '([^         ]):\' ^ $1 ^ $2 ^ :
> }
>
> - erik
>

Yes, I now see yours and Roger Peppe's idea to build a regexps and then use it.
That's true.
Only as I look at your code, not sure if it can stand possible spaces
at the beginning of a line, like
     1 2     3

Thanks
Ruda

Re: [9fans] awk help; not plan9 matter

[roger peppe]

2009/9/17 Rudolf Sykora <rudolf.sykora@gmail.com>:
> Yes, I now see yours and Roger Peppe's idea to build a regexps and then use it.
> That's true.
> Only as I look at your code, not sure if it can stand possible spaces
> at the beginning of a line, like
>     1 2     3

just change the regexp as required.

Re: [9fans] awk help; not plan9 matter

[Rudolf Sykora]

2009/9/17 roger peppe <rogpeppe@gmail.com>:
> just change the regexp as required.

Ok. I decided (although just for game now) to play a little with the
idea of building the regexp.
Starting from Eric's post I slowly progressed to sth. quite similar to
your post :) :

fn buildre {
       re = 's:^([ 	]*)('
       for(i in `{seq 1 `{hoc -e $1-1}})
               re = $re ^ '[^ 	]+[ 	]+'
       re = $re ^ ')[^ 	]+:\1\2' ^ $2:
}

This works (even takes care about leading spaces) unless I want to use
it for the 1st field. Then the produced regexp (buildre 1 hell)is

s:^([ 	]*)()[^ 	]+:\1\2hell:

and sed says
sed: Command garbled: s:^([ 	]*)()[^ 	]+:\1\2hell:

Apparently it dislikes the empty group, (). Is there any reason?
I tried it in linux, there with

echo '1 28	 	3' | sed 's:^\([ 	]*\)\(\)[^ 	][^ 	]*:\1\2hell:'

and it works as expected...

Thanks
Ruda

Re: [9fans] awk help; not plan9 matter

[erik quanstrom]

i don't think you need an extra () for the leading
white space.  just tack it on in with the leading expression.

the hoc is unnecessary.  just start with 2.

fn buildre {
	re = 's:^([ 	]*'
	for(i in `{seq 2 $1})
		re = $re ^ '[^ 	]+[ 	]+'
	re = $re ^ ')([^ 	]+):\1' ^ $2:
}

; buildre 1 hell
; whatis re
re='s:^([ 	]*)([^ 	]+):\1hell:'
; buildre 2 hell
; whatis re
re='s:^([ 	]*[^ 	]+[ 	]+)([^ 	]+):\1hell:'

- erik

Re: [9fans] awk help; not plan9 matter

[Rudolf Sykora]

2009/9/17 erik quanstrom <quanstro@quanstro.net>:
> i don't think you need an extra () for the leading
> white space.  just tack it on in with the leading expression.

see below

> the hoc is unnecessary.  just start with 2.
true :)

> fn buildre {
>        re = 's:^([     ]*'
>        for(i in `{seq 2 $1})
>                re = $re ^ '[^  ]+[     ]+'
>        re = $re ^ ')([^        ]+):\1' ^ $2:
> }
>
> ; buildre 1 hell
> ; whatis re
> re='s:^([       ]*)([^  ]+):\1hell:'

already here is a problem:
1) I am to save the leading space --- so it shouldn't be an argument to s.
2) only the 2nd () --- the word with no spaces in itself --- is to be changed
... since the spaces are to be preserved, none of them should be an
argument to the s command, I think. That's why it was treated
seperately by me.

Anyway, what do you think about that problem with the empty group? Why
linux is ok with it while plan 9 is not? Is there any reason? Is that
a bug in plan9 sed?

Thanks
Ruda

> ; buildre 2 hell
> ; whatis re
> re='s:^([       ]*[^    ]+[     ]+)([^  ]+):\1hell:'
>
> - erik
>
>

Re: [9fans] awk help; not plan9 matter

[Rudolf Sykora]

forget about my previous post regarding the leading space,
I am tired, it was wrong...

Anyway the question about () is still hot...
R

2009/9/17 Rudolf Sykora <rudolf.sykora@gmail.com>:
> 2009/9/17 erik quanstrom <quanstro@quanstro.net>:
>> i don't think you need an extra () for the leading
>> white space.  just tack it on in with the leading expression.
>
> see below
>
>> the hoc is unnecessary.  just start with 2.
> true :)
>
>> fn buildre {
>>        re = 's:^([     ]*'
>>        for(i in `{seq 2 $1})
>>                re = $re ^ '[^  ]+[     ]+'
>>        re = $re ^ ')([^        ]+):\1' ^ $2:
>> }
>>
>> ; buildre 1 hell
>> ; whatis re
>> re='s:^([       ]*)([^  ]+):\1hell:'
>
> already here is a problem:
> 1) I am to save the leading space --- so it shouldn't be an argument to s.
> 2) only the 2nd () --- the word with no spaces in itself --- is to be changed
> ... since the spaces are to be preserved, none of them should be an
> argument to the s command, I think. That's why it was treated
> seperately by me.
>
> Anyway, what do you think about that problem with the empty group? Why
> linux is ok with it while plan 9 is not? Is there any reason? Is that
> a bug in plan9 sed?
>
> Thanks
> Ruda
>
>> ; buildre 2 hell
>> ; whatis re
>> re='s:^([       ]*[^    ]+[     ]+)([^  ]+):\1hell:'
>>
>> - erik
>>
>>
>

Re: [9fans] awk help; not plan9 matter

[erik quanstrom]

> Anyway, what do you think about that problem with the empty group? Why
> linux is ok with it while plan 9 is not? Is there any reason? Is that
> a bug in plan9 sed?

it is not a bug.  see regexp(6).  there is no production in the
grammar that allows ().

- erik

Re: [9fans] awk help; not plan9 matter

[Noah Evans]

Since you're doing character processing rather than record processing,
isn't C your best tool for the job here?

This is what I whipped out YMMV:

#include <u.h>
#include <libc.h>
#include <bio.h>

char *str;
int ntok;

#define	WHITESPACE(c)		((c) == ' ' || (c) == '\t' || (c) == '\n')

void
chgtok(Biobuf *bin, Biobuf *bout)
{
	int seentok, waswhite, c;
	seentok = 1;
	waswhite = 0;

	while((c = Bgetc(bin)) != Beof){
		switch(c){
		case ' ':
		case '\t':
			if(!waswhite){
				seentok++;
				waswhite = 1;
			}
			break;
		case '\n':
			seentok = 1;
			break;
		default:
			waswhite = 0;
			if(seentok == ntok){
				Bprint(bout, str);
				while((c = Bgetc(bin)) != Beof)
					if(WHITESPACE(c))
						break;
				Bungetc(bin);
			}
			break;			
		}
		Bputc(bout, c);
	}
	Bflush(bout);
}

void
main(int argc, char **argv)
{
	Biobuf bin, bout;

	ARGBEGIN{
	}ARGEND;
	if(argc != 2)
		sysfatal("usage");
	ntok = atoi(argv[0]);
	str = argv[1];
	Binit(&bin, 0, OREAD);
	Binit(&bout, 1, OWRITE);
	chgtok(&bin, &bout);
	exits(0);	
}




On Thu, Sep 17, 2009 at 10:23 AM, Rudolf Sykora <rudolf.sykora@gmail.com> wrote:
> Hello,
>
> simple task.
> I want to change the 2nd field on each line of a file, but preserve
> the spacing of the lines.
>
> If I do
>  awk '{$2="hell"; print}' file
> the field gets changed, but all the spacing of the lines is gone; i.e.
> any space is now just ' ' like this:
> 1      3 4           8
> changes to
> 1 hell 4 8
> while I need
> 1      hell 4           8.
>
> Any help?
> Thanks
> Ruda
>
>

Re: [9fans] awk help; not plan9 matter

[roger peppe]

another approach, build up a regexp and use that:

fn changefield {
	n = $1
	repl = $2
	s=')([^ 	]+)(.*)'
	for(i in `{seq 1 `{echo $n 1 -p | dc}}){
		s='[^ 	]+[ 	]+'^$s
	}
	s='('^$s
	sed 's/'^$s^'/\1'^$repl^'\3/'
}

(N.B. if the replacement string
might contain / or \, those characters would need
quoting)

Re: [9fans] awk help; not plan9 matter

[Rudolf Sykora]

2009/9/17 Noah Evans <noah.evans@gmail.com>:
> Since you're doing character processing rather than record processing,
> isn't C your best tool for the job here?
>
> This is what I whipped out YMMV:
>
> #include <u.h>
> #include <libc.h>
> #include <bio.h>
>
> char *str;
> int ntok;
>
> #define WHITESPACE(c)           ((c) == ' ' || (c) == '\t' || (c) == '\n')
>
> void
> chgtok(Biobuf *bin, Biobuf *bout)
> {
>        int seentok, waswhite, c;
>        seentok = 1;
>        waswhite = 0;
>
>        while((c = Bgetc(bin)) != Beof){
>                switch(c){
>                case ' ':
>                case '\t':
>                        if(!waswhite){
>                                seentok++;
>                                waswhite = 1;
>                        }
>                        break;
>                case '\n':
>                        seentok = 1;
>                        break;
>                default:
>                        waswhite = 0;
>                        if(seentok == ntok){
>                                Bprint(bout, str);
>                                while((c = Bgetc(bin)) != Beof)
>                                        if(WHITESPACE(c))
>                                                break;
>                                Bungetc(bin);
>                        }
>                        break;
>                }
>                Bputc(bout, c);
>        }
>        Bflush(bout);
> }
>
> void
> main(int argc, char **argv)
> {
>        Biobuf bin, bout;
>
>        ARGBEGIN{
>        }ARGEND;
>        if(argc != 2)
>                sysfatal("usage");
>        ntok = atoi(argv[0]);
>        str = argv[1];
>        Binit(&bin, 0, OREAD);
>        Binit(&bout, 1, OWRITE);
>        chgtok(&bin, &bout);
>        exits(0);
> }
>

Thanks, terrific job. :)
But awk finally works (see my post at 12:46 or so) just fine and the
code is just straightforward 9 lines.
The only problem was to realize how one can proceed, i.e. here to
remember the individual spaces and reconstruct the line from the
fields.

Nonetheless, your solution is almost surely faster.

Thanks
Ruda