[9fans] venti backup example vs. 2GB

[9fans] venti backup example vs. 2GB

[Robert Raschke]

Hi,

I've been using a backup script modelled on the example in
/sys/src/cmd/venti/backup.example and the other day it "just" stopped
working.  On debugging I found that I have now reached the beginning
of the fifth 500MB arena.  The script uses awk to calculate the
starting byte offset and that fails with a floating point error (!)
when we go beyond the 2G mark.

I resorted to a fairly brutal use of bc(1) and that seems to work.
Here's what I use now (I flipped the 2nd and 3rd parameter to the x fn
to make printing from bc easier):

#!/bin/rc

rfork e
. bkup.info
fn x {
	echo x $*
	y=$1
	if(~ $#$y 0){
		$y=0
	}
	echo venti/wrarena -h venti_backup_server -o $3 $2 $$y
	end=`{venti/wrarena -h venti_backup_server -o $3 $2 $$y | grep '^end offset ' | sed 's/^end offset //'}
	if(~ $#end 1 && ! ~ $$y $end){
		$y=$end
		echo '#' `{date} >>bkup.info
		whatis $y >>bkup.info
	}
}
hget http://127.1:8000/index | 
awk '
/^index=/ { blockSize=0+substr($3, 11); }
/^arena=/ { arena=substr($1, 7); }
/^	arena=/ {
		start=substr($5, 2, index($5, ",")-2);
		printf("\"x \"; \"%s \"; \"%s \"; s=%s-%s; print s;", arena, $3, start, blockSize);
	}
' |bc -s |rc


Robby

Re: [9fans] venti backup example vs. 2GB

[geoff]

I think that the floating-point exception is from converting the
internal floating-point representation of start to an int for printing
with %d.  Numbers above 2^31 won't fit in a signed int, thus the
exception.  awk doesn't seem to have any form of vlong, nor %lld;
perhaps it should.

Re: [9fans] venti backup example vs. 2GB

[geoff]

... or perhaps %g suffices.

Re: [9fans] venti backup example vs. 2GB

[erik quanstrom]

perhaps awk numbers should be 64 bits?

- erik

Re: [9fans] venti backup example vs. 2GB

[geoff]

They are 64 bits, just floating-point.  Changing them to an integral
type would break lots of existing awk programs.

Re: [9fans] venti backup example vs. 2GB

[erik quanstrom]

i guess i didn't say what i ment.  there are places where the Awkfloat
is treated as an integer and manipulated with a signed int or long.
i was suggesting replacing that with vlong.

- erik

Re: [9fans] venti backup example vs. 2GB

[Russ Cox]

You can use %.0f to print the number from awk.

Russ


On 9/25/06, Robert Raschke <rrplan9@tombob.com> wrote:
> Hi,
>
> I've been using a backup script modelled on the example in
> /sys/src/cmd/venti/backup.example and the other day it "just" stopped
> working.  On debugging I found that I have now reached the beginning
> of the fifth 500MB arena.  The script uses awk to calculate the
> starting byte offset and that fails with a floating point error (!)
> when we go beyond the 2G mark.
>
> I resorted to a fairly brutal use of bc(1) and that seems to work.
> Here's what I use now (I flipped the 2nd and 3rd parameter to the x fn
> to make printing from bc easier):
>
> #!/bin/rc
>
> rfork e
> . bkup.info
> fn x {
>         echo x $*
>         y=$1
>         if(~ $#$y 0){
>                 $y=0
>         }
>         echo venti/wrarena -h venti_backup_server -o $3 $2 $$y
>         end=`{venti/wrarena -h venti_backup_server -o $3 $2 $$y | grep '^end offset ' | sed 's/^end offset //'}
>         if(~ $#end 1 && ! ~ $$y $end){
>                 $y=$end
>                 echo '#' `{date} >>bkup.info
>                 whatis $y >>bkup.info
>         }
> }
> hget http://127.1:8000/index |
> awk '
> /^index=/ { blockSize=0+substr($3, 11); }
> /^arena=/ { arena=substr($1, 7); }
> /^      arena=/ {
>                 start=substr($5, 2, index($5, ",")-2);
>                 printf("\"x \"; \"%s \"; \"%s \"; s=%s-%s; print s;", arena, $3, start, blockSize);
>         }
> ' |bc -s |rc
>
>
> Robby
>
>