I guess I'm really hammering away at this. I do have quite a bit of stuff, Some images, tables in html the rest is markdown. cat index.md | wc -l 128 128 lines is still low. Any thoughts? I can't go past this not even by one character memem user overflow pool sbrkmem block 4eed10 hdr 0a110c09 00002020 0020cabc 00206fa7 3e703c0a 69206548 tail 66647361 3e702f3c 3e703c0a 66647361 3e702f3c faf00000 | ef1300be 00002020 user data 73 64 66 3c 2f 70 3e 00 | 00 f0 fa be 00 13 ef 20 panic: pool panic sys: trap: fault read addr=0x0 pc=0x2150a6 md2html.awk 9863: suicide: sys: trap: fault read addr=0x0 pc=0x2150a6 Thu Mar 10 22:19:08 PST 2022 :: williamgunnells.com :: GET /pub/style/style.css HTTP/1.1 :: Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/605.1.15 (KHTML, like Gecko) Version/15.2 Safari/605.1.15 :: 200 :: http://williamgunnells.com/
Quoth william@thinktankworkspaces.com:
> I guess I'm really hammering away at this. I do have quite a bit of stuff, Some images,
> tables in html the rest is markdown.
>
> cat index.md | wc -l
> 128
>
> 128 lines is still low. Any thoughts? I can't go past this not even by one character
>
> memem user overflow
> pool sbrkmem block 4eed10
> hdr 0a110c09 00002020 0020cabc 00206fa7 3e703c0a 69206548
> tail 66647361 3e702f3c 3e703c0a 66647361 3e702f3c faf00000 | ef1300be 00002020
> user data 73 64 66 3c 2f 70 3e 00 | 00 f0 fa be 00 13 ef 20
> panic: pool panic
> sys: trap: fault read addr=0x0 pc=0x2150a6
> md2html.awk 9863: suicide: sys: trap: fault read addr=0x0 pc=0x2150a6
> Thu Mar 10 22:19:08 PST 2022 :: williamgunnells.com :: GET /pub/style/style.css HTTP/1.1 :: Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/605.1.15 (KHTML, like Gecko) Version/15.2 Safari/605.1.15 :: 200 :: http://williamgunnells.com/
Can you get a snap of the dead process, or at least a stack trace? (see snap(4))
Quoth william@thinktankworkspaces.com:
> I guess I'm really hammering away at this. I do have quite a bit of stuff, Some images,
> tables in html the rest is markdown.
>
> cat index.md | wc -l
> 128
>
> 128 lines is still low. Any thoughts? I can't go past this not even by one character
>
> memem user overflow
> pool sbrkmem block 4eed10
> hdr 0a110c09 00002020 0020cabc 00206fa7 3e703c0a 69206548
> tail 66647361 3e702f3c 3e703c0a 66647361 3e702f3c faf00000 | ef1300be 00002020
> user data 73 64 66 3c 2f 70 3e 00 | 00 f0 fa be 00 13 ef 20
> panic: pool panic
> sys: trap: fault read addr=0x0 pc=0x2150a6
> md2html.awk 9863: suicide: sys: trap: fault read addr=0x0 pc=0x2150a6
> Thu Mar 10 22:19:08 PST 2022 :: williamgunnells.com :: GET /pub/style/style.css HTTP/1.1 :: Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/605.1.15 (KHTML, like Gecko) Version/15.2 Safari/605.1.15 :: 200 :: http://williamgunnells.com/
Do you think you can grab a stack trace, or better,
a snap?
% acid 9863
; lstk()
<prints stack trace>
or
% snap -o /tmp/crash 9863
% <upload snap somewhere>
for debugging?
thanks.
> tail 66647361 3e702f3c 3e703c0a 66647361 3e702f3c that is fdsa>p/<>p< fdsa>p/< in ascii, and it looks like an overflow. it seems like a bug in awk. are you able to get a snap(4) of the process, or a stack trace with acid? On Thu, Mar 10, 2022 at 10:30 PM <william@thinktankworkspaces.com> wrote: > > I guess I'm really hammering away at this. I do have quite a bit of stuff, Some images, > tables in html the rest is markdown. > > cat index.md | wc -l > 128 > > 128 lines is still low. Any thoughts? I can't go past this not even by one character > > memem user overflow > pool sbrkmem block 4eed10 > hdr 0a110c09 00002020 0020cabc 00206fa7 3e703c0a 69206548 > tail 66647361 3e702f3c 3e703c0a 66647361 3e702f3c faf00000 | ef1300be 00002020 > user data 73 64 66 3c 2f 70 3e 00 | 00 f0 fa be 00 13 ef 20 > panic: pool panic > sys: trap: fault read addr=0x0 pc=0x2150a6 > md2html.awk 9863: suicide: sys: trap: fault read addr=0x0 pc=0x2150a6 > Thu Mar 10 22:19:08 PST 2022 :: williamgunnells.com :: GET /pub/style/style.css HTTP/1.1 :: Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/605.1.15 (KHTML, like Gecko) Version/15.2 Safari/605.1.15 :: 200 :: http://williamgunnells.com/
that seems to be an awk bug. ¿off by one? should be deterministic to reproduce. could you capture the input file(s) for md2html.awk and then see which one reproduces the crash? you can make a wrapper script that first dumps the inputs to a temporary file (named by $pid) and then runs md2html.awk on that: #!/bin/rc cat > /tmp/input.$pid md2html.awk $* < /tmp/input.$pid then we should have a reproducer. -- cinap
it doesnt matter so much. it is a memory corruption. > hdr 0a110c09 00002020 0020cabc 00206fa7 3e703c0a 69206548 the callerpc of the allocation being corrupted is 0x20cabc (this is amd64, not 386) acid: src(0x20cabc) /sys/src/cmd/awk/run.c:1898 1893 char *buf; 1894 void *p; 1895 int mflag, num; 1896 int bufsz = recsize; 1897 >1898 if ((buf = (char *)malloc(bufsz)) == nil) 1899 FATAL("out of memory in gsub"); 1900 mflag = 0; /* if mflag == 0, can replace empty string */ 1901 num = 0; 1902 x = execute(a[3]); /* target string */ 1903 c = t = getsval(x); which is in gsub() function. from the tail, we can see that it overwrote one single '\0' byte past the buffer. it would be much better to have a reproducer for this bug, not a stacktrace. we already have all the information. now i could stare at this code for a day or you give me a reproducer and then we can fix gsub() and make sure the bug is fixed. -- cinap
actually, i think i found it (by staring at the code). the code at the done label was unconditionally inserting NUL terminator, without the final adjbuf() ensuring theres space for it. the patch gets rid of the label, so we wont skip the final adjbuf(). diff d52f25ecdcf1dc8ee8d278c8da44159d82d8dd8f uncommitted --- a/sys/src/cmd/awk/run.c +++ b/sys/src/cmd/awk/run.c @@ -1934,7 +1934,7 @@ } } if (*c == 0) /* at end */ - goto done; + break; adjbuf(&buf, &bufsz, 2+pb-buf, recsize, &pb, "gsub"); *pb++ = *c++; if (pb > buf + bufsz) /* BUG: not sure of this test */ @@ -1962,8 +1962,12 @@ *pb++ = *sptr++; } c = patbeg + patlen; - if ((c[-1] == 0) || (*c == 0)) - goto done; + if (c[-1] == 0){ + c--; + break; + } + if (*c == 0) + break; if (pb > buf + bufsz) FATAL("gsub result1 %.30s too big; can't happen", buf); mflag = 1; @@ -1973,7 +1977,7 @@ adjbuf(&buf, &bufsz, 1+strlen(sptr)+pb-buf, 0, &pb, "gsub"); while ((*pb++ = *sptr++) != 0) ; - done: if (pb > buf + bufsz) + if (pb > buf + bufsz) FATAL("gsub result2 %.30s too big; can't happen", buf); *pb = '\0'; setsval(x, buf); /* BUG: should be able to avoid copy + free */ -- cinap
arg, also have to not do this: - *pb = '\0'; setsval(x, buf); /* BUG: should be able to avoid copy + free */ was already been done by the copy loop before: while ((*pb++ = *sptr++) != 0) ; i commited the fix now. please report if that fixes the issue. -- cinap
forward the fix to mr kernighan - perhaps you will get a prize? ;-)
-Steve
> On 12 Mar 2022, at 11:58 am, cinap_lenrek@felloff.net wrote:
>
> actually, i think i found it (by staring at the code).
>
> the code at the done label was unconditionally inserting
> NUL terminator, without the final adjbuf() ensuring
> theres space for it.
>
> the patch gets rid of the label, so we wont skip the
> final adjbuf().
>
> diff d52f25ecdcf1dc8ee8d278c8da44159d82d8dd8f uncommitted
> --- a/sys/src/cmd/awk/run.c
> +++ b/sys/src/cmd/awk/run.c
> @@ -1934,7 +1934,7 @@
> }
> }
> if (*c == 0) /* at end */
> - goto done;
> + break;
> adjbuf(&buf, &bufsz, 2+pb-buf, recsize, &pb, "gsub");
> *pb++ = *c++;
> if (pb > buf + bufsz) /* BUG: not sure of this test */
> @@ -1962,8 +1962,12 @@
> *pb++ = *sptr++;
> }
> c = patbeg + patlen;
> - if ((c[-1] == 0) || (*c == 0))
> - goto done;
> + if (c[-1] == 0){
> + c--;
> + break;
> + }
> + if (*c == 0)
> + break;
> if (pb > buf + bufsz)
> FATAL("gsub result1 %.30s too big; can't happen", buf);
> mflag = 1;
> @@ -1973,7 +1977,7 @@
> adjbuf(&buf, &bufsz, 1+strlen(sptr)+pb-buf, 0, &pb, "gsub");
> while ((*pb++ = *sptr++) != 0)
> ;
> - done: if (pb > buf + bufsz)
> + if (pb > buf + bufsz)
> FATAL("gsub result2 %.30s too big; can't happen", buf);
> *pb = '\0';
> setsval(x, buf); /* BUG: should be able to avoid copy + free */
>
> --
> cinap
Steve Simon wrote in <C5AF8865-7335-46CE-A101-EE8F1DF9501C@quintile.net>: |forward the fix to mr kernighan - perhaps you will get a prize? ;-) Mr. BW. K's awk is now maintained by someone else at https://github.com/onetrueawk/awk.git (after Arnold Robbins had a long stint). Cool that 9front git shows the updates again btw!!! Though my last one is facb0e757ac63f763bd942a2714f979538b99eb0 now, from 2021-12-22? |-Steve | |> On 12 Mar 2022, at 11:58 am, cinap_lenrek@felloff.net wrote: |> |> actually, i think i found it (by staring at the code). |> |> the code at the done label was unconditionally inserting |> NUL terminator, without the final adjbuf() ensuring |> theres space for it. |> |> the patch gets rid of the label, so we wont skip the |> final adjbuf(). |> |> diff d52f25ecdcf1dc8ee8d278c8da44159d82d8dd8f uncommitted |> --- a/sys/src/cmd/awk/run.c |> +++ b/sys/src/cmd/awk/run.c |> @@ -1934,7 +1934,7 @@ |>} |>} |> if (*c == 0) /* at end */ |> - goto done; |> + break; At least that is still there. |> adjbuf(&buf, &bufsz, 2+pb-buf, recsize, &pb, "gsub"); |> *pb++ = *c++; |> if (pb > buf + bufsz) /* BUG: not sure of this test */ |> @@ -1962,8 +1962,12 @@ |> *pb++ = *sptr++; |>} |> c = patbeg + patlen; |> - if ((c[-1] == 0) || (*c == 0)) |> - goto done; |> + if (c[-1] == 0){ |> + c--; |> + break; |> + } |> + if (*c == 0) |> + break; That is different now: |> if (pb > buf + bufsz) |> FATAL("gsub result1 %.30s too big; can't happen", \ |> buf); |> mflag = 1; } else *pb++ = *sptr++; } t = patbeg + patlen; if (patlen == 0 || *t == '\0' || *(t-1) == '\0') goto done; if (pb > buf + bufsz) FATAL("gsub result1 %.30s too big; can't happen", buf); mflag = 1; Here too: |> @@ -1973,7 +1977,7 @@ |> adjbuf(&buf, &bufsz, 1+strlen(sptr)+pb-buf, 0, &pb, "gsub"); |> while ((*pb++ = *sptr++) != 0) |4> ; |> - done: if (pb > buf + bufsz) |> + if (pb > buf + bufsz) |> FATAL("gsub result2 %.30s too big; can't happen", buf); done: if (pb < buf + bufsz) *pb = '\0'; else if (*(pb-1) != '\0') FATAL("gsub result2 %.30s truncated; can't happen", buf); |> *pb = '\0'; |> setsval(x, buf); /* BUG: should be able to avoid copy + free */ |> |> -- |> cinap --End of <C5AF8865-7335-46CE-A101-EE8F1DF9501C@quintile.net> --steffen | |Der Kragenbaer, The moon bear, |der holt sich munter he cheerfully and one by one |einen nach dem anderen runter wa.ks himself off |(By Robert Gernhardt)
Sorry I got caught up doing other stuff. I did try acid but had a permission
error which is probably expected or maybe it was a read error I can't remember.
But acid wasn't printing lst() or anything else.
The wrapper would have been the best step. However I did another pull and
merged your branch and 'mk install' The fix seems to work great.
On a side not how do I make a push not that I have anything to push considering
i'm new to this. Do I need to fork a branch and submit a pr or what.
I suspect I need an account or I can create another branch and just paste a diff
into an email. NOT that I have anything to contribute at this point. Just curious.
Regards,
-Will
Quoth cinap_lenrek@felloff.net:
> arg, also have to not do this:
>
> - *pb = '\0';
> setsval(x, buf); /* BUG: should be able to avoid copy + free */
>
> was already been done by the copy loop before:
>
> while ((*pb++ = *sptr++) != 0)
> ;
>
> i commited the fix now.
>
> please report if that fixes the issue.
>
> --
> cinap
>
no. everything is already fixed and commited upstream. i just need you to sysupdate and recompile awk and confirm that this fixed the problem: sysupdate cd /sys/src/cmd/awk mk install -- cinap