* rc hangs forever concatenating a huge list
@ 2014-07-23 6:00 sl
2014-07-23 13:29 ` [9front] " cinap_lenrek
0 siblings, 1 reply; 2+ messages in thread
From: sl @ 2014-07-23 6:00 UTC (permalink / raw)
To: 9front
Yesterday, mischief tried to upload a rather large amount of text to okturing.com,
which seemed to hang forever (no error, but the upload just continued to upload,
forever, until interrupted). We were both able to reproduce the hang reliably simply
by uploading large amounts of text. Small uploads succeeded as normal.
After poking around a bit I determined that the fault probably lies with okturing's
CGI, which attempts to store that huge input in a single list, and then later attempts
to concatenate that list into a single string.
Here is a test example that should be reproducible on any system:
; cd /lib/rfc
; wc rfc-index.xml
229468 695874 9953256 rfc-index.xml
; tail -1 rfc-index.xml
</rfc-index>
; a=`{cat rfc-index.xml}
; echo $#a
695874
; echo $a($#a)
</rfc-index>
As you can see, the entire contents of the file is accessible from list $a. No
problem so far.
However, when we attempt to concatenate the list as a string:
; echo $"a
This hangs indefinitely. Additionally, the Del key now seems to have no effect.
Interestingly, the echo command is absent from the output of ps. So, maybe rc is
choking while trying to process its arguments and has not yet attempted to run the
echo.
While hung, the rc process in question will fall into a state similar to this:
; ps | grep 339327
sl 339327 4:11 0:01 191808K Running rc
; cat /proc/339327/fd
/lib/rfc
0 r M 79379 (0000000000000001 0 00) 8192 74 /mnt/term/dev/cons
1 w M 79379 (0000000000000001 0 00) 8192 258 /mnt/term/dev/cons
2 w M 79379 (0000000000000001 0 00) 8192 258 /mnt/term/dev/cons
3 r M 79363 (00000000000080dd 2 00) 8192 512 /rc/lib/rcmain
4 r M 79379 (0000000000000001 0 00) 8192 74 /mnt/term/dev/cons
5 w M 79379 (0000000000000004 0 00) 8192 8 /dev/wdir
; acid 339327
/proc/339327/text:amd64 plan 9 executable
/sys/lib/acid/port
/sys/lib/acid/amd64
acid: lstk()
strcat(p2=0xa733158)+0x1c /sys/src/libc/amd64/strcat.s:22
Xqdol()+0x122 /sys/src/cmd/rc/exec.c:672
s=0x428b68
a=0xaf4df08
n=0x428b680077d1f6
p=0xa733118
main(argv=0x7ffffeffef98,argc=0x1)+0x3d2 /sys/src/cmd/rc/exec.c:184
rcmain=0x400dd8
num=0x373233393333
bootstrap=0x2
i=0x400dd800000000
_main+0x40 /sys/src/libc/amd64/main9.s:15
acid:
I experimented with concatenating incrementally smaller $a lists and discovered that
past a certain threshold, the echo does eventually succeed, but only after a very
long delay. This suggests that the "hangs indefinitely" observed above may in fact
just be an extended period spent processing the concatenation.
Yesterday, mischief ran [unspecified profiling] on a similar test and concluded:
mischief → Xqdol()+0x122 /sys/src/cmd/rc/exec.c:672
mischief → it counts the total length of each word in the list
mischief → allocates that
mischief → and then concatenates each word together
mischief → my poor man's profiler indicates thats where the time is spent
mischief → it spends a shit ton of time just doing the strcat'ing
There seems to be no problem managing the huge list. But squashing the huge list
into a string spins our wheels.
What gives?
sl
^ permalink raw reply [flat|nested] 2+ messages in thread
* Re: [9front] rc hangs forever concatenating a huge list
2014-07-23 6:00 rc hangs forever concatenating a huge list sl
@ 2014-07-23 13:29 ` cinap_lenrek
0 siblings, 0 replies; 2+ messages in thread
From: cinap_lenrek @ 2014-07-23 13:29 UTC (permalink / raw)
To: 9front
that makes sense, the longer the old string in the destination buffer,
the longer strcat() takes.
whats funny is that the code isnt really needed. list2str() already
does what Xqdol() does and it does it in linear time. :)
fixed.
--
cinap
^ permalink raw reply [flat|nested] 2+ messages in thread
end of thread, other threads:[~2014-07-23 13:29 UTC | newest]
Thread overview: 2+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2014-07-23 6:00 rc hangs forever concatenating a huge list sl
2014-07-23 13:29 ` [9front] " cinap_lenrek
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).