* cache: sendfile(2) usage results in short writes to pipes
@ 2022-12-18 18:38 Leah Neukirchen
2022-12-19 15:17 ` Jason A. Donenfeld
0 siblings, 1 reply; 3+ messages in thread
From: Leah Neukirchen @ 2022-12-18 18:38 UTC (permalink / raw)
To: cgit; +Cc: valdis.vitolins
Hi,
in cache.c:print_slot, sendfile is called only once, but not checked
for short writes. These happen for example when serving cgit via
fcgiwrap, which uses a pipe, so the write only writes 65k:
[pid 31211] openat(AT_FDCWD, "/var/cache/cgit/b9200000", O_RDONLY) = 4
[pid 31211] fstat(4, {st_mode=S_IFREG|0600, st_size=19824215, ...}) = 0
[pid 31211] read(4, "mirror/glibc/snapshot/glibc-3aae"..., 4096) = 4096
[pid 31211] sendfile(1, 4, [77] => [65536], 19824138) = 65459
[pid 31211] close(4 <unfinished ...>
This results in truncated cache responses, and therefore broken
downloads from snapshots (larger than 65k).
I assume this is also the cause for the bug reported in
<0ef89b1e-6543-e726-2cf0-b68e7167fabe@odo.lv>, I added the author to Cc:.
I recommend running sendfile in a loop until the whole file is sent.
I also recommend detecting sendfile errors and falling back to the
default read/write implementation, as there are some Linux filesystems
where sendfile fails permanently.
Disabling HAVE_LINUX_SENDFILE on build serves as a workaround.
cu,
--
Leah Neukirchen <leah@vuxu.org> https://leahneukirchen.org/
^ permalink raw reply [flat|nested] 3+ messages in thread
* Re: cache: sendfile(2) usage results in short writes to pipes
2022-12-18 18:38 cache: sendfile(2) usage results in short writes to pipes Leah Neukirchen
@ 2022-12-19 15:17 ` Jason A. Donenfeld
0 siblings, 0 replies; 3+ messages in thread
From: Jason A. Donenfeld @ 2022-12-19 15:17 UTC (permalink / raw)
To: Leah Neukirchen; +Cc: cgit, valdis.vitolins
Fixed in the master branch now.
^ permalink raw reply [flat|nested] 3+ messages in thread
* Re: cache: sendfile(2) usage results in short writes to pipes
[not found] <87wn6opniu.fsf@vuxu.org>
@ 2022-12-22 8:58 ` Valdis Vītoliņš
0 siblings, 0 replies; 3+ messages in thread
From: Valdis Vītoliņš @ 2022-12-22 8:58 UTC (permalink / raw)
To: cgit; +Cc: Leah Neukirchen
Hi, Leah!
Do I understand you correctly that, when compiling and deploying cgit
according to https://git.zx2c4.com/cgit/tree/README
before compilation I should add in Makefile:
HAVE_LINUX_SENDFILE = 1
Thanks!
Valdis
> Hi,
>
> in cache.c:print_slot, sendfile is called only once, but not checked
> for short writes. These happen for example when serving cgit via
> fcgiwrap, which uses a pipe, so the write only writes 65k:
>
> [pid 31211] openat(AT_FDCWD, "/var/cache/cgit/b9200000", O_RDONLY) = 4
> [pid 31211] fstat(4, {st_mode=S_IFREG|0600, st_size=19824215, ...}) = 0
> [pid 31211] read(4, "mirror/glibc/snapshot/glibc-3aae"..., 4096) = 4096
> [pid 31211] sendfile(1, 4, [77] => [65536], 19824138) = 65459
> [pid 31211] close(4 <unfinished ...>
>
> This results in truncated cache responses, and therefore broken
> downloads from snapshots (larger than 65k).
>
> I assume this is also the cause for the bug reported in
> <0ef89b1e-6543-e726-2cf0-b68e7167fabe@odo.lv>, I added the author to Cc:.
>
> I recommend running sendfile in a loop until the whole file is sent.
> I also recommend detecting sendfile errors and falling back to the
> default read/write implementation, as there are some Linux filesystems
> where sendfile fails permanently.
>
> Disabling HAVE_LINUX_SENDFILE on build serves as a workaround.
>
> cu,
^ permalink raw reply [flat|nested] 3+ messages in thread
end of thread, other threads:[~2022-12-22 8:58 UTC | newest]
Thread overview: 3+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2022-12-18 18:38 cache: sendfile(2) usage results in short writes to pipes Leah Neukirchen
2022-12-19 15:17 ` Jason A. Donenfeld
[not found] <87wn6opniu.fsf@vuxu.org>
2022-12-22 8:58 ` Valdis Vītoliņš
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).