From: Leah Neukirchen <leah@vuxu.org>
To: <cgit@lists.zx2c4.com>
Cc: <valdis.vitolins@odo.lv>
Subject: cache: sendfile(2) usage results in short writes to pipes
Date: Sun, 18 Dec 2022 19:38:56 +0100 [thread overview]
Message-ID: <87v8m8pnen.fsf@vuxu.org> (raw)
Hi,
in cache.c:print_slot, sendfile is called only once, but not checked
for short writes. These happen for example when serving cgit via
fcgiwrap, which uses a pipe, so the write only writes 65k:
[pid 31211] openat(AT_FDCWD, "/var/cache/cgit/b9200000", O_RDONLY) = 4
[pid 31211] fstat(4, {st_mode=S_IFREG|0600, st_size=19824215, ...}) = 0
[pid 31211] read(4, "mirror/glibc/snapshot/glibc-3aae"..., 4096) = 4096
[pid 31211] sendfile(1, 4, [77] => [65536], 19824138) = 65459
[pid 31211] close(4 <unfinished ...>
This results in truncated cache responses, and therefore broken
downloads from snapshots (larger than 65k).
I assume this is also the cause for the bug reported in
<0ef89b1e-6543-e726-2cf0-b68e7167fabe@odo.lv>, I added the author to Cc:.
I recommend running sendfile in a loop until the whole file is sent.
I also recommend detecting sendfile errors and falling back to the
default read/write implementation, as there are some Linux filesystems
where sendfile fails permanently.
Disabling HAVE_LINUX_SENDFILE on build serves as a workaround.
cu,
--
Leah Neukirchen <leah@vuxu.org> https://leahneukirchen.org/
next reply other threads:[~2022-12-18 18:39 UTC|newest]
Thread overview: 3+ messages / expand[flat|nested] mbox.gz Atom feed top
2022-12-18 18:38 Leah Neukirchen [this message]
2022-12-19 15:17 ` Jason A. Donenfeld
[not found] <87wn6opniu.fsf@vuxu.org>
2022-12-22 8:58 ` Valdis Vītoliņš
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=87v8m8pnen.fsf@vuxu.org \
--to=leah@vuxu.org \
--cc=cgit@lists.zx2c4.com \
--cc=valdis.vitolins@odo.lv \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).