From mboxrd@z Thu Jan 1 00:00:00 1970 X-Spam-Checker-Version: SpamAssassin 3.4.4 (2020-01-24) on inbox.vuxu.org X-Spam-Level: X-Spam-Status: No, score=-3.4 required=5.0 tests=DKIM_SIGNED,DKIM_VALID, DKIM_VALID_AU,MAILING_LIST_MULTI,NICE_REPLY_A,RCVD_IN_DNSWL_MED autolearn=ham autolearn_force=no version=3.4.4 Received: (qmail 22434 invoked from network); 22 Dec 2022 08:58:13 -0000 Received: from lists.zx2c4.com (165.227.139.114) by inbox.vuxu.org with ESMTPUTF8; 22 Dec 2022 08:58:13 -0000 Received: by lists.zx2c4.com (ZX2C4 Mail Server) with ESMTP id c247b508; Thu, 22 Dec 2022 08:57:55 +0000 (UTC) Return-Path: Received: from odo.lv (odo.lv [52.58.11.108]) by lists.zx2c4.com (ZX2C4 Mail Server) with ESMTPS id 67e3024a (TLSv1.3:TLS_AES_256_GCM_SHA384:256:NO) for ; Thu, 22 Dec 2022 08:57:53 +0000 (UTC) Received: from [10.0.0.11] (unknown [213.226.141.197]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by odo.lv (Postfix) with ESMTPSA id B092C1014D3; Thu, 22 Dec 2022 10:57:52 +0200 (EET) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=odo.lv; s=mail; t=1671699472; bh=t1fYk9G8nRnU3ZOYgQLqZGM/iLXSrcT2vD5H9c4gtsg=; h=Date:Reply-To:Subject:To:References:Cc:From:In-Reply-To:From; b=vh4uyvhf5tPUzVONhMR7EPckbyKUrnQbjjHGdkiNSRvWnALusSXAKbsncbkFakYtD 3FSmAVnWBo0cjrfe1/t/1/zGIhmPbH35JLIcQzQws/P+0CQYPXkhm1hL5uJvzjN05d xUHwDmKh5HvteokUVskJjWHAe0SLm/LzfmJbRnLQ= Message-ID: <8839a585-09fd-f672-871c-5fe58e98f82c@odo.lv> Date: Thu, 22 Dec 2022 10:58:00 +0200 MIME-Version: 1.0 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Thunderbird/102.5.1 Subject: Re: cache: sendfile(2) usage results in short writes to pipes Content-Language: lv, en-US To: cgit@lists.zx2c4.com References: <87wn6opniu.fsf@vuxu.org> Cc: Leah Neukirchen From: =?UTF-8?B?VmFsZGlzIFbEq3RvbGnFhsWh?= Organization: Odo SIA In-Reply-To: <87wn6opniu.fsf@vuxu.org> Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 7bit X-BeenThere: cgit@lists.zx2c4.com X-Mailman-Version: 2.1.30rc1 Precedence: list List-Id: List for cgit developers and users List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: valdis.vitolins@odo.lv Errors-To: cgit-bounces@lists.zx2c4.com Sender: "CGit" Hi, Leah! Do I understand you correctly that, when compiling and deploying cgit according to https://git.zx2c4.com/cgit/tree/README before compilation I should add in Makefile: HAVE_LINUX_SENDFILE = 1 Thanks! Valdis > Hi, > > in cache.c:print_slot, sendfile is called only once, but not checked > for short writes. These happen for example when serving cgit via > fcgiwrap, which uses a pipe, so the write only writes 65k: > > [pid 31211] openat(AT_FDCWD, "/var/cache/cgit/b9200000", O_RDONLY) = 4 > [pid 31211] fstat(4, {st_mode=S_IFREG|0600, st_size=19824215, ...}) = 0 > [pid 31211] read(4, "mirror/glibc/snapshot/glibc-3aae"..., 4096) = 4096 > [pid 31211] sendfile(1, 4, [77] => [65536], 19824138) = 65459 > [pid 31211] close(4 > > This results in truncated cache responses, and therefore broken > downloads from snapshots (larger than 65k). > > I assume this is also the cause for the bug reported in > <0ef89b1e-6543-e726-2cf0-b68e7167fabe@odo.lv>, I added the author to Cc:. > > I recommend running sendfile in a loop until the whole file is sent. > I also recommend detecting sendfile errors and falling back to the > default read/write implementation, as there are some Linux filesystems > where sendfile fails permanently. > > Disabling HAVE_LINUX_SENDFILE on build serves as a workaround. > > cu,