From mboxrd@z Thu Jan 1 00:00:00 1970 X-Spam-Checker-Version: SpamAssassin 3.4.4 (2020-01-24) on inbox.vuxu.org X-Spam-Level: X-Spam-Status: No, score=-1.1 required=5.0 tests=DKIM_SIGNED,DKIM_VALID, DKIM_VALID_AU,FREEMAIL_FROM,MAILING_LIST_MULTI autolearn=ham autolearn_force=no version=3.4.4 Received: (qmail 23882 invoked from network); 7 May 2022 12:49:54 -0000 Received: from lists.zx2c4.com (165.227.139.114) by inbox.vuxu.org with ESMTPUTF8; 7 May 2022 12:49:54 -0000 Received: by lists.zx2c4.com (OpenSMTPD) with ESMTP id 619c6ba7; Sat, 7 May 2022 12:49:37 +0000 (UTC) Return-Path: Received: from mail-ej1-x629.google.com (mail-ej1-x629.google.com [2a00:1450:4864:20::629]) by lists.zx2c4.com (OpenSMTPD) with ESMTPS id af237400 (TLSv1.3:AEAD-AES256-GCM-SHA384:256:NO) for ; Sat, 7 May 2022 12:49:36 +0000 (UTC) Received: by mail-ej1-x629.google.com with SMTP id i19so18931009eja.11 for ; Sat, 07 May 2022 05:49:35 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=mime-version:from:date:message-id:subject:to; bh=8UxxVT+COJZO0N2XDTXlVAUNDbKrOHKKX77jqKRNKoI=; b=Ei4YX/4qf2C0Omf5nybcf7+vTupSAC8xdryKd2A3ssAkNMWNdA5sT275jN8npFXko2 Weg9RAnuBqqyX2gbIQXovM0VMWGUfjqhj0Ad9u+fQc70z/AaAPwGX3qxBnY5wtrpCRWn kUhNTtOhWb2PTloYk3OQ0lBV7NXAsSQPXsH8uaLBOnpvK4ZUsxAZyMXQYHEsLN8w7OXB un5l142zHBQcWWH7xYlR+Sw7IU2CtddaOMSWutO0YLCY8mefmfYs4qg0Vq0+axPVZP3u FRStCXTrxFyOYe5zidZE/VP2khIDLT1XZ1H93Mkpp6MgJ2A7PrHtdOwJopYoBLjDsZOC lcqA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:mime-version:from:date:message-id:subject:to; bh=8UxxVT+COJZO0N2XDTXlVAUNDbKrOHKKX77jqKRNKoI=; b=uZi322ldhCze/oH3PgGAN5oG10TeHJmxNfpdK5kSNhD4iKlatud6y0LcYQrrqkzFbp fYZzeIRD5hTqHAs9WOcLrvSEtKMq7VPQaEUsy81gXO2Ow2tXpzTnoNTtBsQJDQI26Dhm Y9m7btW9oMhBLNKJXvEY8nWLa+lX9mZJPYb9eRF68YVEhtXHXYeW8By3ukYHAu2FQc38 liv1sF26D84Ae6gxVgn3eg8DnSQk7XJ1P2jNk5zGWOt43YyNKgpkqAsi2n5VquNr2qLX o7lm2r7mmU0H+GHjRcw8FCCUWA9bdA7RE9cLL72xQ4URZ+016zEdZjvq/BfARzSGMe5i LqBA== X-Gm-Message-State: AOAM532620fSLyZMHBLyeFlEompRq36BtMhz+cQKP2oDjIOt3S5NwH/9 5Gs05J8psDa34b/eoZV72nMUq7nZLiNnlFtMsil0X3yYVYk= X-Google-Smtp-Source: ABdhPJzFxNw4QiuttFjSIm7+3JPKLwWzh3Bu6s+r8Oky77ZbqYTuHTLSxz4aReCwDw8si0Zq+SFq7Slfd++bwSN2tZQ= X-Received: by 2002:a17:906:9743:b0:6d8:632a:a42d with SMTP id o3-20020a170906974300b006d8632aa42dmr7291964ejy.157.1651927775641; Sat, 07 May 2022 05:49:35 -0700 (PDT) MIME-Version: 1.0 From: Filips R Date: Sat, 7 May 2022 15:49:23 +0300 Message-ID: Subject: Truncated output when writing to a pipe using sendfile To: cgit@lists.zx2c4.com Content-Type: text/plain; charset="UTF-8" X-BeenThere: cgit@lists.zx2c4.com X-Mailman-Version: 2.1.30rc1 Precedence: list List-Id: List for cgit developers and users List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: cgit-bounces@lists.zx2c4.com Sender: "CGit" Hello! I've been investigating an issue on my Alpine linux server running cgit 1.2.3-r2. When I call the CGI program and redirect to a regular file (or just output to tty), the output is correct, however if the standard output is a pipe, the output is truncated at 65523 bytes. Here is the end of the diff between strace outputs: open("/var/cache/cgit/a1000000", O_RDONLY|O_LARGEFILE) = 3 fstat(3, {st_mode=S_IFREG|0600, st_size=157003, ...}) = 0 read(3, "st/tree/st.c\0Content-Type: text/"..., 4096) = 4096 -sendfile(1, 3, [13] => [157003], 156990) = 156990 +sendfile(1, 3, [13] => [65536], 156990) = 65523 close(3) = 0 exit_group(0) = ? +++ exited with 0 +++ I have managed to work around the issue by wrapping cgit and redirecting to a temporary file. Looks like the sendfile call comes from cache.c:95 - https://git.zx2c4.com/cgit/tree/cache.c?h=v1.2.3&id=55fa25adb097d2681607d8b0f51a0c393cc9af1a#n95 senfile(2) says that a successful call may write fewer bytes than requested; the caller should be prepared to retry the call if there were unsent bytes