From mboxrd@z Thu Jan 1 00:00:00 1970 X-Spam-Checker-Version: SpamAssassin 3.4.4 (2020-01-24) on inbox.vuxu.org X-Spam-Level: X-Spam-Status: No, score=-3.3 required=5.0 tests=DKIM_SIGNED,DKIM_VALID, MAILING_LIST_MULTI,RCVD_IN_DNSWL_MED,T_SCC_BODY_TEXT_LINE, UNPARSEABLE_RELAY autolearn=ham autolearn_force=no version=3.4.4 Received: (qmail 18847 invoked from network); 19 Aug 2022 02:12:02 -0000 Received: from zero.zsh.org (2a02:898:31:0:48:4558:7a:7368) by inbox.vuxu.org with ESMTPUTF8; 19 Aug 2022 02:12:02 -0000 ARC-Seal: i=1; cv=none; a=rsa-sha256; d=zsh.org; s=rsa-20210803; t=1660875123; b=bFV/gomPUUyOwtOuriCuMdFYPOy+T1Yprhd6z1+2mQZaj3xNJjuXLcfFcxhrRQ77yEy0/TGN4X qMyXgss2BlgzCky4V7zcDfasdaVxQxnYQddzz0568cz9ixW0vwpPmH8TnIqnc3zVxGjc7vfIvR bCg7v9hJOSJ4xRgpkqcWyB8IhE1Y5qXktAkliAVgj8r57XGN3m44GJMFWvOsVW7Il5Zt+qHIH2 iveKtF+yUGsUN2XuShlcxVwbZJtmeNLw/99RkEek8/QpXsBa3hVtEQ/7SNnAtKZ8x3hiP6WQ/y BuhcsVamSruNNZfyLoopKsuBM0GRran85rTK1eyfUVKwvA==; ARC-Authentication-Results: i=1; zsh.org; iprev=pass (mail-pl1-f176.google.com) smtp.remote-ip=209.85.214.176; dkim=pass header.d=awkwork-com.20210112.gappssmtp.com header.s=20210112 header.a=rsa-sha256; dmarc=none header.from=awkwork.com; arc=none ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed; d=zsh.org; s=rsa-20210803; t=1660875123; bh=qT3zN/fWvYbGLDx0dyR6pWOnN+YxmiIJfb5ucfB1JDc=; h=List-Archive:List-Owner:List-Post:List-Unsubscribe:List-Subscribe:List-Help: List-Id:Sender:Content-Type:To:Subject:Message-ID:Date:From:MIME-Version: DKIM-Signature:DKIM-Signature; b=fEoUP348O3dI7GQLLd/QWeEy5veOn5F2vM7R5gSVDLqKrvaVvH+KGQx3r78bfVhQYgW4MZ1t9T I4KLf+/huxZ6rsus8jbiMr1TUpWpxcpx98TkjIN3UYhmcVZqWtx3TyvbYWZ/wCxFlPDZHEtxFi 1NctcCgVDlMVxBaSonLfm8xOcOO2/6RxdoJPNfa2Si7ngCXIo44Zh+I0vjVLn8Q/jBYLOvzINV BYHyKLnLNrlPsE2d64FcAJLoSsKhB7SL/1UgF/WGB//ISwloqPF1DljO98y3JJ1Oz/XflvGVEK nEchwNH6mLhoPyCOWfyjnSKHDa2MZZoOTXUYxTb8fPKVVQ==; DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=zsh.org; s=rsa-20210803; h=List-Archive:List-Owner:List-Post:List-Unsubscribe: List-Subscribe:List-Help:List-Id:Sender:Content-Type:To:Subject:Message-ID: Date:From:MIME-Version:Reply-To:Cc:Content-Transfer-Encoding:Content-ID: Content-Description:Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc :Resent-Message-ID:In-Reply-To:References; bh=w6yGUQRlKwBZbkREDCixPhiUyvRrGIFM6By67juyW5A=; b=EFDGVp09ZcCO26UGnUZv3qKKH1 /AxkM+QzAOsm7lLEQWNMm1LWcq1XAwPEXerUQJ51sOPjozr7P1zn8EHl6dMDN7mLacQx4nW4H7/Tv Cab3sCUbE+UVVNiPGIV42HyEmJLX6DUcgrtug10t2wFIu/shRfjBCIZ39/VT7SjP34ZyGPMVOpo8s E4TtH/fG00+m5YYczV27YwGUUXNk9BZAAFW/Y6MYxAJv/KkBngVOCM7avURzR6/Owujqz/fTbztmT gRhZuxFYlTjjXZxOVcbMewL+wgr4ZR5ETg+dFXT1t+7g+tAwUgKffJ2JIm2rDNu5hWR8ViuG5p1PH jzBOIv3Q==; Received: from authenticated user by zero.zsh.org with local id 1oOrUQ-0008ET-Ih; Fri, 19 Aug 2022 02:12:02 +0000 Authentication-Results: zsh.org; iprev=pass (mail-pl1-f176.google.com) smtp.remote-ip=209.85.214.176; dkim=pass header.d=awkwork-com.20210112.gappssmtp.com header.s=20210112 header.a=rsa-sha256; dmarc=none header.from=awkwork.com; arc=none Received: from mail-pl1-f176.google.com ([209.85.214.176]:41911) by zero.zsh.org with esmtps (TLS1.3:TLS_AES_128_GCM_SHA256:128) id 1oOrPJ-0007lk-6R; Fri, 19 Aug 2022 02:06:45 +0000 Received: by mail-pl1-f176.google.com with SMTP id p18so2977176plr.8 for ; Thu, 18 Aug 2022 19:06:45 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=awkwork-com.20210112.gappssmtp.com; s=20210112; h=to:subject:message-id:date:from:mime-version:from:to:cc; bh=w6yGUQRlKwBZbkREDCixPhiUyvRrGIFM6By67juyW5A=; b=pPNc324L0yFtxl+INyLGiqQa7X2tlU2bnX3ThMQNYnslJ/iRg1OTF/fPjE/WCjqRFQ ZLSf+7LqbRZneOOBFADPx5vaByi5q3vetuImvF5tKy0mRBPIRtC8lrodkbCMBMgy63z0 DJv3ZSouC88nxL8cDj2/RCdQRMpx76Vbsei60M8PG+PMVCwfk3CEsltWxNSt1PkiUlq1 w4ZJ55TI8/7hdrgzc2cGXaSBLskKna4Uj9r/52vRoHoHUnXC6HxHhEMCX+qvTyriuD5Q 9f/oNNPVZY+bkNzKCVgi1bJOg7C7I8W+EjIr5qwKRjhJS0dDB0oycH+Ix5O+HS0ir5a/ 13Zg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=to:subject:message-id:date:from:mime-version:x-gm-message-state :from:to:cc; bh=w6yGUQRlKwBZbkREDCixPhiUyvRrGIFM6By67juyW5A=; b=KxOcD1GsKf+pU3icfUiMn8neVIBoZPxi5PEVv47X3ujTVfPxYz1mvhsfbAOHc2egXr 4fFtQYwAfT1z12AErQ8L0VNesTLFE5Sl84YpbShey0RY3k+znV7RvfkzlBRYaSghsW0Q qeX8fvTmbiW1NPMFwbDW0yTNCa6vj7yv8UA/K749726flJ/lCwpqwO4w2lqIhLld2Rck aiHcxxHM9y9QdQroH7UoFaP07mEKWYL3evKQXXvKW2DtFW6NEN/3ctMThuVvhoDaUdEj MRF4OVO4k+RLDO1qzaoIqj6n2ye1r0gOSe6lbajhTusvLkQvNWPnED6FI9wuL1Jmueu+ qqgA== X-Gm-Message-State: ACgBeo3fovWBizdrVJEXJLEU9Hc2fWhpwnk9XRFpX8gpzxvzw9TVeMcu Wt7w3dL/1zeLDgQzMu2SLshkItAiDUDsMyvI6j4DEB7KjWo= X-Google-Smtp-Source: AA6agR7g7iui6D0U58QSosjj3TF89qQFsyPbXtErp/aefNSuI66pb3b1k6nJWzQg6qIbtKF+ishwuUnrDGHHXVmKQE8= X-Received: by 2002:a17:902:d151:b0:172:9361:2a11 with SMTP id t17-20020a170902d15100b0017293612a11mr5167130plt.148.1660874803536; Thu, 18 Aug 2022 19:06:43 -0700 (PDT) MIME-Version: 1.0 From: Alan Wagner-Krankel Date: Thu, 18 Aug 2022 21:06:16 -0500 Message-ID: Subject: zargs error with some combinations of arguments To: zsh-workers@zsh.org Content-Type: text/plain; charset="UTF-8" X-Seq: 50486 Archived-At: X-Loop: zsh-workers@zsh.org Errors-To: zsh-workers-owner@zsh.org Precedence: list Precedence: bulk Sender: zsh-workers-request@zsh.org X-no-archive: yes List-Id: List-Help: List-Subscribe: List-Unsubscribe: List-Post: List-Owner: List-Archive: I've run across some conditions that apparently trip up calls to zargs. A simplified example: zsh -f <<\eof vals=(${(l:79::x:):-{001..256}}) print count:${#vals} length:${(c)#vals} autoload -Uz zargs zargs -- $vals -- print -l eof The output: count:256 length:20479 zargs: cannot fit single argument within size limit This error occurs when: - the combined length of the arguments that are going to be used in a call is less than the max argument size (-s, default is 20480), and - the length of the command string plus the arguments is greater than the -s size value. It looks like zargs was set up this way to emulate the behavior of gxargs from GNU findutils. With the right options, you can get a similar error: gxargs -L 256 -s 20480 -x echo <<<${(f)${(l:79::x:):-{001..256}}} On my system this returns: gxargs: argument list too long I wasn't able to find an input that gave this result when using the (large) defaults for gxargs. On the other hand, POSIX seems to want the constructed command to fit within all of the various size arguments, if I'm reading this snippet in the spec correctly: -s size Invoke utility using as many standard input arguments as possible yielding a command line length less than size (a positive decimal integer) bytes. Fewer arguments shall be used if: - The total number of arguments exceeds that specified by the -n option. - The total number of lines exceeds that specified by the -L option. I've run into the zargs error a couple of times recently with real-world data, so my system is now using a version that is hopefully a bit closer to the POSIX behavior. The diff listing is below. Another option would have been to use a large value for -s, and significantly reduce the likelihood of hitting a set of inputs that result in an error. That seems to be the gxargs approach. Any thoughts? Is there something else I missed, or is a change like I've described something that you want to consider for a future version of zargs? Thank you for reading this far. Hope I didn't make too many mistakes in my first post to the list :). Thanks, Awk diff --context=1 zsh.5.9/Functions/Misc/zargs ~/.zfunctions/zargs *** zsh.5.9/Functions/Misc/zargs --- ~/.zfunctions/zargs *************** *** 178,180 **** ! local -i end c=0 if [[ $eof == -(e|-eof) ]]; then ((end=ARGC+1)) --- 178,180 ---- ! local -i end a c=0 if [[ $eof == -(e|-eof) ]]; then ((end=ARGC+1)) *************** *** 249,250 **** --- 249,257 ---- + (( a = s - ${#:-"$command "} )) + if (( a <= 0 )) + then + print -u2 'zargs: value for max-chars must be >= command length' + return 1 + fi + l=${${${l##*-(l|L|-max-lines(=|))}[-1]}:-${${l[1]:+1}:-$ARGC}} *************** *** 291,293 **** ((ARGC)) || break ! for (( end=l; end && ${(c)#argv[1,end]} > s; end/=2 )) { } (( end > n && ( end = n ) )) --- 298,300 ---- ((ARGC)) || break ! for (( end=l; end && ${(c)#argv[1,end]} > a; end/=2 )) { } (( end > n && ( end = n ) ))