From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: (qmail 25142 invoked by alias); 13 Sep 2016 10:21:53 -0000 Mailing-List: contact zsh-workers-help@zsh.org; run by ezmlm Precedence: bulk X-No-Archive: yes List-Id: Zsh Workers List List-Post: List-Help: X-Seq: 39306 Received: (qmail 26345 invoked from network); 13 Sep 2016 10:21:53 -0000 X-Qmail-Scanner-Diagnostics: from mailout3.w1.samsung.com by f.primenet.com.au (envelope-from , uid 7791) with qmail-scanner-2.11 (clamdscan: 0.99.2/21882. spamassassin: 3.4.1. Clear:RC:0(210.118.77.13):SA:0(-2.2/5.0):. Processed in 0.526666 secs); 13 Sep 2016 10:21:53 -0000 X-Spam-Checker-Version: SpamAssassin 3.4.1 (2015-04-28) on f.primenet.com.au X-Spam-Level: X-Spam-Status: No, score=-2.2 required=5.0 tests=RP_MATCHES_RCVD autolearn=unavailable autolearn_force=no version=3.4.1 X-Envelope-From: p.stephenson@samsung.com X-Qmail-Scanner-Mime-Attachments: | X-Qmail-Scanner-Zip-Files: | Received-SPF: none (ns1.primenet.com.au: domain at samsung.com does not designate permitted sender hosts) X-AuditID: cbfec7f5-f79ce6d000004c54-6b-57d7d3384b5b Date: Tue, 13 Sep 2016 11:21:41 +0100 From: Peter Stephenson To: zsh-workers@zsh.org Subject: Re: compset -q oddities Message-id: <20160913112141.1243fbde@pwslap01u.europe.root.pri> In-reply-to: <160912232853.ZM27002@torch.brasslantern.com> Organization: Samsung Cambridge Solution Centre X-Mailer: Claws Mail 3.7.9 (GTK+ 2.22.0; i386-redhat-linux-gnu) MIME-version: 1.0 Content-type: text/plain; charset=US-ASCII Content-transfer-encoding: 7bit X-Brightmail-Tracker: H4sIAAAAAAAAA+NgFnrDIsWRmVeSWpSXmKPExsWy7djP87qWl6+HG5x+wmRxsPkhkwOjx6qD H5gCGKO4bFJSczLLUov07RK4MrbNXM5U8FGk4vbb6WwNjEcEuhg5OSQETCQmf1/LAmGLSVy4 t56ti5GLQ0hgKaPEgcWH2SGcXiaJKUsWMcF0fF62HqxDSGAZo8TLST4QRdOYJE7eboFqP8Mo 8frjRVYI5yyjxM8F39lBWlgEVCUWH74C1s4mYCgxddNsRhBbREBc4uza82BxYQEFiR0rljCD 2LwC9hLHvv4GinNwcApYSazeBHYFv4C+xNW/n6AuspeYeeUMI0S5oMSPyffAxjAL6Ehs2/aY HcKWl9i85i0zyD0SAs3sEv8u3WcGmSkhICux6QAzxBwXifYpTewQtrDEq+NboGwZic6Og1C7 +hklnnT7QsyZwShx+swONoiEtUTf7YuMEMv4JCZtmw41n1eio00IosRD4til7VDljhJPN85k ncCoOAvJ2bOQnD0LydkLGJlXMYqklhbnpqcWm+oVJ+YWl+al6yXn525iBKaB0/+Of93BuPSY 1SFGAQ5GJR7ehtXXwoVYE8uKK3MPMUpwMCuJ8Epeuh4uxJuSWFmVWpQfX1Sak1p8iFGag0VJ nHfPgivhQgLpiSWp2ampBalFMFkmDk6pBkavqqhQ1wa7K9r5uUumTj3aedzCyMZK7LCY88Wb O7a/235syZ4XDLkHfxlLqi8v3H1s1d9DsYr3C6aqzbaQ3uc49/ibT7zfqr9r2jnx3gn81cjE FaR69tPVOhkF7vT//xb8vLKs6kLZuuBSXvlPBs7hagb3T6df6zGMWvfDmLOCUdrM6XizjbwS S3FGoqEWc1FxIgDsltrt/wIAAA== X-Brightmail-Tracker: H4sIAAAAAAAAA+NgFlrGIsWRmVeSWpSXmKPExsVy+t/xy7pcl6+HGzx6JGNxsPkhkwOjx6qD H5gCGKPcbDJSE1NSixRS85LzUzLz0m2VQkPcdC2UFPISc1NtlSJ0fUOClBTKEnNKgTwjAzTg 4BzgHqykb5fglrFt5nKmgo8iFbffTmdrYDwi0MXIySEhYCLxedl6FghbTOLCvfVsXYxcHEIC SxglXv77zALhzGCS+H/0MCuEc45RYsPxT4wQzllGidbpXewg/SwCqhKLD18Bm8UmYCgxddNs RhBbREBc4uza82BxYQEFiR0rljCD2LwC9hLHvv4GinNwcApYSazexAQxcz6TROucI2D1/AL6 Elf/fmKCuM9eYuaVM4wQvYISPybfA6thFtCS2LytiRXClpfYvOYt2HwhAXWJG3d3s09gFJ6F pGUWkpZZSFoWMDKvYhRJLS3OTc8tNtQrTswtLs1L10vOz93ECIyjbcd+bt7BeGlj8CFGAQ5G JR7ehtXXwoVYE8uKK3MPMUpwMCuJ8Epeuh4uxJuSWFmVWpQfX1Sak1p8iNEUGDATmaVEk/OB MZ5XEm9oYmhuaWhkbGFhbmSkJM5b8uFKuJBAemJJanZqakFqEUwfEwenVAMj837NXMsZhw2e fryVXW286v2i/MKAIlPpFU6dsmJZZ/ZfW6iZ/oxj5q03Fy/0m65rjNvw79/R6IYd7PFLXPXl nz5cvayZ45WtywyL1Q9KfGO/rm3M3lilfcI8LK9M9yr/tG8lx1acd/q87uL86+ZHEhk/TH67 Vc5mgkxUxe0YAVuO8/olV4zFlFiKMxINtZiLihMBaZO30bkCAAA= X-MTR: 20000000000000000@CPGS X-CMS-MailID: 20160913102144eucas1p2ed8869ffd3c3736d177f5cef2f6a5bd2 X-Msg-Generator: CA X-Sender-IP: 182.198.249.179 X-Local-Sender: =?UTF-8?B?UGV0ZXIgU3RlcGhlbnNvbhtTQ1NDLURhdGEgUGxhbmUb?= =?UTF-8?B?7IK87ISx7KCE7J6QG1ByaW5jaXBhbCBFbmdpbmVlciwgU29mdHdhcmU=?= X-Global-Sender: =?UTF-8?B?UGV0ZXIgU3RlcGhlbnNvbhtTQ1NDLURhdGEgUGxhbmUbU2Ft?= =?UTF-8?B?c3VuZyBFbGVjdHJvbmljcxtQcmluY2lwYWwgRW5naW5lZXIsIFNvZnR3YXJl?= X-Sender-Code: =?UTF-8?B?QzEwG0VIURtDMTBDRDA1Q0QwNTAwNTg=?= CMS-TYPE: 201P X-HopCount: 7 X-CMS-RootMailID: 20160913063712eucas1p2deca64bf6437fe218152a2ced73328a4 X-RootMTR: 20160913063712eucas1p2deca64bf6437fe218152a2ced73328a4 References: <20160911073031.GA19137@fujitsu.shahaf.local2> <160911191422.ZM21970@torch.brasslantern.com> <20160912230632.GA29577@fujitsu.shahaf.local2> <160912232853.ZM27002@torch.brasslantern.com> On Mon, 12 Sep 2016 23:28:53 -0700 Bart Schaefer wrote: > This one -- > % _g() { compset -q } > % compdef _g g > % g $'\' > compcore.c:1657: expecting 'x' at offset 2 of "'x"> > > -- seems to be a legit problem with counting bytes when looking for an > unbalanced $'...'. I don't think fixing that would change the outcome, > that is $'\' --> \' (see quoting converted, above). I'm not confident > of how to fix it; PWS was last here in workers/22026 (git 34381548). The missing offset seems to come because we have an input string (omitting double quotes) " \\'x" even before we do the double quote processing. The \\ goes aways with the double quote processing leaving the ', which seems to be fine. It looks like it ought to modify tl to give the length we set to zlemetall later on, since that comes from the modified tmp (great names), but it doesn't; however, that doesn't seem to be the problem. When we run the lexer over this and get an error, it then ignores that initial " ", and I think that's what's giving the bad offset. tokstr is now "'x " except with ' tokenised (that's not a problem here); we make efforts to deal with the added final " " but not the fact the previous initial space has (in technical language) gorn. I don't know where that initial space is coming from. I don't know if this means the lexerr stuff is therefore too brittle about characters it found on an unterminated string to be useful and we should simply give up in some more well-defined way. It might be sensible instead of using tokstr to take the input string when we encounter an error, but I presume we'd need to know at least the start of input for the current iteration of the loop containing ctxtlex() (we do have mechanisms for copying literal input during lexing for use with the new-improved $(...) parsing but that's largely orthogonal to what's happening here, where "largely" means "actually I don't really know what's going on in terms of hierarchy because it's way too complicated"). Anyway, I suspect it ought to be possible to do better. I've seen this case crash intermittently as we're accessing bad memory --- we could probably at least fix up accesses off the end of the string after the DPUTS for safety. However, I'm not sure we want to fiddle with this before a release which we really ought to be making imminently as this is just one of a series of incrementally partially fixed problems in this area (This whole interface is horribly brittle anyway, of course, but it's unlikely anyone's going to dare to rewrite it any time soon.) pws