From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: (qmail 10643 invoked by alias); 20 Feb 2015 03:16:11 -0000 Mailing-List: contact zsh-workers-help@zsh.org; run by ezmlm Precedence: bulk X-No-Archive: yes List-Id: Zsh Workers List List-Post: List-Help: X-Seq: 34578 Received: (qmail 14883 invoked from network); 20 Feb 2015 03:16:09 -0000 X-Spam-Checker-Version: SpamAssassin 3.3.2 (2011-06-06) on f.primenet.com.au X-Spam-Level: X-Spam-Status: No, score=-2.7 required=5.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,FREEMAIL_FROM,RCVD_IN_DNSWL_LOW autolearn=ham version=3.3.2 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type:content-transfer-encoding; bh=1zvbz0Q/SHVcIejdiWYHaFZkJUz7Tc/dIC3GlUWfnR0=; b=OgBRSy+AbiJgMzKYblmZO8wU3HH8kxTUESJEXYnU5v4o90xOk+zDlRthll8ZSJHxX0 h5nSo1Y7ta8MqsRBt2tYzhe4Cs5XNfuyHWzjjNNiaxREiPQRAL7Lu3IZ/TF8lHSrP43E WDWAHQUdPUI7AD0K87MdUntGh2g8VvKLOVN1QXiqGivjYxSd09Xg7RwIP1974CA+ILNC k2hyVXVjXn4jqiBvx5cX8nSNm50S/5xyIgw1nKzoex+dYCnHnGrRKA0BK7fw77saY68U u2A6Gp7hPyU0onG+tRkfAGPOiGXBjFNWxdr/KTCcQrqW9wDfAtWRP+BIA0kwXnI0s2F7 W0YA== MIME-Version: 1.0 X-Received: by 10.50.114.4 with SMTP id jc4mr1549819igb.14.1424402165031; Thu, 19 Feb 2015 19:16:05 -0800 (PST) In-Reply-To: <20150219220311.7dfdc4ec@ntlworld.com> References: <20150219101315.477f7f95@pwslap01u.europe.root.pri> <20150219220311.7dfdc4ec@ntlworld.com> Date: Fri, 20 Feb 2015 04:16:04 +0100 Message-ID: Subject: Re: PATCH: parse from even deeper in hell From: Mikael Magnusson To: Peter Stephenson Cc: "Zsh Hackers' List" Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: quoted-printable On Thu, Feb 19, 2015 at 11:03 PM, Peter Stephenson wrote: > On Thu, 19 Feb 2015 22:47:12 +0100 > Mikael Magnusson wrote: >> I get a crapton of "bad(2) wordsplit reading history:" with this >> patch. It seems like all the failed lines have metafied characters in >> them, if that's a hint. Most don't contain any syntax characters at >> all, for example: >> hist.c:3499: bad(2) wordsplit reading history: mp3info =E5=A5=BD=E3=81= =8D=E3=81=AB=E3=81=AA=E3=82=8A\M-c\M-^A=E3=81=84.mp3 >> at: =E5=A5=BD=E3=81=8D=E3=81=AB=E3=81=AA=E3=82=8A\M-c\M-^A=E3=81=84.mp3s >> word: =E5=A5=BD=E3=81=8D=E3=81=AB=E3=81=AA=E3=82=8A\M-c\M-^A=E3=81=84.mp= 3 > > Unless I'm missing something, I don't think you've said what the real > characters you're expecting are. The broken ones aren't much use for > testing. > >> The (2) means it's the second of the two bad=3D1; assignments >> triggering. > > At line 3490? Yes. >> I'm also not sure why the utf8 is slightly mishandled in the output >> there. It has at least been unmetafied, the raw string in the history >> file is more or less: >> mp3info =E5=A5=BD=E3=81=83=EF=BF=BD=E3=81=AB=E3=81=AA=E3=82=83=EF=BF=BD= =E3=81=9F=E3=81=83=EF=BF=BD.mp3 > > So those aren't actually valid characters? Does that mean metafied > characters are getting into the history? I've made it necessary for two > more bytes to be metafied, so if the shell was expecting them to be > metafied in the history file they won't be. The bytes are 0x9e and > 0x9f. I guess we could special case those, but do we really output > metafied characters to the history file? The actual line in the history is mp3info =E5=A5=BD=E3=81=8D=E3=81=AB=E3=81=AA=E3=82=8A=E3=81=9F=E3=81=84.mp3 but in the history _file_, it's stored metafied, which is hard to paste into an email. I'm not sure why pasting the original string didn't occur to me. AFAIK, history files have always been metafied. I'm not sure why the =E3=81=9F is mangled in the error message is what I tr= ied to say originally. The final byte is 9f which I suppose is an esc with the 8th bit set. Maybe something is trying to double unmetafy? Running it through unmetafy() twice doesn't cause any problems though... --=20 Mikael Magnusson