Re: PATCH: parse from even deeper in hell

[Mikael Magnusson <mikachu@gmail.com>]

On Thu, Feb 19, 2015 at 11:03 PM, Peter Stephenson
<p.w.stephenson@ntlworld.com> wrote:
> On Thu, 19 Feb 2015 22:47:12 +0100
> Mikael Magnusson <mikachu@gmail.com> wrote:
>> I get a crapton of "bad(2) wordsplit reading history:" with this
>> patch. It seems like all the failed lines have metafied characters in
>> them, if that's a hint. Most don't contain any syntax characters at
>> all, for example:
>>  hist.c:3499: bad(2) wordsplit reading history: mp3info 好きになり\M-c\M-^Aい.mp3
>> at: 好きになり\M-c\M-^Aい.mp3s
>> word: 好きになり\M-c\M-^Aい.mp3
>
> Unless I'm missing something, I don't think you've said what the real
> characters you're expecting are.  The broken ones aren't much use for
> testing.
>
>> The (2) means it's the second of the two bad=1; assignments
>> triggering.
>
> At line 3490?

Yes.

>> I'm also not sure why the utf8 is slightly mishandled in the output
>> there. It has at least been unmetafied, the raw string in the history
>> file is more or less:
>> mp3info 好ぃ�になゃ�たぃ�.mp3
>
> So those aren't actually valid characters?  Does that mean metafied
> characters are getting into the history?  I've made it necessary for two
> more bytes to be metafied, so if the shell was expecting them to be
> metafied in the history file they won't be.  The bytes are 0x9e and
> 0x9f.  I guess we could special case those, but do we really output
> metafied characters to the history file?

The actual line in the history is
mp3info 好きになりたい.mp3
but in the history _file_, it's stored metafied, which is hard to
paste into an email. I'm not sure why pasting the original string
didn't occur to me. AFAIK, history files have always been metafied.
I'm not sure why the た is mangled in the error message is what I tried
to say originally. The final byte is 9f which I suppose is an esc with
the 8th bit set. Maybe something is trying to double unmetafy? Running
it through unmetafy() twice doesn't cause any problems though...

-- 
Mikael Magnusson