From mboxrd@z Thu Jan  1 00:00:00 1970
Return-Path: <zsh-users-return-15832-mason-zsh=primenet.com.au@zsh.org>
Received: (qmail 12366 invoked by alias); 2 Mar 2011 10:01:05 -0000
Mailing-List: contact zsh-users-help@zsh.org; run by ezmlm
Precedence: bulk
X-No-Archive: yes
List-Id: Zsh Users List <zsh-users.zsh.org>
List-Post: <mailto:zsh-users@zsh.org>
List-Help: <mailto:zsh-users-help@zsh.org>
X-Seq: 15832
Received: (qmail 18430 invoked from network); 2 Mar 2011 10:00:53 -0000
X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on f.primenet.com.au
X-Spam-Level: 
X-Spam-Status: No, score=-0.0 required=5.0 tests=BAYES_00,DEAR_SOMETHING,
	DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,FREEMAIL_FROM,RCVD_IN_DNSWL_NONE
	autolearn=no version=3.3.1
Received-SPF: pass (ns1.primenet.com.au: SPF record at _spf.google.com designates 209.85.220.171 as permitted sender)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=gmail.com; s=gamma;
        h=domainkey-signature:mime-version:in-reply-to:references:date
         :message-id:subject:from:to:cc:content-type
         :content-transfer-encoding;
        bh=PtZTgBLpPBO4I4c8H/cidtxp1wwbCSDUoOXcfOtNntA=;
        b=i01eNiR3RAl3K4XMGJoQ+9/bjLxZsfmlboH5kSqqh+Ct2IyuGcK97qKDfxj+Cu/RBk
         U8KQGXDQqBmeNm+rVSYRljXQvYb2Xb0Lh4wbn6zxRIiJvhjlU2pP8S874EUjgFta2PyR
         gBENP8a5ukBeVU3YpL5b9MfLisP5csJ1+YihE=
DomainKey-Signature: a=rsa-sha1; c=nofws;
        d=gmail.com; s=gamma;
        h=mime-version:in-reply-to:references:date:message-id:subject:from:to
         :cc:content-type:content-transfer-encoding;
        b=ow8zn4OR6LSnv+wQxqTfJ6pasTiKadEWwWGHeA048T6KGtd021Eqg1mwrmseU2SQKR
         sKzgl5daTVgqtwcBjwb8Wa/I2CbnA3ENJr2l5f1oOW7cqG9L+pqzJaFb8TwKlPkpLkx9
         y69zDIJ4DlfCgfn7iohRnypBbPihL+6/1vyFc=
MIME-Version: 1.0
In-Reply-To: <20110302094336.62001793@pwslap01u.europe.root.pri>
References: <20110302094336.62001793@pwslap01u.europe.root.pri>
Date: Wed, 2 Mar 2011 11:00:45 +0100
Message-ID: <AANLkTi=9ARVQeU556YcrSB9=x4MmFU7=RqdxZ8GxgxUv@mail.gmail.com>
Subject: Re: Fw: ZSH history file VS. UTF-8 data
From: Mikael Magnusson <mikachu@gmail.com>
To: Peter Stephenson <pws@csr.com>, Kawabata Taichi <kawabata.taichi@gmail.com>
Cc: "Zsh Users' List" <zsh-users@zsh.org>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: quoted-printable

> Dear sir,
>
> I'm sorry to disturb you.
>
> I've been a user of ZSH  for a long time and I really
> appreciate for the developer of this great software.
>
> Recently, I've got one trouble in ZSH history file, and as I tried to
> solve it by
> surveying Webs and Sources, with no help.  As of it, knowing that this
> disturbs you,
> I'm trying to ask for any help.
>
> The trouble is as follows:
>
> When I try to use UTF-8 file name in shell command, ZHS history file
> seems to save
> it with "meta code".
>
> For example, executing
>  $ ls \346\226\207\345\255\227  (octal expression of "ls =E6=96=87=E5=AD=
=97=E2=80=9D=EF=BC=89
> results in histfile
>  $ ls \346\203\266\203\247\2\345\255\203\267
>
> That is, when 0x80-0x9F characters are used, then always 0x83 Meta
> character is inserted and following character is bit shifted, resulting
> garbage history.
>
> Any way to avoid this situation?
>
> Any help is really appreciated,

This isn't a bug, the history file is saved in metafied format. If you
want to print it outside zsh you can use this simple program.

#define Meta ((char) 0x83)

#define _GNU_SOURCE
#include <stdio.h>
#include <stdlib.h>

/* from zsh utils.c */
char *unmetafy(char *s, int *len)
{
  char *p, *t;

  for (p =3D s; *p && *p !=3D Meta; p++);
  for (t =3D p; (*t =3D *p++);)
    if (*t++ =3D=3D Meta)
      t[-1] =3D *p++ ^ 32;
  if (len)
    *len =3D t - s;
  return s;
}

int main(int argc, char *argv[]) {
  char *line =3D NULL;
  size_t size;

  while (getline(&line, &size, stdin) !=3D -1) {
    unmetafy(line, NULL);
    printf("%s", line);
  }

  if (line) free(line);
  return EXIT_SUCCESS;
}


--=20
Mikael Magnusson