From mboxrd@z Thu Jan  1 00:00:00 1970
Return-Path: <zsh-workers-return-31492-mason-zsh=primenet.com.au@zsh.org>
Received: (qmail 13438 invoked by alias); 23 Jun 2013 21:24:23 -0000
Mailing-List: contact zsh-workers-help@zsh.org; run by ezmlm
Precedence: bulk
X-No-Archive: yes
List-Id: Zsh Workers List <zsh-workers.zsh.org>
List-Post: <mailto:zsh-workers@zsh.org>
List-Help: <mailto:zsh-workers-help@zsh.org>
X-Seq: 31492
Received: (qmail 27783 invoked from network); 23 Jun 2013 21:24:09 -0000
X-Spam-Checker-Version: SpamAssassin 3.3.2 (2011-06-06) on f.primenet.com.au
X-Spam-Level: 
X-Spam-Status: No, score=-2.7 required=5.0 tests=BAYES_00,DKIM_SIGNED,
	DKIM_VALID,DKIM_VALID_AU,FREEMAIL_FROM,RCVD_IN_DNSWL_LOW autolearn=ham
	version=3.3.2
Received-SPF: pass (ns1.primenet.com.au: SPF record at _netblocks.google.com designates 209.85.223.174 as permitted sender)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=gmail.com; s=20120113;
        h=mime-version:in-reply-to:references:date:message-id:subject:from:to
         :content-type;
        bh=CxwsS7QR+9tq34S22zpQjBPYUt2gtOt4vHboqDx37tM=;
        b=p5V8E5SZ8oX58SsRQqRUrLXSAB6elZrWX7Je6Yi+o9cZVLbPbOdZypYMJlYFYaqC3K
         Wl701xQqCZr08LdXX6YVRsqpM5o7GKgskSOm6L/snzM1rbkkbr/KOZ81+9ePY8bCt7B3
         6yAaNqtD90luIDT136awLRgTCK99yk8CyX2Qlm2kzy31ixd8HZmWVHhmw6P8CkLlogZN
         DDSB8lXivRw6dRBwVjBsxdMnHYwqsM9ZDfEDj/7Befj2TNxe1jBPh+tVlJygMr2ykjUI
         NsFnZ6RMxDrC7pz0CxBVX0UtNXTE17yCkC75v+cAMVsRoGRYgDFgVL3CdclzHloFuIxH
         RmxQ==
MIME-Version: 1.0
X-Received: by 10.50.32.10 with SMTP id e10mr4134737igi.10.1372022644183; Sun,
 23 Jun 2013 14:24:04 -0700 (PDT)
In-Reply-To: <CAH+w=7ZQiT8g3uU+rzmKNWC+WE-xOrVCX_LUHZKJq5tPnOxgtw@mail.gmail.com>
References: <20130622184554.GA32128@adz-acer>
	<CAHYJk3RqD-JfWKUeG08Atrbhcoz9XZEXvBP_b_cP21EMy5RunA@mail.gmail.com>
	<CAH+w=7ZQiT8g3uU+rzmKNWC+WE-xOrVCX_LUHZKJq5tPnOxgtw@mail.gmail.com>
Date: Sun, 23 Jun 2013 23:24:04 +0200
Message-ID: <CAHYJk3SFaF8Y3+2SPKXjmp3BRS3fE6FqKj7YGcWA5EaidoMmnw@mail.gmail.com>
Subject: Re: [bug] history corrupt on utf8
From: Mikael Magnusson <mikachu@gmail.com>
To: zsh workers <zsh-workers@zsh.org>
Content-Type: text/plain; charset=UTF-8

On 23 June 2013 20:33, Bart Schaefer <schaefer@brasslantern.com> wrote:
> On Jun 22, 2013 7:07 PM, "Mikael Magnusson" <mikachu@gmail.com> wrote:
>>
>> The history file isn't saved verbatim, since the command line can
>> contain non-utf8, nuls, etc.
>
> To be more specific, the history file is saved in the same format used to
> represent history lines internally, except for prefixing newlines with
> backslash.  The file is only "corrupted" if zsh itself can't re-read it.
> We gave up on sharing history files with other shells a long time ago.

If you want, you can use this small program to "uncorrupt" it
(obviously don't overwrite the history file with the result),

#define Meta ((char) 0x83)

#define _GNU_SOURCE
#include <stdio.h>
#include <stdlib.h>

/* from zsh utils.c */
char *unmetafy(char *s, int *len)
{
  char *p, *t;

  for (p = s; *p && *p != Meta; p++);
  for (t = p; (*t = *p++);)
    if (*t++ == Meta)
      t[-1] = *p++ ^ 32;
  if (len)
    *len = t - s;
  return s;
}

int main(int argc, char *argv[]) {
  char *line = NULL;
  size_t size;

  while (getline(&line, &size, stdin) != -1) {
    unmetafy(line, NULL);
    printf("%s", line);
  }

  if (line) free(line);
  return EXIT_SUCCESS;
}

--
Mikael Magnusson