* Re: Fw: ZSH history file VS. UTF-8 data
2011-03-02 9:43 Fw: ZSH history file VS. UTF-8 data Peter Stephenson
@ 2011-03-02 10:00 ` Mikael Magnusson
2011-03-02 10:37 ` lilydjwg
1 sibling, 0 replies; 3+ messages in thread
From: Mikael Magnusson @ 2011-03-02 10:00 UTC (permalink / raw)
To: Peter Stephenson, Kawabata Taichi; +Cc: Zsh Users' List
> Dear sir,
>
> I'm sorry to disturb you.
>
> I've been a user of ZSH for a long time and I really
> appreciate for the developer of this great software.
>
> Recently, I've got one trouble in ZSH history file, and as I tried to
> solve it by
> surveying Webs and Sources, with no help. As of it, knowing that this
> disturbs you,
> I'm trying to ask for any help.
>
> The trouble is as follows:
>
> When I try to use UTF-8 file name in shell command, ZHS history file
> seems to save
> it with "meta code".
>
> For example, executing
> $ ls \346\226\207\345\255\227 (octal expression of "ls 文字”）
> results in histfile
> $ ls \346\203\266\203\247\2\345\255\203\267
>
> That is, when 0x80-0x9F characters are used, then always 0x83 Meta
> character is inserted and following character is bit shifted, resulting
> garbage history.
>
> Any way to avoid this situation?
>
> Any help is really appreciated,
This isn't a bug, the history file is saved in metafied format. If you
want to print it outside zsh you can use this simple program.
#define Meta ((char) 0x83)
#define _GNU_SOURCE
#include <stdio.h>
#include <stdlib.h>
/* from zsh utils.c */
char *unmetafy(char *s, int *len)
{
char *p, *t;
for (p = s; *p && *p != Meta; p++);
for (t = p; (*t = *p++);)
if (*t++ == Meta)
t[-1] = *p++ ^ 32;
if (len)
*len = t - s;
return s;
}
int main(int argc, char *argv[]) {
char *line = NULL;
size_t size;
while (getline(&line, &size, stdin) != -1) {
unmetafy(line, NULL);
printf("%s", line);
}
if (line) free(line);
return EXIT_SUCCESS;
}
--
Mikael Magnusson
^ permalink raw reply [flat|nested] 3+ messages in thread* Re: Fw: ZSH history file VS. UTF-8 data
2011-03-02 9:43 Fw: ZSH history file VS. UTF-8 data Peter Stephenson
2011-03-02 10:00 ` Mikael Magnusson
@ 2011-03-02 10:37 ` lilydjwg
1 sibling, 0 replies; 3+ messages in thread
From: lilydjwg @ 2011-03-02 10:37 UTC (permalink / raw)
To: zsh-users
[-- Attachment #1: Type: text/plain, Size: 1602 bytes --]
On Wed, Mar 02, 2011 at 09:43:36AM +0000, Peter Stephenson wrote:
> I got this. I haven't investigated. Replies will need to be copied
> directly.
>
>
> Begin forwarded message:
>
> Date: Wed, 2 Mar 2011 11:49:29 +0900
> From: ıɥɔıɐʇ ɐʇɐqɐʍɐʞ <kawabata.taichi@gmail.com>
> To: <pws@csr.com>
> Subject: ZSH history file VS. UTF-8 data
>
>
> Dear sir,
>
> I'm sorry to disturb you.
>
> I've been a user of ZSH for a long time and I really
> appreciate for the developer of this great software.
>
> Recently, I've got one trouble in ZSH history file, and as I tried to
> solve it by
> surveying Webs and Sources, with no help. As of it, knowing that this
> disturbs you,
> I'm trying to ask for any help.
>
> The trouble is as follows:
>
> When I try to use UTF-8 file name in shell command, ZHS history file
> seems to save
> it with "meta code".
>
> For example, executing
> $ ls \346\226\207\345\255\227 (octal expression of "ls 文字”）
> results in histfile
> $ ls \346\203\266\203\247\2\345\255\203\267
>
> That is, when 0x80-0x9F characters are used, then always 0x83 Meta
> character is inserted and following character is bit shifted, resulting
> garbage history.
>
> Any way to avoid this situation?
>
> Any help is really appreciated,
>
I met the same problem, and had a look at the zsh source code, attached
is the C program I use to deal with this. However, I have not found an
option to turn this feature off or why the history file is encoded in
this way.
--
Best regards,
lilydjwg
Linux Vim Python 我的博客
http://lilydjwg.is-programmer.com/
[-- Attachment #2: zhist.c --]
[-- Type: text/x-csrc, Size: 1584 bytes --]
//=====================================================================
// 让 zsh 的历史记录可读
//=====================================================================
#include<unistd.h>
#include<stdio.h>
#include<stdlib.h>
#include<string.h>
//---------------------------------------------------------------------
void readhist();
void writehist();
void usage();
static int ismeta(unsigned char ch);
//---------------------------------------------------------------------
int main(int argc, char** argv){
if(argc == 1)
readhist();
else if(argc == 2){
if(!strcmp(argv[1], "read"))
readhist();
else if(!strcmp(argv[1], "write"))
writehist();
else
usage();
}else
usage();
return 0;
}
void readhist(){
unsigned char c, d;
int change = 0;
while(read(0, &c, 1)){
if(c != 0x83){
if(change){
d = c ^ 32;
}else
d = c;
write(1, &d, 1);
change = 0;
}else
change = 1;
}
}
void writehist(){
unsigned char c, d;
while(read(0, &c, 1)){
if(ismeta(c)){
d = 0x83;
write(1, &d, 1);
d = c ^ 32;
write(1, &d, 1);
}else
write(1, &c, 1);
}
}
void usage(){
fprintf(stderr, "Usage: zhist [read|write]\n"
"\tzhist reads from stdin and outputs to stdout, "
"with the content changes\nto be read by human being "
"or by the zsh program.\n");
exit(1);
}
static int ismeta(unsigned char ch){
return (ch > 0x83 && ch < 0x9e)
|| ch == 0xa0 || ch == 0x83 || ch == 0;
}
//=====================================================================
^ permalink raw reply [flat|nested] 3+ messages in thread