From mboxrd@z Thu Jan  1 00:00:00 1970
Message-ID: <ee9e417a050923091375aed85c@mail.gmail.com>
Date: Fri, 23 Sep 2005 12:13:25 -0400
From: Russ Cox <rsc@swtch.com>
To: ISHWAR RATTAN <rattan@cps.cmich.edu>
Subject: Re: [9fans] deleting files?
In-Reply-To: <Pine.GSO.4.61.0509231142420.2660@cps210.cps.cmich.edu.>
MIME-Version: 1.0
Content-Type: text/plain; charset=ISO-8859-1
Content-Transfer-Encoding: quoted-printable
Content-Disposition: inline
References: <Pine.GSO.4.61.0509221013120.2321@cps210.cps.cmich.edu.>
	<ee9e417a05092207214395b629@mail.gmail.com>
	<Pine.GSO.4.61.0509231142420.2660@cps210.cps.cmich.edu.>
Cc: Fans of the OS Plan 9 from Bell Labs <9fans@cse.psu.edu>
Topicbox-Message-UUID: 90cfd69a-ead0-11e9-9d60-3106f5b1d025

>    The out put of
>
> ls | grep H.*test1.c | xd -b -c
>    is
>
> 0000000  48 ef bf bd 74 65 73 74 31 2e 63 0a
>        0   H ef bf bd  t  e  s  t  1  .  c \n
> 000000c

Bad UTF got into the file system some how,
and so ls is showing you an FFFD (RuneError) instead.
(Now I understand what you meant by [?].)

It looks like fossil doesn't check aggressively enough
to find bad UTF before using the name.  We should fix
this, but until then, you can use the program below,
fixbadutf, to rename the troublesome files.

Russ

#include <u.h>
#include <libc.h>

void
usage(void)
{
=09fprint(2, "usage: fixbadutf [dir]\n");
=09exits("usage");
}

int
isbadutf(char *s)
{
=09Rune r;
=09int n;

=09r =3D Runeerror;
=09for(; *s; s+=3Dn){
=09=09n =3D chartorune(&r, s);
=09=09if(r =3D=3D Runeerror && n =3D=3D 1)
=09=09=09return 1;
=09}
=09return 0;
}

char*
mkgoodname(char *name)
{
=09char *gname, *p, *s;
=09int i, n;
=09Rune r;

=09gname =3D malloc(strlen(name)+20);
=09if(gname =3D=3D nil)
=09=09sysfatal("out of memory");
=09s =3D gname;
=09p =3D name;
=09for(; *p; p+=3Dn){
=09=09n =3D chartorune(&r, p);
=09=09if(r =3D=3D Runeerror && n =3D=3D 1)
=09=09=09*s++ =3D '?';
=09=09else{
=09=09=09for(i=3D0; i<n; i++)
=09=09=09=09*s++ =3D p[i];
=09=09}
=09}
=09*s =3D 0;
=09for(i=3D0; i<100; i++){
=09=09if(access(gname, AEXIST) < 0)
=09=09=09return gname;
=09=09sprint(s, ".%d", i);
=09}
=09fprint(2, "cannot translate %s\n", name);
=09return nil;
}

void
main(int argc, char **argv)
{
=09char *dir, *errors, *newname, *oldname;
=09int fd, i, j, n;
=09Dir *d, xd;

=09ARGBEGIN{
=09default:
=09=09usage();
=09}ARGEND

=09if(argc > 1)
=09=09usage();
=09if(argc =3D=3D 1)
=09=09dir =3D argv[0];
=09else
=09=09dir =3D ".";
=09if((fd =3D open(dir, OREAD)) < 0)
=09=09sysfatal("open %s: %r", dir);

=09errors =3D nil;
=09n =3D dirreadall(fd, &d);
=09for(i=3D0; i<n; i++){
=09=09if(isbadutf(d[i].name)){
=09=09=09oldname =3D malloc(strlen(dir)+1+strlen(d[i].name)+1);
=09=09=09if(oldname =3D=3D nil)
=09=09=09=09sysfatal("out of memory");
=09=09=09strcpy(oldname, dir);
=09=09=09strcat(oldname, "/");
=09=09=09strcat(oldname, d[i].name);
=09=09=09newname =3D mkgoodname(d[i].name);
=09=09=09if(newname =3D=3D nil)
=09=09=09=09continue;
=09=09=09nulldir(&xd);
=09=09=09xd.name =3D newname;
=09=09=09print("%s =3D> %s\n", oldname, newname);
=09=09=09if(dirwstat(oldname, &xd) < 0){
=09=09=09=09fprint(2, "rename %s to %s: %r\n", oldname, newname);
=09=09=09=09errors =3D "errors";
=09=09=09}
=09=09=09free(oldname);
=09=09=09free(newname);
=09=09}
=09}
=09exits(errors);
}