From mboxrd@z Thu Jan  1 00:00:00 1970
Received: from sigma.offblast.org ([199.191.58.44]) by ttr; Sun Feb 16 09:46:26 EST 2014
Received: from iota.offblast.org ([66.109.99.211]) by sigma; Sun Feb 16 09:46:23 EST 2014
Date: Sun, 16 Feb 2014 06:46:18 -0800
From: Nick Owens <mischief@9.offblast.org>
To: 9front@9front.org
Subject: Re: [9front] mothra not respecting charset=utf-8
Message-ID: <20140216144618.GE25422@iota.offblast.org>
List-ID: <9front.9front.org>
X-Glyph: ➈
X-Bullshit: agile database SSL storage API
References: <1392547636.22281.83982753.3496171A@webmail.messagingengine.com>
 <1392549916.30058.83991793.135CF34A@webmail.messagingengine.com>
MIME-Version: 1.0
Content-Type: multipart/signed; micalg=pgp-sha1;
	protocol="application/pgp-signature"; boundary="R6sEYoIZpp9JErk7"
Content-Disposition: inline
In-Reply-To: <1392549916.30058.83991793.135CF34A@webmail.messagingengine.com>
User-Agent: Mutt/1.5.21 (2010-09-15)


--R6sEYoIZpp9JErk7
Content-Type: text/plain; charset=us-ascii
Content-Disposition: inline
Content-Transfer-Encoding: quoted-printable

On Sun, Feb 16, 2014 at 11:25:16AM +0000, Ethan Grammatikidis wrote:
> mischief reports (and i've confirmed) the following patch works. i'm
> almost scared to ask why.
>=20
> diff -r 709e18f21cad sys/src/cmd/mothra/rdhtml.c
> --- a/sys/src/cmd/mothra/rdhtml.c       Sat Feb 15 17:18:58 2014 -0500
> +++ b/sys/src/cmd/mothra/rdhtml.c       Sat Feb 15 03:13:05 2014 -0800
> @@ -166,7 +166,7 @@
>  		g->hbufp=3Dg->hbuf;
>  		g->ehbuf=3Dg->hbuf+n;
>  	}
> -       c=3D*g->hbufp++&255;
> +       c=3D*g->hbufp++;
>  	if(c=3D=3D'\n') g->lineno++;
>  	return c;
>  }
>=20

i'm not sure why i said that was a good idea.
maybe this one isn't either.

here we actually read full runes from the fd.

this seems to fix visiting
https://www.cl.cam.ac.uk/~mgk25/ucs/examples/UTF-8-demo.txt
in mothra.


diff -r 709e18f21cad sys/src/cmd/mothra/rdhtml.c
--- a/sys/src/cmd/mothra/rdhtml.c	Sat Feb 15 17:18:58 2014 -0500
+++ b/sys/src/cmd/mothra/rdhtml.c	Sat Feb 15 04:10:34 2014 -0800
@@ -154,6 +154,7 @@
 	int n, c;
 	char err[1024];
 	if(g->hbufp=3D=3Dg->ehbuf){
+doread:
 		n=3Dread(g->hfd, g->hbuf, NHBUF);
 		if(n<=3D0){
 			if(n<0){
@@ -166,7 +167,11 @@
 		g->hbufp=3Dg->hbuf;
 		g->ehbuf=3Dg->hbuf+n;
 	}
-	c=3D*g->hbufp++&255;
+	if(!fullrune(g->hbufp, g->ehbuf - g->hbufp)) {
+		goto doread;
+	}
+
+	g->hbufp +=3D chartorune((Rune*)&c, g->hbufp);
 	if(c=3D=3D'\n') g->lineno++;
 	return c;
 }


--R6sEYoIZpp9JErk7
Content-Type: application/pgp-signature

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.13 (OpenBSD)

iQIcBAEBAgAGBQJTAM86AAoJEAlFY6k1Rs31gSsQAJcV7ZpgbQTlbeVtBRTLc40J
V/6kgmj8VqHTb2S1DCQUrDwtgWlVMv3Wona8dXyLMLbxpOccI/IuSJf+o3uvNPKE
YLT4fRw9f4Mx20FN7nMmT/1vvmxiVg6wd++K3aSlwCxcX912hkjpd3IWZZO4xisK
PQqlMjhxVfQDi3hIjYXW/Ci1d+vvHR1WN8J4Pt/w1VCzE0gTLbvMfoFfavn5y4+g
ro/muUCYt+lwPLOxL/9onfdDROMfFj5PI/ym4TXS0vBmuoi1eIhMWnWYAOdKCkVg
AnYw7X6irziFqptpbCXn7AhoXvNFRryfYmipUdrTo1pSptVRWjHCktI3nabx5Gir
OWXeOce7VUskRdJbXlLa7rA8+BXoyoZiJxZKpWsssUJmQn8/fXuZbqIuZbmXNsD1
gnajwDkn5l1cx/T6/HyMY9gb8PYuyjijVW9zmz1OQwH+6mCnyHDbMs1D5qN1SXlT
0efMpZW5fUlcrC6weMPIRqJhB3YX/02W3Kgcu/rGdEiCILrG2w9oGvOpqU1LU7K3
q+r3gaRVWWLPpOOeKmNuzdDE36LD4fwyRfuTjDV64RFYI0d+Z9usD+AwbLts/0rv
l/Zdyw4HCsVFh+p/xeaM7NK9nogmxb21lU3HY3aaGZnUc9PrUiaQ/sstd++CR8Xh
qbR03oEmeCEoYRuuWzh1
=Jydg
-----END PGP SIGNATURE-----

--R6sEYoIZpp9JErk7--