[9fans] fmt and unicode text

[9fans] fmt and unicode text

[mirtchov]

This diff makes 'fmt' understand utf text without cutting it short:

	home% diff fmt.c /sys/src/cmd/fmt.c
	197c197
	< 			col += utflen(w[i]->text);
	---
	> 			col += strlen(w[i]->text);
	203c203
	< 			if(col+nsp+utflen(w[i]->text) > extraindent+length)
	---
	> 			if(col+nsp+strlen(w[i]->text) > extraindent+length)
	home% 


here's an example:

This is utf text formatted with the stock fmt:

  Зарегистрируйтесь сейчас на Десятую
  Международную Конференцию по Unicode,
  которая состоится 10-12 марта 1997 года в
  Майнце в Германии.  

This is utf text formatted with the new fmt:

  Зарегистрируйтесь сейчас на Десятую Международную Конференцию по
  Unicode, которая состоится 10-12 марта 1997 года в Майнце в
  Германии.  

Note that the russian text above appears to be formatted longer than
this english text.  That is due to the default cyrillic fonts being
used in acme rendered slightly larger, even the fixed-size ones.  If
you use something completely fixed-size such as
/lib/font/bit/10646/7x13/7x13.font you'll find that the texts are of
proper length :)

cheers, andrey

ps: the patch(1) system isn't operational still..
  
pps: a similar change may be required for cb(1), but since most code
is written in plain english I doubt it's that necessary.

Re: [9fans] fmt and unicode text

[Russ Cox]

> ps: the patch(1) system isn't operational still..

i don't know what to do about that.
i don't want to get rid of the write
group on sources.  the write group lets me not
worry too much about getting the permissions right
since the fs is r/o for the vast majority of users.
i could change the logs to be 444 and override their
modes in the proto file, but there might still be
775 directories or just plain mistakes.  the write
group makes those mistakes not costly.

since users and groups are per-fossil and not per-file system,
i'd have to set up another server (either on the same machine
running on an alternate port/ip, or on an alternate machine
entirely).  but then i'd have to worry about adding new users
to both the sources /adm/users and the patches /adm/users.
the addition of new users is already clumsy enough without
worrying about doing it in two places.

so instead i'm doing nothing and waiting for a better idea
to come along.  probably what will happen is enough time will
go by that i come to terms with the fact that:

	- patches must be another, small, file server
	- we need a programmatic way to issue
	  console commands and get responses

and i will implement that programmatic way, set up the
small file server on an alternate ip address running on
the same machine that now hosts sources, and we'll be
happy.

but all that is a lot more work than it was to write
the patch(1) system to start with, and i have not gotten
to it.

what little time i have to work on plan 9 i've been
spending trying to make venti faster and fossil better.

russ

Re: [9fans] fmt and unicode text

[mirtchov]

how about creating yet another group on sources?  call it 'patch' and
add the 10-15 people who more or less regularly submit patches to it.
this creates a hierarchy similar to the BSD's, where the Bell-Labs
people are similar to the 'core' team -- they decide what stays and
what doesn't; people added to 'patch' are similar to the ones with cvs
commit bit for parts of the tree, and everybody else is nobody.

if a non-patch member wants to submit a patch they can contact one of
the members in the group to do it for them.  if that becomes frequent
enough this person gets added to it.

it lifts the burdain of having to monitor many 9fans and 9trouble for
you guys, allows us to see what goes in and why stuff is rejected,
keeps a trail of external submissions and keeps this list cleaner of
code and free for offtopic discussions :)

andrey

Re: [9fans] fmt and unicode text

[Russ Cox]

> how about creating yet another group on sources?  call it 'patch' and
> add the 10-15 people who more or less regularly submit patches to it.
> this creates a hierarchy similar to the BSD's, where the Bell-Labs
> people are similar to the 'core' team -- they decide what stays and
> what doesn't; people added to 'patch' are similar to the ones with cvs
> commit bit for parts of the tree, and everybody else is nobody.
>
> if a non-patch member wants to submit a patch they can contact one of
> the members in the group to do it for them.  if that becomes frequent
> enough this person gets added to it.
>
> it lifts the burdain of having to monitor many 9fans and 9trouble for
> you guys, allows us to see what goes in and why stuff is rejected,
> keeps a trail of external submissions and keeps this list cleaner of
> code and free for offtopic discussions :)

I'd rather have a way that can (continue to) treat everyone equally.
Anyone who wants to submit a patch should be able to do it themselves,
without having to find someone to introduce it for them.  What you
suggest sounds reasonable at first glance but might be enough extra
work that it would discourage newcomers from bothering.  And newcomers
bearing patches are always a pleasant surprise, one that I'd prefer
not to discourage.  What you suggest introduces a new level of politics
that I'd like to avoid.

The main advantage of patch to us is that we don't have to edit
the source files manually to take them.  What I said in the last
message will probably happen eventually, perhaps in December.

Re: [9fans] fmt and unicode text

[Lyndon Nerenberg]

--On Sunday, November 16, 2003 5:22 PM -0500 Russ Cox <rsc@swtch.com>
wrote:

> I'd rather have a way that can (continue to) treat everyone equally.
> Anyone who wants to submit a patch should be able to do it themselves,
> without having to find someone to introduce it for them.  What you
> suggest sounds reasonable at first glance but might be enough extra
> work that it would discourage newcomers from bothering.  And newcomers
> bearing patches are always a pleasant surprise, one that I'd prefer
> not to discourage.

How about an email address (9fans-patches@xxx ?) that just drops
incoming messages into an IMAP mailbox that anyone can access
anonymously? That way everyone can see the patches, test them out, etc.

--lyndon

Re: [9fans] fmt and unicode text

[Russ Cox]

> > I'd rather have a way that can (continue to) treat everyone equally.
> > Anyone who wants to submit a patch should be able to do it themselves,
> > without having to find someone to introduce it for them.  What you
> > suggest sounds reasonable at first glance but might be enough extra
> > work that it would discourage newcomers from bothering.  And newcomers
> > bearing patches are always a pleasant surprise, one that I'd prefer
> > not to discourage.
>
> How about an email address (9fans-patches@xxx ?) that just drops
> incoming messages into an IMAP mailbox that anyone can access
> anonymously? That way everyone can see the patches, test them out, etc.

that exists.  9fans@cse.psu.edu.  (for anonymous access, see google
or gmane.)

the reason i like the patch(1) system is that using the file server
makes it easy to put enough context there that patches can be applied
easily.  sure i could set it up to bundle up the various bits and
mail them somewhere, and write other tools to walk the mime attachments
and pull out the parts, but that's just using a mail system to do a
file system's job.

eventually i'll fix the problem correctly.
until then, i'd rather live without a solution than spend time building
a solution that solves the problem poorly.

Re: [9fans] fmt and unicode text

[David Presotto]

[-- Attachment #1: Type: text/plain, Size: 146 bytes --]

Odd, of the two examples, the old fmt looks reasonable on my screen
and the other has odd long and unbalanced line.  What is the
advantage here?

[-- Attachment #2: Type: message/rfc822, Size: 3992 bytes --]

From: mirtchov@cpsc.ucalgary.ca
To: 9fans@cse.psu.edu
Subject: [9fans] fmt and unicode text
Date: Sun, 16 Nov 2003 12:29:24 -0700
Message-ID: <6a245736ff4fa6f1dcf7d9b2303482bd@plan9.ucalgary.ca>

This diff makes 'fmt' understand utf text without cutting it short:

	home% diff fmt.c /sys/src/cmd/fmt.c
	197c197
	< 			col += utflen(w[i]->text);
	---
	> 			col += strlen(w[i]->text);
	203c203
	< 			if(col+nsp+utflen(w[i]->text) > extraindent+length)
	---
	> 			if(col+nsp+strlen(w[i]->text) > extraindent+length)
	home% 


here's an example:

This is utf text formatted with the stock fmt:

  Зарегистрируйтесь сейчас на Десятую
  Международную Конференцию по Unicode,
  которая состоится 10-12 марта 1997 года в
  Майнце в Германии.  

This is utf text formatted with the new fmt:

  Зарегистрируйтесь сейчас на Десятую Международную Конференцию по
  Unicode, которая состоится 10-12 марта 1997 года в Майнце в
  Германии.  

Note that the russian text above appears to be formatted longer than
this english text.  That is due to the default cyrillic fonts being
used in acme rendered slightly larger, even the fixed-size ones.  If
you use something completely fixed-size such as
/lib/font/bit/10646/7x13/7x13.font you'll find that the texts are of
proper length :)

cheers, andrey

ps: the patch(1) system isn't operational still..
  
pps: a similar change may be required for cb(1), but since most code
is written in plain english I doubt it's that necessary.

Re: [9fans] fmt and unicode text

[David Presotto]

I'll make the change; its clearly correct.  I'm clearly limited by
the cyrillic in my font being way too wide.

Re: [9fans] fmt and unicode text

[mirtchov]

> I'll make the change; its clearly correct.  I'm clearly limited by
> the cyrillic in my font being way too wide.

oops, this arrived after I had posted my prolonged explanation, sorry
for the noise...

Re: [9fans] fmt and unicode text

[mirtchov]

> Odd, of the two examples, the old fmt looks reasonable on my screen
> and the other has odd long and unbalanced line.  What is the
> advantage here?

it is long and unbalanced because the font it's rendered in has
cyrillic letters that are larger than the english ones.  it's bad even
in the default acme fixed-size font.

it looks bad because i used a fairly small sample of text.  the
problem with the older fmt was treating 2- and more-byted utf
character as two 1-byte ones for the purpose of finding where to put
the EOL.  A 36-letter text in russian will thus be broken into two
lines, even though it's way below the 70-char line margin.

if you want to see how it looks with a proper fixed-size font try
/lib/font/bit/10646/7x13/7x13.font:

	http://pages.cpsc.ucalgary.ca/~mirtchov/screenshots/scr-fmt.jpg

andrey