From: "Jack Johnson" <knapjack@gmail.com>
To: "Fans of the OS Plan 9 from Bell Labs" <9fans@cse.psu.edu>
Subject: Re: [9fans] awk, not utf aware...
Date: Mon, 3 Mar 2008 14:48:01 -0900 [thread overview]
Message-ID: <6e35c0620803031548v583c051flb4d9ace76e220998@mail.gmail.com> (raw)
In-Reply-To: <6e3f8d391c4219c9b0acb323d74603d6@quanstro.net>
On Thu, Feb 28, 2008 at 6:10 AM, erik quanstrom <quanstro@quanstro.net> wrote:
> perhaps it would be more effective to break down the concept
> a bit. instead of a general locale hammer, why not expose some
> operations that could go into a locale? for example, have a base-
> character folding switch that allows regexps to fold codpoints into
> base codepoints so that íïìîi -> i. this information is in the unicode
> tables. perhaps the language-dependent character mapping should
> be specified explictly. &c.
Loosely-related tangent:
http://www.mail-archive.com/rsync@lists.samba.org/msg20395.html
> On the LINUX machines running utf-8 the ä is coded as $C3A4 which is
> in utf-8 equal to the character E4. The ä occupies in that way 2 bytes.
>
> I was very astonished, when I copied a mac-filename, pasted into a
> texteditor and looked at the file:
>
> In the mac-filename the letter ä is coded as: $61CC88, which in utf-8
> means the letter "a" followed by a $0308. (Combining diacritical marks)
> So the Mac combines the letter a with the two points above it instead
> using the E4 letter
> Now the things are clear: The filenames are different, in spite of
> looking equally.
So, if folding codepoints is a reasonable tactic, how many
representations do you need to fold? How many binary representations
are needed to fold íïìîi -> i?
-Jack
next prev parent reply other threads:[~2008-03-03 23:48 UTC|newest]
Thread overview: 25+ messages / expand[flat|nested] mbox.gz Atom feed top
2008-02-26 12:18 Gorka Guardiola
2008-02-26 13:16 ` Martin Neubauer
2008-02-26 14:54 ` Gorka Guardiola
2008-02-26 20:24 ` erik quanstrom
2008-02-26 21:08 ` geoff
2008-02-26 21:21 ` Pietro Gagliardi
2008-02-26 21:24 ` erik quanstrom
2008-02-26 21:32 ` Steven Vormwald
2008-02-26 21:40 ` Pietro Gagliardi
2008-02-26 21:42 ` Pietro Gagliardi
2008-02-26 23:59 ` Steven Vormwald
2008-02-27 2:38 ` Joel C. Salomon
2008-02-29 17:00 ` Douglas A. Gwyn
2008-02-26 21:34 ` erik quanstrom
2008-02-27 7:36 ` Gorka Guardiola
2008-02-27 15:54 ` Sape Mullender
2008-02-27 20:01 ` Uriel
2008-02-28 19:06 ` [9fans] localization, unicode, regexps (was: awk, not utf aware...) Tristan Plumb
2008-02-28 15:10 ` [9fans] awk, not utf aware erik quanstrom
2008-03-03 23:48 ` Jack Johnson [this message]
2008-03-04 0:13 ` erik quanstrom
2008-02-27 9:57 erik quanstrom
2008-02-28 18:54 Aharon Robbins
2008-02-28 21:48 ` Uriel
2008-02-28 22:08 ` erik quanstrom
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=6e35c0620803031548v583c051flb4d9ace76e220998@mail.gmail.com \
--to=knapjack@gmail.com \
--cc=9fans@cse.psu.edu \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).