From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: (qmail 17906 invoked by alias); 1 Jun 2014 19:53:41 -0000 Mailing-List: contact zsh-workers-help@zsh.org; run by ezmlm Precedence: bulk X-No-Archive: yes List-Id: Zsh Workers List List-Post: List-Help: X-Seq: 32656 Received: (qmail 10649 invoked from network); 1 Jun 2014 19:53:29 -0000 X-Spam-Checker-Version: SpamAssassin 3.3.2 (2011-06-06) on f.primenet.com.au X-Spam-Level: X-Spam-Status: No, score=-1.9 required=5.0 tests=BAYES_00,RCVD_IN_DNSWL_NONE autolearn=ham version=3.3.2 From: Bart Schaefer Message-id: <140601125323.ZM9969@torch.brasslantern.com> Date: Sun, 01 Jun 2014 12:53:23 -0700 In-reply-to: Comments: In reply to "Jun T." "Re: Unicode, Korean, normalization form, Mac OS X and tab completion" (Jun 2, 2:00am) References: <20140531201617.4ca60ab8@pws-pc.ntlworld.com> <140531142926.ZM556@torch.brasslantern.com> <20140601022527.GD1820@tarsus.local2> <140601005624.ZM3283@torch.brasslantern.com> X-Mailer: OpenZMail Classic (0.9.2 24April2005) To: zsh-workers@zsh.org Subject: Re: Unicode, Korean, normalization form, Mac OS X and tab completion MIME-version: 1.0 Content-type: text/plain; charset=us-ascii On Jun 2, 2:00am, Jun T. wrote: } Subject: Re: Unicode, Korean, normalization form, Mac OS X and tab complet } } There is a patch by a Japanese user which simply converts } file names obtained by readder() into the composed form ("NFC") I tried this and found a couple of problems, mainly that iconv() can return >= 0 but do nothing if the input has no convertible characters. Here is my suggested correction, also eliminates unnecessary temp_name: diff --git a/Src/utils.c b/Src/utils.c index 9439227..8b512bb 100644 --- a/Src/utils.c +++ b/Src/utils.c @@ -4270,6 +4270,12 @@ mod_export char * zreaddir(DIR *dir, int ignoredots) { struct dirent *de; +#if defined(HAVE_ICONV) && defined(__APPLE__) + static iconv_t conv_ds = (iconv_t)0; + static char *conv_name = 0; + char *conv_name_ptr, *orig_name_ptr; + size_t conv_name_len, orig_name_len; +#endif do { de = readdir(dir); @@ -4278,6 +4284,31 @@ zreaddir(DIR *dir, int ignoredots) } while(ignoredots && de->d_name[0] == '.' && (!de->d_name[1] || (de->d_name[1] == '.' && !de->d_name[2]))); +#if defined(HAVE_ICONV) && defined(__APPLE__) + if (!conv_ds) + conv_ds = iconv_open("UTF-8", "UTF-8-MAC"); + if (conv_ds) { + /* Force initial state in case re-using conv_ds */ + (void) iconv(conv_ds, 0, &orig_name_len, 0, &conv_name_len); + + orig_name_ptr = de->d_name; + orig_name_len = strlen(de->d_name); + conv_name = zrealloc(conv_name, orig_name_len+1); + conv_name_ptr = conv_name; + conv_name_len = orig_name_len; + if (iconv(conv_ds, + &orig_name_ptr, &orig_name_len, + &conv_name_ptr, &conv_name_len) >= 0) { + if (orig_name_len == 0) { + /* Completely converted, metafy and return */ + *conv_name_ptr = '\0'; + return metafy(conv_name, -1, META_STATIC); + } + } + /* Error, or conversion incomplete, keep the original name */ + } +#endif + return metafy(de->d_name, -1, META_STATIC); }