From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.4 (2020-01-24) on inbox.vuxu.org X-Spam-Level: X-Spam-Status: No, score=-3.3 required=5.0 tests=DKIM_SIGNED,DKIM_VALID, DKIM_VALID_AU,DKIM_VALID_EF,HEADER_FROM_DIFFERENT_DOMAINS, MAILING_LIST_MULTI,RCVD_IN_DNSWL_MED autolearn=ham autolearn_force=no version=3.4.4 Received: from zero.zsh.org (zero.zsh.org [IPv6:2a02:898:31:0:48:4558:7a:7368]) by inbox.vuxu.org (Postfix) with ESMTP id C68EF2203A for ; Mon, 13 May 2024 11:51:59 +0200 (CEST) ARC-Seal: i=1; cv=none; a=rsa-sha256; d=zsh.org; s=rsa-20210803; t=1715593919; b=Xsz7DFGeSb3VhVbOAWap3Xa2lJu0NwRSZil2YNn5uNKjnQeAs9N6MzskDoD8geDFHWCi0jn6Mw yoHBDeJAZ0eZnQCWVGQ/uq9TkVNrZu5jM8AZl/hiaOzda7LbHyNv8335Zt5k86IeLG4i8hyoxZ 6DRXjVkzM5YZk4NoGnz8kEb4PNYoftm0NJZwdmwZ0EdhxR5PuaZpxkjDYAa3gOKiUb8t8Bm339 mq1yb4FgnmrMWNofyeOpa8alm8zUpCQc1OuLPrAD3QIzk9uXjtthsekfZabSNTS8nEmsqgpEFD FLtFPwCyapezRhFVZ3oz4ess7uNbGbUECqbLzgDkf7Emew==; ARC-Authentication-Results: i=1; zsh.org; iprev=pass (csmtpq1-prd-nl1-vmo.edge.unified.services) smtp.remote-ip=84.116.50.35; dkim=pass header.d=ntlworld.com header.s=meg.feb2017 header.a=rsa-sha256; dmarc=pass header.from=ntlworld.com; arc=none ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed; d=zsh.org; s=rsa-20210803; t=1715593919; bh=4Lo3gESxX0YsWyeAjTJH4JLpvYwEKGWCBXN3yYrUXVk=; h=List-Archive:List-Owner:List-Post:List-Unsubscribe:List-Subscribe:List-Help: List-Id:Sender:Content-Transfer-Encoding:Content-Type:MIME-Version:Subject: References:In-Reply-To:Message-ID:To:From:Date:DKIM-Signature: DKIM-Signature; b=FwJcGndKpcdtcv5tbzH+JR6W/XINo5g+xgnyAnGzDcGlMzm7akPj1nOjBo8vlV9F60lk3azSmy jXIQnWtngh9Opo52IRKjnfC+zaXF8hKACaTtdxTUUXqbZx6V2jzIlag+fuTOhCaJ1g2cNiZoBX fde7hxsiAd1DFJYm3WNqsjbQrMRL16MQvM9VaoXW3AgmQ2JeiGXlj0eyk17FBDGc7T/34F2e4B uzExQygpyXbBxdFFyyLg8R9LzceQD+pUhR5jipCaddYTIh/x699+p2nqBkH842vpAEqBu8bA7r tk1P8gW5UbQ6l1mHtvkbmOmKgrAZmon3Z+Oh+EKNSa2ddQ==; DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=zsh.org; s=rsa-20210803; h=List-Archive:List-Owner:List-Post:List-Unsubscribe: List-Subscribe:List-Help:List-Id:Sender:Content-Transfer-Encoding: Content-Type:MIME-Version:Subject:References:In-Reply-To:Message-ID:To:From: Date:Reply-To:Cc:Content-ID:Content-Description:Resent-Date:Resent-From: Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID; bh=lpV2l1iCunrx6kaa4XnjFDmgox7xDkwzDB/e+Ho9fNw=; b=DQM8FQHJlSU0lzynr4AQjx+O/9 CF3TD7lWC4AStMMKO14OafokCOTu4/pNF4kWdX/PTxyC7ZY+akfmZUrt/s6xCHklOPaK4YiIKOlZA MIgSYnsUF05u5qyMfcFkUXzV8RybI60iOdBoWfdjcSwXMfa2Areec+K7AU4YDy07UHtptKGNArz/K Gl8zJqa9PwvU8R88nTPWo586+nMYZH4C+ZEIqA77irqVrfLpU2iBOvbU5FqeqBiIfS+IsxxZKcsRj hVEyYynhVYb3moXRjejjlOCP9V2O3IUBw22/WbRktz6cU5htsTUhPy89vtQFWmuSJF9t3cX4tM3bE e3o2vWyQ==; Received: by zero.zsh.org with local id 1s6SLe-0002C4-2K; Mon, 13 May 2024 09:51:58 +0000 Authentication-Results: zsh.org; iprev=pass (csmtpq1-prd-nl1-vmo.edge.unified.services) smtp.remote-ip=84.116.50.35; dkim=pass header.d=ntlworld.com header.s=meg.feb2017 header.a=rsa-sha256; dmarc=pass header.from=ntlworld.com; arc=none Received: from csmtpq1-prd-nl1-vmo.edge.unified.services ([84.116.50.35]:25931) by zero.zsh.org with esmtps (TLS1.3:TLS_AES_256_GCM_SHA384:256) id 1s6SL1-0001rP-9J; Mon, 13 May 2024 09:51:21 +0000 Received: from oxsmtp2-prd-nl1-vmo.nl1.unified.services ([100.107.83.137]) by csmtpq1-prd-nl1-vmo.edge.unified.services with esmtp (Exim 4.93) (envelope-from ) id 1s6SL0-00Gd8a-31 for zsh-workers@zsh.org; Mon, 13 May 2024 11:51:18 +0200 Received: from oxbe3-prd-nl1-vmo.nl1.unified.services ([100.107.83.146]) by oxsmtp2-prd-nl1-vmo.nl1.unified.services with ESMTP id 6SL0spcR1ep6K6SL0sNXRC; Mon, 13 May 2024 11:51:18 +0200 X-Env-Mailfrom: p.w.stephenson@ntlworld.com X-Env-Rcptto: zsh-workers@zsh.org X-SourceIP: 100.107.83.146 X-CNFS-Analysis: v=2.4 cv=N+ziFH9B c=1 sm=1 tr=0 ts=6641e296 cx=a_exe a=+uVq5v2AViepsjIBXWI4Ew==:117 a=IvlwODmuRu4A:10 a=IkcTkHD0fZMA:10 a=NEAV23lmAAAA:8 a=AnVPfWlyJOROUXD2B3AA:9 a=QEXdDO2ut3YA:10 a=HbQOABYz3jhqdZF7JfnK:22 X-Authenticated-Sender: p.w.stephenson@ntlworld.com DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ntlworld.com; s=meg.feb2017; t=1715593878; bh=4Lo3gESxX0YsWyeAjTJH4JLpvYwEKGWCBXN3yYrUXVk=; h=Date:From:To:In-Reply-To:References:Subject; b=i3fOD7sgW8KqRwLtuPYG85aLHYayKGhiZG+4poriZ7w2tzCswNEB3riE8MLnhIOYR Ogy+BIjC/Rxj+UTzQEsNDdv9KYj3B+nU8DXbvticY38XYw5lA6MTxL5z6LhLtrffBu KxvUW9XlAYj3dOt1cPre7uRwgKN7tnFk0roEpwn9VxPDzaTxn1Pw62iLbsj62R+gXn m43fsY2We4fMxB8cgvTlMsF3lNlkTV0HrMsCm2/c7Q8c1lSe3QRiT83EN4ADuWV/l+ Hyu5XH6Jdkd6l6eY+khFQbXNua9Y2WZjW+hk00QuNr6cd8TNvYGJwuhW1Lo8eg0VvC rZ0tY9iMnZV6w== Date: Mon, 13 May 2024 10:51:18 +0100 (BST) From: Peter Stephenson To: "Wu, Zhenyu" , zsh-workers@zsh.org Message-ID: <941766233.1602619.1715593878048@mail.virginmedia.com> In-Reply-To: References: Subject: Re: Bug: Unicode character of zsh module will become weird characters MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: quoted-printable X-Priority: 3 Importance: Normal X-Mailer: Open-Xchange Mailer X-Originating-IP: 147.161.224.196 X-Originating-Client: open-xchange-appsuite X-CMAE-Envelope: MS4xfCknqMzN2CKuiCENgFYtT6fI7xgxPWanPNI1m5GuvgUAWnGD8vBfbejpW03eIzQn2aQFiaxoLx1VY+0U4jNw3lHIhcHi0XNL4eUn7jSd7ED30TQ4CffQ tcJNEP1jqMy1J/H6p6dD4OPzvYuLWj3x814SKQTy5uJY62yHMI6Anu0Kd39PLohXamEM0DpMJ0xpBebKCtU14x9tFsi6OohOxjokSeqrkERz9z1+8gl0ZQB7 clcL6JUH/a1ZWx9tudv9v2Q/zx/nghhFVTjBSROm+7Q= X-Seq: 52932 Archived-At: X-Loop: zsh-workers@zsh.org Errors-To: zsh-workers-owner@zsh.org Precedence: list Precedence: bulk Sender: zsh-workers-request@zsh.org X-no-archive: yes List-Id: List-Help: , List-Subscribe: , List-Unsubscribe: , List-Post: List-Owner: List-Archive: > On 11/05/2024 14:40 BST Wu, Zhenyu wrote: > ```zsh > % git clone --depth=3D1 https://github.com/zsh-users/zsh > % cd zsh > % sed -i 's/strparam =3D ztrdup("example");/strparam =3D ztrdup("=E4=BD= =A0=E5=A5=BD");/' Src/Modules/example.c > % autoreconf -vif > % ./configure > % make > ``` I think The problem's actually here. Zsh has a special encoding of strings internally to avoid confusion between normal strings and tokenised strings. This is because special characters are mixed in with normal characters in strings inside the code. This system is known as "metafication". This means you have to be careful when creating strings to be stored inside the system. In fact, this is usually only a problem if strings contain byte 0x80, so it can be a bit of a surprise when you see it. If you're doing normal input and output this should work automatically, but you're updating the source code directly. Try changing the line in the example file to: strparam =3D metafy("=E4=BD=A0=E5=A5=BD", strlen("=E4=BD=A0=E5=A5=BD"),= META_DUP); and see if that works --- looks OK to me with a quick test. cheers pws