From mboxrd@z Thu Jan 1 00:00:00 1970 X-Msuck: nntp://news.gmane.io/gmane.text.pandoc/21656 Path: news.gmane.org!.POSTED!not-for-mail From: John MacFarlane Newsgroups: gmane.text.pandoc Subject: Re: Pandoc + TWiki (writer) output Date: Tue, 11 Dec 2018 11:04:57 -0800 Message-ID: References: <976a9019-374b-4cc9-b9cf-c2e98e04fed9@googlegroups.com> <2ba0d474-f0db-456e-922a-bb7282e60bf5@googlegroups.com> <5bffbda3-8281-4247-99c2-3826011ec9ca@googlegroups.com> <8631e4f1-8744-40b4-a3ce-a8d8e1bb0202@googlegroups.com> Reply-To: pandoc-discuss-/JYPxA39Uh5TLH3MbocFFw@public.gmane.org NNTP-Posting-Host: blaine.gmane.org Mime-Version: 1.0 Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Trace: blaine.gmane.org 1544554986 7530 195.159.176.226 (11 Dec 2018 19:03:06 GMT) X-Complaints-To: usenet@blaine.gmane.org NNTP-Posting-Date: Tue, 11 Dec 2018 19:03:06 +0000 (UTC) To: Joey Dumont , pandoc-discuss Original-X-From: pandoc-discuss+bncBCJZJHG45QDBBZUUYDQAKGQEUM4JUYI-/JYPxA39Uh5TLH3MbocFFw@public.gmane.org Tue Dec 11 20:03:02 2018 Return-path: Envelope-to: gtp-pandoc-discuss@m.gmane.org Original-Received: from mail-it1-f185.google.com ([209.85.166.185]) by blaine.gmane.org with esmtp (Exim 4.84_2) (envelope-from ) id 1gWnJ6-0001p1-E3 for gtp-pandoc-discuss@m.gmane.org; Tue, 11 Dec 2018 20:03:00 +0100 Original-Received: by mail-it1-f185.google.com with SMTP id g7sf3313068itg.7 for ; Tue, 11 Dec 2018 11:05:11 -0800 (PST) ARC-Seal: i=2; a=rsa-sha256; t=1544555111; cv=pass; d=google.com; s=arc-20160816; b=VHcvps/ZS4U/zrHD7R2Ehs7ypn7yQvgVpYmxrN+8RZbtCq8no32vZfoq0zuBgjAJXq sBU52fiHtMncSA8SuBd/Z+reT2LNWeJwHViBi3bSw9dvEzCINwbU34FQ/ZL0vt6iYfLD loFkVy6Oo2dlO22KopEQQoUGQUwqxRkKmRCQi5h2bZsQwzO/k4rtfjovpcgQg3LwkDgt h0hAjyApA4Q6BNSbE5ekyDDDXRL5jedsvfBYoWg/02rsU7JEQczVpJMkeaHK/Vi50Nah SqJc/TYx12RYop1J3oyGx2x0Nnp7DYm8hEu9Il1vyiE9OExx1rmbTuWtZuxS+qtztGCp kbPA== ARC-Message-Signature: i=2; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-unsubscribe:list-subscribe:list-archive:list-help:list-post :list-id:mailing-list:precedence:reply-to:content-transfer-encoding :mime-version:message-id:date:references:in-reply-to:subject:to:from :sender:dkim-signature; bh=IrJJZl9NdaplQwk4OdyxhRWhVadBWMxT44MsWJzrDIY=; b=BGi58/maKzStUeNPwvbuf8WeFLfpCzBB97LldAbjX08IuoNE04bPtJE71A/OxRo9lm 9xhqZRHqMasRE0D2t2N17LTWsyp/+J0RTaVxwbNEiU2MCODbd7A4FHYMYNcd618cptFs /hy3RC0AgRhPRu5i+TDFZUiOTzHr2/8Kgy1yk/McJtu97kYUaA5JODU3hboWM/I0BS/Y Ea/JZ5I4evzLfMj4sPz0qWjk7lPt9p1ViuNkXrYWITqqqDR2CkJHpqcIGR9x5pXw9T4d +3HFbENhT1p5HW1MztL6zWiP4UVgYO4AEscwemIeWbQPD0NuCyHKY/1O7EBFS+Nbqm4B 09PA== ARC-Authentication-Results: i=2; gmr-mx.google.com; dkim=pass header.i=@berkeley-edu.20150623.gappssmtp.com header.s=20150623 header.b=M+48dvbi; spf=pass (google.com: domain of jgm-TVLZxgkOlNX2fBVCVOL8/A@public.gmane.org designates 2607:f8b0:4864:20::533 as permitted sender) smtp.mailfrom=jgm-TVLZxgkOlNX2fBVCVOL8/A@public.gmane.org DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=googlegroups.com; s=20161025; h=sender:from:to:subject:in-reply-to:references:date:message-id :mime-version:content-transfer-encoding:x-original-sender :x-original-authentication-results:reply-to:precedence:mailing-list :list-id:list-post:list-help:list-archive:list-subscribe :list-unsubscribe; bh=IrJJZl9NdaplQwk4OdyxhRWhVadBWMxT44MsWJzrDIY=; b=tcaCke0umNcKQW5KQa7DvsgoC652L4loEyX8mHy5WL0X6dmD4fMB0e68UPaZrl/VZu QbBkwMB+qjcSNtxILV8nfZNqQ60CPqi7ctTFXEgF7wTTpG9PfAq6LWtI1u2ICL8sk3XL 3Mpb18mVyx0+0IGLo2nZXVry51dBYDcequuP/lFvxmTaKkSsRstY5mqGwvU0Vpxkzryi F3DZxvkeCsKrsK1GoKOwNJTRxav2bh45l8eavWmQrs5w16FWftgyh9vhhWST046eSd+r NfAKUmqKczn5rtm0JN/8Yd5XWKU8t7jKWuB/LzNkTSQbKXsdVhTmSxl7hqK7hxupENqZ 7sUQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=sender:x-gm-message-state:from:to:subject:in-reply-to:references :date:message-id:mime-version:content-transfer-encoding :x-original-sender:x-original-authentication-results:reply-to :precedence:mailing-list:list-id:x-spam-checked-in-group:list-post :list-help:list-archive:list-subscribe:list-unsubscribe; bh=IrJJZl9NdaplQwk4OdyxhRWhVadBWMxT44MsWJzrDIY=; b=cPNOJzrOgYPRnYSRiXyVDq4WwQfcavmb9WRhfeqgVJQ368fkFl031VEqvVDT3k/EAi sjKto02wuirHS0fiVwAuuhJIcgNvpDtwHWoF5X3ZK4vgm9O/KYs3ejFpZk3NaWogq78y KZcl0AFuGr6HyBrDBkT8CerCL5/cwbVoGXt6gENgb2fSsor6IszN9PGmk2Qq+AlTz/0Z MkFRh/3YeVrimkYRQa56R4DR/dUcrMJThZVMenhcbwgafwl2onNLhDZlNcF1I7zP701O w3XlV1deyVQfoytgRmn2e3ROI64iRnx/+oLAV5WGZBe/EM+Sr2XI4k6syuG7TBQD07Vx Original-Sender: pandoc-discuss-/JYPxA39Uh5TLH3MbocFFw@public.gmane.org X-Gm-Message-State: AA+aEWYne02q52xD7Dw9JSoYLIRga3tDnSxJzymmGP534ssJNeyP1UZZ ug0ll8do/otFbkVEJegHvQo= X-Google-Smtp-Source: AFSGD/VCiAhwFct6kp40EvRjUXgiGYzCpokwZufgHz0kor5NVvsKJmco3jSK62JH5PEtmTS3KKddeQ== X-Received: by 2002:a02:3b21:: with SMTP id c33mr130968jaa.6.1544555111089; Tue, 11 Dec 2018 11:05:11 -0800 (PST) X-BeenThere: pandoc-discuss-/JYPxA39Uh5TLH3MbocFFw@public.gmane.org Original-Received: by 2002:a5d:8d0c:: with SMTP id p12ls4758592ioj.3.gmail; Tue, 11 Dec 2018 11:05:10 -0800 (PST) X-Received: by 2002:a6b:8e83:: with SMTP id q125mr13203177iod.9.1544555110218; Tue, 11 Dec 2018 11:05:10 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1544555110; cv=none; d=google.com; s=arc-20160816; b=dHYLgBioluX+swjycQ9Vj8EsDiH/6rUxw5Jqc7/pI+u0B1ZPEVUlhSw9LBNHdPrtNX kfTEkw0UIAg//AnrVECLuYFYd26WEnXl4xcXS61Sr2UqxhqOW7CI+uY8H8uPFHAfvQH+ NFdH/xJqTJewD6st/qi2WdHGC6kCaf9PTqCv/BTCBXizjIROjiSCTodwrh4ENJdQv1Hg G5z+/aE4NzfD7dRp3E2t/G44nTgQ4jXrzS93nyWhLMJMd41MtCNppdSCFgEYkaGtsdo4 QoOCcsv5ci5gUVzCx6ks2DZdDbj4I/mp9ICXBpidqCP2TGVqmfneJGuzJNWcGm0o/wWK 6f5w== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=content-transfer-encoding:mime-version:message-id:date:references :in-reply-to:subject:to:from:dkim-signature; bh=sMV0CEzOoXw6s7Tl+s6oXc5rbys75uxSSUD4KlQF05Y=; b=R9mrYTyAvCWhUKa606pbAkm9+OvMj5vCbxVVWAPFjvaj5Dk3AYGepGlFWJvAPFSpzU XBLWeVxzgZDjvnV2DzkHYaGYuCOtcyWIvlxloGwl4siWmYvYpCZ9/kEqxGRbmZ2vQ+pP r0Ne0X/4q6ctOofjQd/qDxHj4HjbKfThIIXHh6cQbYnicQNHnBpnFWHFOi28On5qEHss mxQZcBJaq7Uytkd1/el2Sh4mYgX/kQd3L+bLZcmYP2q+bV5Q5qeRijVVYuWcSrbD1usd zCf/aDmvYlwkWo3S5fTnzmW8HHO6NEmKkxXOi/zgLztPCWTh0u15VfRUpZ7r/QnoeXOI cDAQ== ARC-Authentication-Results: i=1; gmr-mx.google.com; dkim=pass header.i=@berkeley-edu.20150623.gappssmtp.com header.s=20150623 header.b=M+48dvbi; spf=pass (google.com: domain of jgm-TVLZxgkOlNX2fBVCVOL8/A@public.gmane.org designates 2607:f8b0:4864:20::533 as permitted sender) smtp.mailfrom=jgm-TVLZxgkOlNX2fBVCVOL8/A@public.gmane.org Original-Received: from mail-pg1-x533.google.com (mail-pg1-x533.google.com. [2607:f8b0:4864:20::533]) by gmr-mx.google.com with ESMTPS id v67si89161itb.0.2018.12.11.11.05.10 for (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Tue, 11 Dec 2018 11:05:10 -0800 (PST) Received-SPF: pass (google.com: domain of jgm-TVLZxgkOlNX2fBVCVOL8/A@public.gmane.org designates 2607:f8b0:4864:20::533 as permitted sender) client-ip=2607:f8b0:4864:20::533; Original-Received: by mail-pg1-x533.google.com with SMTP id 70so7062974pgh.8 for ; Tue, 11 Dec 2018 11:05:10 -0800 (PST) X-Received: by 2002:a63:8149:: with SMTP id t70mr15992237pgd.172.1544555109722; Tue, 11 Dec 2018 11:05:09 -0800 (PST) Original-Received: from johnmacfarlane.net (li55-134.members.linode.com. [74.82.3.134]) by smtp.gmail.com with ESMTPSA id b27sm21288086pfh.113.2018.12.11.11.05.08 (version=TLS1_2 cipher=ECDHE-RSA-CHACHA20-POLY1305 bits=256/256); Tue, 11 Dec 2018 11:05:08 -0800 (PST) Original-Received: by johnmacfarlane.net (Postfix, from userid 1000) id 9CEC8A15F; Tue, 11 Dec 2018 14:04:57 -0500 (EST) In-Reply-To: <8631e4f1-8744-40b4-a3ce-a8d8e1bb0202-/JYPxA39Uh5TLH3MbocFFw@public.gmane.org> X-Original-Sender: jgm-TVLZxgkOlNX2fBVCVOL8/A@public.gmane.org X-Original-Authentication-Results: gmr-mx.google.com; dkim=pass header.i=@berkeley-edu.20150623.gappssmtp.com header.s=20150623 header.b=M+48dvbi; spf=pass (google.com: domain of jgm-TVLZxgkOlNX2fBVCVOL8/A@public.gmane.org designates 2607:f8b0:4864:20::533 as permitted sender) smtp.mailfrom=jgm-TVLZxgkOlNX2fBVCVOL8/A@public.gmane.org Precedence: list Mailing-list: list pandoc-discuss-/JYPxA39Uh5TLH3MbocFFw@public.gmane.org; contact pandoc-discuss+owners-/JYPxA39Uh5TLH3MbocFFw@public.gmane.org List-ID: X-Spam-Checked-In-Group: pandoc-discuss-/JYPxA39Uh5TLH3MbocFFw@public.gmane.org X-Google-Group-Id: 1007024079513 List-Post: , List-Help: , List-Archive: , List-Unsubscribe: , Xref: news.gmane.org gmane.text.pandoc:21656 Archived-At: The problem is that lua's gsub is not really UTF-8 aware. [<>^\"'] will match a single BYTE, but the quote characters are multibyte. You can use the functions documented in sec 6.5 here: https://www.lua.org/manual/5.3/manual.html#6.5. These allow you to iterate over the characters in a string. for p, c in utf8.codes(s) do -- c is the code point (integer) of the character end You can use this to build up a new string, escaping each code point appropriately. Joey Dumont writes: > I tried that already. My escape function is probably bad then.=20 > > -- Character escaping > local function escape(s, in_attribute) > return s:gsub("[<>&\"']", > function(x) > if x =3D=3D '<' then > return '<' > elseif x =3D=3D '>' then > return '>' > elseif x =3D=3D '&' then > return '&' > elseif x =3D=3D '"' then > return '' > elseif x =3D=3D "'" then > return "'" > else > return x > end > end) > end > > Should I put something more specific in the x =3D=3D "'" and x =3D=3D '"'= fields? > > On Tuesday, December 11, 2018 at 1:49:13 PM UTC-5, John MacFarlane wrote: >> >> >> You can always change the 'escape' function in your=20 >> custom writer so it outputs ASCII entities instead of=20 >> UTF-8 characters.=20 >> >> Joey Dumont > writes:=20 >> >> > Oh.=20 >> >=20 >> > It might be my TWiki installation then. Nothing I can really do about= =20 >> that=20 >> > though.=20 >> >=20 >> > Thanks for the help!=20 >> >=20 >> > On Monday, December 10, 2018 at 2:49:58 PM UTC-5, John MacFarlane wrot= e:=20 >> >>=20 >> >>=20 >> >> This is an encoding issue. It looks like twiki is=20 >> >> generating the right UTF-8 encoded output, but your=20 >> >> browser isn't displaying it as UTF-8. You should=20 >> >> be able to fix this by ensuring that twiki produces=20 >> >> HTML containing a meta tag that sets the charset to=20 >> >> UTF-8.=20 >> >>=20 >> >> Joey Dumont > writes:=20 >> >>=20 >> >> > I'm having trouble with quotes and what I believe are thin spaces,= =20 >> >> however.=20 >> >> >=20 >> >> > For instance, writing "he'll test something" in Markdown, then=20 >> >> converting=20 >> >> > to TWiki yields an HTML curly quote instead of a plain ' quote. TWi= ki=20 >> >> then=20 >> >> > freaks and renders that as "he=C3=A2=E2=82=AC=E2=84=A2ll test somet= hing"*.*=20 >> >> >=20 >> >> > The string "Dec. 5th" seems to yield a thin space between the perio= d=20 >> and=20 >> >> > the decimal, but TWiki renders it as "Dec.=C3=82 5th"=20 >> >> >=20 >> >> > Not sure how to fix this. Tried escaping with escape(s) in multiple= =20 >> >> places,=20 >> >> > such as Str, Plain and Para, but that didn't work. Tried modifying= =20 >> >> > SingleQuoted to return "\'" .. s .. "\'", but that also didn't work= .=20 >> Not=20 >> >> > sure what to do about the thin space either.=20 >> >> >=20 >> >> > On Saturday, February 25, 2017 at 3:29:07 PM UTC-5, ss infod wrote:= =20 >> >> >>=20 >> >> >> Hello,=20 >> >> >>=20 >> >> >> I have a small question about Pandoc.=20 >> >> >>=20 >> >> >> Is there any plan to add the "Twiki Markup" language as an output= =20 >> >> (writer)=20 >> >> >> option ?=20 >> >> >>=20 >> >> >> If not, do you think it is a big job to add it ?=20 >> >> >> Is there some kind of dictionnary or script to create if I want to= =20 >> add=20 >> >> a=20 >> >> >> new output format ?=20 >> >> >>=20 >> >> >> Would it be a big task to modify dokuwiki format to Twiki ? (both= =20 >> seems=20 >> >> >> rather similar..)=20 >> >> >>=20 >> >> >> Thanks for your input.=20 >> >> >>=20 >> >> >=20 >> >> > --=20 >> >> > You received this message because you are subscribed to the Google= =20 >> >> Groups "pandoc-discuss" group.=20 >> >> > To unsubscribe from this group and stop receiving emails from it,= =20 >> send=20 >> >> an email to pandoc-discus...-/JYPxA39Uh5TLH3MbocFFw@public.gmane.org .=20 >> >> > To post to this group, send email to pandoc-...-/JYPxA39Uh5TLH3MbocFFw@public.gmane.org=20 >> >> .=20 >> >> > To view this discussion on the web visit=20 >> >>=20 >> https://groups.google.com/d/msgid/pandoc-discuss/2ba0d474-f0db-456e-922a= -bb7282e60bf5%40googlegroups.com.=20 >> >> >>=20 >> >> > For more options, visit https://groups.google.com/d/optout.=20 >> >>=20 >> >=20 >> > --=20 >> > You received this message because you are subscribed to the Google=20 >> Groups "pandoc-discuss" group.=20 >> > To unsubscribe from this group and stop receiving emails from it, send= =20 >> an email to pandoc-discus...-/JYPxA39Uh5TLH3MbocFFw@public.gmane.org .=20 >> > To post to this group, send email to pandoc-...-/JYPxA39Uh5TLH3MbocFFw@public.gmane.org=20 >> .=20 >> > To view this discussion on the web visit=20 >> https://groups.google.com/d/msgid/pandoc-discuss/5bffbda3-8281-4247-99c2= -3826011ec9ca%40googlegroups.com.=20 >> >> > For more options, visit https://groups.google.com/d/optout.=20 >> > > --=20 > You received this message because you are subscribed to the Google Groups= "pandoc-discuss" group. > To unsubscribe from this group and stop receiving emails from it, send an= email to pandoc-discuss+unsubscribe-/JYPxA39Uh5TLH3MbocFF+G/Ez6ZCGd0@public.gmane.org > To post to this group, send email to pandoc-discuss-/JYPxA39Uh5TLH3MbocFF+G/Ez6ZCGd0@public.gmane.org > To view this discussion on the web visit https://groups.google.com/d/msgi= d/pandoc-discuss/8631e4f1-8744-40b4-a3ce-a8d8e1bb0202%40googlegroups.com. > For more options, visit https://groups.google.com/d/optout. --=20 You received this message because you are subscribed to the Google Groups "= pandoc-discuss" group. To unsubscribe from this group and stop receiving emails from it, send an e= mail to pandoc-discuss+unsubscribe-/JYPxA39Uh5TLH3MbocFF+G/Ez6ZCGd0@public.gmane.org To post to this group, send email to pandoc-discuss-/JYPxA39Uh5TLH3MbocFF+G/Ez6ZCGd0@public.gmane.org To view this discussion on the web visit https://groups.google.com/d/msgid/= pandoc-discuss/yh480k5zvzj352.fsf%40johnmacfarlane.net. For more options, visit https://groups.google.com/d/optout.