Nice! I added this section to escape(), -- For TWiki installations that don't support Unicode. -- We replace some common Unicode-only symbols by their closest ASCII -- counterpart. for p, c in utf8.codes(s) do if (c == 0x2019 or c == 0x2018) then new_string = new_string .. "'" elseif (c == 0x201C or c == 0x201D) then new_string = new_string .. '"' elseif (c == 0x00A0) then new_string = new_string .. ' ' else new_string = new_string .. utf8.char(c) end end return new_string end I haven't gone and fixed accents and stuff, I'll probably add to this as I need more. Thanks John for all the help, it is appreciated. I've attached the full custom writer in case somebody wants to build on it. Cheers, On Tuesday, December 11, 2018 at 2:05:12 PM UTC-5, John MacFarlane wrote: > > > The problem is that lua's gsub is not really UTF-8 > aware. [<>^\"'] will match a single BYTE, but the > quote characters are multibyte. > > You can use the functions documented in sec 6.5 > here: https://www.lua.org/manual/5.3/manual.html#6.5. > > These allow you to iterate over the characters in a > string. > > for p, c in utf8.codes(s) do > -- c is the code point (integer) of the character > end > > You can use this to build up a new string, escaping > each code point appropriately. > > > Joey Dumont > writes: > > > I tried that already. My escape function is probably bad then. > > > > -- Character escaping > > local function escape(s, in_attribute) > > return s:gsub("[<>&\"']", > > function(x) > > if x == '<' then > > return '<' > > elseif x == '>' then > > return '>' > > elseif x == '&' then > > return '&' > > elseif x == '"' then > > return '' > > elseif x == "'" then > > return "'" > > else > > return x > > end > > end) > > end > > > > Should I put something more specific in the x == "'" and x == '"' > fields? > > > > On Tuesday, December 11, 2018 at 1:49:13 PM UTC-5, John MacFarlane > wrote: > >> > >> > >> You can always change the 'escape' function in your > >> custom writer so it outputs ASCII entities instead of > >> UTF-8 characters. > >> > >> Joey Dumont > writes: > >> > >> > Oh. > >> > > >> > It might be my TWiki installation then. Nothing I can really do about > >> that > >> > though. > >> > > >> > Thanks for the help! > >> > > >> > On Monday, December 10, 2018 at 2:49:58 PM UTC-5, John MacFarlane > wrote: > >> >> > >> >> > >> >> This is an encoding issue. It looks like twiki is > >> >> generating the right UTF-8 encoded output, but your > >> >> browser isn't displaying it as UTF-8. You should > >> >> be able to fix this by ensuring that twiki produces > >> >> HTML containing a meta tag that sets the charset to > >> >> UTF-8. > >> >> > >> >> Joey Dumont > writes: > >> >> > >> >> > I'm having trouble with quotes and what I believe are thin spaces, > >> >> however. > >> >> > > >> >> > For instance, writing "he'll test something" in Markdown, then > >> >> converting > >> >> > to TWiki yields an HTML curly quote instead of a plain ' quote. > TWiki > >> >> then > >> >> > freaks and renders that as "he’ll test something"*.* > >> >> > > >> >> > The string "Dec. 5th" seems to yield a thin space between the > period > >> and > >> >> > the decimal, but TWiki renders it as "Dec. 5th" > >> >> > > >> >> > Not sure how to fix this. Tried escaping with escape(s) in > multiple > >> >> places, > >> >> > such as Str, Plain and Para, but that didn't work. Tried modifying > >> >> > SingleQuoted to return "\'" .. s .. "\'", but that also didn't > work. > >> Not > >> >> > sure what to do about the thin space either. > >> >> > > >> >> > On Saturday, February 25, 2017 at 3:29:07 PM UTC-5, ss infod > wrote: > >> >> >> > >> >> >> Hello, > >> >> >> > >> >> >> I have a small question about Pandoc. > >> >> >> > >> >> >> Is there any plan to add the "Twiki Markup" language as an output > >> >> (writer) > >> >> >> option ? > >> >> >> > >> >> >> If not, do you think it is a big job to add it ? > >> >> >> Is there some kind of dictionnary or script to create if I want > to > >> add > >> >> a > >> >> >> new output format ? > >> >> >> > >> >> >> Would it be a big task to modify dokuwiki format to Twiki ? (both > >> seems > >> >> >> rather similar..) > >> >> >> > >> >> >> Thanks for your input. > >> >> >> > >> >> > > >> >> > -- > >> >> > You received this message because you are subscribed to the Google > >> >> Groups "pandoc-discuss" group. > >> >> > To unsubscribe from this group and stop receiving emails from it, > >> send > >> >> an email to pandoc-discus...-/JYPxA39Uh5TLH3MbocFFw@public.gmane.org . > >> >> > To post to this group, send email to pandoc-...-/JYPxA39Uh5TLH3MbocFFw@public.gmane.org > >> >> . > >> >> > To view this discussion on the web visit > >> >> > >> > https://groups.google.com/d/msgid/pandoc-discuss/2ba0d474-f0db-456e-922a-bb7282e60bf5%40googlegroups.com. > > >> > >> >> > >> >> > For more options, visit https://groups.google.com/d/optout. > >> >> > >> > > >> > -- > >> > You received this message because you are subscribed to the Google > >> Groups "pandoc-discuss" group. > >> > To unsubscribe from this group and stop receiving emails from it, > send > >> an email to pandoc-discus...-/JYPxA39Uh5TLH3MbocFFw@public.gmane.org . > >> > To post to this group, send email to pandoc-...-/JYPxA39Uh5TLH3MbocFFw@public.gmane.org > >> . > >> > To view this discussion on the web visit > >> > https://groups.google.com/d/msgid/pandoc-discuss/5bffbda3-8281-4247-99c2-3826011ec9ca%40googlegroups.com. > > >> > >> > For more options, visit https://groups.google.com/d/optout. > >> > > > > -- > > You received this message because you are subscribed to the Google > Groups "pandoc-discuss" group. > > To unsubscribe from this group and stop receiving emails from it, send > an email to pandoc-discus...-/JYPxA39Uh5TLH3MbocFFw@public.gmane.org . > > To post to this group, send email to pandoc-...-/JYPxA39Uh5TLH3MbocFFw@public.gmane.org > . > > To view this discussion on the web visit > https://groups.google.com/d/msgid/pandoc-discuss/8631e4f1-8744-40b4-a3ce-a8d8e1bb0202%40googlegroups.com. > > > For more options, visit https://groups.google.com/d/optout. > -- You received this message because you are subscribed to the Google Groups "pandoc-discuss" group. To unsubscribe from this group and stop receiving emails from it, send an email to pandoc-discuss+unsubscribe-/JYPxA39Uh5TLH3MbocFF+G/Ez6ZCGd0@public.gmane.org To post to this group, send email to pandoc-discuss-/JYPxA39Uh5TLH3MbocFF+G/Ez6ZCGd0@public.gmane.org To view this discussion on the web visit https://groups.google.com/d/msgid/pandoc-discuss/6a8f82dc-1d9a-473e-ab21-c6ab413a75ba%40googlegroups.com. For more options, visit https://groups.google.com/d/optout.