Plain text input

Plain text input

[Neil Youngman]

[-- Attachment #1.1: Type: text/plain, Size: 1123 bytes --]

I'm trying to find something that will convert CJK plain text to a 
graphical format. So far the only likely candidate I have found is Pandoc 
with GNU unifont to convert to PDF and the ghostscript to produce TIFF.

Pandoc does not appear to have a plain text input format, so if I go down 
this route, I may have to preprocess the text and escape any markdown 
characters. Is there an option I've missed to turn off markdown completely, 
a pre existing script to preprocess the text or some other more suitable 
program that I should be considering?

-- 
You received this message because you are subscribed to the Google Groups "pandoc-discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email to pandoc-discuss+unsubscribe-/JYPxA39Uh5TLH3MbocFF+G/Ez6ZCGd0@public.gmane.org
To post to this group, send email to pandoc-discuss-/JYPxA39Uh5TLH3MbocFF+G/Ez6ZCGd0@public.gmane.org
To view this discussion on the web visit https://groups.google.com/d/msgid/pandoc-discuss/8e9e08e4-1eb5-4620-9a0d-5afa1d31166b%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[-- Attachment #1.2: Type: text/html, Size: 1547 bytes --]

Re: Plain text input

[lukshuntim-Re5JQEeQqe8AvxtiuMwx3w]

On 11/17/2015 06:12 PM, Neil Youngman wrote:
> I'm trying to find something that will convert CJK plain text to a
> graphical format. So far the only likely candidate I have found is
> Pandoc with GNU unifont to convert to PDF and the ghostscript to produce
> TIFF.
>
> Pandoc does not appear to have a plain text input format, so if I go
> down this route, I may have to preprocess the text and escape any
> markdown characters. Is there an option I've missed to turn off markdown
> completely, a pre existing script to preprocess the text or some other
> more suitable program that I should be considering?

Hi Neil,

There's should be no problem for pandoc if your CJK text is in utf-8.

Regards,
ST
-- 

Re: Plain text input

[Neil Youngman]

[-- Attachment #1.1: Type: text/plain, Size: 1825 bytes --]



On Tuesday, 17 November 2015 13:45:09 UTC, lukshuntim wrote:
>
> On 11/17/2015 06:12 PM, Neil Youngman wrote: 
> > I'm trying to find something that will convert CJK plain text to a 
> > graphical format. So far the only likely candidate I have found is 
> > Pandoc with GNU unifont to convert to PDF and the ghostscript to produce 
> > TIFF. 
> > 
> > Pandoc does not appear to have a plain text input format, so if I go 
> > down this route, I may have to preprocess the text and escape any 
> > markdown characters. Is there an option I've missed to turn off markdown 
> > completely, a pre existing script to preprocess the text or some other 
> > more suitable program that I should be considering? 
>
> Hi Neil, 
>
> There's should be no problem for pandoc if your CJK text is in utf-8. 
>
> Regards, 
> ST 
> -- 
>
> The problem is making sure that it doesn't interpret any punctuation as 
markdown. Verbatim might be useful, but if I put backticks around Japanese 
characters they disappear from the output. 

An option to turn off markdown completely would be great, but as a 
non-Haskell programmer I can't just dive into the source.

I am currently thinking I will wrap backticks around all characters in the 
ASCII range except backticks.

-- 
You received this message because you are subscribed to the Google Groups "pandoc-discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email to pandoc-discuss+unsubscribe-/JYPxA39Uh5TLH3MbocFF+G/Ez6ZCGd0@public.gmane.org
To post to this group, send email to pandoc-discuss-/JYPxA39Uh5TLH3MbocFF+G/Ez6ZCGd0@public.gmane.org
To view this discussion on the web visit https://groups.google.com/d/msgid/pandoc-discuss/edf3f806-94a1-4ae1-ad9c-0338c5469526%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[-- Attachment #1.2: Type: text/html, Size: 2469 bytes --]

Re: Plain text input

[John MacFarlane]

If you just want the text as one big string, you can try
running this Haskell program:

-- plain.hs
import Text.Pandoc.Definition

main = do
  inp <- getContents
  print $ Pandoc nullMeta [Para [Str inp]]


Run as:  runghc plain.hs < inputfile.txt | pandoc -f native -o my.pdf

But you probably want to retain some structural formatting,
e.g. paragraphs?

+++ Neil Youngman [Nov 17 15 06:15 ]:
>   On Tuesday, 17 November 2015 13:45:09 UTC, lukshuntim wrote:
>
>     On 11/17/2015 06:12 PM, Neil Youngman wrote:
>     > I'm trying to find something that will convert CJK plain text to a
>     > graphical format. So far the only likely candidate I have found is
>     > Pandoc with GNU unifont to convert to PDF and the ghostscript to
>     produce
>     > TIFF.
>     >
>     > Pandoc does not appear to have a plain text input format, so if I
>     go
>     > down this route, I may have to preprocess the text and escape any
>     > markdown characters. Is there an option I've missed to turn off
>     markdown
>     > completely, a pre existing script to preprocess the text or some
>     other
>     > more suitable program that I should be considering?
>     Hi Neil,
>     There's should be no problem for pandoc if your CJK text is in
>     utf-8.
>     Regards,
>     ST
>     --
>
>   The problem is making sure that it doesn't interpret any punctuation as
>   markdown. Verbatim might be useful, but if I put backticks around
>   Japanese characters they disappear from the output.
>   An option to turn off markdown completely would be great, but as a
>   non-Haskell programmer I can't just dive into the source.
>   I am currently thinking I will wrap backticks around all characters in
>   the ASCII range except backticks.
>
>   --
>   You received this message because you are subscribed to the Google
>   Groups "pandoc-discuss" group.
>   To unsubscribe from this group and stop receiving emails from it, send
>   an email to [1]pandoc-discuss+unsubscribe-/JYPxA39Uh5TLH3MbocFF+G/Ez6ZCGd0@public.gmane.org
>   To post to this group, send email to
>   [2]pandoc-discuss-/JYPxA39Uh5TLH3MbocFF+G/Ez6ZCGd0@public.gmane.org
>   To view this discussion on the web visit
>   [3]https://groups.google.com/d/msgid/pandoc-discuss/edf3f806-94a1-4ae1-
>   ad9c-0338c5469526%40googlegroups.com.
>   For more options, visit [4]https://groups.google.com/d/optout.
>
>References
>
>   1. mailto:pandoc-discuss+unsubscribe-/JYPxA39Uh5TLH3MbocFFw@public.gmane.org
>   2. mailto:pandoc-discuss-/JYPxA39Uh5TLH3MbocFFw@public.gmane.org
>   3. https://groups.google.com/d/msgid/pandoc-discuss/edf3f806-94a1-4ae1-ad9c-0338c5469526-/JYPxA39Uh5TLH3MbocFFw@public.gmane.org?utm_medium=email&utm_source=footer
>   4. https://groups.google.com/d/optout

Re: Plain text input

[Neil Youngman]

[-- Attachment #1.1: Type: text/plain, Size: 946 bytes --]



> But you probably want to retain some structural formatting, 
> e.g. paragraphs? 
>
> I want it as close as possible to what I'd get running cat in an xterm, so 
I want line breaks retained, but long lines should be wrapped. I don;t want 
adjacent lines to be treated as a paragraph and run together.

I'll have a play tomorrow and see if I can get that Haskell to work.

-- 
You received this message because you are subscribed to the Google Groups "pandoc-discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email to pandoc-discuss+unsubscribe-/JYPxA39Uh5TLH3MbocFF+G/Ez6ZCGd0@public.gmane.org
To post to this group, send email to pandoc-discuss-/JYPxA39Uh5TLH3MbocFF+G/Ez6ZCGd0@public.gmane.org
To view this discussion on the web visit https://groups.google.com/d/msgid/pandoc-discuss/944f76c0-db57-4272-a137-d4fb793f5de8%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[-- Attachment #1.2: Type: text/html, Size: 1522 bytes --]

Re: Plain text input

[John MACFARLANE]

Just put the whole thing in a code block, then, and set your
desired font as `monofont`.

To do this, create a file 'border' containing just a line
with 3 backticks

```

Then, assuming input.txt contains your Chinese text,

pandoc -o output.pdf border input.txt border

+++ Neil Youngman [Nov 17 15 14:17 ]:
>     But you probably want to retain some structural formatting,
>     e.g. paragraphs?
>
>   I want it as close as possible to what I'd get running cat in an xterm,
>   so I want line breaks retained, but long lines should be wrapped. I
>   don;t want adjacent lines to be treated as a paragraph and run
>   together.
>   I'll have a play tomorrow and see if I can get that Haskell to work.
>
>   --
>   You received this message because you are subscribed to the Google
>   Groups "pandoc-discuss" group.
>   To unsubscribe from this group and stop receiving emails from it, send
>   an email to [1]pandoc-discuss+unsubscribe-/JYPxA39Uh5TLH3MbocFF+G/Ez6ZCGd0@public.gmane.org
>   To post to this group, send email to
>   [2]pandoc-discuss-/JYPxA39Uh5TLH3MbocFF+G/Ez6ZCGd0@public.gmane.org
>   To view this discussion on the web visit
>   [3]https://groups.google.com/d/msgid/pandoc-discuss/944f76c0-db57-4272-
>   a137-d4fb793f5de8%40googlegroups.com.
>   For more options, visit [4]https://groups.google.com/d/optout.
>
>References
>
>   1. mailto:pandoc-discuss+unsubscribe-/JYPxA39Uh5TLH3MbocFFw@public.gmane.org
>   2. mailto:pandoc-discuss-/JYPxA39Uh5TLH3MbocFFw@public.gmane.org
>   3. https://groups.google.com/d/msgid/pandoc-discuss/944f76c0-db57-4272-a137-d4fb793f5de8-/JYPxA39Uh5TLH3MbocFFw@public.gmane.org?utm_medium=email&utm_source=footer
>   4. https://groups.google.com/d/optout

Re: Plain text input

[Daniel Staal]

--As of November 17, 2015 2:12:05 AM -0800, Neil Youngman is alleged to 
have said:

> I'm trying to find something that will convert CJK plain text to a
> graphical format. So far the only likely candidate I have found is Pandoc
> with GNU unifont to convert to PDF and the ghostscript to produce TIFF.
>
> Pandoc does not appear to have a plain text input format, so if I go down
> this route, I may have to preprocess the text and escape any markdown
> characters. Is there an option I've missed to turn off markdown
> completely,  a pre existing script to preprocess the text or some other
> more suitable  program that I should be considering?

--As for the rest, it is mine.

Another thought would be to tell pandoc that it's in some other (more 
intrusive) markup format - one that doesn't overlap with what is in your 
documents, but is still plain-text enough to let through your content 
without causing issues. MediaWiki would look like a good bet - it's markup 
is a bit more oddball and less likely to be something that would cause 
issues.

Daniel T. Staal

---------------------------------------------------------------
This email copyright the author.  Unless otherwise noted, you
are expressly allowed to retransmit, quote, or otherwise use
the contents for non-commercial purposes.  This copyright will
expire 5 years after the author's death, or in 30 years,
whichever is longer, unless such a period is in excess of
local copyright law.
---------------------------------------------------------------

Re: Plain text input

[Neil Youngman]

[-- Attachment #1.1: Type: text/plain, Size: 1084 bytes --]

On Tuesday, 17 November 2015 16:17:03 UTC, John MacFarlane wrote:
>
> If you just want the text as one big string, you can try 
> running this Haskell program: 
>
> -- plain.hs 
> import Text.Pandoc.Definition 
>
> main = do 
>   inp <- getContents 
>   print $ Pandoc nullMeta [Para [Str inp]] 
>
>
> Run as:  runghc plain.hs < inputfile.txt | pandoc -f native -o my.pdf 
>
>
Pandoc doesn't like the  output. It complains about undefined control 
characters \r, \n and \3 (start of escaped Japanese character).

-- 
You received this message because you are subscribed to the Google Groups "pandoc-discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email to pandoc-discuss+unsubscribe-/JYPxA39Uh5TLH3MbocFF+G/Ez6ZCGd0@public.gmane.org
To post to this group, send email to pandoc-discuss-/JYPxA39Uh5TLH3MbocFF+G/Ez6ZCGd0@public.gmane.org
To view this discussion on the web visit https://groups.google.com/d/msgid/pandoc-discuss/fdfdd54d-aecc-4f57-9722-ef63716ff2a9%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[-- Attachment #1.2: Type: text/html, Size: 1689 bytes --]

Re: Plain text input

[Neil Youngman]

[-- Attachment #1.1: Type: text/plain, Size: 1028 bytes --]

On Tuesday, 17 November 2015 22:30:09 UTC, John MacFarlane wrote:
>
> Just put the whole thing in a code block, then, and set your 
> desired font as `monofont`. 
>
> To do this, create a file 'border' containing just a line 
> with 3 backticks 
>
> ``` 
>
> Then, assuming input.txt contains your Chinese text, 
>
> pandoc -o output.pdf border input.txt border 
>
>
The code blocks don't wrap the long lines. Other than that the results are 
pretty good.

-- 
You received this message because you are subscribed to the Google Groups "pandoc-discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email to pandoc-discuss+unsubscribe-/JYPxA39Uh5TLH3MbocFF+G/Ez6ZCGd0@public.gmane.org
To post to this group, send email to pandoc-discuss-/JYPxA39Uh5TLH3MbocFF+G/Ez6ZCGd0@public.gmane.org
To view this discussion on the web visit https://groups.google.com/d/msgid/pandoc-discuss/8a370e82-be54-4f05-99a7-07cc11de0025%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[-- Attachment #1.2: Type: text/html, Size: 1629 bytes --]

Re: Plain text input

[John MacFarlane]

+++ Neil Youngman [Nov 18 15 02:32 ]:
>   On Tuesday, 17 November 2015 16:17:03 UTC, John MacFarlane wrote:
>
>     If you just want the text as one big string, you can try
>     running this Haskell program:
>     -- plain.hs
>     import Text.Pandoc.Definition
>     main = do
>       inp <- getContents
>       print $ Pandoc nullMeta [Para [Str inp]]
>     Run as:  runghc plain.hs < inputfile.txt | pandoc -f native -o
>     my.pdf
>
>   Pandoc doesn't like the  output. It complains about undefined control
>   characters \r, \n and \3 (start of escaped Japanese character).

Try the code block approach I outlined in my later message.

Re: Plain text input

[Neil Youngman]

[-- Attachment #1.1: Type: text/plain, Size: 830 bytes --]



On Wednesday, 18 November 2015 16:53:23 UTC, John MacFarlane wrote:
>
>
>
> Try the code block approach I outlined in my later message. 
>
>
As I said earlier, that doesn't wrap long lines. I guess I could sort that 
with some preprocessing.

Thanks

Neil

-- 
You received this message because you are subscribed to the Google Groups "pandoc-discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email to pandoc-discuss+unsubscribe-/JYPxA39Uh5TLH3MbocFF+G/Ez6ZCGd0@public.gmane.org
To post to this group, send email to pandoc-discuss-/JYPxA39Uh5TLH3MbocFF+G/Ez6ZCGd0@public.gmane.org
To view this discussion on the web visit https://groups.google.com/d/msgid/pandoc-discuss/6fd2263e-73ba-4975-84b8-4103c022e756%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[-- Attachment #1.2: Type: text/html, Size: 1419 bytes --]

Re: Plain text input

[John MacFarlane]

+++ Neil Youngman [Nov 18 15 02:32 ]:
>   On Tuesday, 17 November 2015 16:17:03 UTC, John MacFarlane wrote:
>
>     If you just want the text as one big string, you can try
>     running this Haskell program:
>     -- plain.hs
>     import Text.Pandoc.Definition
>     main = do
>       inp <- getContents
>       print $ Pandoc nullMeta [Para [Str inp]]
>     Run as:  runghc plain.hs < inputfile.txt | pandoc -f native -o
>     my.pdf
>
>   Pandoc doesn't like the  output. It complains about undefined control
>   characters \r, \n and \3 (start of escaped Japanese character).

I don't know about the `\3` (is your text UTF8?)

As for the rest, we can try something like:

-- plain.hs
import Text.Pandoc.Definition
import Text.Pandoc.Builder
import qualified Data.Text as T
import qualified Data.Text.IO as TIO
import Data.Monoid

main = do
  inp <- TIO.getContents
  -- break into paragraphs
  let pars = T.splitOn (T.pack "\n\n") $ T.filter (/= '\r') inp
  print $ doc $ mconcat $ map (\par -> para (text (T.unpack par))) pars

Re: Plain text input

[Neil Youngman]

[-- Attachment #1.1: Type: text/plain, Size: 794 bytes --]


Thanks for all the input. Someone has now suggested GrahicsMagick Convert, 
which works for my simple test case, provided I specify a suitable font.

Gm convert doesn't wrap, but the fold utility will handle that for me.

-- 
You received this message because you are subscribed to the Google Groups "pandoc-discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email to pandoc-discuss+unsubscribe-/JYPxA39Uh5TLH3MbocFF+G/Ez6ZCGd0@public.gmane.org
To post to this group, send email to pandoc-discuss-/JYPxA39Uh5TLH3MbocFF+G/Ez6ZCGd0@public.gmane.org
To view this discussion on the web visit https://groups.google.com/d/msgid/pandoc-discuss/328b3b44-c747-4494-9b3e-c8e7aa0e304e%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[-- Attachment #1.2: Type: text/html, Size: 1222 bytes --]