Why is Pandoc apparently producing inconsistent rows when I convert to PDF?

Why is Pandoc apparently producing inconsistent rows when I convert to PDF?

[tutiluren via pandoc-discuss]

[-- Attachment #1.1: Type: text/plain, Size: 1294 bytes --]

Command used:

pandoc --pdf-engine=xelatex -V pagestyle="empty" -V mainfont="Courier New" -V monofont="Courier New" -V fontsize=10pt -V papersize="A4" -V margin-top="2cm" -V margin-right="2cm" -V margin-bottom="2cm" -V margin-left="2cm" "test.md" -o "out.pdf"

Contents of test.md:

```
--------------------------------------------------------------------------------
Test                                                                        Test
--------------------------------------------------------------------------------
testttt
```

Result:

(see screenshot.png)

Why are the two first rows visually not lined up with the third and further rows? Courier New is supposed to be a monospace font, and appears to be, except for the first rows... I don't even know how to ask the question because it's so absurd.

-- 
You received this message because you are subscribed to the Google Groups "pandoc-discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email to pandoc-discuss+unsubscribe-/JYPxA39Uh5TLH3MbocFF+G/Ez6ZCGd0@public.gmane.org
To view this discussion on the web visit https://groups.google.com/d/msgid/pandoc-discuss/M-pXqpk--3-2%40tutanota.com.

[-- Attachment #1.2: Type: text/html, Size: 2194 bytes --]

[-- Attachment #2: screenshot.png --]
[-- Type: image/png, Size: 10331 bytes --]

Re: Why is Pandoc apparently producing inconsistent rows when I convert to PDF?

[Jeremy Theler]

Dear Tutiluren

I do not know how to write a private email to you.

I think you should read a little bit more about what Donald Knuth has 
done over the last sixty years and complain a lot less about Pandoc.
Both TeX and Pandoc are way out of your league.



On Tue, 2020-02-11 at 19:38 +0100, tutiluren via pandoc-discuss wrote:
> 
> Why are the two first rows visually not lined up with the third and
> further rows? Courier New is supposed to be a monospace font, and
> appears to be, except for the first rows... I don't even know how to
> ask the question because it's so absurd.

Re: Why is Pandoc apparently producing inconsistent rows when I convert to PDF?

[tutiluren via pandoc-discuss]

[-- Attachment #1: Type: text/plain, Size: 1855 bytes --]

Huh? Where did that come from?

It appears as if basic human decency is "way out of your league", but kindly refrain from spewing your equally unwarranted as baffling insults in the future, whether privately or in public. (I guess e-mail is also "way out of your league" since you don't know how to send this to me only.)


Feb 11, 2020, 19:51 by jeremy-24em0bpozeFWk0Htik3J/w@public.gmane.org:

> Dear Tutiluren
>
> I do not know how to write a private email to you.
>
> I think you should read a little bit more about what Donald Knuth has 
> done over the last sixty years and complain a lot less about Pandoc.
> Both TeX and Pandoc are way out of your league.
>
>
>
> On Tue, 2020-02-11 at 19:38 +0100, tutiluren via pandoc-discuss wrote:
>
>>
>> Why are the two first rows visually not lined up with the third and
>> further rows? Courier New is supposed to be a monospace font, and
>> appears to be, except for the first rows... I don't even know how to
>> ask the question because it's so absurd.
>>
>
>
> -- 
> You received this message because you are subscribed to the Google Groups "pandoc-discuss" group.
> To unsubscribe from this group and stop receiving emails from it, send an email to pandoc-discuss+unsubscribe-/JYPxA39Uh5TLH3MbocFF+G/Ez6ZCGd0@public.gmane.org
> To view this discussion on the web visit https://groups.google.com/d/msgid/pandoc-discuss/2d7cced8da4adf4ba1f03d33c48c71f43fc94522.camel%40seamplex.com.
>

-- 
You received this message because you are subscribed to the Google Groups "pandoc-discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email to pandoc-discuss+unsubscribe-/JYPxA39Uh5TLH3MbocFF+G/Ez6ZCGd0@public.gmane.org
To view this discussion on the web visit https://groups.google.com/d/msgid/pandoc-discuss/M-prYFv--3-2%40tutanota.com.

[-- Attachment #2: Type: text/html, Size: 2714 bytes --]

Re: Why is Pandoc apparently producing inconsistent rows when I convert to PDF?

[Joost Kremers]

On Tue, Feb 11 2020, tutiluren via pandoc-discuss wrote:
> Command used:
>
> pandoc --pdf-engine=xelatex -V pagestyle="empty" -V 
> mainfont="Courier New" -V monofont="Courier New" -V 
> fontsize=10pt -V papersize="A4" -V margin-top="2cm" -V 
> margin-right="2cm" -V margin-bottom="2cm" -V margin-left="2cm" 
> "test.md" -o "out.pdf"
>
> Contents of test.md:
>
> ```
> --------------------------------------------------------------------------------
> Test 
> Test
> --------------------------------------------------------------------------------
> testttt
> ```
>
> Result:
>
> (see screenshot.png)
>
> Why are the two first rows visually not lined up with the third 
> and further rows? Courier New is supposed to be a monospace 
> font, and appears to be, except for the first rows... I don't 
> even know how to ask the question because it's so absurd.

I can't test with Courier New because I don't have that font, but 
substituting DejaVu Sans Mono I get the same effect. It goes away 
if I remove the variable `mainfont` from the call to Pandoc.

There's no real need to set the main font to a monospace font, 
since you're using backticks around the part of the text that 
needs to be monospace.

If you do want to set the main font to a monospace font, the 
effect also goes away if you remove the following lines from the 
default template:

    \IfFileExists{microtype.sty}{% use microtype if available
      \usepackage[]{microtype}
      \UseMicrotypeSet[protrusion]{basicmath} % disable protrusion 
      for tt fonts
    }{}

I don't know why those lines are added and I don't know why they 
cause the effect you are seeing in combination with a monospace 
main font but not in combination with a variable-width font, but 
that's how it is. LaTeX is a complex beast and weird interactions 
between different packages or options sometimes occur, especially 
if you do things that are very non-standard (such as setting the 
main document font to a monospace font...)

HTH

-- 
Joost Kremers
Life has its moments

Re: Why is Pandoc apparently producing inconsistent rows when I convert to PDF?

[John MacFarlane]

Sorry, I can't reproduce this what you're seeing.  I don't get
the vertical bars or the alignment issue.

Re: Why is Pandoc apparently producing inconsistent rows when I convert to PDF?

['Nick Bart' via pandoc-discuss]

I can reproduce the alignment issue.

Solution: RTFM. The microtype manual states “You might want to disable protrusion in verbatim environments.” and recommends using `\AtBeginEnvironment{verbatim}{\microtypesetup{activate=false}}` - which indeed does seem to fix the alignment problem.

We might want to add this to the `default.latex` template.


‐‐‐‐‐‐‐ Original Message ‐‐‐‐‐‐‐
On Wednesday, February 12, 2020 1:31 AM, John MacFarlane <jgm-TVLZxgkOlNX2fBVCVOL8/A@public.gmane.org> wrote:

> Sorry, I can't reproduce this what you're seeing. I don't get
> the vertical bars or the alignment issue.
>
> --------------------------------------------------------------------------------------------------------
>
> You received this message because you are subscribed to the Google Groups "pandoc-discuss" group.
> To unsubscribe from this group and stop receiving emails from it, send an email to pandoc-discuss+unsubscribe-/JYPxA39Uh5TLH3MbocFF+G/Ez6ZCGd0@public.gmane.org
> To view this discussion on the web visit https://groups.google.com/d/msgid/pandoc-discuss/yh480k4kvw4mly.fsf%40johnmacfarlane.net.


-- 
You received this message because you are subscribed to the Google Groups "pandoc-discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email to pandoc-discuss+unsubscribe-/JYPxA39Uh5TLH3MbocFF+G/Ez6ZCGd0@public.gmane.org
To view this discussion on the web visit https://groups.google.com/d/msgid/pandoc-discuss/hto9hF1z74jUNG2QE6wxdQbOsvXDnTjv-rd29u1oSbbdjavJawCSV1VRZpmiPqnPWN4Yj71xGM2FkKCEltVxDYFBhCGnSuFKByWq9JPdUQQ%3D%40protonmail.com.

Re: Why is Pandoc apparently producing inconsistent rows when I convert to PDF?

[tutiluren via pandoc-discuss]

[-- Attachment #1: Type: text/plain, Size: 3166 bytes --]

I don't know what you mean by "the microtype manual", but the quoted text is nowhere on https://pandoc.org/MANUAL.html , nor do I understand what it's saying. So "RTFM" doesn't do any good in this case, which of course I have already done to get this far in the first place.

As somebody suggested, not including/setting the "mainfont" somehow made this glitch go away (but instead introduced other issues with the margins, which I've "hacked away" by guessing values until it looks right).

At this point, it's clear that this is way more voodoo than science, given that even my ultra-simple kind of document has taken forever to carefully fine-tune and work around various bizarre glitches. I guess it "works" now, but it sure doesn't feel "solid". It feels like it can mess up in any way from changing any minor option. Of course, this kind of tool is very complicated to accomplish, but after all, I'm using Pandoc and not the "underlying tools" directly, precisely to avoid headaches like this.


Feb 12, 2020, 07:32 by pandoc-discuss-/JYPxA39Uh5TLH3MbocFFw@public.gmane.org:

> I can reproduce the alignment issue.
>
> Solution: RTFM. The microtype manual states “You might want to disable protrusion in verbatim environments.” and recommends using `\AtBeginEnvironment{verbatim}{\microtypesetup{activate=false}}` - which indeed does seem to fix the alignment problem.
>
> We might want to add this to the `default.latex` template.
>
>
> ‐‐‐‐‐‐‐ Original Message ‐‐‐‐‐‐‐
> On Wednesday, February 12, 2020 1:31 AM, John MacFarlane <jgm-TVLZxgkOlNWn+EJxYGL2xA@public.gmane.orgu> wrote:
>
>> Sorry, I can't reproduce this what you're seeing. I don't get
>> the vertical bars or the alignment issue.
>>
>> --------------------------------------------------------------------------------------------------------
>>
>> You received this message because you are subscribed to the Google Groups "pandoc-discuss" group.
>> To unsubscribe from this group and stop receiving emails from it, send an email to pandoc-discuss+unsubscribe-/JYPxA39Uh5TLH3MbocFF+G/Ez6ZCGd0@public.gmane.org
>> To view this discussion on the web visit https://groups.google.com/d/msgid/pandoc-discuss/yh480k4kvw4mly.fsf%40johnmacfarlane.net.
>>
>
>
> -- 
> You received this message because you are subscribed to the Google Groups "pandoc-discuss" group.
> To unsubscribe from this group and stop receiving emails from it, send an email to pandoc-discuss+unsubscribe-/JYPxA39Uh5TLH3MbocFF+G/Ez6ZCGd0@public.gmane.org
> To view this discussion on the web visit https://groups.google.com/d/msgid/pandoc-discuss/hto9hF1z74jUNG2QE6wxdQbOsvXDnTjv-rd29u1oSbbdjavJawCSV1VRZpmiPqnPWN4Yj71xGM2FkKCEltVxDYFBhCGnSuFKByWq9JPdUQQ%3D%40protonmail.com.
>

-- 
You received this message because you are subscribed to the Google Groups "pandoc-discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email to pandoc-discuss+unsubscribe-/JYPxA39Uh5TLH3MbocFF+G/Ez6ZCGd0@public.gmane.org
To view this discussion on the web visit https://groups.google.com/d/msgid/pandoc-discuss/M-t079Y--3-2%40tutanota.com.

[-- Attachment #2: Type: text/html, Size: 4086 bytes --]

Re: Why is Pandoc apparently producing inconsistent rows when I convert to PDF?

[Joost Kremers]

On Wed, Feb 12 2020, tutiluren via pandoc-discuss wrote:
> At this point, it's clear that this is way more voodoo than 
> science, given that even my ultra-simple kind of document has 
> taken forever to carefully fine-tune and work around various 
> bizarre glitches. I guess it "works" now, but it sure doesn't 
> feel "solid". It feels like it can mess up in any way from 
> changing any minor option. Of course, this kind of tool is very 
> complicated to accomplish, but after all, I'm using Pandoc and 
> not the "underlying tools" directly, precisely to avoid 
> headaches like this.

If you want to avoid such headaches, you should stick to the 
defaults that Pandoc provides and not mess with any LaTeX 
settings. As soon as you do that, you run the risk of running into 
situations that can only be dealt with at the LaTeX level.

In this case you seem to have run into an issue with verbatim 
blocks in a document that uses a monospace font as its main 
document font. Arguably, the solution suggested by Nick Bart 
should be added to Pandoc's default LaTeX template, but your use 
case is so rare that it's no surprise that no-one has encountered 
this issue before and, consequently, that the fix hasn't been 
included in Pandoc.

Please keep in mind that Pandoc is free software, created 
essentially by a single person in their spare time. Your e-mails 
convey (whether intended or not) a sense of entitlement and 
indignation at the fact that Pandoc doesn't work as seamlessly as 
you expect it to, which might be understandable if you'd payed 
good money for it, but in the present case it's not conducive to 
solving your issue.

HTH


-- 
Joost Kremers
Life has its moments

Re: Why is Pandoc apparently producing inconsistent rows when I convert to PDF?

[John MacFarlane]

Good find!

We already have

  \UseMicrotypeSet[protrusion]{basicmath} % disable protrusion for tt fonts

See https://github.com/jgm/pandoc/issues/1549

This is supposed to disable protrusion for all monospace fonts.
I would have thought this would have fixed the issue for verbatim
environments too. But maybe this doesn't work for system fonts,
or something?

If the problem only occurs when the main font is set to a fixed-width
font, then I'm not particularly worried about it and we might prefer
to avoid another complication in the default template.  But if
affects more normal uses, then we could consider putting this in.

"'Nick Bart' via pandoc-discuss"
<pandoc-discuss-/JYPxA39Uh5TLH3MbocFFw@public.gmane.org> writes:

> I can reproduce the alignment issue.
>
> Solution: RTFM. The microtype manual states “You might want to disable protrusion in verbatim environments.” and recommends using `\AtBeginEnvironment{verbatim}{\microtypesetup{activate=false}}` - which indeed does seem to fix the alignment problem.
>
> We might want to add this to the `default.latex` template.
>

-- 
You received this message because you are subscribed to the Google Groups "pandoc-discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email to pandoc-discuss+unsubscribe-/JYPxA39Uh5TLH3MbocFF+G/Ez6ZCGd0@public.gmane.org
To view this discussion on the web visit https://groups.google.com/d/msgid/pandoc-discuss/m2pnejsp7v.fsf%40johnmacfarlane.net.

Re: Why is Pandoc apparently producing inconsistent rows when I convert to PDF?

[John MacFarlane]

If you don't need math or fine typesetting support that LaTeX
offers, and just want to format simple things like in your
example, then you might try the wkhtmltopdf pdf-engine (which
uses an HTML intermediary rather than LaTeX). See the manual
under --pdf-engine and "Variables for wkhtmltopdf" -- and of
course you'll need to install wkhtmltopdf. For setting the font
with this approach you'd use regular CSS (--css option).

tutiluren via pandoc-discuss <pandoc-discuss-/JYPxA39Uh5TLH3MbocFFw@public.gmane.org>
writes:

> I don't know what you mean by "the microtype manual", but the quoted text is nowhere on https://pandoc.org/MANUAL.html , nor do I understand what it's saying. So "RTFM" doesn't do any good in this case, which of course I have already done to get this far in the first place.
>
> As somebody suggested, not including/setting the "mainfont" somehow made this glitch go away (but instead introduced other issues with the margins, which I've "hacked away" by guessing values until it looks right).
>
> At this point, it's clear that this is way more voodoo than science, given that even my ultra-simple kind of document has taken forever to carefully fine-tune and work around various bizarre glitches. I guess it "works" now, but it sure doesn't feel "solid". It feels like it can mess up in any way from changing any minor option. Of course, this kind of tool is very complicated to accomplish, but after all, I'm using Pandoc and not the "underlying tools" directly, precisely to avoid headaches like this.
>
>
> Feb 12, 2020, 07:32 by pandoc-discuss-/JYPxA39Uh5TLH3MbocFFw@public.gmane.org:
>
>> I can reproduce the alignment issue.
>>
>> Solution: RTFM. The microtype manual states “You might want to disable protrusion in verbatim environments.” and recommends using `\AtBeginEnvironment{verbatim}{\microtypesetup{activate=false}}` - which indeed does seem to fix the alignment problem.
>>
>> We might want to add this to the `default.latex` template.
>>
>>
>> ‐‐‐‐‐‐‐ Original Message ‐‐‐‐‐‐‐
>> On Wednesday, February 12, 2020 1:31 AM, John MacFarlane <jgm@berkeley.edu> wrote:
>>
>>> Sorry, I can't reproduce this what you're seeing. I don't get
>>> the vertical bars or the alignment issue.
>>>
>>> --------------------------------------------------------------------------------------------------------
>>>
>>> You received this message because you are subscribed to the Google Groups "pandoc-discuss" group.
>>> To unsubscribe from this group and stop receiving emails from it, send an email to pandoc-discuss+unsubscribe-/JYPxA39Uh5TLH3MbocFF+G/Ez6ZCGd0@public.gmane.org
>>> To view this discussion on the web visit https://groups.google.com/d/msgid/pandoc-discuss/yh480k4kvw4mly.fsf%40johnmacfarlane.net.
>>>
>>
>>
>> -- 
>> You received this message because you are subscribed to the Google Groups "pandoc-discuss" group.
>> To unsubscribe from this group and stop receiving emails from it, send an email to pandoc-discuss+unsubscribe-/JYPxA39Uh5TLH3MbocFF+G/Ez6ZCGd0@public.gmane.org
>> To view this discussion on the web visit https://groups.google.com/d/msgid/pandoc-discuss/hto9hF1z74jUNG2QE6wxdQbOsvXDnTjv-rd29u1oSbbdjavJawCSV1VRZpmiPqnPWN4Yj71xGM2FkKCEltVxDYFBhCGnSuFKByWq9JPdUQQ%3D%40protonmail.com.
>>
>
> -- 
> You received this message because you are subscribed to the Google Groups "pandoc-discuss" group.
> To unsubscribe from this group and stop receiving emails from it, send an email to pandoc-discuss+unsubscribe-/JYPxA39Uh5TLH3MbocFF+G/Ez6ZCGd0@public.gmane.org
> To view this discussion on the web visit https://groups.google.com/d/msgid/pandoc-discuss/M-t079Y--3-2%40tutanota.com.

-- 
You received this message because you are subscribed to the Google Groups "pandoc-discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email to pandoc-discuss+unsubscribe-/JYPxA39Uh5TLH3MbocFF+G/Ez6ZCGd0@public.gmane.org
To view this discussion on the web visit https://groups.google.com/d/msgid/pandoc-discuss/m2mu9nsp0l.fsf%40johnmacfarlane.net.

Re: Why is Pandoc apparently producing inconsistent rows when I convert to PDF?

[tutiluren via pandoc-discuss]

[-- Attachment #1: Type: text/plain, Size: 1287 bytes --]

 > your use case is so rare that it's no surprise that no-one has encountered this issue before and, consequently, that the fix hasn't been included in Pandoc.

My use case, converting a plaintext document to PDF, is rare? You'd think that would be the #1 use case, or at least top-3...

> Your e-mails convey (whether intended or not) a sense of entitlement and indignation at the fact that Pandoc doesn't work as seamlessly as you expect it to

It wouldn't matter how many layers of politeness I'd add -- somebody would still make that "feeling entitled" claim. There is no way to avoid it. As for "indignation", that's right. Am I expected to enjoy it when things don't work as expected/desired? Especially after spending such enormous amounts of time and energy trying countless ways to solve this one problem, before Pandoc. And it's just one in a huge pile of other, unrelated problems.

-- 
You received this message because you are subscribed to the Google Groups "pandoc-discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email to pandoc-discuss+unsubscribe-/JYPxA39Uh5TLH3MbocFF+G/Ez6ZCGd0@public.gmane.org
To view this discussion on the web visit https://groups.google.com/d/msgid/pandoc-discuss/M007DOi--B-2%40tutanota.com.

[-- Attachment #2: Type: text/html, Size: 1766 bytes --]

Re: Why is Pandoc apparently producing inconsistent rows when I convert to PDF?

[Joost Kremers]

On Fri, Feb 14 2020, tutiluren via pandoc-discuss wrote:
>  > your use case is so rare that it's no surprise that no-one 
>  > has encountered this issue before and, consequently, that the 
>  > fix hasn't been included in Pandoc.
>
> My use case, converting a plaintext document to PDF, is rare? 
> You'd think that would be the #1 use case, or at least top-3...

You want the pdf to still look like a plain-text document. The 
main purpose of Pandoc's LaTeX conversion, however, is to take a 
plain-text document and turn it Thisinto a properly type-set 
document, something that does *not* look like plain text. This 
assumes, for one, that the main document font is a proportional 
(usually serif) font. You want to use a monospace font as the main 
document font, which is a very rare use-case indeed. This means 
that such a configuration is not tested very much and is more 
likely to turn up bugs. Which in your case it has.

It's not impossible, it just takes some tweaking, something that 
you seem to be unwilling to accept.

>> Your e-mails convey (whether intended or not) a sense of 
>> entitlement and indignation at the fact that Pandoc doesn't 
>> work as seamlessly as you expect it to
>
> It wouldn't matter how many layers of politeness I'd add -- 
> somebody would still make that "feeling entitled" claim. There 
> is no way to avoid it.

Trust me, there is. It's clear from your messages that you 
basically think Pandoc is crappy software because it doesn't do 
exactly what you want it to do. If you would instead consider the 
possibility that you may simply be aiming for something unusual 
that takes some special configuration and tweaking, your messages 
would sound very different.

> As for "indignation", that's right. Am I expected to enjoy it 
> when things don't work as expected/desired?

No, but it would help if you didn't automatically think that the 
software is crappy and exhume that attitude on the mailing list. 
Consider the possibility that your expectations may be wrong and 
limit your messages to describing what you want to achieve, what 
you've tried and what went wrong.

> Especially after spending such enormous amounts of time and 
> energy trying countless ways to solve this one problem, before 
> Pandoc. And it's just one in a huge pile of other, unrelated 
> problems.

So you're taking your frustration with other programs out on 
Pandoc. That's obviously not going to help.

-- 
Joost Kremers
Life has its moments