Debugging a seemingly infinitely loop when processing a LaTeX input file

Debugging a seemingly infinitely loop when processing a LaTeX input file

[Joey Dumont]

[-- Attachment #1.1: Type: text/plain, Size: 986 bytes --]

Hi, 

I am trying to convert a LaTeX input file to docx. However, pandoc is 
having issues doing just that. It just allocates more memory until the 
system runs out and never completes. 

Is there a way to debug that to see where the portion of the file that 
makes pandoc hang?

Also, I'd rather not post the file publicly, so if you want the file to 
test for yourself, I'll arrange a private transfer.

Thanks!

-- 
You received this message because you are subscribed to the Google Groups "pandoc-discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email to pandoc-discuss+unsubscribe-/JYPxA39Uh5TLH3MbocFF+G/Ez6ZCGd0@public.gmane.org
To post to this group, send email to pandoc-discuss-/JYPxA39Uh5TLH3MbocFF+G/Ez6ZCGd0@public.gmane.org
To view this discussion on the web visit https://groups.google.com/d/msgid/pandoc-discuss/18db4113-1c4e-4be1-971e-6368ee3d54d2%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[-- Attachment #1.2: Type: text/html, Size: 1521 bytes --]

Re: Debugging a seemingly infinitely loop when processing a LaTeX input file

[Jesse Rosenthal]

You don't really give us much to work with (pandoc version, os)
here. But here are a few steps toward helping us solve the bug:

1. First convert the file to native representation (`pandoc
secret.tex -o secret.native`)

2. If that works, try converting the native file to a docx file (`pandoc
secret.native -o secret.docx`)

This will help you figure out whether the problem is in the LaTeX reader
or the docx writer. Once you figure that out, you can try to

3. produce a MWE (minimal working example) that produces the same issue,
by removing parts of the file, and replacing the sensitive text with
other text.

Once you have that MWE, you can post it, along with pandoc version and
os, you can check to see whether there is a similar issue on the github
issue tracker already. If not, you can post it as a bug.

Best,
Jesse

Joey Dumont <joey.dumont-Re5JQEeQqe8AvxtiuMwx3w@public.gmane.org> writes:

> Hi, 
>
> I am trying to convert a LaTeX input file to docx. However, pandoc is 
> having issues doing just that. It just allocates more memory until the 
> system runs out and never completes. 
>
> Is there a way to debug that to see where the portion of the file that 
> makes pandoc hang?
>
> Also, I'd rather not post the file publicly, so if you want the file to 
> test for yourself, I'll arrange a private transfer.
>
> Thanks!
>
> -- 
> You received this message because you are subscribed to the Google Groups "pandoc-discuss" group.
> To unsubscribe from this group and stop receiving emails from it, send an email to pandoc-discuss+unsubscribe-/JYPxA39Uh5TLH3MbocFF+G/Ez6ZCGd0@public.gmane.org
> To post to this group, send email to pandoc-discuss-/JYPxA39Uh5TLH3MbocFF+G/Ez6ZCGd0@public.gmane.org
> To view this discussion on the web visit https://groups.google.com/d/msgid/pandoc-discuss/18db4113-1c4e-4be1-971e-6368ee3d54d2%40googlegroups.com.
> For more options, visit https://groups.google.com/d/optout.

Re: Debugging a seemingly infinitely loop when processing a LaTeX input file

[Joey Dumont]

[-- Attachment #1.1: Type: text/plain, Size: 3169 bytes --]

pandoc 2.7.2, CentOS 7 (I downloaded the latest release from GitHub).

I cannot convert to a native representation.

I'm not sure on where to start to create a MWE. My input file is about 2000 
lines long, so I was hoping that there was a way to know approximately 
where the issue is happening in the file. That's why I didn't include a lot 
of detail, I just wanted to know if there was a debugging feature in 
pandoc. I couldn't find anything in the manual.

Thanks!

On Wednesday, May 1, 2019 at 9:45:03 AM UTC-4, Jesse Rosenthal wrote:
>
> You don't really give us much to work with (pandoc version, os) 
> here. But here are a few steps toward helping us solve the bug: 
>
> 1. First convert the file to native representation (`pandoc 
> secret.tex -o secret.native`) 
>
> 2. If that works, try converting the native file to a docx file (`pandoc 
> secret.native -o secret.docx`) 
>
> This will help you figure out whether the problem is in the LaTeX reader 
> or the docx writer. Once you figure that out, you can try to 
>
> 3. produce a MWE (minimal working example) that produces the same issue, 
> by removing parts of the file, and replacing the sensitive text with 
> other text. 
>
> Once you have that MWE, you can post it, along with pandoc version and 
> os, you can check to see whether there is a similar issue on the github 
> issue tracker already. If not, you can post it as a bug. 
>
> Best, 
> Jesse 
>
> Joey Dumont <joey....-Re5JQEeQqe8AvxtiuMwx3w@public.gmane.org <javascript:>> writes: 
>
> > Hi, 
> > 
> > I am trying to convert a LaTeX input file to docx. However, pandoc is 
> > having issues doing just that. It just allocates more memory until the 
> > system runs out and never completes. 
> > 
> > Is there a way to debug that to see where the portion of the file that 
> > makes pandoc hang? 
> > 
> > Also, I'd rather not post the file publicly, so if you want the file to 
> > test for yourself, I'll arrange a private transfer. 
> > 
> > Thanks! 
> > 
> > -- 
> > You received this message because you are subscribed to the Google 
> Groups "pandoc-discuss" group. 
> > To unsubscribe from this group and stop receiving emails from it, send 
> an email to pandoc-...-/JYPxA39Uh5TLH3MbocFFw@public.gmane.org <javascript:>. 
> > To post to this group, send email to pandoc-...-/JYPxA39Uh5TLH3MbocFFw@public.gmane.org 
> <javascript:>. 
> > To view this discussion on the web visit 
> https://groups.google.com/d/msgid/pandoc-discuss/18db4113-1c4e-4be1-971e-6368ee3d54d2%40googlegroups.com. 
>
> > For more options, visit https://groups.google.com/d/optout. 
>

-- 
You received this message because you are subscribed to the Google Groups "pandoc-discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email to pandoc-discuss+unsubscribe-/JYPxA39Uh5TLH3MbocFF+G/Ez6ZCGd0@public.gmane.org
To post to this group, send email to pandoc-discuss-/JYPxA39Uh5TLH3MbocFF+G/Ez6ZCGd0@public.gmane.org
To view this discussion on the web visit https://groups.google.com/d/msgid/pandoc-discuss/21ca689e-db3c-4c3b-9035-21592df59f8a%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[-- Attachment #1.2: Type: text/html, Size: 5246 bytes --]

Re: Debugging a seemingly infinitely loop when processing a LaTeX input file

[Joost Kremers]

On Wed, May 01 2019, Joey Dumont wrote:
> I'm not sure on where to start to create a MWE. My input file is 
> about 2000 
> lines long,

Bisecting. Copy your input file to a backup, then simply remove 
the second half and run it through Pandoc. If you run into the 
same problem, you know the cause is in the first half. Otherwise 
try the second half of your input file. Then split the part where 
the problem occurs into two halves again and run those through 
Pandoc. You should get to the part that causes the problem pretty 
quickly that way.

HTH

-- 
Joost Kremers
Life has its moments

Re: Debugging a seemingly infinitely loop when processing a LaTeX input file

[John MacFarlane]

Yes!  There is a --trace option which will give you
some idea of what the parser is doing.  This may or
may not help.

Joey Dumont <joey.dumont-Re5JQEeQqe8AvxtiuMwx3w@public.gmane.org> writes:

> pandoc 2.7.2, CentOS 7 (I downloaded the latest release from GitHub).
>
> I cannot convert to a native representation.
>
> I'm not sure on where to start to create a MWE. My input file is about 2000 
> lines long, so I was hoping that there was a way to know approximately 
> where the issue is happening in the file. That's why I didn't include a lot 
> of detail, I just wanted to know if there was a debugging feature in 
> pandoc. I couldn't find anything in the manual.

Re: Debugging a seemingly infinitely loop when processing a LaTeX input file

[Joey Dumont]

[-- Attachment #1.1: Type: text/plain, Size: 1891 bytes --]

Alright, bisecting worked pretty good. I'm so used to gcc telling me 
exactly where I need to look that I didn't think of doing that :)

I had multiple issues all stemming from what I think is a root cause: macro 
redefinitions. I had redefined figure and texttt, causing pandoc to not 
recognize the _ character.

I also learned that when using \mintinline{sql}{CREATE DATABASE 
<db_name>;}, everything should be on the same line, otherwise pandoc 
errors. 

Thanks for the --trace option tip, that's very helpful.

On Wednesday, May 1, 2019 at 12:15:20 PM UTC-4, John MacFarlane wrote:
>
>
> Yes!  There is a --trace option which will give you 
> some idea of what the parser is doing.  This may or 
> may not help. 
>
> Joey Dumont <joey....-Re5JQEeQqe8AvxtiuMwx3w@public.gmane.org <javascript:>> writes: 
>
> > pandoc 2.7.2, CentOS 7 (I downloaded the latest release from GitHub). 
> > 
> > I cannot convert to a native representation. 
> > 
> > I'm not sure on where to start to create a MWE. My input file is about 
> 2000 
> > lines long, so I was hoping that there was a way to know approximately 
> > where the issue is happening in the file. That's why I didn't include a 
> lot 
> > of detail, I just wanted to know if there was a debugging feature in 
> > pandoc. I couldn't find anything in the manual. 
>

-- 
You received this message because you are subscribed to the Google Groups "pandoc-discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email to pandoc-discuss+unsubscribe-/JYPxA39Uh5TLH3MbocFF+G/Ez6ZCGd0@public.gmane.org
To post to this group, send email to pandoc-discuss-/JYPxA39Uh5TLH3MbocFF+G/Ez6ZCGd0@public.gmane.org
To view this discussion on the web visit https://groups.google.com/d/msgid/pandoc-discuss/cab75b11-e6fa-46af-aae6-a6927b79d88e%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[-- Attachment #1.2: Type: text/html, Size: 2823 bytes --]

Re: Debugging a seemingly infinitely loop when processing a LaTeX input file

[John MacFarlane]

Joey Dumont <joey.dumont-Re5JQEeQqe8AvxtiuMwx3w@public.gmane.org> writes:

> I also learned that when using \mintinline{sql}{CREATE DATABASE 
> <db_name>;}, everything should be on the same line, otherwise pandoc 
> errors. 

That is surprising to hear, can you give a minimal
test case to reproduce?

Re: Debugging a seemingly infinitely loop when processing a LaTeX input file

[Joey Dumont]

[-- Attachment #1.1: Type: text/plain, Size: 1280 bytes --]

\documentclass{article}

\usepackage{minted}

\begin{document}

\mintinline{sql}{CREATE DATABASE 
cb_keys;}

\end{document}

yields

$ pandoc pandoc_test.tex -o pandoc_test.native

Error at "source" (line 7, column 33):
unexpected 

\mintinline{sql}{CREATE DATABASE
                                ^





On Wednesday, May 1, 2019 at 1:56:35 PM UTC-4, John MacFarlane wrote:
>
> Joey Dumont <joey....-Re5JQEeQqe8AvxtiuMwx3w@public.gmane.org <javascript:>> writes: 
>
> > I also learned that when using \mintinline{sql}{CREATE DATABASE 
> > <db_name>;}, everything should be on the same line, otherwise pandoc 
> > errors. 
>
> That is surprising to hear, can you give a minimal 
> test case to reproduce? 
>

-- 
You received this message because you are subscribed to the Google Groups "pandoc-discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email to pandoc-discuss+unsubscribe-/JYPxA39Uh5TLH3MbocFF+G/Ez6ZCGd0@public.gmane.org
To post to this group, send email to pandoc-discuss-/JYPxA39Uh5TLH3MbocFF+G/Ez6ZCGd0@public.gmane.org
To view this discussion on the web visit https://groups.google.com/d/msgid/pandoc-discuss/46c7ed1e-607b-4348-bcda-3e8dfa281320%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[-- Attachment #1.2: Type: text/html, Size: 6331 bytes --]

Re: Debugging a seemingly infinitely loop when processing a LaTeX input file

[John MacFarlane]

Thanks, I've pushed a fix.

Joey Dumont <joey.dumont-Re5JQEeQqe8AvxtiuMwx3w@public.gmane.org> writes:

> \documentclass{article}
>
> \usepackage{minted}
>
> \begin{document}
>
> \mintinline{sql}{CREATE DATABASE 
> cb_keys;}
>
> \end{document}
>
> yields
>
> $ pandoc pandoc_test.tex -o pandoc_test.native
>
> Error at "source" (line 7, column 33):
> unexpected 
>
> \mintinline{sql}{CREATE DATABASE
>                                 ^
>
>
>
>
>
> On Wednesday, May 1, 2019 at 1:56:35 PM UTC-4, John MacFarlane wrote:
>>
>> Joey Dumont <joey....-Re5JQEeQqe8AvxtiuMwx3w@public.gmane.org <javascript:>> writes: 
>>
>> > I also learned that when using \mintinline{sql}{CREATE DATABASE 
>> > <db_name>;}, everything should be on the same line, otherwise pandoc 
>> > errors. 
>>
>> That is surprising to hear, can you give a minimal 
>> test case to reproduce? 
>>
>
> -- 
> You received this message because you are subscribed to the Google Groups "pandoc-discuss" group.
> To unsubscribe from this group and stop receiving emails from it, send an email to pandoc-discuss+unsubscribe-/JYPxA39Uh5TLH3MbocFF+G/Ez6ZCGd0@public.gmane.org
> To post to this group, send email to pandoc-discuss-/JYPxA39Uh5TLH3MbocFF+G/Ez6ZCGd0@public.gmane.org
> To view this discussion on the web visit https://groups.google.com/d/msgid/pandoc-discuss/46c7ed1e-607b-4348-bcda-3e8dfa281320%40googlegroups.com.
> For more options, visit https://groups.google.com/d/optout.