Pandoc doesn't define references in PDF output with --biblatex (or --natbib)

Pandoc doesn't define references in PDF output with --biblatex (or --natbib)

[per.review-cl+VPiYnx/1AfugRpC6u6w]

I want to create a PDF file with a bibliography, e.g. `echo "See @test."
| pandoc -o min.pdf --bibliography=min.bib --biblatex`

With `min.bib`:

~~~
@article{test,
   author = "Tester, T.",
   title = "Test",
   journal = "Testing",
   year = "2018",
}
~~~

With the `--verbose` option I can see that Pandoc generates a LaTeX
document with

~~~
\usepackage[]{biblatex}
\addbibresource{min.bib}

\date{}

\begin{document}

See \textcite{test}.

\printbibliography

\end{document}
~~~

It runs makePDF three times, and every time it says (some empty lines
removed):

~~~
No file input.bbl.

LaTeX Warning: Citation 'test' on page 1 undefined on input line 61.

LaTeX Warning: Empty bibliography on input line 63.

[1{/var/lib/texmf/fonts/map/pdftex/updmap/pdftex.map}]
(./tex2pdf.4369/input.aux)

LaTeX Warning: There were undefined references.

Package biblatex Warning: Please (re)run Biber on the file:
(biblatex)                input
(biblatex)                and rerun LaTeX afterwards.
~~~

Unsurprisingly, the citation remains undefined. Biber doesn't run. The
same happens when I run Pandoc with `--natbib` instead of `--biblatex`,
or with `--pdf-engine=xelatex`. (With pandoc-citeproc it works, since
the bibliography is already generated in the .tex code makePDF starts
out with.)

What is going on?

Having pandoc output a .tex file instead of a PDF document, then running
xelatex, biber and xelatex works as expected. (This is not an option for
my more complicated real-world example, which also depends on pandoc to
convert figures for the PDF output and on pandoc-crossref, among other
things.)

I'm pretty sure this wasn't a problem until recently, but I don't know
when it arose. I'm using the TeX Live distribution with Linux, but the
problem is identical with MiKTeX in Windows.

~~~
$ pandoc -v
pandoc 2.2.1
Compiled with pandoc-types 1.17.4.2, texmath 0.10.1.1, skylighting 0.6
$ biber -v
biber version: 2.11
~~~

If this warrants a bug report on Github, feel free to make it into one.
I can't get rid of my suspicion that it's simply a basic human error on
my side.

-- 
Per

Re: Pandoc doesn't define references in PDF output with --biblatex (or --natbib)

[John MacFarlane]

Pandoc doesn't run bibtex/biber for you when producing
a PDF.  If you use pandoc-citeproc, you can get a nice
bibliography in PDF output (and maybe that's what you
were doing when it worked before), but if you want to use
native biblatex, you'll have to produce a .tex file
and run latex and biber yourself.

Note:  it's helpful to search closed issues before
submitting a report.  You would have found #4105,
#1193, #681, #2795, ...

But perhaps pandoc should issue a warning when you
use --natbib or --biblatex with output to a PDF file.

per.review-cl+VPiYnx/1AfugRpC6u6w@public.gmane.org writes:

> I want to create a PDF file with a bibliography, e.g. `echo "See @test."
> | pandoc -o min.pdf --bibliography=min.bib --biblatex`
>
> With `min.bib`:
>
> ~~~
> @article{test,
>    author = "Tester, T.",
>    title = "Test",
>    journal = "Testing",
>    year = "2018",
> }
> ~~~
>
> With the `--verbose` option I can see that Pandoc generates a LaTeX
> document with
>
> ~~~
> \usepackage[]{biblatex}
> \addbibresource{min.bib}
>
> \date{}
>
> \begin{document}
>
> See \textcite{test}.
>
> \printbibliography
>
> \end{document}
> ~~~
>
> It runs makePDF three times, and every time it says (some empty lines
> removed):
>
> ~~~
> No file input.bbl.
>
> LaTeX Warning: Citation 'test' on page 1 undefined on input line 61.
>
> LaTeX Warning: Empty bibliography on input line 63.
>
> [1{/var/lib/texmf/fonts/map/pdftex/updmap/pdftex.map}]
> (./tex2pdf.4369/input.aux)
>
> LaTeX Warning: There were undefined references.
>
> Package biblatex Warning: Please (re)run Biber on the file:
> (biblatex)                input
> (biblatex)                and rerun LaTeX afterwards.
> ~~~
>
> Unsurprisingly, the citation remains undefined. Biber doesn't run. The
> same happens when I run Pandoc with `--natbib` instead of `--biblatex`,
> or with `--pdf-engine=xelatex`. (With pandoc-citeproc it works, since
> the bibliography is already generated in the .tex code makePDF starts
> out with.)
>
> What is going on?
>
> Having pandoc output a .tex file instead of a PDF document, then running
> xelatex, biber and xelatex works as expected. (This is not an option for
> my more complicated real-world example, which also depends on pandoc to
> convert figures for the PDF output and on pandoc-crossref, among other
> things.)
>
> I'm pretty sure this wasn't a problem until recently, but I don't know
> when it arose. I'm using the TeX Live distribution with Linux, but the
> problem is identical with MiKTeX in Windows.
>
> ~~~
> $ pandoc -v
> pandoc 2.2.1
> Compiled with pandoc-types 1.17.4.2, texmath 0.10.1.1, skylighting 0.6
> $ biber -v
> biber version: 2.11
> ~~~
>
> If this warrants a bug report on Github, feel free to make it into one.
> I can't get rid of my suspicion that it's simply a basic human error on
> my side.
>
> -- 
> Per
>
> -- 
> You received this message because you are subscribed to the Google Groups "pandoc-discuss" group.
> To unsubscribe from this group and stop receiving emails from it, send an email to pandoc-discuss+unsubscribe-/JYPxA39Uh5TLH3MbocFF+G/Ez6ZCGd0@public.gmane.org
> To post to this group, send email to pandoc-discuss-/JYPxA39Uh5TLH3MbocFF+G/Ez6ZCGd0@public.gmane.org
> To view this discussion on the web visit https://groups.google.com/d/msgid/pandoc-discuss/20180612145052.GA8444%40mailbox.org.
> For more options, visit https://groups.google.com/d/optout.

Re: Pandoc doesn't define references in PDF output with --biblatex (or --natbib)

[John MacFarlane]

Note also that the man page for --biblatex says: "This
option is not for use with the pandoc-citeproc filter
or with PDF output. It is intended for producing a
LaTeX file that can be processed with biblatex or
biber.

Re: Pandoc doesn't define references in PDF output with --biblatex (or --natbib)

[per.review-cl+VPiYnx/1AfugRpC6u6w]

John MacFarlane <jgm-TVLZxgkOlNX2fBVCVOL8/A@public.gmane.org> [2018-06-12 18:02 +0200]:
> Pandoc doesn't run bibtex/biber for you when producing
> a PDF.  If you use pandoc-citeproc, you can get a nice
> bibliography in PDF output (and maybe that's what you
> were doing when it worked before), but if you want to use
> native biblatex, you'll have to produce a .tex file
> and run latex and biber yourself.
> 
> Note:  it's helpful to search closed issues before
> submitting a report.  You would have found #4105,
> #1193, #681, #2795, ...

Thanks for the quick answer! I saw the explanation of the --biblatex
option in the manual, but was so certain I could remember Pandoc doing
this. Turns my usage of rmarkdown/knitr, which *does* handle this, had
contaminated my memory (and led me to only look for recent issues).

However, Pandoc(-citeproc)'s implementation of CSL styles still has a
long way to go in terms of customisation and styling. If the PDF output
has indented paragraphs, bibliography entries will be styled the same
way. I've posted a hack to deal with this:
https://github.com/jgm/pandoc/issues/335#issuecomment-396739093 I don't
know what you think about including anything like in the LaTeX template?

(I understand that better support for CSL attributes are not a priority,
but I think it's a problem when something as simple a paragraph
indentation breaks the styling.)

-- 
Per

Re: Pandoc doesn't define references in PDF output with --biblatex (or --natbib)

[John MacFarlane]

pandoc-citeproc wraps the whole bibliography in a Div
element with class "references."

This is currently just ignored by the LaTeX writer,
but you can use a lua filter to insert a custom
environment or some custom commands.

We might think about having the LaTeX writer do
something here by default.  But what?  The most
flexible thing would be inserting the whole thing
into a special environment, but then we'd need to
add something to the default template to make sure
this environment is defined.

per.review-cl+VPiYnx/1AfugRpC6u6w@public.gmane.org writes:

> John MacFarlane <jgm-TVLZxgkOlNX2fBVCVOL8/A@public.gmane.org> [2018-06-12 18:02 +0200]:
>> Pandoc doesn't run bibtex/biber for you when producing
>> a PDF.  If you use pandoc-citeproc, you can get a nice
>> bibliography in PDF output (and maybe that's what you
>> were doing when it worked before), but if you want to use
>> native biblatex, you'll have to produce a .tex file
>> and run latex and biber yourself.
>> 
>> Note:  it's helpful to search closed issues before
>> submitting a report.  You would have found #4105,
>> #1193, #681, #2795, ...
>
> Thanks for the quick answer! I saw the explanation of the --biblatex
> option in the manual, but was so certain I could remember Pandoc doing
> this. Turns my usage of rmarkdown/knitr, which *does* handle this, had
> contaminated my memory (and led me to only look for recent issues).
>
> However, Pandoc(-citeproc)'s implementation of CSL styles still has a
> long way to go in terms of customisation and styling. If the PDF output
> has indented paragraphs, bibliography entries will be styled the same
> way. I've posted a hack to deal with this:
> https://github.com/jgm/pandoc/issues/335#issuecomment-396739093 I don't
> know what you think about including anything like in the LaTeX template?
>
> (I understand that better support for CSL attributes are not a priority,
> but I think it's a problem when something as simple a paragraph
> indentation breaks the styling.)
>
> -- 
> Per
>
> -- 
> You received this message because you are subscribed to the Google Groups "pandoc-discuss" group.
> To unsubscribe from this group and stop receiving emails from it, send an email to pandoc-discuss+unsubscribe-/JYPxA39Uh5TLH3MbocFF+G/Ez6ZCGd0@public.gmane.org
> To post to this group, send email to pandoc-discuss-/JYPxA39Uh5TLH3MbocFF+G/Ez6ZCGd0@public.gmane.org
> To view this discussion on the web visit https://groups.google.com/d/msgid/pandoc-discuss/20180612214644.GB8444%40mailbox.org.
> For more options, visit https://groups.google.com/d/optout.

Re: Pandoc doesn't define references in PDF output with --biblatex (or --natbib)

['Nick Bart' via pandoc-discuss]

I have been using a Haskell filter for years now (see https://github.com/njbart/pandoc-tools/blob/master/addRefEnvItem.hs) to replace the Div element with class "references" by a LaTeX "references" environment, and to prepend "\item" to each block.

(Traditionally, both bibtex and biblatex use such itemized lists for the references section, and LaTeX itself uses this internally for "quote" and many other environments -- so using "\item" is nothing particularly unusual.)

I’m defining the LaTeX "references" environment in the template as follows:

\newenvironment{references} {\list{}{%
    \leftmargin1.5em%
    \itemindent-\leftmargin%
$if(indent)$
    \itemsep0ex%
$else$
    \itemsep\parsep%
$endif$
    \parsep0pt%
    \setlength{\topsep}{0pt}
    \setlength{\partopsep}{0pt}
    }}
    {\endlist}

I think it’d be great if the LaTeX writer could do this by default.

Note that my definition always creates a hanging indent. It might be useful if pandoc set a "hanging" (or "csl-hanging-indent") variable that could be evaluated in LaTeX (and possibly other) templates.

As to the other CSL "Whitespace" options (see http://docs.citationstyles.org/en/1.0.1/specification.html#whitespace), "line-spacing" and "entry-spacing" could be made to be set via a variable, too. I tend to think this could be made to work for "second-field-align", too, though I’m not sure at the moment how exactly the list environment would have to look like to make this somewhat table-like alignment possible.

‐‐‐‐‐‐‐ Original Message ‐‐‐‐‐‐‐

On June 13, 2018 5:28 PM, John MacFarlane <jgm-TVLZxgkOlNX2fBVCVOL8/A@public.gmane.org> wrote:

> pandoc-citeproc wraps the whole bibliography in a Div
>
> element with class "references."
>
> This is currently just ignored by the LaTeX writer,
>
> but you can use a lua filter to insert a custom
>
> environment or some custom commands.
>
> We might think about having the LaTeX writer do
>
> something here by default. But what? The most
>
> flexible thing would be inserting the whole thing
>
> into a special environment, but then we'd need to
>
> add something to the default template to make sure
>
> this environment is defined.
>
> per.review-cl+VPiYnx/1AfugRpC6u6w@public.gmane.org writes:
>
> > John MacFarlane jgm-TVLZxgkOlNX2fBVCVOL8/A@public.gmane.org [2018-06-12 18:02 +0200]:
> >
> > > Pandoc doesn't run bibtex/biber for you when producing
> > >
> > > a PDF. If you use pandoc-citeproc, you can get a nice
> > >
> > > bibliography in PDF output (and maybe that's what you
> > >
> > > were doing when it worked before), but if you want to use
> > >
> > > native biblatex, you'll have to produce a .tex file
> > >
> > > and run latex and biber yourself.
> > >
> > > Note: it's helpful to search closed issues before
> > >
> > > submitting a report. You would have found #4105,
> > >
> > > #1193, #681, #2795, ...
> >
> > Thanks for the quick answer! I saw the explanation of the --biblatex
> >
> > option in the manual, but was so certain I could remember Pandoc doing
> >
> > this. Turns my usage of rmarkdown/knitr, which does handle this, had
> >
> > contaminated my memory (and led me to only look for recent issues).
> >
> > However, Pandoc(-citeproc)'s implementation of CSL styles still has a
> >
> > long way to go in terms of customisation and styling. If the PDF output
> >
> > has indented paragraphs, bibliography entries will be styled the same
> >
> > way. I've posted a hack to deal with this:
> >
> > https://github.com/jgm/pandoc/issues/335#issuecomment-396739093 I don't
> >
> > know what you think about including anything like in the LaTeX template?
> >
> > (I understand that better support for CSL attributes are not a priority,
> >
> > but I think it's a problem when something as simple a paragraph
> >
> > indentation breaks the styling.)
> >
> > --
> >
> > Per
> >
> > --
> >
> > You received this message because you are subscribed to the Google Groups "pandoc-discuss" group.
> >
> > To unsubscribe from this group and stop receiving emails from it, send an email to pandoc-discuss+unsubscribe-/JYPxA39Uh5TLH3MbocFF+G/Ez6ZCGd0@public.gmane.org
> >
> > To post to this group, send email to pandoc-discuss-/JYPxA39Uh5TLH3MbocFF+G/Ez6ZCGd0@public.gmane.org
> >
> > To view this discussion on the web visit https://groups.google.com/d/msgid/pandoc-discuss/20180612214644.GB8444%40mailbox.org.
> >
> > For more options, visit https://groups.google.com/d/optout.
>
> --
>
> You received this message because you are subscribed to the Google Groups "pandoc-discuss" group.
>
> To unsubscribe from this group and stop receiving emails from it, send an email to pandoc-discuss+unsubscribe-/JYPxA39Uh5TLH3MbocFF+G/Ez6ZCGd0@public.gmane.org
>
> To post to this group, send email to pandoc-discuss-/JYPxA39Uh5TLH3MbocFF+G/Ez6ZCGd0@public.gmane.org
>
> To view this discussion on the web visit https://groups.google.com/d/msgid/pandoc-discuss/m2fu1qzjdf.fsf%40johnmacfarlane.net.
>
> For more options, visit https://groups.google.com/d/optout.


-- 
You received this message because you are subscribed to the Google Groups "pandoc-discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email to pandoc-discuss+unsubscribe-/JYPxA39Uh5TLH3MbocFF+G/Ez6ZCGd0@public.gmane.org
To post to this group, send email to pandoc-discuss-/JYPxA39Uh5TLH3MbocFF+G/Ez6ZCGd0@public.gmane.org
To view this discussion on the web visit https://groups.google.com/d/msgid/pandoc-discuss/Qg6kaM969Hyzb8mp4AWqajN8RTCullV-EevCrKIItI4aZRBOWydJ--tFaW69q744i99DiydmIKeIvthe8iFIu7ghAjZ4STy8dyi9FAmSs1U%3D%40protonmail.com.
For more options, visit https://groups.google.com/d/optout.