Re: pandoc/citeproc issues: multiple bibliographies, nocite, citeonly

[John MacFarlane <fiddlosopher-Re5JQEeQqe8AvxtiuMwx3w@public.gmane.org>]

+++ Andrea Rossato [Dec 01 10 14:03 ]:
> On Tue, Nov 30, 2010 at 07:25:56PM -0800, John MacFarlane wrote:
> > I guess I'm tempted to:
> > - allow multiple bibliographies
> 
> That would be easy, indeed.
> 
> > - specify the source file in the markdown text, as above
> 
> keep in mind that multiple bibliographic databases should be allowed
> (for instance to mix bibtex and json data).
> 
> > I'm not positive this will work, though. As I mentioned, there are technical
> > hurdles to having the bibliography file specified in the text. The markdown
> > reader itself can't do IO, so it can't read the file. So we can't read the
> > bibliography (or even verify that it can be found) until we've parsed the
> > markdown source. That means that we can't check potential citations as we
> > parse, to see if they're in the bibliography; instead, we have to parse
> > everything that might be a citation as a citation. But what if we have a
> > citation with id "foo" (from "@foo" in the text), but the bibliography
> > contains no corresponding item? Then we need to reinsert the literal text.
> > It's easy enough to generate "@foo" from a Cite inline, but what if the Cite
> > was generated by parsing latex + bibtex? This gets complicated.
> > 
> > I did think of one solution, which I'd like to get Andrea's feedback on (as
> > it might require citeproc changes). Currently when we parse a citation, we
> > just leave the [Inline] part empty; this gets filled in by citeproc when it
> > processes the citations with the bibliography.
> > 
> > My suggestion: instead of leaving it empty, fill in the [Inline] part of the
> > Cite with the literal text that should be included in the document if the
> > citation isn't found in the database. This way, if citeproc doesn't find the
> > item, it can simply leave the Cite alone (rather than raising an error). That
> > seems simple; it just adds a bit of complexity to the parsers, which have to
> > generate the "replacement text" when they parse a Cite.
> > 
> > Thoughts?
> 
> That would be fine with me. But I just wanted to let you know, though,
> that I modified citeproc so that, when a citation is not found in the
> bibliographic data, an error is emitted:
> 
>    "[CSL BIBLIOGRAPHIC DATA ERROR: reference " ++ show (citationId c) ++ " not found.]"

This is fine the way the code currently works, since pandoc *shouldn't*
be passing citations that aren't in the bibliography to citeproc.
But it would have to be changed if we did things the way I sketched
above.  With this approach, citeproc would simply leave Citations alone
if the entry was not found. (There might be difficulties I'm not
anticipating; I don't know the code.)

> I don't know if this is going to solve your problem.
> 
> More generally I'd prefer the metadata approach you were talking
> about. If this requires more time to get mature I think we could now
> stick with the present approach: a single bibliography automatically
> added at the end of the Pandoc document, and multiple bibliographic
> databases via the command line --bibliography option.
> 
> That would leave us free to take every possible path for implementing
> multiple bibliographies and for setting the source biblio data in the
> source document.

The only drawback I see to proceeding this way is that these future
changes would require changes in existing documents, unless we kept
the --bibliography option in addition to whatever mechanism we come
up with for multiply bibliographies and in-text specification of
the source files.

But maybe we could just do that:  keep --bibliography for legacy
files.  That would allow us to do a release sooner, without settling
all of these issues.

John