Citeproc-lua

Citeproc-lua

[Denis Maier via ntg-context]

[-- Attachment #1.1: Type: text/plain, Size: 466 bytes --]

Hi,

I just came across this: https://github.com/zepinglee/citeproc-lua
Thats's an lua implemenation of the CSL language targeting LaTeX.
I don't know if the processor itself contains any LaTeX specific code, but it looks like the integration basically happens here : https://github.com/zepinglee/citeproc-lua/blob/main/latex/citation-style-language.sty
So, at least in theory, shouldn't it be possible to make this workable also with context?

Best,
Denis

[-- Attachment #1.2: Type: text/html, Size: 2529 bytes --]

[-- Attachment #2: Type: text/plain, Size: 493 bytes --]

___________________________________________________________________________________
If your question is of interest to others as well, please add an entry to the Wiki!

maillist : ntg-context@ntg.nl / http://www.ntg.nl/mailman/listinfo/ntg-context
webpage  : http://www.pragma-ade.nl / http://context.aanhet.net
archive  : https://bitbucket.org/phg/context-mirror/commits/
wiki     : http://contextgarden.net
___________________________________________________________________________________

Re: Citeproc-lua

[Taco Hoekwater via ntg-context]

Hi,

I had a quick look at this and it seems very extensible if-not-quite ConTeXt-ready right away.

On the input side, it reads a LaTeX .aux file to parse things like options, databases, and requested citations.
For this, it uses a dedicated “citeproc” lua script that should be easy to modify/copy for someone who knows what
is actually in the ConTeXt .tuc file.

On the output side, it uses a lookup table to generate the desired markup (in citeproc-formats.lua). Right now,
there are definitions provided for HTML and LaTeX, but that is very simple to extend.

There is a LaTeX package for the integration (making sure the right commands appear in the aux file, 
option processing, et cetera), and that file I do not understand *at all*. It seems to be completely isolated 
from the core citeproc processing, though.

Very promising stuff, I think. 

But: I personally do not understand how to do the integration with ConTeXt’s new biblio system. I only understand
half of what ConTeXt does these days, and much less of what the LaTeX package is trying to achieve i.r.t. options.

Best wishes,
Taco

> On 3 May 2022, at 22:00, Denis Maier via ntg-context <ntg-context@ntg.nl> wrote:
> 
> Hi,
> 
> I just came across this: https://github.com/zepinglee/citeproc-lua
> Thats’s an lua implemenation of the CSL language targeting LaTeX.
> I don’t know if the processor itself contains any LaTeX specific code, but it looks like the integration basically happens here :https://github.com/zepinglee/citeproc-lua/blob/main/latex/citation-style-language.sty
> So, at least in theory, shouldn’t it be possible to make this workable also with context?
> 
> Best,
> Denis 
> ___________________________________________________________________________________
> If your question is of interest to others as well, please add an entry to the Wiki!
> 
> maillist : ntg-context@ntg.nl / http://www.ntg.nl/mailman/listinfo/ntg-context
> webpage  : http://www.pragma-ade.nl / http://context.aanhet.net
> archive  : https://bitbucket.org/phg/context-mirror/commits/
> wiki     : http://contextgarden.net
> ___________________________________________________________________________________

Taco Hoekwater
Elvenkind BV




___________________________________________________________________________________
If your question is of interest to others as well, please add an entry to the Wiki!

maillist : ntg-context@ntg.nl / http://www.ntg.nl/mailman/listinfo/ntg-context
webpage  : http://www.pragma-ade.nl / http://context.aanhet.net
archive  : https://bitbucket.org/phg/context-mirror/commits/
wiki     : http://contextgarden.net
___________________________________________________________________________________

Re: Citeproc-lua

[Hans Hagen via ntg-context]

On 5/6/2022 2:57 PM, Taco Hoekwater via ntg-context wrote:

> But: I personally do not understand how to do the integration with ConTeXt’s new biblio system. I only understand
> half of what ConTeXt does these days, and much less of what the LaTeX package is trying to achieve i.r.t. options.
i'll have a look at the citeproc spec to see how far away it is from 
what we have

Hans


-----------------------------------------------------------------
                                           Hans Hagen | PRAGMA ADE
               Ridderstraat 27 | 8061 GH Hasselt | The Netherlands
        tel: 038 477 53 69 | www.pragma-ade.nl | www.pragma-pod.nl
-----------------------------------------------------------------
___________________________________________________________________________________
If your question is of interest to others as well, please add an entry to the Wiki!

maillist : ntg-context@ntg.nl / http://www.ntg.nl/mailman/listinfo/ntg-context
webpage  : http://www.pragma-ade.nl / http://context.aanhet.net
archive  : https://bitbucket.org/phg/context-mirror/commits/
wiki     : http://contextgarden.net
___________________________________________________________________________________

Re: Citeproc-lua

[Denis Maier via ntg-context]

Hi,
thanks for your assessment. Indeed, that looks very promising.

I've now had a chance to look at this a bit more too.  Not that I understand that stuff in detail, but anyway...

I also have the impression that extending citeproc-formats.lua should be the easiest part. As an aside, I think the HTML is in there because that is needed for the test suite, but for ConTeXt we should be able to copy the LaTeX mapping and tweak a few things here and there.

The LaTeX package is hard to understand, at least if you're not used to the "new" latex3 programming syntax and style. I also can only guess what that stuff does. But as you say this is rather isolated from the citeproc processing, so I guess the main task would be to come up with an equivalent ConTeXt module that writes the citations to the tuc file, calls the citeproc engine, and uses the results for the output. Right? (I.e., AFAICS, everything that lives under https://github.com/zepinglee/citeproc-lua/tree/main/citeproc should pretty much just work, while https://github.com/zepinglee/citeproc-lua/tree/main/latex must be adapted.)

For the record: the relevant stuff in the aux file looks like this:
\bibstyle {apa}
\bibdata {examples.bib}
\csloptions {}
\citation {{texbook@1}{{id={texbook},label={page},locator={12}}}{1}}
\citation {{latex2e@1}{{id={latex2e},prefix={see }}}{1}}
\citation {{texbook@2}{{id={texbook},prefix={see },label={page},locator={12}}}{1}}
\citation {{texbook@3}{{id={texbook},prefix={see },suffix={, everywhere}}}{1}}
\citation {{texbook@4}{{id={texbook},prefix={see },label={page},locator={12, and passim}}}{1}}

But I guess that's the way these things work in latex, and adding that stuff to a lua table will probably save us much trouble anyway. (E.g., no need for https://github.com/zepinglee/citeproc-lua/blob/main/latex/citeproc#L81, or https://github.com/zepinglee/citeproc-lua/blob/main/latex/csl-core.lua#L168)

Regarding the citation commands, I've noticed that while biblatex's syntax model (\cite[prenote][postnote]{key}, e.g. \cite[see][34]{doe}) is supported, the preferred method is actually \cite[prefix={See }, page={34}]{doe}. But that's of course not so important since the important question is how that stuff is passed to the citeproc.

Best,
Denis



> -----Ursprüngliche Nachricht-----
> Von: Taco Hoekwater <taco@elvenkind.com>
> Gesendet: Freitag, 6. Mai 2022 14:57
> An: mailing list for ConTeXt users <ntg-context@ntg.nl>
> Cc: Maier, Denis Christian (UB) <denis.maier@unibe.ch>
> Betreff: Re: [NTG-context] Citeproc-lua
> 
> Hi,
> 
> I had a quick look at this and it seems very extensible if-not-quite ConTeXt-
> ready right away.
> 
> On the input side, it reads a LaTeX .aux file to parse things like options,
> databases, and requested citations.
> For this, it uses a dedicated “citeproc” lua script that should be easy to
> modify/copy for someone who knows what is actually in the ConTeXt .tuc
> file.
> 
> On the output side, it uses a lookup table to generate the desired markup (in
> citeproc-formats.lua). Right now, there are definitions provided for HTML
> and LaTeX, but that is very simple to extend.
> 
> There is a LaTeX package for the integration (making sure the right
> commands appear in the aux file, option processing, et cetera), and that file I
> do not understand *at all*. It seems to be completely isolated from the core
> citeproc processing, though.
> 
> Very promising stuff, I think.
> 
> But: I personally do not understand how to do the integration with ConTeXt’s
> new biblio system. I only understand half of what ConTeXt does these days,
> and much less of what the LaTeX package is trying to achieve i.r.t. options.
> 
> Best wishes,
> Taco
> 
> > On 3 May 2022, at 22:00, Denis Maier via ntg-context <ntg-
> context@ntg.nl> wrote:
> >
> > Hi,
> >
> > I just came across this: https://github.com/zepinglee/citeproc-lua
> > Thats’s an lua implemenation of the CSL language targeting LaTeX.
> > I don’t know if the processor itself contains any LaTeX specific code,
> > but it looks like the integration basically happens here
> > :https://github.com/zepinglee/citeproc-lua/blob/main/latex/citation-st
> > yle-language.sty So, at least in theory, shouldn’t it be possible to
> > make this workable also with context?
> >
> > Best,
> > Denis
> >
> ______________________________________________________________
> ________
> > _____________ If your question is of interest to others as well,
> > please add an entry to the Wiki!
> >
> > maillist : ntg-context@ntg.nl /
> > http://www.ntg.nl/mailman/listinfo/ntg-context
> > webpage  : http://www.pragma-ade.nl / http://context.aanhet.net
> > archive  : https://bitbucket.org/phg/context-mirror/commits/
> > wiki     : http://contextgarden.net
> >
> ______________________________________________________________
> ________
> > _____________
> 
> Taco Hoekwater
> Elvenkind BV
> 
> 
> 

___________________________________________________________________________________
If your question is of interest to others as well, please add an entry to the Wiki!

maillist : ntg-context@ntg.nl / http://www.ntg.nl/mailman/listinfo/ntg-context
webpage  : http://www.pragma-ade.nl / http://context.aanhet.net
archive  : https://bitbucket.org/phg/context-mirror/commits/
wiki     : http://contextgarden.net
___________________________________________________________________________________

Re: Citeproc-lua

[Aditya Mahajan via ntg-context]

On Fri, 6 May 2022, Denis Maier via ntg-context wrote:

> Hi,
> thanks for your assessment. Indeed, that looks very promising.
> 
> I've now had a chance to look at this a bit more too.  Not that I understand that stuff in detail, but anyway...
> 
> I also have the impression that extending citeproc-formats.lua should be the easiest part. As an aside, I think the HTML is in there because that is needed for the test suite, but for ConTeXt we should be able to copy the LaTeX mapping and tweak a few things here and there.
>
> The LaTeX package is hard to understand, at least if you're not used to the "new" latex3 programming syntax and style. I also can only guess what that stuff does. But as you say this is rather isolated from the citeproc processing, so I guess the main task would be to come up with an equivalent ConTeXt module that writes the citations to the tuc file, calls the citeproc engine, and uses the results for the output. Right? (I.e., AFAICS, everything that lives under https://github.com/zepinglee/citeproc-lua/tree/main/citeproc should pretty much just work, while https://github.com/zepinglee/citeproc-lua/tree/main/latex must be adapted.)

Does citeproc-lua/biblatex support using CSL files (https://citationstyles.org/). IMO, that is currently the biggest limitation of the context bib module that only a few citation styles are supported. Developing a new citation style is relatively easy but a lot of work. Being able to use CSL will simplify things (from the point of view of the user) a lot.

Aditya
___________________________________________________________________________________
If your question is of interest to others as well, please add an entry to the Wiki!

maillist : ntg-context@ntg.nl / http://www.ntg.nl/mailman/listinfo/ntg-context
webpage  : http://www.pragma-ade.nl / http://context.aanhet.net
archive  : https://bitbucket.org/phg/context-mirror/commits/
wiki     : http://contextgarden.net
___________________________________________________________________________________

Re: Citeproc-lua

[Hans Hagen via ntg-context]

On 5/6/2022 11:09 PM, Aditya Mahajan via ntg-context wrote:
> 
> 
> On Fri, 6 May 2022, Denis Maier via ntg-context wrote:
> 
>> Hi,
>> thanks for your assessment. Indeed, that looks very promising.
>>
>> I've now had a chance to look at this a bit more too.  Not that I understand that stuff in detail, but anyway...
>>
>> I also have the impression that extending citeproc-formats.lua should be the easiest part. As an aside, I think the HTML is in there because that is needed for the test suite, but for ConTeXt we should be able to copy the LaTeX mapping and tweak a few things here and there.
>>
>> The LaTeX package is hard to understand, at least if you're not used to the "new" latex3 programming syntax and style. I also can only guess what that stuff does. But as you say this is rather isolated from the citeproc processing, so I guess the main task would be to come up with an equivalent ConTeXt module that writes the citations to the tuc file, calls the citeproc engine, and uses the results for the output. Right? (I.e., AFAICS, everything that lives under https://github.com/zepinglee/citeproc-lua/tree/main/citeproc should pretty much just work, while https://github.com/zepinglee/citeproc-lua/tree/main/latex must be adapted.)
> 
> Does citeproc-lua/biblatex support using CSL files (https://citationstyles.org/). IMO, that is currently the biggest limitation of the context bib module that only a few citation styles are supported. Developing a new citation style is relatively easy but a lot of work. Being able to use CSL will simplify things (from the point of view of the user) a lot.

the main question is 'how easy is it to convertt one of these xml defs 
into a context rendering' ... probably somethng we can do stepwise

using some external solution creates a dependency (and also maintainance 
issue i guess) and there is a good change for interferences

(btw, in the apa one that we support the main complication is that it 
has many variants / border cases)

Hans

-----------------------------------------------------------------
                                           Hans Hagen | PRAGMA ADE
               Ridderstraat 27 | 8061 GH Hasselt | The Netherlands
        tel: 038 477 53 69 | www.pragma-ade.nl | www.pragma-pod.nl
-----------------------------------------------------------------
___________________________________________________________________________________
If your question is of interest to others as well, please add an entry to the Wiki!

maillist : ntg-context@ntg.nl / http://www.ntg.nl/mailman/listinfo/ntg-context
webpage  : http://www.pragma-ade.nl / http://context.aanhet.net
archive  : https://bitbucket.org/phg/context-mirror/commits/
wiki     : http://contextgarden.net
___________________________________________________________________________________

Re: Citeproc-lua

[Hans Hagen via ntg-context]

On 5/6/2022 11:02 PM, Denis Maier via ntg-context wrote:

> I also have the impression that extending citeproc-formats.lua should be the easiest part. As an aside, I think the HTML is in there because that is needed for the test suite, but for ConTeXt we should be able to copy the LaTeX mapping and tweak a few things here and there.

> The LaTeX package is hard to understand, at least if you're not used to the "new" latex3 programming syntax and style. I also can only guess what that stuff does. But as you say this is rather isolated from the citeproc processing, so I guess the main task would be to come up with an equivalent ConTeXt module that writes the citations to the tuc file, calls the citeproc engine, and uses the results for the output. Right? (I.e., AFAICS, everything that lives under https://github.com/zepinglee/citeproc-lua/tree/main/citeproc should pretty much just work, while https://github.com/zepinglee/citeproc-lua/tree/main/latex must be adapted.)

For the record: i don't think you should write something to the tuc file 
that doesn't come from context itself because you can mess it up (also 
performaance wise). Of course you could have another file (just like a 
bib file is independent). One problem could be that you need to make 
some extra installation to make it work as we're not going to add all 
kind of code to the distribution (we tend to go smaller) and someone 
needs to maintain that moduie then because users depend in it working.

Did you look into what pandoc provides? It might be easier to take that 
output and include it. Some kind of html? That's easy to render.

A bit like this:

- Convert the whole bibliography database to html using pandoc in the 
preferred cs rendering. That should be fast.

- Use the normal context commands for referencing a citation (the cite 
part is normally easy as there is not much variation in that; if needed 
one can cheat and also pregenerate that). That's then just some 
relatively small plugin mode.

- When placing the bibliography, filter the right entries from that html 
file (easy) using info that got stored in the tuc.

It is also fast. The only depdency then is pandoc but that is widely 
available (irr). But I would need to see an example of that kind of out 
first. We basically treat the (formatted) bibliography as an external 
resource but in some format that we can easily parse (and if needed 
tweak).

Hans


-----------------------------------------------------------------
                                           Hans Hagen | PRAGMA ADE
               Ridderstraat 27 | 8061 GH Hasselt | The Netherlands
        tel: 038 477 53 69 | www.pragma-ade.nl | www.pragma-pod.nl
-----------------------------------------------------------------
___________________________________________________________________________________
If your question is of interest to others as well, please add an entry to the Wiki!

maillist : ntg-context@ntg.nl / http://www.ntg.nl/mailman/listinfo/ntg-context
webpage  : http://www.pragma-ade.nl / http://context.aanhet.net
archive  : https://bitbucket.org/phg/context-mirror/commits/
wiki     : http://contextgarden.net
___________________________________________________________________________________

Re: Citeproc-lua

[Denis Maier via ntg-context]

Yes, Aditya. Citeproc-lua uses csl styles. Test suite coverage is not complete yet, though.

Denis



________________________________________
Von: ntg-context <ntg-context-bounces@ntg.nl> im Auftrag von Hans Hagen via ntg-context <ntg-context@ntg.nl>
Gesendet: Freitag, 6. Mai 2022 23:44:38
An: Denis Maier via ntg-context
Cc: Hans Hagen
Betreff: Re: [NTG-context] Citeproc-lua

On 5/6/2022 11:02 PM, Denis Maier via ntg-context wrote:

> I also have the impression that extending citeproc-formats.lua should be the easiest part. As an aside, I think the HTML is in there because that is needed for the test suite, but for ConTeXt we should be able to copy the LaTeX mapping and tweak a few things here and there.

> The LaTeX package is hard to understand, at least if you're not used to the "new" latex3 programming syntax and style. I also can only guess what that stuff does. But as you say this is rather isolated from the citeproc processing, so I guess the main task would be to come up with an equivalent ConTeXt module that writes the citations to the tuc file, calls the citeproc engine, and uses the results for the output. Right? (I.e., AFAICS, everything that lives under https://github.com/zepinglee/citeproc-lua/tree/main/citeproc should pretty much just work, while https://github.com/zepinglee/citeproc-lua/tree/main/latex must be adapted.)

For the record: i don't think you should write something to the tuc file
that doesn't come from context itself because you can mess it up (also
performaance wise). Of course you could have another file (just like a
bib file is independent). One problem could be that you need to make
some extra installation to make it work as we're not going to add all
kind of code to the distribution (we tend to go smaller) and someone
needs to maintain that moduie then because users depend in it working.

Did you look into what pandoc provides? It might be easier to take that
output and include it. Some kind of html? That's easy to render.

A bit like this:

- Convert the whole bibliography database to html using pandoc in the
preferred cs rendering. That should be fast.

- Use the normal context commands for referencing a citation (the cite
part is normally easy as there is not much variation in that; if needed
one can cheat and also pregenerate that). That's then just some
relatively small plugin mode.

- When placing the bibliography, filter the right entries from that html
file (easy) using info that got stored in the tuc.

It is also fast. The only depdency then is pandoc but that is widely
available (irr). But I would need to see an example of that kind of out
first. We basically treat the (formatted) bibliography as an external
resource but in some format that we can easily parse (and if needed
tweak).

Hans


-----------------------------------------------------------------
                                           Hans Hagen | PRAGMA ADE
               Ridderstraat 27 | 8061 GH Hasselt | The Netherlands
        tel: 038 477 53 69 | www.pragma-ade.nl | www.pragma-pod.nl
-----------------------------------------------------------------
___________________________________________________________________________________
If your question is of interest to others as well, please add an entry to the Wiki!

maillist : ntg-context@ntg.nl / http://www.ntg.nl/mailman/listinfo/ntg-context
webpage  : http://www.pragma-ade.nl / http://context.aanhet.net
archive  : https://bitbucket.org/phg/context-mirror/commits/
wiki     : http://contextgarden.net
___________________________________________________________________________________
___________________________________________________________________________________
If your question is of interest to others as well, please add an entry to the Wiki!

maillist : ntg-context@ntg.nl / http://www.ntg.nl/mailman/listinfo/ntg-context
webpage  : http://www.pragma-ade.nl / http://context.aanhet.net
archive  : https://bitbucket.org/phg/context-mirror/commits/
wiki     : http://contextgarden.net
___________________________________________________________________________________

Re: Citeproc-lua

[Denis Maier via ntg-context]

> -----Ursprüngliche Nachricht-----
> Von: ntg-context <ntg-context-bounces@ntg.nl> Im Auftrag von Hans Hagen
> via ntg-context
> Gesendet: Freitag, 6. Mai 2022 23:45
> An: Denis Maier via ntg-context <ntg-context@ntg.nl>
> Cc: Hans Hagen <j.hagen@xs4all.nl>
> Betreff: Re: [NTG-context] Citeproc-lua
> 
> [...]
> 
> For the record: i don't think you should write something to the tuc file that
> doesn't come from context itself because you can mess it up (also
> performaance wise). 

Really? But what about that stuff? Anyway, even better if the data coming from context itself can be used. 

Of course you could have another file (just like a bib file
> is independent). One problem could be that you need to make some extra
> installation to make it work as we're not going to add all kind of code to the
> distribution (we tend to go smaller) and someone needs to maintain that
> moduie then because users depend in it working.
> 
> Did you look into what pandoc provides? It might be easier to take that
> output and include it. Some kind of html? That's easy to render.

Pandoc can output context as well, so we can just take that.

> 
> A bit like this:
> 
> - Convert the whole bibliography database to html using pandoc in the
> preferred cs rendering. That should be fast.

Using the whole bibliography might give problems with regards to disambiguation.

> 
> - Use the normal context commands for referencing a citation (the cite part
> is normally easy as there is not much variation in that; if needed one can
> cheat and also pregenerate that). That's then just some relatively small
> plugin mode.
> 
> - When placing the bibliography, filter the right entries from that html file
> (easy) using info that got stored in the tuc.

As said above, that might create problems with regards to disambiguation. Better just use the right entries.

> 
> It is also fast. The only depdency then is pandoc but that is widely available
> (irr). But I would need to see an example of that kind of out first. We
> basically treat the (formatted) bibliography as an external resource but in
> some format that we can easily parse (and if needed tweak).

So, that would work a bit like Aditya's filter module: https://github.com/adityam/filter
Right?

Another option would be to just use pandoc's citeproc directly, as described here: https://github.com/jgm/citeproc/blob/master/man/citeproc.1.md

We'd have to pass a list of citations as a JSON object to that citeproc and use the results. 
The results will include citations as well as the bibliography (all as JSON, content can be HTML formatted).

Denis

___________________________________________________________________________________
If your question is of interest to others as well, please add an entry to the Wiki!

maillist : ntg-context@ntg.nl / http://www.ntg.nl/mailman/listinfo/ntg-context
webpage  : http://www.pragma-ade.nl / http://context.aanhet.net
archive  : https://bitbucket.org/phg/context-mirror/commits/
wiki     : http://contextgarden.net
___________________________________________________________________________________

Re: Citeproc-lua

[Hans Hagen via ntg-context]

On 5/7/2022 12:33 AM, denis.maier@unibe.ch wrote:
>> -----Ursprüngliche Nachricht-----
>> Von: ntg-context <ntg-context-bounces@ntg.nl> Im Auftrag von Hans Hagen
>> via ntg-context
>> Gesendet: Freitag, 6. Mai 2022 23:45
>> An: Denis Maier via ntg-context <ntg-context@ntg.nl>
>> Cc: Hans Hagen <j.hagen@xs4all.nl>
>> Betreff: Re: [NTG-context] Citeproc-lua
>>
>> [...]
>>
>> For the record: i don't think you should write something to the tuc file that
>> doesn't come from context itself because you can mess it up (also
>> performaance wise).
> 
> Really? But what about that stuff? Anyway, even better if the data coming from context itself can be used.

A bibliography is just about (cross) referencing data. If that data 
comes from elsewhere it can best kept indepdendent. One can of course 
store some reference but why the data if it is not used in some special 
way.

> Of course you could have another file (just like a bib file
>> is independent). One problem could be that you need to make some extra
>> installation to make it work as we're not going to add all kind of code to the
>> distribution (we tend to go smaller) and someone needs to maintain that
>> moduie then because users depend in it working.
>>
>> Did you look into what pandoc provides? It might be easier to take that
>> output and include it. Some kind of html? That's easy to render.
> 
> Pandoc can output context as well, so we can just take that.

Sure, but some parseble format might be better if one wants control over 
rendering. Basically we're talking about some pseudo formatted 
bibliography entry with some unique tag as reference.

> Using the whole bibliography might give problems with regards to disambiguation.

In what sense? Aren't the 'tags' or id's unique? Anyway, one can

- let \cite write some refe to soem file
- that file gets pandoc'd to a bibliography list
- that then gets included

if that list is in some parsable format (like html width classes) we can 
easily pick up info (if needed) and do nmore

> So, that would work a bit like Aditya's filter module: https://github.com/adityam/filter
> Right?

sort of

> Another option would be to just use pandoc's citeproc directly, as described here: https://github.com/jgm/citeproc/blob/master/man/citeproc.1.md
> 
> We'd have to pass a list of citations as a JSON object to that citeproc and use the results.
> The results will include citations as well as the bibliography (all as JSON, content can be HTML formatted).
we can handle json indeed, so hwo does such a json look like? can you 
make an example?

Hans

-----------------------------------------------------------------
                                           Hans Hagen | PRAGMA ADE
               Ridderstraat 27 | 8061 GH Hasselt | The Netherlands
        tel: 038 477 53 69 | www.pragma-ade.nl | www.pragma-pod.nl
-----------------------------------------------------------------
___________________________________________________________________________________
If your question is of interest to others as well, please add an entry to the Wiki!

maillist : ntg-context@ntg.nl / http://www.ntg.nl/mailman/listinfo/ntg-context
webpage  : http://www.pragma-ade.nl / http://context.aanhet.net
archive  : https://bitbucket.org/phg/context-mirror/commits/
wiki     : http://contextgarden.net
___________________________________________________________________________________

Re: Citeproc-lua

[Denis Maier via ntg-context]

> -----Ursprüngliche Nachricht-----
> Von: Hans Hagen <j.hagen@xs4all.nl>
> Gesendet: Samstag, 7. Mai 2022 01:15
> An: Maier, Denis Christian (UB) <denis.maier@unibe.ch>; ntg-context@ntg.nl
> Betreff: Re: AW: [NTG-context] Citeproc-lua
> 
> On 5/7/2022 12:33 AM, denis.maier@unibe.ch wrote:
> >> -----Ursprüngliche Nachricht-----
> >> Von: ntg-context <ntg-context-bounces@ntg.nl> Im Auftrag von Hans
> >> Hagen via ntg-context
> >> Gesendet: Freitag, 6. Mai 2022 23:45
> >> An: Denis Maier via ntg-context <ntg-context@ntg.nl>
> >> Cc: Hans Hagen <j.hagen@xs4all.nl>
> >> Betreff: Re: [NTG-context] Citeproc-lua
> >>
> >> [...]
> >>
> >> For the record: i don't think you should write something to the tuc
> >> file that doesn't come from context itself because you can mess it up
> >> (also performaance wise).
> >
> > Really? But what about that stuff? Anyway, even better if the data coming
> from context itself can be used.
> 
> A bibliography is just about (cross) referencing data. If that data comes from
> elsewhere it can best kept indepdendent. One can of course store some
> reference but why the data if it is not used in some special way.

I think we're on the same page here. I was just talking about the info about which items have been cited, which pages, prefixes, suffixes, etc. All that stuff that is basically already there...

> 
> > Of course you could have another file (just like a bib file
> >> is independent). One problem could be that you need to make some
> >> extra installation to make it work as we're not going to add all kind
> >> of code to the distribution (we tend to go smaller) and someone needs
> >> to maintain that moduie then because users depend in it working.
> >>
> >> Did you look into what pandoc provides? It might be easier to take
> >> that output and include it. Some kind of html? That's easy to render.
> >
> > Pandoc can output context as well, so we can just take that.
> 
> Sure, but some parseble format might be better if one wants control over
> rendering. Basically we're talking about some pseudo formatted bibliography
> entry with some unique tag as reference.

I see...

> 
> > Using the whole bibliography might give problems with regards to
> disambiguation.
> 
> In what sense? Aren't the 'tags' or id's unique? 

Tags/Ids are unique, for sure. But, if you render a bibliography based on everything in the bib-file, you might end up with 
Doe 2005a
Doe 2005b
etc.
but in the end you'll remove Doe 2005b, so the whole a/b stuff isn't necessary anymore.

Anyway, one can
> 
> - let \cite write some refe to soem file
> - that file gets pandoc'd to a bibliography list
> - that then gets included

Yes, looks reasonable.

> 
> if that list is in some parsable format (like html width classes) we can easily pick
> up info (if needed) and do nmore
> 
> > So, that would work a bit like Aditya's filter module:
> > https://github.com/adityam/filter Right?
> 
> sort of
> 
> > Another option would be to just use pandoc's citeproc directly, as
> > described here:
> > https://github.com/jgm/citeproc/blob/master/man/citeproc.1.md
> >
> > We'd have to pass a list of citations as a JSON object to that citeproc and use
> the results.
> > The results will include citations as well as the bibliography (all as JSON,
> content can be HTML formatted).
> we can handle json indeed, so hwo does such a json look like? can you make an
> example?

There are some examples on this page https://github.com/jgm/citeproc/blob/master/man/citeproc.1.md
Does that help or do you need to know more?

Anyway, this adds the citeproc executable as a dependency. Maybe they can make that functionality from pandoc proper.

Denis
___________________________________________________________________________________
If your question is of interest to others as well, please add an entry to the Wiki!

maillist : ntg-context@ntg.nl / http://www.ntg.nl/mailman/listinfo/ntg-context
webpage  : http://www.pragma-ade.nl / http://context.aanhet.net
archive  : https://bitbucket.org/phg/context-mirror/commits/
wiki     : http://contextgarden.net
___________________________________________________________________________________

Re: Citeproc-lua

[Rik Kabel via ntg-context]

[-- Attachment #1.1: Type: text/plain, Size: 1767 bytes --]


On 2022-05-06 17:44, Hans Hagen via ntg-context wrote:
> [...]
> A bit like this:
>
> - Convert the whole bibliography database to html using pandoc in the 
> preferred cs rendering. That should be fast.
>
> - Use the normal context commands for referencing a citation (the cite 
> part is normally easy as there is not much variation in that; if 
> needed one can cheat and also pregenerate that). That's then just some 
> relatively small plugin mode.
>
> - When placing the bibliography, filter the right entries from that 
> html file (easy) using info that got stored in the tuc.
>
> It is also fast. The only depdency then is pandoc but that is widely 
> available (irr). But I would need to see an example of that kind of 
> out first. We basically treat the (formatted) bibliography as an 
> external resource but in some format that we can easily parse (and if 
> needed tweak).
>
> Hans
>
This is similar to my strategy today. I create a pandoc markdown file 
from my ConTeXt source using a simple script and generate ConTeXt output 
containing just the bibliography using pandoc. This output needs some 
minor tweaks, again scripted, to do things like replace the space in 
/Vol. 999/ with a non-breaking space to keep the following number on the 
same line and to convert /\em/ and such to taggable macros that I define 
in my environment. The result is ConText source that can be further 
edited (if necessary) and inserted where needed/./ I have not extended 
it to multiple bibliographies in  a single document, although that 
should not be difficult.

Another possibility would be for ConTeXt, under control of a flag or 
setting, to generate an editable bibliography file that could be 
included under control of a second flag or setting.

-- 
Rik

[-- Attachment #1.2: Type: text/html, Size: 2485 bytes --]

[-- Attachment #2: Type: text/plain, Size: 493 bytes --]

___________________________________________________________________________________
If your question is of interest to others as well, please add an entry to the Wiki!

maillist : ntg-context@ntg.nl / http://www.ntg.nl/mailman/listinfo/ntg-context
webpage  : http://www.pragma-ade.nl / http://context.aanhet.net
archive  : https://bitbucket.org/phg/context-mirror/commits/
wiki     : http://contextgarden.net
___________________________________________________________________________________