public inbox archive for pandoc-discuss@googlegroups.com
 help / color / mirror / Atom feed
From: John MacFarlane <jgm-TVLZxgkOlNX2fBVCVOL8/A@public.gmane.org>
To: pandoc-discuss-/JYPxA39Uh5TLH3MbocFFw@public.gmane.org
Subject: WIP: better citation processing
Date: Sun, 16 Aug 2020 15:59:34 -0700	[thread overview]
Message-ID: <m2mu2udwo9.fsf@johnmacfarlane.net> (raw)


I've been working this summer on a replacement to pandoc-citeproc.

Part of this is a new library, citeproc, which is
not specific to pandoc. This already passes a larger portion
of the CSL test suite than pandoc-citeproc, and it should be
straightforward to improve it further. It will be easier to
maintain than pandoc-citeproc, more accurate, and faster.

I've used this library to create a new filter,
new-pandoc-citeproc, which now passes most of the pandoc-citeproc
test suite but runs around 6 times faster.

I'm inclined, though, not to release this as a new filter,
but instead to depend on the citeproc library and build the
citation processing capabilities into pandoc itself. This will
cut down the binaries we need to distribute from two to one, and
it will simplify things for users, who won't have to worry about
filters. It will also be more performant, as we'll avoid the
overhead of JSON serialization and deserialization.

I'm not quite ready to release any of this code, but I hope
to do so in the next month or two.  This is just a teaser.

The new library is pure Haskell and won't depend on bibutils
(a wrapper around a C library).  That means we'll only support
bibtex/biblatex, pandoc yaml, and CSL JSON as bibliography
formats.  Those who need others will have to convert them using
standalone bibutils.  But support for other formats was never
great, so I don't think this is a big loss.


             reply	other threads:[~2020-08-16 22:59 UTC|newest]

Thread overview: 43+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2020-08-16 22:59 John MacFarlane [this message]
2020-08-17 14:55 ` OT: " Anton Shepelev
     [not found] ` <m2mu2udwo9.fsf-pgq/RBwaQ+zq8tPRBa0AtqxOck334EZe@public.gmane.org>
2020-08-17 14:31   ` James
     [not found]     ` <6F6F5A78-7473-473E-927F-46E2382FE979-Re5JQEeQqe8AvxtiuMwx3w@public.gmane.org>
2020-08-17 23:17       ` Priv.-Doz. Dr. Maria Shinoto
2020-08-18  9:58   ` Denis Maier
     [not found]     ` <360dbe5b-f1ef-17f6-32e6-8c9f85204844-cl+VPiYnx/1AfugRpC6u6w@public.gmane.org>
2020-08-18 15:39       ` jcr
     [not found]         ` <46e97135-ea1f-469a-898a-eb24876c0708o-/JYPxA39Uh5TLH3MbocFFw@public.gmane.org>
2020-08-18 16:59           ` Denis Maier
     [not found]             ` <1ee49c20-ee12-4984-abdf-e00a9e4414e6o-/JYPxA39Uh5TLH3MbocFFw@public.gmane.org>
2020-08-18 18:13               ` FI Apps
     [not found]                 ` <D2ED4685-A7ED-4BB4-B651-D362B2564F62-Re5JQEeQqe8AvxtiuMwx3w@public.gmane.org>
2020-08-18 21:14                   ` Denis Maier
     [not found]                     ` <d59896f4-b360-9ef4-e751-d9dc912ee700-Re5JQEeQqe8AvxtiuMwx3w@public.gmane.org>
2020-08-19  8:37                       ` FI Apps
2020-08-19 12:47               ` James P. Ascher
     [not found]                 ` <1edca1a5-bb73-42b7-a61a-b02d2a1ec5dco-/JYPxA39Uh5TLH3MbocFFw@public.gmane.org>
2020-08-19 13:12                   ` Denis Maier
     [not found]                     ` <5d4b596a-b3ef-744c-bbae-564e490eb598-Re5JQEeQqe8AvxtiuMwx3w@public.gmane.org>
2020-08-19 13:37                       ` James P. Ascher
2020-08-21 19:41       ` John MacFarlane
     [not found]         ` <m2mu2nbxcv.fsf-pgq/RBwaQ+zq8tPRBa0AtqxOck334EZe@public.gmane.org>
2020-08-22  9:33           ` Denis Maier
     [not found]             ` <865f327e-d305-4409-dd6f-1659f6a884cc-cl+VPiYnx/1AfugRpC6u6w@public.gmane.org>
2020-09-08 18:15               ` John MacFarlane
     [not found]                 ` <m2bligt9ra.fsf-jF64zX8BO08an7k8zZ43ob9bIa4KchGshsV+eolpW18@public.gmane.org>
2020-09-09  7:51                   ` Gabriel Nützi
2020-09-09  9:10                   ` FI Apps
     [not found]                     ` <CAGOSsd=Xs6u6Xup9YugR-zd6FC_29QFs1asEcARA4m2UPts_vQ-JsoAwUIsXosN+BqQ9rBEUg@public.gmane.org>
2020-09-11 15:12                       ` John MacFarlane
     [not found]                         ` <m2tuw4jqj1.fsf-jF64zX8BO08an7k8zZ43ob9bIa4KchGshsV+eolpW18@public.gmane.org>
2020-09-11 16:02                           ` Albert Krewinkel
     [not found]                             ` <87imckl2tg.fsf-9EawChwDxG8hFhg+JK9F0w@public.gmane.org>
2020-09-11 16:24                               ` John MacFarlane
     [not found]                                 ` <m2blicjn7p.fsf-jF64zX8BO08an7k8zZ43ob9bIa4KchGshsV+eolpW18@public.gmane.org>
2020-09-11 16:46                                   ` Albert Krewinkel
     [not found]                                     ` <87ft7ol0rq.fsf-9EawChwDxG8hFhg+JK9F0w@public.gmane.org>
2020-09-11 18:51                                       ` proposed --citeproc option John MacFarlane
     [not found]                                         ` <m21rj8jge2.fsf-jF64zX8BO08an7k8zZ43ob9bIa4KchGshsV+eolpW18@public.gmane.org>
2020-09-12 15:56                                           ` BPJ
2020-09-11 20:31                                       ` WIP: better citation processing Denis Maier
     [not found]                                         ` <1adfb0c1-d745-a6dc-bf14-f790e3f1280f-cl+VPiYnx/1AfugRpC6u6w@public.gmane.org>
2020-09-11 20:58                                           ` John MacFarlane
     [not found]                                             ` <m2v9gkhvya.fsf-jF64zX8BO08an7k8zZ43ob9bIa4KchGshsV+eolpW18@public.gmane.org>
2020-09-11 21:14                                               ` Denis Maier
     [not found]                                                 ` <64087dc0-cf6d-4c8d-c579-9b4312baec2e-cl+VPiYnx/1AfugRpC6u6w@public.gmane.org>
2020-09-11 21:21                                                   ` Denis Maier
2020-09-17 17:18                                                   ` John MacFarlane
     [not found]                                                     ` <d7317035-fba6-466b-8d6e-699f82ae5445n-/JYPxA39Uh5TLH3MbocFFw@public.gmane.org>
2020-09-17 17:30                                                       ` Denis Maier
     [not found]                                                         ` <m27dssjnvu.fsf@MacBook-Pro.hsd1.ca.comcast.net>
     [not found]                                                           ` <m24knwjnm5.fsf@MacBook-Pro.hsd1.ca.comcast.net>
     [not found]                                                             ` <86c0bfc6-4795-718c-5ddd-0e3ef0f91403@mailbox.org>
     [not found]                                                               ` <86c0bfc6-4795-718c-5ddd-0e3ef0f91403-cl+VPiYnx/1AfugRpC6u6w@public.gmane.org>
2020-09-17 19:20                                                                 ` Denis Maier
2020-09-12  6:45                                               ` Denis Maier
2020-09-12  6:51                                               ` Denis Maier
2020-09-12  6:26                       ` 'Nick Bart' via pandoc-discuss
2020-08-20 12:29   ` Joseph Reagle
     [not found]     ` <e94c6f9d-e6d8-099a-4bf2-7aed30476a6c-T1oY19WcHSwdnm+yROfE0A@public.gmane.org>
2020-08-21 19:06       ` John MacFarlane
     [not found]         ` <m2v9hbbyyu.fsf-pgq/RBwaQ+zq8tPRBa0AtqxOck334EZe@public.gmane.org>
2020-08-21 19:12           ` John MacFarlane
2020-08-21 21:21           ` Joseph Reagle
     [not found]             ` <91c2cfed-7211-4194-72cf-5c01abf2315c-T1oY19WcHSwdnm+yROfE0A@public.gmane.org>
2020-08-21 23:03               ` John MacFarlane
2021-06-16 18:14           ` Joseph
     [not found]             ` <30abf74d-df6f-4df8-aaee-a493331d4e92n-/JYPxA39Uh5TLH3MbocFFw@public.gmane.org>
2021-06-16 19:33               ` John MacFarlane
     [not found]                 ` <m24kdx38qp.fsf-jF64zX8BO0+FqBokazbCQ6OPv3vYUT2dxr7GGTnW70NeoWH0uzbU5w@public.gmane.org>
2021-06-17 14:01                   ` Joseph Reagle
2020-09-24 16:10   ` WIP: better citation processing - nightlies now available for testing! John MacFarlane

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=m2mu2udwo9.fsf@johnmacfarlane.net \
    --to=jgm-tvlzxgkolnx2fbvcvol8/a@public.gmane.org \
    --cc=pandoc-discuss-/JYPxA39Uh5TLH3MbocFFw@public.gmane.org \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).