Thank you for this, it's instructive. I can't say I'm enthusiastic about starting from scratch on a parser, even a limited one. After thinking some more, calling Lua's read function also won't really work. The Rust code still has to run outside a Lua context: in JSON mode, as a BibTeX <-> CSL-JSON <-> YAML metadata converter, and to fulfil the role pandoc-citeproc takes in things like vim-pandoc's citekey completion. These are all fairly non-negotiable features. (Also, because it is Rust code that would be parsing reference data, you would need a way back into the Lua VM, which is more complicated than just being Lua->Rust only.) It might actually be possible to emulate the current pandoc-citeproc approach (Text.CSL.Compat) of linking against Pandoc itself. There is some promising work (broken but fixable) over at https://github.com/mgattozzi/curryrs with linking the GHC runtime and simplifying interop using the type system(s), so it might be possible to just define some FFI-able Inline types in a little haskell wrapper library much like today's Text.CSL.Compat, and link it in. 2x link-time optimisations hopefully wouldn't even pull in that much of Pandoc. In that way, users committed to pandoc output might even get to choose their input syntax through a metadata option; you could use the Pandoc HTML parser instead of micro-HTML, and even let people use Markdown or LaTeX in their Zotero entries and any emitted CSL-JSON. I think I would leave the citeproc-js micro-HTML syntax around for other output formats, because that means all of the pandoc baggage can still be stripped out for WebAssembly distribution. I also think for ease of development I won't dive too hard into the Lua approach, because correctness doesn't depend on it, it can be added later, and all of the JSON interop is already functional. It also turns out there's another serde_json-supporting pandoc-types Rust crate that already has tree-walking here , so I think that problem might be mostly solved. On Thursday, December 13, 2018 at 8:07:17 AM UTC+11, Paulo Ney de Souza wrote: > > Here is a sample of elaborate titles that "really" happen in the wild -- > these are taken from the Proceedings of the ICM 2018: > > title = {{$\bold Z$-theory: chasing ${\mathfrak m}/f$ theory}}, > title = {New examples of complete Calabi--Yau metrics on > $\mathbb{C}^n$ for $n\ge 3$}, > title = {Uniqueness of the group measure space decomposition for > {P}opa's {$\mathscr{HT}$} factors}, > title = {Actions of {$\mathbb F_\infty$} whose {${\rm II}_1$} factors > and orbit equivalence relations have prescribed fundamental group}, > title = {Cocycle and orbit superrigidity for lattices in {${\rm > SL}(n,\mathbb R)$} acting on homogeneous spaces}, > title = {Profinite rigidity of $\mathbf{PGL}(2,{\Z}[\omega])$ and > $\mathbf{PSL}(2,{\Z}[\omega])$}, > title = {Representation of measures with polynomial denseness in > {$L_p(\mathbb R,d\mu)$}, {$0 moment problems}, > title = {Zimmer's conjecture for actions of > $\mathrm{SL}(m,\mathbb{Z})$}, > title = {Can lattices in {${\rm SL}(n,\mathbb R)$} act on the circle?}, > title = {Higher {T}eichm\"uller spaces: from {${\rm SL}(2,\mathbb R)$} > to other {L}ie groups}, > title = {Exponential decay of connection probabilities for subcritical > Voronoi percolation in $\mathbb{R}^d$}, > title = {A {KAM} scheme for {${\rm SL}(2,\mathbb R)$} cocycles with > {L}iouvillean frequencies}, > title = {On dynamics of {$Out(F_n)$} on {$\mathrm{PSL}_2({\mathbb > C})$} characters}, > title = {General topology meets model theory, on {$\mathfrak p$} and > {$\mathfrak t$}},^M > title = {New classes of {${\mathcal L}\sp{p}$}-spaces}, > title = {A class of special {${\mathcal L}\sb{\infty }$}\ spaces}, > title = {More {$\ell_r$} saturated {$\mathscr L^\infty$} spaces}, > title = {The {${\mathcal L}\sb{p}$} spaces}, > title = {A remark on bases in {${\mathcal L}\sb{p}$}-spaces with an > application to complementably universal {${\mathcal L}\sb{\infty > }$}-spaces}, > title = {{${\rm SL}(2,\mathbb C)$} {C}hern-{S}imons theory and the > asymptotic behavior of the colored {J}ones polynomial}, > title = {K-polystability of {${\mathbb Q}$}-{F}ano varieties admitting > {K}\"ahler-{E}instein metrics}, > title = {Weak geodesic rays in the space of {K}\"ahler potentials and > the class {$\mathcal{E}(X,\omega)$}}, > title = {Operator-algebraic superridigity for {${\rm SL}_n(\mathbb > Z)$}, {$n\geq 3$}}, > title = {The space of closed subgroups of {$\mathbb R^n$} is > stratified and simply connected}, > title = {The irreducible representations of the {L}ie algebra > {${\mathfrak s}{\mathfrak l}(2)$}\ and of the {W}eyl algebra}, > title = {Singular {G}elfand-{T}setlin modules of > {${\mathfrak{gl}}(n)$}}, > title = {Families of irreducible singular Gelfand-Tsetlin modules of > $\mathfrak{gl}(n)$}, > title = {Infinite-dimensional representations of the {L}ie algebra > {$\mathfrak{gl}(n,{\mathbb C})$} related to complex analogs of the > {G}elfand-{T}setlin patterns and general hypergeometric functions on the > {L}ie group {${\rm GL}(n,{\mathbb C})$}}, > title = {A geometric approach to 1-singular {G}elfand--{T}setlin > {$\mathfrak{gl}_n$}-modules}, > title = {Geometric approach to $p$-singular Gelfand--Tsetlin > $\mathfrak {gl}_n$-modules}, > title = {On some {B}ruhat decomposition and the structure of the > {H}ecke rings of {${\mathfrak p}$}-adic {C}hevalley groups}, > title = {Stable $s$-minimal cones in $\mathbb{R}^3$ are flat for > $s\sim 1$}, > title = {Delaunay type domains for an overdetermined elliptic problem > in {$\mathbb S^n\times\mathbb R$} and {$\Bbb H^n\times\Bbb R$}}, > title = {A {KAM} scheme for {${\rm SL}(2,\mathbb R)$} cocycles with > {L}iouvillean frequencies}, > > > Paulo Ney > > > On Wed, Dec 12, 2018 at 12:39 PM John MacFarlane > wrote: > >> Cormac Relf > writes: >> >> > The BibTex parsing is a tricky one, though. There’s this >> > for the main syntax, at >> least. I >> > wouldn’t want to fork out to Pandoc for every single latex text field, >> but >> > maybe the Lua API’s read would help here. >> >> That's an interesting point. If your parser just >> parsed the fields as RawInline (Format "latex") ---, >> you could have the lua filter do a separate pass at >> the beginning to try to convert all of these into >> native pandoc inlines using read. >> >> > It might be simpler to support >> > both citeproc-js’ micro-HTML and a similarly limited micro-LaTeX with a >> > simple Rust-based parser, but not at the same time. What do people use >> > backslash commands for in BibTeX? Are there names and document titles >> out >> > there that really need the whole power of LaTeX to render? I might have >> to >> > think about this some more. Perhaps a successor to CSL-JSON that >> accepts >> > arbitrary JSON objects wherever the old one accepts strings. >> >> In practice, a fairly small subset of LaTeX would be >> enough to handle most of what you find in bibtex >> bibliographies. >> >> Certainly you will find things like `\emph`, inline >> math, and lots of escape characters like `\"{a}`. >> >> -- >> You received this message because you are subscribed to the Google Groups >> "pandoc-discuss" group. >> To unsubscribe from this group and stop receiving emails from it, send an >> email to pandoc-discus...-/JYPxA39Uh5TLH3MbocFFw@public.gmane.org . >> To post to this group, send email to pandoc-...-/JYPxA39Uh5TLH3MbocFFw@public.gmane.org >> . >> To view this discussion on the web visit >> https://groups.google.com/d/msgid/pandoc-discuss/yh480kk1keeazt.fsf%40johnmacfarlane.net >> . >> For more options, visit https://groups.google.com/d/optout. >> > -- You received this message because you are subscribed to the Google Groups "pandoc-discuss" group. To unsubscribe from this group and stop receiving emails from it, send an email to pandoc-discuss+unsubscribe-/JYPxA39Uh5TLH3MbocFF+G/Ez6ZCGd0@public.gmane.org To post to this group, send email to pandoc-discuss-/JYPxA39Uh5TLH3MbocFF+G/Ez6ZCGd0@public.gmane.org To view this discussion on the web visit https://groups.google.com/d/msgid/pandoc-discuss/786c8104-1297-465e-9cd9-d3c720e6685e%40googlegroups.com. For more options, visit https://groups.google.com/d/optout.