Pandoc JSON schema

Pandoc JSON schema

[Albert Krewinkel]

If anyone is interested: I've created a JSON schema for pandoc's JSON
output.  This should make it possible to validate the output of JSON
filters without having to invoke pandoc.

https://gist.github.com/tarleb/bcb4b31e35263fcbc6cf7e533670c1cf

I intended for the schema to also serve as documentation, but I find it
difficult to read. It might be possible to improve that, though.

Re: Pandoc JSON schema

[John MACFARLANE]

This is great.  It shoud go somewhere in the pandoc
repository (or perhaps pandoc-types), and be made available
via pandoc.org.

+++ Albert Krewinkel [Oct 24 17 13:16 ]:
>If anyone is interested: I've created a JSON schema for pandoc's JSON
>output.  This should make it possible to validate the output of JSON
>filters without having to invoke pandoc.
>
>https://gist.github.com/tarleb/bcb4b31e35263fcbc6cf7e533670c1cf
>
>I intended for the schema to also serve as documentation, but I find it
>difficult to read. It might be possible to improve that, though.
>
>-- 
>You received this message because you are subscribed to the Google Groups "pandoc-discuss" group.
>To unsubscribe from this group and stop receiving emails from it, send an email to pandoc-discuss+unsubscribe-/JYPxA39Uh5TLH3MbocFF+G/Ez6ZCGd0@public.gmane.org
>To post to this group, send email to pandoc-discuss-/JYPxA39Uh5TLH3MbocFF+G/Ez6ZCGd0@public.gmane.org
>To view this discussion on the web visit https://groups.google.com/d/msgid/pandoc-discuss/87bmkwdcdr.fsf%40zeitkraut.de.
>For more options, visit https://groups.google.com/d/optout.

Re: Pandoc JSON schema

[John MACFARLANE]

PS. I wonder if there is any way to autogenerate this from the
FromJSON/ToJSON instances for pandoc types?  That would
allow us to ensure that this is always in sync with changes.

+++ John MACFARLANE [Oct 24 17 09:47 ]:
>This is great.  It shoud go somewhere in the pandoc
>repository (or perhaps pandoc-types), and be made available
>via pandoc.org.
>
>+++ Albert Krewinkel [Oct 24 17 13:16 ]:
>>If anyone is interested: I've created a JSON schema for pandoc's JSON
>>output.  This should make it possible to validate the output of JSON
>>filters without having to invoke pandoc.
>>
>>https://gist.github.com/tarleb/bcb4b31e35263fcbc6cf7e533670c1cf
>>
>>I intended for the schema to also serve as documentation, but I find it
>>difficult to read. It might be possible to improve that, though.
>>
>>-- 
>>You received this message because you are subscribed to the Google Groups "pandoc-discuss" group.
>>To unsubscribe from this group and stop receiving emails from it, send an email to pandoc-discuss+unsubscribe-/JYPxA39Uh5TLH3MbocFF+G/Ez6ZCGd0@public.gmane.org
>>To post to this group, send email to pandoc-discuss-/JYPxA39Uh5TLH3MbocFF+G/Ez6ZCGd0@public.gmane.org
>>To view this discussion on the web visit https://groups.google.com/d/msgid/pandoc-discuss/87bmkwdcdr.fsf%40zeitkraut.de.
>>For more options, visit https://groups.google.com/d/optout.
>
>-- 
>You received this message because you are subscribed to the Google Groups "pandoc-discuss" group.
>To unsubscribe from this group and stop receiving emails from it, send an email to pandoc-discuss+unsubscribe-/JYPxA39Uh5TLH3MbocFF+G/Ez6ZCGd0@public.gmane.org
>To post to this group, send email to pandoc-discuss-/JYPxA39Uh5TLH3MbocFF+G/Ez6ZCGd0@public.gmane.org
>To view this discussion on the web visit https://groups.google.com/d/msgid/pandoc-discuss/20171024164746.GC90469%40protagoras.
>For more options, visit https://groups.google.com/d/optout.

Re: Pandoc JSON schema

[Albert Krewinkel]

John MACFARLANE <jgm-TVLZxgkOlNX2fBVCVOL8/A@public.gmane.org> writes:

> PS. I wonder if there is any way to autogenerate this from the
> FromJSON/ToJSON instances for pandoc types?  That would
> allow us to ensure that this is always in sync with changes.

Autogeneration is the reason why I didn't open a PR for this yet. I'm
planning to experiment with generics and alternative JSON
representations, this schema was merely created to get me started.

It was also suggested to me to use the [Concise Data Definition
Language] instead, which is currently in I-D status, *much* less
verbose, and actually readable. I may try to use generics to generate a
CDDL definition, which could then be translated into a JSON schema. That
is going to take me a while, though.

Long story short: I'll open a PR on jgm/pandoc-website for now, if
that's ok with you. I don't think it's ready for the other repos yet,
neither for pandoc nor pandoc-types.

[Concise Data Definition Language]: https://datatracker.ietf.org/doc/draft-ietf-cbor-cddl/

Re: Pandoc JSON schema

[Harsh Donga]

[-- Attachment #1.1: Type: text/plain, Size: 1968 bytes --]

Hey folks, 
Do we have any schema validation now, I would like to use it in my custom 
script which creates pandoc JSON eventually to convert into different 
formats (HTML. DOCX...).

Of course, a way to go is run pandoc command and we can get first erroneous 
object in json and continuing till they are all resolved.
Was hoping to have better solution, if there was any other way 
 
Thanks in advance!

On Wednesday, 25 October 2017 at 13:18:43 UTC+5:30 Albert Krewinkel wrote:

> John MACFARLANE <j...-TVLZxgkOlNX2fBVCVOL8/A@public.gmane.org> writes:
>
> > PS. I wonder if there is any way to autogenerate this from the
> > FromJSON/ToJSON instances for pandoc types? That would
> > allow us to ensure that this is always in sync with changes.
>
> Autogeneration is the reason why I didn't open a PR for this yet. I'm
> planning to experiment with generics and alternative JSON
> representations, this schema was merely created to get me started.
>
> It was also suggested to me to use the [Concise Data Definition
> Language] instead, which is currently in I-D status, *much* less
> verbose, and actually readable. I may try to use generics to generate a
> CDDL definition, which could then be translated into a JSON schema. That
> is going to take me a while, though.
>
> Long story short: I'll open a PR on jgm/pandoc-website for now, if
> that's ok with you. I don't think it's ready for the other repos yet,
> neither for pandoc nor pandoc-types.
>
> [Concise Data Definition Language]: 
> https://datatracker.ietf.org/doc/draft-ietf-cbor-cddl/
>
>

-- 
You received this message because you are subscribed to the Google Groups "pandoc-discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email to pandoc-discuss+unsubscribe-/JYPxA39Uh5TLH3MbocFF+G/Ez6ZCGd0@public.gmane.org
To view this discussion on the web visit https://groups.google.com/d/msgid/pandoc-discuss/8d53751b-a59c-43fb-a7e1-4f5079903e28n%40googlegroups.com.

[-- Attachment #1.2: Type: text/html, Size: 3005 bytes --]

Re: Pandoc JSON schema

[Albert Krewinkel]

Harsh Donga <harsh-7+aFW328pE6p1wGUEcWPqti2O/JbrIOy@public.gmane.org> writes:

> Do we have any schema validation now, I would like to use it in my
> custom script which creates pandoc JSON eventually to convert into
> different formats (HTML. DOCX...).

The schema posted before does not include the new table structure yet.
Other than that it should remain usable.

I believe that our best path forward would be to use autodocodec:
https://github.com/NorfairKing/autodocodec


-- 
Albert Krewinkel
GPG: 8eed e3e2 e8c5 6f18 81fe  e836 388d c0b2 1f63 1124

Re: Pandoc JSON schema

[Harsh Donga]

[-- Attachment #1.1: Type: text/plain, Size: 1247 bytes --]

Seems great, will try it out
Anything non-haskell? Not so great with coding in haskell yet.

Again, thanks! It has been best to work around with pandoc and it's 
community!

On Wednesday, 2 November 2022 at 20:44:06 UTC+5:30 Albert Krewinkel wrote:

>
> Harsh Donga <ha...-7+aFW328pE6p1wGUEcWPqti2O/JbrIOy@public.gmane.org> writes:
>
> > Do we have any schema validation now, I would like to use it in my
> > custom script which creates pandoc JSON eventually to convert into
> > different formats (HTML. DOCX...).
>
> The schema posted before does not include the new table structure yet.
> Other than that it should remain usable.
>
> I believe that our best path forward would be to use autodocodec:
> https://github.com/NorfairKing/autodocodec
>
>
> -- 
> Albert Krewinkel
> GPG: 8eed e3e2 e8c5 6f18 81fe e836 388d c0b2 1f63 1124
>

-- 
You received this message because you are subscribed to the Google Groups "pandoc-discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email to pandoc-discuss+unsubscribe-/JYPxA39Uh5TLH3MbocFF+G/Ez6ZCGd0@public.gmane.org
To view this discussion on the web visit https://groups.google.com/d/msgid/pandoc-discuss/81054dc7-a893-4f84-9ce4-ea90f997b12fn%40googlegroups.com.

[-- Attachment #1.2: Type: text/html, Size: 2142 bytes --]

Re: Pandoc JSON schema

[Albert Krewinkel]

Harsh Donga <harsh-7+aFW328pE6p1wGUEcWPqti2O/JbrIOy@public.gmane.org> writes:

> Seems great, will try it out
> Anything non-haskell? Not so great with coding in haskell yet.

I'm afraid that Haskell is the only good choice here; otherwise we'd
risk the schema to become outdated without us noticing.

> Again, thanks! It has been best to work around with pandoc and it's
> community!

💙


-- 
Albert Krewinkel
GPG: 8eed e3e2 e8c5 6f18 81fe  e836 388d c0b2 1f63 1124

-- 
You received this message because you are subscribed to the Google Groups "pandoc-discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email to pandoc-discuss+unsubscribe-/JYPxA39Uh5TLH3MbocFF+G/Ez6ZCGd0@public.gmane.org
To view this discussion on the web visit https://groups.google.com/d/msgid/pandoc-discuss/87leot5otd.fsf%40zeitkraut.de.

Re: Pandoc JSON schema

[John MacFarlane]

Albert, what program does one use to validate JSON against the schema you made before?
https://gist.github.com/tarleb/bcb4b31e35263fcbc6cf7e533670c1cf



> On Nov 2, 2022, at 8:58 AM, Albert Krewinkel <albert+pandoc-9EawChwDxG8hFhg+JK9F0w@public.gmane.org> wrote:
> 
> 
> Harsh Donga <harsh-7+aFW328pE6p1wGUEcWPqti2O/JbrIOy@public.gmane.org> writes:
> 
>> Seems great, will try it out
>> Anything non-haskell? Not so great with coding in haskell yet.
> 
> I'm afraid that Haskell is the only good choice here; otherwise we'd
> risk the schema to become outdated without us noticing.
> 
>> Again, thanks! It has been best to work around with pandoc and it's
>> community!
> 
> 💙
> 
> 
> -- 
> Albert Krewinkel
> GPG: 8eed e3e2 e8c5 6f18 81fe  e836 388d c0b2 1f63 1124
> 
> -- 
> You received this message because you are subscribed to the Google Groups "pandoc-discuss" group.
> To unsubscribe from this group and stop receiving emails from it, send an email to pandoc-discuss+unsubscribe-/JYPxA39Uh5TLH3MbocFF+G/Ez6ZCGd0@public.gmane.org
> To view this discussion on the web visit https://groups.google.com/d/msgid/pandoc-discuss/87leot5otd.fsf%40zeitkraut.de.

-- 
You received this message because you are subscribed to the Google Groups "pandoc-discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email to pandoc-discuss+unsubscribe-/JYPxA39Uh5TLH3MbocFF+G/Ez6ZCGd0@public.gmane.org
To view this discussion on the web visit https://groups.google.com/d/msgid/pandoc-discuss/AC7F0C0B-122B-4BF6-BC02-81FC4BA4B37C%40gmail.com.

Re: Pandoc JSON schema

[Albert Krewinkel]

John MacFarlane <fiddlosopher-Re5JQEeQqe8AvxtiuMwx3w@public.gmane.org> writes:

> Albert, what program does one use to validate JSON against the schema
> you made before?
> https://gist.github.com/tarleb/bcb4b31e35263fcbc6cf7e533670c1cf

I think I used some kind of Python library when I did that, but don't
actually remember. It might have been
https://python-jsonschema.readthedocs.io/


-- 
Albert Krewinkel
GPG: 8eed e3e2 e8c5 6f18 81fe  e836 388d c0b2 1f63 1124

Re: Pandoc JSON schema

[Harsh Donga]

[-- Attachment #1.1: Type: text/plain, Size: 1026 bytes --]

PFA: pandoc AST json schema (pandoc-types 1.22)


On Tuesday, 6 December 2022 at 21:18:37 UTC+5:30 Albert Krewinkel wrote:

>
> John MacFarlane <fiddlo...-Re5JQEeQqe8AvxtiuMwx3w@public.gmane.org> writes:
>
> > Albert, what program does one use to validate JSON against the schema
> > you made before?
> > https://gist.github.com/tarleb/bcb4b31e35263fcbc6cf7e533670c1cf
>
> I think I used some kind of Python library when I did that, but don't
> actually remember. It might have been
> https://python-jsonschema.readthedocs.io/
>
>
> -- 
> Albert Krewinkel
> GPG: 8eed e3e2 e8c5 6f18 81fe e836 388d c0b2 1f63 1124
>

-- 
You received this message because you are subscribed to the Google Groups "pandoc-discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email to pandoc-discuss+unsubscribe-/JYPxA39Uh5TLH3MbocFF+G/Ez6ZCGd0@public.gmane.org
To view this discussion on the web visit https://groups.google.com/d/msgid/pandoc-discuss/4e6dd89d-1dd0-40bb-a760-e3468c5b2403n%40googlegroups.com.

[-- Attachment #1.2: Type: text/html, Size: 2212 bytes --]

[-- Attachment #2: pandoc_schema.json --]
[-- Type: application/json, Size: 31089 bytes --]

Re: Pandoc JSON schema

[John MacFarlane]

Is there a way to programatically test a document against this schema?

> On Dec 28, 2022, at 1:44 AM, Harsh Donga <harsh-7+aFW328pE6p1wGUEcWPqti2O/JbrIOy@public.gmane.org> wrote:
> 
> PFA: pandoc AST json schema (pandoc-types 1.22)
> 

Re: Pandoc JSON schema

[Jörn K]

[-- Attachment #1.1: Type: text/plain, Size: 1939 bytes --]

Hi!

Is stumbled upon this while trying to use json-schema to enforce a certain 
format and content of a markdown document. Our use-case is enforcing the 
layout and data entered in standardized technical reports for a project of 
our institute.

Basically I have a json-schema prepared and have a test in which a python 
script (using jsonschema) validated Pandoc's json output. 
So far it has proven quite mighty, but sadly it is cumbersome to write the 
schema manually as there is quite some nesting of properties involved.

 Currently I am pondering if one could use a (Lua) writer to generate such 
a schema. One could lookout for certain attributes to set json-schema 
validation options. This way the feature of validating the contents of a 
document against a schema would come one step closer to less json prone 
users.

Is there a way to get the pandoc type string of a pandoc object in Lua? I 
mean the value which goes into the t property of json. I mean one can 
simply map the output of pandoc.util.type with knowledge of the current 
section, but this seems less desirable than getting the string directly 
from pandoc.

Thanks!

Jörn

John MacFarlane schrieb am Donnerstag, 29. Dezember 2022 um 21:37:08 UTC+1:

> Is there a way to programatically test a document against this schema?
>
> > On Dec 28, 2022, at 1:44 AM, Harsh Donga <ha...-7+aFW328pE6p1wGUEcWPqti2O/JbrIOy@public.gmane.org> 
> wrote:
> > 
> > PFA: pandoc AST json schema (pandoc-types 1.22)
> > 
>
>

-- 
You received this message because you are subscribed to the Google Groups "pandoc-discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email to pandoc-discuss+unsubscribe-/JYPxA39Uh5TLH3MbocFF+G/Ez6ZCGd0@public.gmane.org
To view this discussion on the web visit https://groups.google.com/d/msgid/pandoc-discuss/16e8b004-f3c8-4a4e-a6e6-856f45f3bdd5n%40googlegroups.com.

[-- Attachment #1.2: Type: text/html, Size: 2666 bytes --]