Converting org to ipynb

Converting org to ipynb

[Joost Kremers]

Hi list,

I'm using Emacs' Org mode to write Python code combined with notes, and I was
hoping I'd be able to convert such files to Jupyter .ipynb files. So I tried the
following:

    pandoc -f org -t ipynb -o pyfile.ipynb pyfile.org

This did create an .ipynb file that I can open with Jupyter Lab, but the entire
file was converted into one big Markdown cell, including the source blocks.

Is there a way to convert an Org file to a Jupyter notebook that renders the
source blocks as code blocks in the notebook? Pandoc's user manual only talks
about converting Markdown to Jupyter notebooks, and I guess I could convert the
Org files to Markdown, add the `code` class to the source code blocks and then
convert to .ipynb, but that seems a little tedious...

TIA

Joost


-- 
Joost Kremers
Life has its moments

Re: Converting org to ipynb

[John MacFarlane]

Joost Kremers <joostkremers-97jfqw80gc6171pxa8y+qA@public.gmane.org> writes:

> Is there a way to convert an Org file to a Jupyter notebook that renders the
> source blocks as code blocks in the notebook? Pandoc's user manual only talks
> about converting Markdown to Jupyter notebooks, and I guess I could convert the
> Org files to Markdown, add the `code` class to the source code blocks and then
> convert to .ipynb, but that seems a little tedious...

You should be able to do it from org, as long as the structure of
your org file follows the structure the manual describes for
markdown.

If you're in doubt how to do that, you could always convert the
markdown structure to native, and then find an org structure
that produces the same AST.

Maybe converting markdown -> org would lead you in the right
direction.  Sorry, I've forgotten the details of the ipynb
conventions; someone else may be able to give more precise
advice.

Re: Converting org to ipynb

[Joost Kremers]

On Mon, 7 Feb 2022, at 02:14, John MacFarlane wrote:
> If you're in doubt how to do that, you could always convert the
> markdown structure to native, and then find an org structure
> that produces the same AST.

A very quick test suggests that the main difference is in the type of a code block. If I have the following code block in a Markdown file:

~~~~~~
``` code
print("Hello, World!")
```
~~~~~~

converting it to native gives me the following:


```
CodeBlock
  ( "" , [ "code" ] , [] ) "print('Hello, World!\")"
```

Converting the following Org snippet:

```
#+begin_src python
  print("Hello, World!")
#+end_src
```

gives me:

```
CodeBlock
  ( "" , [ "python" ] , [] ) "print(\"Hello, World!\")\n"
```

So they're very close, except for the "source" vs. "python" tag (and the final newline, it seems). I could change the Org source to say "code" instead of "python", of course, but that would mean the source block can no longer be executed in Org.

In case you're not aware of this (I sometimes need to remind myself that not everybody in the world uses Emacs :D), it's possible in Org mode to execute code blocks, much like in a Jupyter notebook, with the output being added to the Org file. I was hoping I would be able to convert that into a Jupyter notebook in a way that correctly recognises the code blocks and their outputs, but I realise now that's probably not realistic.

-- 
Joost Kremers
Life has its moments

-- 
You received this message because you are subscribed to the Google Groups "pandoc-discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email to pandoc-discuss+unsubscribe-/JYPxA39Uh5TLH3MbocFF+G/Ez6ZCGd0@public.gmane.org
To view this discussion on the web visit https://groups.google.com/d/msgid/pandoc-discuss/b34203fa-dd18-4084-95b9-824d50fdb0bf%40www.fastmail.com.

Re: Converting org to ipynb

[Albert Krewinkel]

"Joost Kremers" <joostkremers-97jfqw80gc6171pxa8y+qA@public.gmane.org> writes:

> On Mon, 7 Feb 2022, at 02:14, John MacFarlane wrote:
>> If you're in doubt how to do that, you could always convert the
>> markdown structure to native, and then find an org structure
>> that produces the same AST.
>
> So they're very close, except for the "source" vs. "python" tag (and
> the final newline, it seems). I could change the Org source to say
> "code" instead of "python", of course, but that would mean the source
> block can no longer be executed in Org.

Adding a `code` class to all code blocks is good use-case for a Lua
filter. You probably won't want `#+result` blocks as code cells, so we
won't add it to those blocks:

    function CodeBlock (cb)
      -- #+results blocks have class 'example', ignore those.
      if not cb.classes:includes 'example'
        cb.classes:insert 'code'
      end
      return cb
    end

I believe that should be enough to produce acceptable results.

-- 
Albert Krewinkel
GPG: 8eed e3e2 e8c5 6f18 81fe  e836 388d c0b2 1f63 1124

Re: Converting org to ipynb

[Joost]

On Mon, 7 Feb 2022, at 08:52, Albert Krewinkel wrote:
> Adding a `code` class to all code blocks is good use-case for a Lua
> filter. You probably won't want `#+result` blocks as code cells, so we
> won't add it to those blocks:
>
>     function CodeBlock (cb)
>       -- #+results blocks have class 'example', ignore those.
>       if not cb.classes:includes 'example'
>         cb.classes:insert 'code'
>       end
>       return cb
>     end
>
> I believe that should be enough to produce acceptable results.

Hi, thanks for the tip and the example code! I'd have to make sure that `#+result:` code is always wrapped in an example block (by default, that's not the case) but that would be fairly easy to do.

I just found out that there is an ipynb exporter for Org mode as well at <https://github.com/jkitchin/ox-ipynb>,[1] which I'll be trying out, too. It seems to have its own limitations, so I'll have to see which option is better in my case.

Anyway, thanks for the pointers!

Joost




Footnotes:
[1]  Actually, there's a second one at <https://github.com/zaeph/ox-ipynb>, which claims to have better integration with Org's export mechanism but which looks like it might be abandoned.

-- 
Joost Kremers
Life has its moments

Re: Converting org to ipynb

[Albert Krewinkel]

Joost <joostkremers-97jfqw80gc6171pxa8y+qA@public.gmane.org> writes:

> On Mon, 7 Feb 2022, at 08:52, Albert Krewinkel wrote:
>> Adding a `code` class to all code blocks is good use-case for a Lua
>> filter. You probably won't want `#+result` blocks as code cells, so we
>> won't add it to those blocks:
>>
>>     function CodeBlock (cb)
>>       -- #+results blocks have class 'example', ignore those.
>>       if not cb.classes:includes 'example'
>>         cb.classes:insert 'code'
>>       end
>>       return cb
>>     end
>>
>> I believe that should be enough to produce acceptable results.
>
> Hi, thanks for the tip and the example code! I'd have to make sure
> that `#+result:` code is always wrapped in an example block (by
> default, that's not the case) but that would be fairly easy to do.

I think pandoc does that by default (modeled after the ox-html
exporter).

> I just found out that there is an ipynb exporter for Org mode as well
> at <https://github.com/jkitchin/ox-ipynb>,[1] which I'll be trying
> out, too. It seems to have its own limitations, so I'll have to see
> which option is better in my case.

That will probably be the better option in this case.

Please let us know how it goes!

-- 
Albert Krewinkel
GPG: 8eed e3e2 e8c5 6f18 81fe  e836 388d c0b2 1f63 1124