Bugs in XML export (for ePub)

Bugs in XML export (for ePub)

[Henning Hraban Ramm]

Hi again.

I understand you don’t want to export some arbitrary (X)HTML. That’s fine, I can make my own XSLT.
And I’m glad that \setupexport doesn’t kill hyphenation any more.

But there are serious errors in export, that have nothing to do with „there’s no usable standard“.


[1] root node

First, the exported XML file lacks a root node!
Just call it <export> or <context> or whatever you like, but it MUST be there, otherwise it’s no wellformed XML.

If I enable xhtml mode like
\setupbackend[export=export.xml,xhtml=yes]
the exported file stops after a few lines.

If I set xhtml=somefile.xhtml, I get the broken export.xml and the complete somefile.xhtml, so I guess xhtml=yes causes overwriting the complete file with the incomplete one.

As far as I remember previous discussions, this incomplete content is caused by the project structure (using project, product, environment); from single file it seemed to work, at least at that time, I didn’t check.

Since xhtml mode doesn’t do anything useful anyway, I can just drop the setting.


[2] metadata

The metadata from \setupexport is completely ignored.
The metadata from \settaggedmetadata is exported just fine, but it ends up within the first block of the first page, if that’s not empty.
e.g. my text starts with \startlines Title etc. \stoplines, then <metadata><metavariable> etc. is exported within <lines>.
If I add something like \startparagraph \stopparagraph to contain the metadata, it disappears. Of course I don’t want to add something visible (then it works again).


[3] cover picture

The firstpage key of setupexport is supposed to define a cover picture for the ePub, but it doesn’t do anything, as far as I can tell.
Or is something wrong in:
\setupexport[firstpage={img/cover.jpg}]
?


[4] delimited

If \quotation{} contains more than one paragraph (in my case it’s a quote), the <delimited detail="quotation“> is ended after the first paragraph.


(As always, working with latest MkIV beta on OSX.)

Greetlings, Hraban
---
http://www.fiee.net/texnique/
http://wiki.contextgarden.net
https://www.cacert.org (I'm an assurer)

___________________________________________________________________________________
If your question is of interest to others as well, please add an entry to the Wiki!

maillist : ntg-context@ntg.nl / http://www.ntg.nl/mailman/listinfo/ntg-context
webpage  : http://www.pragma-ade.nl / http://tex.aanhet.net
archive  : http://foundry.supelec.fr/projects/contextrev/
wiki     : http://contextgarden.net
___________________________________________________________________________________

Re: Bugs in XML export (for ePub)

[Thangalin]

Hi Hraban,

I tried to create ePubs using ConTeXt late last year (it was a driving
factor for me to switch from LaTeX).

In practice, ConTeXt cannot actually produce ePub content of any
sufficient complexity that an ePub reader (such as Kindle) can read. I
abandoned that development line a while ago. That said, ConTeXt is
orders of magnitude easier to work with than LaTeX (for me) and has an
architecture that holds a promising future.

Since my project exports XML, I began an ePub solution using XSLT and
DocBook. The ant4docbook Java application can produce ePub files from
DocBook using an ant task, making the API simple to use. The resulting
ePub archive can be read by various e-readers.

http://ant4docbook.sourceforge.net/

Unless anyone says otherwise (or there have been momentous
improvements in this area), I would recommend that you do not use
ConTeXt to create ePub files. While technically it can produce an ePub
archive, you will waste countless hours hitting wall after wall.
___________________________________________________________________________________
If your question is of interest to others as well, please add an entry to the Wiki!

maillist : ntg-context@ntg.nl / http://www.ntg.nl/mailman/listinfo/ntg-context
webpage  : http://www.pragma-ade.nl / http://tex.aanhet.net
archive  : http://foundry.supelec.fr/projects/contextrev/
wiki     : http://contextgarden.net
___________________________________________________________________________________

Re: Bugs in XML export (for ePub)

[Henning Hraban Ramm]

Am 2014-03-11 um 12:42 schrieb Thangalin <thangalin@gmail.com>:

> In practice, ConTeXt cannot actually produce ePub content of any
> sufficient complexity that an ePub reader (such as Kindle) can read. I
> abandoned that development line a while ago. That said, ConTeXt is
> orders of magnitude easier to work with than LaTeX (for me) and has an
> architecture that holds a promising future.
> ...
> Unless anyone says otherwise (or there have been momentous
> improvements in this area), I would recommend that you do not use
> ConTeXt to create ePub files. While technically it can produce an ePub
> archive, you will waste countless hours hitting wall after wall.

I was thinking about starting with markdown or the like, but I *want* to use ConTeXt, and if it should be able to produce ePub, then I want it to succeed.
And maybe I can get on Hans’ and Wolfgang’s nerves enough to make it possible ;-)


Greetlings, Hraban
---
http://www.fiee.net/texnique/
http://wiki.contextgarden.net
https://www.cacert.org (I'm an assurer)

___________________________________________________________________________________
If your question is of interest to others as well, please add an entry to the Wiki!

maillist : ntg-context@ntg.nl / http://www.ntg.nl/mailman/listinfo/ntg-context
webpage  : http://www.pragma-ade.nl / http://tex.aanhet.net
archive  : http://foundry.supelec.fr/projects/contextrev/
wiki     : http://contextgarden.net
___________________________________________________________________________________

Re: Bugs in XML export (for ePub)

[Thangalin]

Hi, Hraban.

Here is the thread from last year for your reference:

https://www.mail-archive.com/ntg-context@ntg.nl/msg71376.html
___________________________________________________________________________________
If your question is of interest to others as well, please add an entry to the Wiki!

maillist : ntg-context@ntg.nl / http://www.ntg.nl/mailman/listinfo/ntg-context
webpage  : http://www.pragma-ade.nl / http://tex.aanhet.net
archive  : http://foundry.supelec.fr/projects/contextrev/
wiki     : http://contextgarden.net
___________________________________________________________________________________

Re: Bugs in XML export (for ePub)

[Henning Hraban Ramm]

Am 2014-03-11 um 23:54 schrieb Thangalin <thangalin@gmail.com>:

> Here is the thread from last year for your reference:
> https://www.mail-archive.com/ntg-context@ntg.nl/msg71376.html

Thank you, that’s what I was referring to.
Hans says, ConTeXt would export XML, but that isn’t true, since the exported „XML“ lacks a root element.
(Issue [1] in my previous post.)

Following Wolfang’s advice I’ll provide minimal examples.

Greetlings, Hraban
---
http://www.fiee.net/texnique/
http://wiki.contextgarden.net
https://www.cacert.org (I'm an assurer)

___________________________________________________________________________________
If your question is of interest to others as well, please add an entry to the Wiki!

maillist : ntg-context@ntg.nl / http://www.ntg.nl/mailman/listinfo/ntg-context
webpage  : http://www.pragma-ade.nl / http://tex.aanhet.net
archive  : http://foundry.supelec.fr/projects/contextrev/
wiki     : http://contextgarden.net
___________________________________________________________________________________

Re: Bugs in XML export (for ePub)

[Henning Hraban Ramm]

Am 2014-03-12 um 09:24 schrieb Henning Hraban Ramm <texml@fiee.net>:

> Am 2014-03-11 um 23:54 schrieb Thangalin <thangalin@gmail.com>:
> 
>> Here is the thread from last year for your reference:
>> https://www.mail-archive.com/ntg-context@ntg.nl/msg71376.html
> 
> Thank you, that’s what I was referring to.
> Hans says, ConTeXt would export XML, but that isn’t true, since the exported „XML“ lacks a root element.
> (Issue [1] in my previous post.)

Ok, I take it partially back: If I have just one standalone document, the output is ok.
So the problem is mostly in the combination of export with project structure.

But … please see below.

Minimal succeeding example:

———
%
\setupbackend[export=export.xml]
\settaggedmetadata[
	title={Export Test Solo},
	author={Hraban},
	version=\currentdate % why that doesn’t work?
]


\starttext

\startchapter[title={Chapter One}]

\startsection[title={Section One}]

\input tufte

\stopsection

\startsection[title={Section Two}]

\input knuth

\stopsection

\stopchapter

\stoptext
———

gives 

———
<?xml version='1.0' encoding='UTF-8' standalone='yes' ?>

<!-- input filename   : solo              -->
<!-- processing date  : Wed Mar 12 10:10:20 2014 -->
<!-- context version  : 2014.03.07 11:42  -->
<!-- exporter version : 0.30              -->


 <document language="en" file="solo" date="Wed Mar 12 10:10:20 2014" context="2014.03.07 11:42" version="0.30" xmlns:m="http://www.w3.org/1998/Math/MathML">
 <metadata>
  <metavariable name="author">Hraban</metavariable>
  <metavariable name="title">Export Test Solo</metavariable>
  <metavariable name="version">\currentdate </metavariable>
 </metadata>
  <section detail="chapter" location='aut:1'>
    <sectionnumber>1</sectionnumber> 
   <sectiontitle>Chapter One</sectiontitle> 
   <sectioncontent>
      <section detail="section">
        <sectionnumber>1.1</sectionnumber> 
     <sectiontitle>Section One</sectiontitle> 
     <sectioncontent>
We thrive in information--thick worlds because of our marvelous and everyday capacity to select, edit, single out, structure, highlight, group, pair, merge, harmonize, synthesize, focus, organize, condense, reduce, boil down, choose, categorize, catalog, classify, list, abstract, scan, look into, idealize, isolate, discriminate, distinguish, screen, pigeonhole, pick over, sort, integrate, blend, inspect, filter, lump, skip, smooth, chunk, average, approximate, cluster, aggregate, outline, summarize, itemize, review, dip into, flip through, browse, glance into, leaf through, skim, refine, enumerate, glean, synopsize, winnow the wheat from the chaff and separate the sheep from the goats.
     </sectioncontent>
    </section>
    <section detail="section" location='aut:3'>
        <sectionnumber>1.2</sectionnumber> 
     <sectiontitle>Section Two</sectiontitle> 
     <sectioncontent>
Thus, I came to the conclusion that the designer of a new system must not only be the implementer and first large--scale user; the designer should also write the first user manual.
      <break/>
The separation of any of these four components would have hurt TEX significantly. If I had not participated fully in all these activities, literally hundreds of improvements would never have been made, because I would never have thought of them or perceived why they were important.
      <break/>
But a system cannot be successful if it is too strongly influenced by a single person. Once the initial design is complete and fairly robust, the real test begins as people with many different viewpoints undertake their own experiments.
     </sectioncontent>
    </section>
   </sectioncontent>
  </section>
 </document>
———

The wiki page about ePub says, it should be possible to define metadata in \setupexport like this:

\setupexport[
	firstpage={koe.pdf},
	title={My Fine Title},
]

But that is completely ignored - it does neither show up in the export, nor in a produced ePub.

If I include an image with \externalfigure[koe][width=\textwidth], I get in export:
<image name="koe" id='image-1' width='15.000cm' height='21.625cm'></image>

That’s not really usable without file extension. In my case, it was even a GIF „giraffe.gif“ that got converted to „m_k_i_v_giraffe.gif.pdf“ .
Would it be possible to set the actually used file name as an additional attribute in <image>?


I’ll document the issues with export in project structure in my next post.


Greetlings, Hraban
___________________________________________________________________________________
If your question is of interest to others as well, please add an entry to the Wiki!

maillist : ntg-context@ntg.nl / http://www.ntg.nl/mailman/listinfo/ntg-context
webpage  : http://www.pragma-ade.nl / http://tex.aanhet.net
archive  : http://foundry.supelec.fr/projects/contextrev/
wiki     : http://contextgarden.net
___________________________________________________________________________________

Re: Bugs in XML export (for ePub)

[Henning Hraban Ramm]

more issues:

[5] emphasis

How to markup emphasize (\em, \bf, \it, \sl) so that it gets exported? {\em Something} doesn’t.


And hey, can I bribe you to look into that issues? Some nice felt hats from Kyrgyzstan perhaps?

The ePub/export stuff is not in a workable state. If you don’t want to work at it, we should really remove it to avoid raising false hopes.

I can’t tell how easy it is to fix/enhance. Would it make sense for me to start somewhere?


Greetlings, Hraban
---
http://www.fiee.net/texnique/
http://wiki.contextgarden.net
https://www.cacert.org (I'm an assurer)

___________________________________________________________________________________
If your question is of interest to others as well, please add an entry to the Wiki!

maillist : ntg-context@ntg.nl / http://www.ntg.nl/mailman/listinfo/ntg-context
webpage  : http://www.pragma-ade.nl / http://tex.aanhet.net
archive  : http://foundry.supelec.fr/projects/contextrev/
wiki     : http://contextgarden.net
___________________________________________________________________________________

Re: Bugs in XML export (for ePub)

[Wolfgang Schuster]

Am 11.03.2014 um 09:09 schrieb Henning Hraban Ramm <texml@fiee.net>:

> more issues:
> 
> [5] emphasis
> 
> How to markup emphasize (\em, \bf, \it, \sl) so that it gets exported? {\em Something} doesn’t.

You have to create a command for them which add tags, e.g.

  \definehighlight[important][style=italic]

or

  \definestartstop[important][style=italic]

and use

  normal text \important{important text} normal text

> And hey, can I bribe you to look into that issues? Some nice felt hats from Kyrgyzstan perhaps?
> 
> The ePub/export stuff is not in a workable state. If you don’t want to work at it, we should really remove it to avoid raising false hopes.
> 
> I can’t tell how easy it is to fix/enhance. Would it make sense for me to start somewhere?

You can make a few minimal examples with bugs in export mechanism.

Wolfgang
___________________________________________________________________________________
If your question is of interest to others as well, please add an entry to the Wiki!

maillist : ntg-context@ntg.nl / http://www.ntg.nl/mailman/listinfo/ntg-context
webpage  : http://www.pragma-ade.nl / http://tex.aanhet.net
archive  : http://foundry.supelec.fr/projects/contextrev/
wiki     : http://contextgarden.net
___________________________________________________________________________________

Re: Bugs in XML export (for ePub)

[Henning Hraban Ramm]

[-- Attachment #1: Type: text/plain, Size: 2218 bytes --]

Hi again,
please find attached a rather minimal project structure including the produced XML export.


> [1] root node
> 
> First, the exported XML file lacks a root node!
> Just call it <export> or <context> or whatever you like, but it MUST be there, otherwise it’s no wellformed XML.

The <document> root node is just fine from standalone documents, but is missing in exports from project structure.

> If I enable xhtml mode ...
> the exported file stops after a few lines.

I didn’t trace that yet, since xhtml mode make so sense anyway.


> [2] metadata
> 
> The metadata from \setupexport is completely ignored.

still true - is \setupexport deprecated?

> The metadata from \settaggedmetadata is exported just fine, but it ends up within the first block of the first page, if that’s not empty.

still true - in this test case, within „section“; I guess <metadata> is inserted into the tree after the first tag, that should be <document>. 
So [2] should be fixed if [1] is.


> [3] cover picture
> 
> The firstpage key of setupexport is supposed to define a cover picture for the ePub, but it doesn’t do anything

probably part of [2] - \setupexport doesn’t do anything. Since the cover picture probably doesn’t need to end up in the exported XML but in one of the ePub auxiliary files, I don’t know how it should (or used to) work at all.


> [4] delimited
> 
> If \quotation{} contains more than one paragraph (in my case it’s a quote), the <delimited detail="quotation“> is ended after the first paragraph.

Can’t reproduce that in my example, must investigate further.


> [5] emphasis
> 
> How to markup emphasize (\em, \bf, \it, \sl) so that it gets exported? {\em Something} doesn’t.

Ok, \definehighlight[important][style=italic] is feasible.


[6] paragraphs

But using \startparagraph … \stopparagraph is still a bit too much, writing XML or HTML would be more comfortable.
Since ConTeXt knows about paragraphs (or does it?), would it be possible to make it an option to handle implicite paragraphs like explicite?


Greetlings, Hraban
---
http://www.fiee.net/texnique/
http://wiki.contextgarden.net
https://www.cacert.org (I'm an assurer)

[-- Attachment #2: epub.zip --]
[-- Type: application/zip, Size: 36227 bytes --]

[-- Attachment #3: Type: text/plain, Size: 485 bytes --]

___________________________________________________________________________________
If your question is of interest to others as well, please add an entry to the Wiki!

maillist : ntg-context@ntg.nl / http://www.ntg.nl/mailman/listinfo/ntg-context
webpage  : http://www.pragma-ade.nl / http://tex.aanhet.net
archive  : http://foundry.supelec.fr/projects/contextrev/
wiki     : http://contextgarden.net
___________________________________________________________________________________