TeX in \xmlsetentity and DTDs in DOCTYPEs

TeX in \xmlsetentity and DTDs in DOCTYPEs

[Kate F]

Hi. I'm using MkIV's XML stuff. I have two related questions.

Firstly, can I have entities in XML expand out to TeX markup?

I thought this was how you'd do it, but this produces a document that
says literally "xxx~yyy\thinspace zzz", whereas I want the ~ and
\thinspace to be executed as TeX commands. Test case below.

Secondly, is it possible to have ConTeXt load such entities from a
DTD? If I use a DOCTYPE in my XML, entities from that DTD seem to not
be loaded.

Thanks.

a.xml:
<?xml version="1.0"?>
<a>xxx&nbsp;yyy&thinsp;zzz</a>

a.tex:
\startxmlsetups xml:mysetups
\xmlsetsetup{\xmldocument}{a}{xml:*}
\stopxmlsetups
\xmlregistersetup{xml:mysetups}

\startxmlsetups xml:a
\xmlflush{#1}
\stopxmlsetups

\xmlsetentity{nbsp}{~}
\xmlsetentity{thinsp}{\thinspace}

\starttext
\xmlprocessfile{main}{a.xml}{}
\stoptext

-- 
Kate
___________________________________________________________________________________
If your question is of interest to others as well, please add an entry to the Wiki!

maillist : ntg-context@ntg.nl / http://www.ntg.nl/mailman/listinfo/ntg-context
webpage  : http://www.pragma-ade.nl / http://tex.aanhet.net
archive  : http://foundry.supelec.fr/projects/contextrev/
wiki     : http://contextgarden.net
___________________________________________________________________________________

Re: TeX in \xmlsetentity and DTDs in DOCTYPEs

[Thomas A. Schmitz]

On 01/15/2016 06:21 PM, Kate F wrote:
> Hi. I'm using MkIV's XML stuff. I have two related questions.
>
> Firstly, can I have entities in XML expand out to TeX markup?
>
> I thought this was how you'd do it, but this produces a document that
> says literally "xxx~yyy\thinspace zzz", whereas I want the ~ and
> \thinspace to be executed as TeX commands. Test case below.
>
> Secondly, is it possible to have ConTeXt load such entities from a
> DTD? If I use a DOCTYPE in my XML, entities from that DTD seem to not
> be loaded.

What do you mean? The way ConText processes entities is completely 
independent from your DTD.

> Thanks.
>
> a.xml:
> <?xml version="1.0"?>
> <a>xxx&nbsp;yyy&thinsp;zzz</a>
>
> a.tex:
> \startxmlsetups xml:mysetups
> \xmlsetsetup{\xmldocument}{a}{xml:*}
> \stopxmlsetups
> \xmlregistersetup{xml:mysetups}
>
> \startxmlsetups xml:a
> \xmlflush{#1}
> \stopxmlsetups

\enabledirectives[lxml.escapedentities]
\xmltexentity{nbsp}{~}
\xmltexentity{thinsp}{\thinspace}

> \xmlsetentity{nbsp}{~}
> \xmlsetentity{thinsp}{\thinspace}
>
> \starttext
> \xmlprocessfile{main}{a.xml}{}
> \stoptext

Thomas

___________________________________________________________________________________
If your question is of interest to others as well, please add an entry to the Wiki!

maillist : ntg-context@ntg.nl / http://www.ntg.nl/mailman/listinfo/ntg-context
webpage  : http://www.pragma-ade.nl / http://tex.aanhet.net
archive  : http://foundry.supelec.fr/projects/contextrev/
wiki     : http://contextgarden.net
___________________________________________________________________________________

Re: TeX in \xmlsetentity and DTDs in DOCTYPEs

[Kate F]

On 15 January 2016 at 17:34, Thomas A. Schmitz
<thomas.schmitz@uni-bonn.de> wrote:
> On 01/15/2016 06:21 PM, Kate F wrote:
>>
>> Hi. I'm using MkIV's XML stuff. I have two related questions.
>>
>> Firstly, can I have entities in XML expand out to TeX markup?
>>
>> I thought this was how you'd do it, but this produces a document that
>> says literally "xxx~yyy\thinspace zzz", whereas I want the ~ and
>> \thinspace to be executed as TeX commands. Test case below.
>>
>> Secondly, is it possible to have ConTeXt load such entities from a
>> DTD? If I use a DOCTYPE in my XML, entities from that DTD seem to not
>> be loaded.
>
>
> What do you mean? The way ConText processes entities is completely
> independent from your DTD.

So I see! But many DTDs contain definitions for entities, which I
would like to use, rather than repeating those definitions by
\xmlsetentity in ConTeXt. Some XML documents also have their own
document-specific entities inside the DOCTYPE at the top of the file.

For example in legal agreements, I have something like:

<?xml version="1.0"?>
<!DOCTYPE blah SYSTEM "blah.dtd" [
    <!ENTITY us   "the company">
    <!ENTITY you "the customer">
]>

Where the stuff inside [...] is considered part of a document-specific DTD.
libxml2 handles this sort of thing with its "dtdattr" options; I
presume lxml has something similar.


>
>> a.xml:
>> <?xml version="1.0"?>
>> <a>xxx&nbsp;yyy&thinsp;zzz</a>
>>
>> a.tex:
>> \startxmlsetups xml:mysetups
>> \xmlsetsetup{\xmldocument}{a}{xml:*}
>> \stopxmlsetups
>> \xmlregistersetup{xml:mysetups}
>>
>> \startxmlsetups xml:a
>> \xmlflush{#1}
>> \stopxmlsetups
>
>
> \enabledirectives[lxml.escapedentities]
> \xmltexentity{nbsp}{~}
> \xmltexentity{thinsp}{\thinspace}

So that's how to use \xmltexentity! I had seen that, but couldn't find
how to use it. Thanks!

-- 
Kate
___________________________________________________________________________________
If your question is of interest to others as well, please add an entry to the Wiki!

maillist : ntg-context@ntg.nl / http://www.ntg.nl/mailman/listinfo/ntg-context
webpage  : http://www.pragma-ade.nl / http://tex.aanhet.net
archive  : http://foundry.supelec.fr/projects/contextrev/
wiki     : http://contextgarden.net
___________________________________________________________________________________

Re: TeX in \xmlsetentity and DTDs in DOCTYPEs

[Thomas A. Schmitz]

On 01/15/2016 06:58 PM, Kate F wrote:
> So I see! But many DTDs contain definitions for entities, which I
> would like to use, rather than repeating those definitions by
> \xmlsetentity in ConTeXt. Some XML documents also have their own
> document-specific entities inside the DOCTYPE at the top of the file.
>
> For example in legal agreements, I have something like:
>
> <?xml version="1.0"?>
> <!DOCTYPE blah SYSTEM "blah.dtd" [
>      <!ENTITY us   "the company">
>      <!ENTITY you "the customer">
> ]>
>
> Where the stuff inside [...] is considered part of a document-specific DTD.
> libxml2 handles this sort of thing with its "dtdattr" options; I
> presume lxml has something similar.

AFAIK, ConTeXt has its own xml parser based on lua lpeg, see file 
lxml-tab.lua. There is code there that treats entities, but I have never 
used this approach, so you'll probably have to wait till Hans looks at 
your question (or understand the lua code...)

Thomas
___________________________________________________________________________________
If your question is of interest to others as well, please add an entry to the Wiki!

maillist : ntg-context@ntg.nl / http://www.ntg.nl/mailman/listinfo/ntg-context
webpage  : http://www.pragma-ade.nl / http://tex.aanhet.net
archive  : http://foundry.supelec.fr/projects/contextrev/
wiki     : http://contextgarden.net
___________________________________________________________________________________

Re: TeX in \xmlsetentity and DTDs in DOCTYPEs

[Hans Hagen]

On 1/15/2016 9:20 PM, Thomas A. Schmitz wrote:
> On 01/15/2016 06:58 PM, Kate F wrote:
>> So I see! But many DTDs contain definitions for entities, which I
>> would like to use, rather than repeating those definitions by
>> \xmlsetentity in ConTeXt. Some XML documents also have their own
>> document-specific entities inside the DOCTYPE at the top of the file.
>>
>> For example in legal agreements, I have something like:
>>
>> <?xml version="1.0"?>
>> <!DOCTYPE blah SYSTEM "blah.dtd" [
>>      <!ENTITY us   "the company">
>>      <!ENTITY you "the customer">
>> ]>
>>
>> Where the stuff inside [...] is considered part of a document-specific
>> DTD.
>> libxml2 handles this sort of thing with its "dtdattr" options; I
>> presume lxml has something similar.
>
> AFAIK, ConTeXt has its own xml parser based on lua lpeg, see file
> lxml-tab.lua. There is code there that treats entities, but I have never
> used this approach, so you'll probably have to wait till Hans looks at
> your question (or understand the lua code...)

i'll come back to it ... fyi: these doctype entities are already parsed 
in mkiv .. just not used (or maybe it got lost when fulfilling other 
wishes)

Hans

-----------------------------------------------------------------
                                           Hans Hagen | PRAGMA ADE
               Ridderstraat 27 | 8061 GH Hasselt | The Netherlands
       tel: 038 477 53 69 | www.pragma-ade.com | www.pragma-pod.nl
-----------------------------------------------------------------
___________________________________________________________________________________
If your question is of interest to others as well, please add an entry to the Wiki!

maillist : ntg-context@ntg.nl / http://www.ntg.nl/mailman/listinfo/ntg-context
webpage  : http://www.pragma-ade.nl / http://tex.aanhet.net
archive  : http://foundry.supelec.fr/projects/contextrev/
wiki     : http://contextgarden.net
___________________________________________________________________________________

Re: TeX in \xmlsetentity and DTDs in DOCTYPEs

[Kate F]

On 16 January 2016 at 00:33, Hans Hagen <pragma@wxs.nl> wrote:
> On 1/15/2016 9:20 PM, Thomas A. Schmitz wrote:
>>
>> On 01/15/2016 06:58 PM, Kate F wrote:
>>>
>>> So I see! But many DTDs contain definitions for entities, which I
>>> would like to use, rather than repeating those definitions by
>>> \xmlsetentity in ConTeXt. Some XML documents also have their own
>>> document-specific entities inside the DOCTYPE at the top of the file.
>>>
>>> For example in legal agreements, I have something like:
>>>
>>> <?xml version="1.0"?>
>>> <!DOCTYPE blah SYSTEM "blah.dtd" [
>>>      <!ENTITY us   "the company">
>>>      <!ENTITY you "the customer">
>>> ]>
>>>
>>> Where the stuff inside [...] is considered part of a document-specific
>>> DTD.
>>> libxml2 handles this sort of thing with its "dtdattr" options; I
>>> presume lxml has something similar.
>>
>>
>> AFAIK, ConTeXt has its own xml parser based on lua lpeg, see file
>> lxml-tab.lua. There is code there that treats entities, but I have never
>> used this approach, so you'll probably have to wait till Hans looks at
>> your question (or understand the lua code...)
>
>
> i'll come back to it ... fyi: these doctype entities are already parsed in
> mkiv .. just not used (or maybe it got lost when fulfilling other wishes)
>

Much appreciated, if you would. Thank you!
Is it just a case of having that parsing call \xmlsetentity?

-- 
Kate
___________________________________________________________________________________
If your question is of interest to others as well, please add an entry to the Wiki!

maillist : ntg-context@ntg.nl / http://www.ntg.nl/mailman/listinfo/ntg-context
webpage  : http://www.pragma-ade.nl / http://tex.aanhet.net
archive  : http://foundry.supelec.fr/projects/contextrev/
wiki     : http://contextgarden.net
___________________________________________________________________________________

Re: TeX in \xmlsetentity and DTDs in DOCTYPEs

[Hans Hagen]

On 1/16/2016 4:55 PM, Kate F wrote:
> On 16 January 2016 at 00:33, Hans Hagen <pragma@wxs.nl> wrote:
>> On 1/15/2016 9:20 PM, Thomas A. Schmitz wrote:
>>>
>>> On 01/15/2016 06:58 PM, Kate F wrote:
>>>>
>>>> So I see! But many DTDs contain definitions for entities, which I
>>>> would like to use, rather than repeating those definitions by
>>>> \xmlsetentity in ConTeXt. Some XML documents also have their own
>>>> document-specific entities inside the DOCTYPE at the top of the file.
>>>>
>>>> For example in legal agreements, I have something like:
>>>>
>>>> <?xml version="1.0"?>
>>>> <!DOCTYPE blah SYSTEM "blah.dtd" [
>>>>       <!ENTITY us   "the company">
>>>>       <!ENTITY you "the customer">
>>>> ]>
>>>>
>>>> Where the stuff inside [...] is considered part of a document-specific
>>>> DTD.
>>>> libxml2 handles this sort of thing with its "dtdattr" options; I
>>>> presume lxml has something similar.
>>>
>>>
>>> AFAIK, ConTeXt has its own xml parser based on lua lpeg, see file
>>> lxml-tab.lua. There is code there that treats entities, but I have never
>>> used this approach, so you'll probably have to wait till Hans looks at
>>> your question (or understand the lua code...)
>>
>>
>> i'll come back to it ... fyi: these doctype entities are already parsed in
>> mkiv .. just not used (or maybe it got lost when fulfilling other wishes)
>>
>
> Much appreciated, if you would. Thank you!
> Is it just a case of having that parsing call \xmlsetentity?

it should work in the in beta now


-----------------------------------------------------------------
                                           Hans Hagen | PRAGMA ADE
               Ridderstraat 27 | 8061 GH Hasselt | The Netherlands
       tel: 038 477 53 69 | www.pragma-ade.com | www.pragma-pod.nl
-----------------------------------------------------------------
___________________________________________________________________________________
If your question is of interest to others as well, please add an entry to the Wiki!

maillist : ntg-context@ntg.nl / http://www.ntg.nl/mailman/listinfo/ntg-context
webpage  : http://www.pragma-ade.nl / http://tex.aanhet.net
archive  : http://foundry.supelec.fr/projects/contextrev/
wiki     : http://contextgarden.net
___________________________________________________________________________________

Re: TeX in \xmlsetentity and DTDs in DOCTYPEs

[Thomas A. Schmitz]

On 01/17/2016 07:24 PM, Hans Hagen wrote:
> it should work in the in beta now

Hi Hans,

now I have a problem :-) What should take precedence if an entity is 
both defined in the dtd and as a \xmltexentity? The way I see it, the 
latter: e.g., in the DTD, I might declare something for use in a browser 
but require a different solution when typesetting with ConTeXt. The 
latest and greatest now takes my DTD definitions instead of the 
\xmltexentities, which did not happen before. Is that an unwanted side 
effect or the new default?

Thomas
___________________________________________________________________________________
If your question is of interest to others as well, please add an entry to the Wiki!

maillist : ntg-context@ntg.nl / http://www.ntg.nl/mailman/listinfo/ntg-context
webpage  : http://www.pragma-ade.nl / http://tex.aanhet.net
archive  : http://foundry.supelec.fr/projects/contextrev/
wiki     : http://contextgarden.net
___________________________________________________________________________________

Re: TeX in \xmlsetentity and DTDs in DOCTYPEs

[Kate F]

On 18 January 2016 at 13:30, Thomas A. Schmitz
<thomas.schmitz@uni-bonn.de> wrote:
> On 01/17/2016 07:24 PM, Hans Hagen wrote:
>>
>> it should work in the in beta now

Wonderful! Thank you so much.


> Hi Hans,
>
> now I have a problem :-) What should take precedence if an entity is both
> defined in the dtd and as a \xmltexentity? The way I see it, the latter:
> e.g., in the DTD, I might declare something for use in a browser but require
> a different solution when typesetting with ConTeXt. The latest and greatest
> now takes my DTD definitions instead of the \xmltexentities, which did not
> happen before. Is that an unwanted side effect or the new default?

I was just about to ask the same!

I agree that \xmltexentity ought to have precedence. I have an example:

    <!ENTITY nbsp "&#160;">

Which for some reason is being rendered as U+FFFD REPLACEMENT
CHARACTER (a question-mark in a diamond). I don't know why that comes
out as U+FFFD actually, since if I type a literal non-breaking space
character instead, the PDF comes out just as if I'd used ~ in TeX.
Maybe that's a bug somewhere?

In any case, I think that sometimes it is helpful to override entities
specifically for TeX processing, which aren't relevant for processing
the XML source in other ways.

-- 
Kate
___________________________________________________________________________________
If your question is of interest to others as well, please add an entry to the Wiki!

maillist : ntg-context@ntg.nl / http://www.ntg.nl/mailman/listinfo/ntg-context
webpage  : http://www.pragma-ade.nl / http://tex.aanhet.net
archive  : http://foundry.supelec.fr/projects/contextrev/
wiki     : http://contextgarden.net
___________________________________________________________________________________

Re: TeX in \xmlsetentity and DTDs in DOCTYPEs

[Kate F]

On 18 January 2016 at 13:30, Thomas A. Schmitz
<thomas.schmitz@uni-bonn.de> wrote:
> On 01/17/2016 07:24 PM, Hans Hagen wrote:
>>
>> it should work in the in beta now
>
>
> Hi Hans,
>
> now I have a problem :-) What should take precedence if an entity is both
> defined in the dtd and as a \xmltexentity? The way I see it, the latter:
> e.g., in the DTD, I might declare something for use in a browser but require
> a different solution when typesetting with ConTeXt. The latest and greatest
> now takes my DTD definitions instead of the \xmltexentities, which did not
> happen before. Is that an unwanted side effect or the new default?
>

Ah, there's a bug:

    <!ENTITY i.opt "<option>-i</option>">

This should produce an <option> node in the DOM tree, just as if you'd
typed that out where the entity is used. Currently ConTeXt takes that
as literal text, as if you'd typed "&lt;option&gt;-i&lt;option/&gt;"

Often I wish XML weren't so complex...

-- 
Kate
___________________________________________________________________________________
If your question is of interest to others as well, please add an entry to the Wiki!

maillist : ntg-context@ntg.nl / http://www.ntg.nl/mailman/listinfo/ntg-context
webpage  : http://www.pragma-ade.nl / http://tex.aanhet.net
archive  : http://foundry.supelec.fr/projects/contextrev/
wiki     : http://contextgarden.net
___________________________________________________________________________________

Re: TeX in \xmlsetentity and DTDs in DOCTYPEs

[Hans Hagen]

On 1/18/2016 5:22 PM, Kate F wrote:
> On 18 January 2016 at 13:30, Thomas A. Schmitz
> <thomas.schmitz@uni-bonn.de> wrote:
>> On 01/17/2016 07:24 PM, Hans Hagen wrote:
>>>
>>> it should work in the in beta now
>>
>>
>> Hi Hans,
>>
>> now I have a problem :-) What should take precedence if an entity is both
>> defined in the dtd and as a \xmltexentity? The way I see it, the latter:
>> e.g., in the DTD, I might declare something for use in a browser but require
>> a different solution when typesetting with ConTeXt. The latest and greatest
>> now takes my DTD definitions instead of the \xmltexentities, which did not
>> happen before. Is that an unwanted side effect or the new default?
>>
>
> Ah, there's a bug:
>
>      <!ENTITY i.opt "<option>-i</option>">
>
> This should produce an <option> node in the DOM tree, just as if you'd
> typed that out where the entity is used. Currently ConTeXt takes that
> as literal text, as if you'd typed "&lt;option&gt;-i&lt;option/&gt;"
>
> Often I wish XML weren't so complex...

are you sure? i've never seen that

Hans

-----------------------------------------------------------------
                                           Hans Hagen | PRAGMA ADE
               Ridderstraat 27 | 8061 GH Hasselt | The Netherlands
       tel: 038 477 53 69 | www.pragma-ade.com | www.pragma-pod.nl
-----------------------------------------------------------------
___________________________________________________________________________________
If your question is of interest to others as well, please add an entry to the Wiki!

maillist : ntg-context@ntg.nl / http://www.ntg.nl/mailman/listinfo/ntg-context
webpage  : http://www.pragma-ade.nl / http://tex.aanhet.net
archive  : http://foundry.supelec.fr/projects/contextrev/
wiki     : http://contextgarden.net
___________________________________________________________________________________

Re: TeX in \xmlsetentity and DTDs in DOCTYPEs

[Hans Hagen]

On 1/18/2016 2:30 PM, Thomas A. Schmitz wrote:
> On 01/17/2016 07:24 PM, Hans Hagen wrote:
>> it should work in the in beta now
>
> Hi Hans,
>
> now I have a problem :-) What should take precedence if an entity is

so you have an example ?

> both defined in the dtd and as a \xmltexentity? The way I see it, the
> latter: e.g., in the DTD, I might declare something for use in a browser
> but require a different solution when typesetting with ConTeXt. The
> latest and greatest now takes my DTD definitions instead of the
> \xmltexentities, which did not happen before. Is that an unwanted side
> effect or the new default?

we probably have to make it configureable

-----------------------------------------------------------------
                                           Hans Hagen | PRAGMA ADE
               Ridderstraat 27 | 8061 GH Hasselt | The Netherlands
       tel: 038 477 53 69 | www.pragma-ade.com | www.pragma-pod.nl
-----------------------------------------------------------------
___________________________________________________________________________________
If your question is of interest to others as well, please add an entry to the Wiki!

maillist : ntg-context@ntg.nl / http://www.ntg.nl/mailman/listinfo/ntg-context
webpage  : http://www.pragma-ade.nl / http://tex.aanhet.net
archive  : http://foundry.supelec.fr/projects/contextrev/
wiki     : http://contextgarden.net
___________________________________________________________________________________

Re: TeX in \xmlsetentity and DTDs in DOCTYPEs

[Kate F]

On 18 January 2016 at 19:13, Hans Hagen <pragma@wxs.nl> wrote:
> On 1/18/2016 5:22 PM, Kate F wrote:
>>
>> On 18 January 2016 at 13:30, Thomas A. Schmitz
>> <thomas.schmitz@uni-bonn.de> wrote:
>>>
>>> On 01/17/2016 07:24 PM, Hans Hagen wrote:
>>>>
>>>>
>>>> it should work in the in beta now
>>>
>>>
>>>
>>> Hi Hans,
>>>
>>> now I have a problem :-) What should take precedence if an entity is both
>>> defined in the dtd and as a \xmltexentity? The way I see it, the latter:
>>> e.g., in the DTD, I might declare something for use in a browser but
>>> require
>>> a different solution when typesetting with ConTeXt. The latest and
>>> greatest
>>> now takes my DTD definitions instead of the \xmltexentities, which did
>>> not
>>> happen before. Is that an unwanted side effect or the new default?
>>>
>>
>> Ah, there's a bug:
>>
>>      <!ENTITY i.opt "<option>-i</option>">
>>
>> This should produce an <option> node in the DOM tree, just as if you'd
>> typed that out where the entity is used. Currently ConTeXt takes that
>> as literal text, as if you'd typed "&lt;option&gt;-i&lt;option/&gt;"
>>
>> Often I wish XML weren't so complex...
>
>
> are you sure? i've never seen that
>
> Hans

Yep!

These are called "internal parsed entities". "Parsed" requires that
any tags *inside* the entity must be balanced, unlike in SGML
entities.

Sorry I can't find a clear explanation in the XML spec; it's a pretty
confusing document.
But here's some random person's slide illustrating an example:
http://images.slideplayer.com/23/6622270/slides/slide_47.jpg

libxml2 deals with these correctly, which is what I've been using
(xsltproc and friends) for my documents which use them. I generally
trust libxml2 to get things right.

I use these entities to centralise often-repeated fragments between
documents, kind of like how you might use a primitive macro in TeX.

So for example in one external DTD I have some general things:

    <!ENTITY macro.arg  "<replaceable>macro</replaceable>">
    <!ENTITY equal.lit      "<literal>=</literal>">

And then in one specific document's internal entities, something which
uses them:

        <!ENTITY D2.opt
"<option>-D</option>&macro.arg;&equal.lit;<replaceable>defn</replaceable>">

Then if I change my mind about how I want to mark up "=", for example,
I only have one place to change it. This makes life with XML a little
bit less painful.

-- 
Kate
___________________________________________________________________________________
If your question is of interest to others as well, please add an entry to the Wiki!

maillist : ntg-context@ntg.nl / http://www.ntg.nl/mailman/listinfo/ntg-context
webpage  : http://www.pragma-ade.nl / http://tex.aanhet.net
archive  : http://foundry.supelec.fr/projects/contextrev/
wiki     : http://contextgarden.net
___________________________________________________________________________________

Re: TeX in \xmlsetentity and DTDs in DOCTYPEs

[Kate F]

On 18 January 2016 at 20:07, Hans Hagen <pragma@wxs.nl> wrote:
> On 1/18/2016 2:30 PM, Thomas A. Schmitz wrote:
>>
>> On 01/17/2016 07:24 PM, Hans Hagen wrote:
>>>
>>> it should work in the in beta now
>>
>>
>> Hi Hans,
>>
>> now I have a problem :-) What should take precedence if an entity is
>
>
> so you have an example ?
>
>> both defined in the dtd and as a \xmltexentity? The way I see it, the
>> latter: e.g., in the DTD, I might declare something for use in a browser
>> but require a different solution when typesetting with ConTeXt. The
>> latest and greatest now takes my DTD definitions instead of the
>> \xmltexentities, which did not happen before. Is that an unwanted side
>> effect or the new default?
>
>
> we probably have to make it configureable

I don't think it's necessary to have the precedence configurable - I
think it would make sense to have \xmltexentity always take
precedence.

The usual case would be that there are no user-specified \xmltexentity
definitions present at all, and the document's DTD entities would be
used (by the feature you just added).

So if somebody does have their own \xmltexentity specified, it's
because they want that to be used instead. That would allow a user to
do something specific for ConTeXt processing, but I don't have a good
example for that. Personally I hope I can delete all my \xmltexentity
calls now.

-- 
Kate
___________________________________________________________________________________
If your question is of interest to others as well, please add an entry to the Wiki!

maillist : ntg-context@ntg.nl / http://www.ntg.nl/mailman/listinfo/ntg-context
webpage  : http://www.pragma-ade.nl / http://tex.aanhet.net
archive  : http://foundry.supelec.fr/projects/contextrev/
wiki     : http://contextgarden.net
___________________________________________________________________________________

Re: TeX in \xmlsetentity and DTDs in DOCTYPEs

[Hans Hagen]

On 1/18/2016 9:49 PM, Kate F wrote:
> On 18 January 2016 at 19:13, Hans Hagen <pragma@wxs.nl> wrote:
>> On 1/18/2016 5:22 PM, Kate F wrote:
>>>
>>> On 18 January 2016 at 13:30, Thomas A. Schmitz
>>> <thomas.schmitz@uni-bonn.de> wrote:
>>>>
>>>> On 01/17/2016 07:24 PM, Hans Hagen wrote:
>>>>>
>>>>>
>>>>> it should work in the in beta now
>>>>
>>>>
>>>>
>>>> Hi Hans,
>>>>
>>>> now I have a problem :-) What should take precedence if an entity is both
>>>> defined in the dtd and as a \xmltexentity? The way I see it, the latter:
>>>> e.g., in the DTD, I might declare something for use in a browser but
>>>> require
>>>> a different solution when typesetting with ConTeXt. The latest and
>>>> greatest
>>>> now takes my DTD definitions instead of the \xmltexentities, which did
>>>> not
>>>> happen before. Is that an unwanted side effect or the new default?
>>>>
>>>
>>> Ah, there's a bug:
>>>
>>>       <!ENTITY i.opt "<option>-i</option>">
>>>
>>> This should produce an <option> node in the DOM tree, just as if you'd
>>> typed that out where the entity is used. Currently ConTeXt takes that
>>> as literal text, as if you'd typed "&lt;option&gt;-i&lt;option/&gt;"
>>>
>>> Often I wish XML weren't so complex...
>>
>>
>> are you sure? i've never seen that
>>
>> Hans
>
> Yep!
>
> These are called "internal parsed entities". "Parsed" requires that
> any tags *inside* the entity must be balanced, unlike in SGML
> entities.
>
> Sorry I can't find a clear explanation in the XML spec; it's a pretty
> confusing document.
> But here's some random person's slide illustrating an example:
> http://images.slideplayer.com/23/6622270/slides/slide_47.jpg
>
> libxml2 deals with these correctly, which is what I've been using
> (xsltproc and friends) for my documents which use them. I generally
> trust libxml2 to get things right.
>
> I use these entities to centralise often-repeated fragments between
> documents, kind of like how you might use a primitive macro in TeX.
>
> So for example in one external DTD I have some general things:
>
>      <!ENTITY macro.arg  "<replaceable>macro</replaceable>">
>      <!ENTITY equal.lit      "<literal>=</literal>">
>
> And then in one specific document's internal entities, something which
> uses them:
>
>          <!ENTITY D2.opt
> "<option>-D</option>&macro.arg;&equal.lit;<replaceable>defn</replaceable>">
>
> Then if I change my mind about how I want to mark up "=", for example,
> I only have one place to change it. This makes life with XML a little
> bit less painful.

well, i've learned not to trust all these docs on the web too much and 
applications can do what they want (and thereby even influence standards)

xml pocket reference:

- parsed entity: replacement text that can be referenced

   internal: literal string to be injected (then the example
             shows only text and entities

in your example you use a (decimal) character entity ... the link you 
give says that you cannot use & and % as part of the entities value so 
that would mean your example is wrong

of course we can consider an option to parse the entity as xml

(we can consider < as a trigger for parsing thereby kin dof 
automatically adapting)


Hans

-----------------------------------------------------------------
                                           Hans Hagen | PRAGMA ADE
               Ridderstraat 27 | 8061 GH Hasselt | The Netherlands
       tel: 038 477 53 69 | www.pragma-ade.com | www.pragma-pod.nl
-----------------------------------------------------------------
___________________________________________________________________________________
If your question is of interest to others as well, please add an entry to the Wiki!

maillist : ntg-context@ntg.nl / http://www.ntg.nl/mailman/listinfo/ntg-context
webpage  : http://www.pragma-ade.nl / http://tex.aanhet.net
archive  : http://foundry.supelec.fr/projects/contextrev/
wiki     : http://contextgarden.net
___________________________________________________________________________________

Re: TeX in \xmlsetentity and DTDs in DOCTYPEs

[Hans Hagen]

On 1/18/2016 9:56 PM, Kate F wrote:
> On 18 January 2016 at 20:07, Hans Hagen <pragma@wxs.nl> wrote:
>> On 1/18/2016 2:30 PM, Thomas A. Schmitz wrote:
>>>
>>> On 01/17/2016 07:24 PM, Hans Hagen wrote:
>>>>
>>>> it should work in the in beta now
>>>
>>>
>>> Hi Hans,
>>>
>>> now I have a problem :-) What should take precedence if an entity is
>>
>>
>> so you have an example ?
>>
>>> both defined in the dtd and as a \xmltexentity? The way I see it, the
>>> latter: e.g., in the DTD, I might declare something for use in a browser
>>> but require a different solution when typesetting with ConTeXt. The
>>> latest and greatest now takes my DTD definitions instead of the
>>> \xmltexentities, which did not happen before. Is that an unwanted side
>>> effect or the new default?
>>
>>
>> we probably have to make it configureable
>
> I don't think it's necessary to have the precedence configurable - I
> think it would make sense to have \xmltexentity always take
> precedence.

all is configureable because there's always someone who wants it different

> The usual case would be that there are no user-specified \xmltexentity
> definitions present at all, and the document's DTD entities would be
> used (by the feature you just added).
>
> So if somebody does have their own \xmltexentity specified, it's
> because they want that to be used instead. That would allow a user to
> do something specific for ConTeXt processing, but I don't have a good
> example for that. Personally I hope I can delete all my \xmltexentity
> calls now.

of course that assumes that you don't need to revert to tex for 
rendering them

Hans


-----------------------------------------------------------------
                                           Hans Hagen | PRAGMA ADE
               Ridderstraat 27 | 8061 GH Hasselt | The Netherlands
       tel: 038 477 53 69 | www.pragma-ade.com | www.pragma-pod.nl
-----------------------------------------------------------------
___________________________________________________________________________________
If your question is of interest to others as well, please add an entry to the Wiki!

maillist : ntg-context@ntg.nl / http://www.ntg.nl/mailman/listinfo/ntg-context
webpage  : http://www.pragma-ade.nl / http://tex.aanhet.net
archive  : http://foundry.supelec.fr/projects/contextrev/
wiki     : http://contextgarden.net
___________________________________________________________________________________

Re: TeX in \xmlsetentity and DTDs in DOCTYPEs

[Hans Hagen]

On 1/18/2016 2:30 PM, Thomas A. Schmitz wrote:
> On 01/17/2016 07:24 PM, Hans Hagen wrote:
>> it should work in the in beta now
>
> Hi Hans,
>
> now I have a problem :-) What should take precedence if an entity is
> both defined in the dtd and as a \xmltexentity? The way I see it, the
> latter: e.g., in the DTD, I might declare something for use in a browser
> but require a different solution when typesetting with ConTeXt. The
> latest and greatest now takes my DTD definitions instead of the
> \xmltexentities, which did not happen before. Is that an unwanted side
> effect or the new default?

i already feared that ...

% \disabledirectives[lxml.entities.escaped]
% \disabledirectives[lxml.entities.prefertex]

\setupbodyfont[dejavu]

\xmltexentity{oeps}{\textdegree}

\startbuffer[test]
<?xml version="1.0"?>
<!DOCTYPE blah SYSTEM "blah.dtd" [
      <!ENTITY oeps "indeed">
      <!ENTITY more "&#160;">
]>
<text>
     Random text with an entity: &oeps; test &amp; test

     Random text with an entity: &more; test
</text>
\stopbuffer

\startxmlsetups xml:testsetups
     \xmlsetsetup{#1}{text}{xml:*}
\stopxmlsetups

\xmlregistersetup{xml:testsetups}

\startxmlsetups xml:text
     \xmlflush{#1}\par
\stopxmlsetups

\starttext
     \xmlprocessbuffer{main}{test}{}
\stoptext

i still wonder what happens in tuc files (roundtrip mess)

beta coming

Hans



-----------------------------------------------------------------
                                           Hans Hagen | PRAGMA ADE
               Ridderstraat 27 | 8061 GH Hasselt | The Netherlands
       tel: 038 477 53 69 | www.pragma-ade.com | www.pragma-pod.nl
-----------------------------------------------------------------
___________________________________________________________________________________
If your question is of interest to others as well, please add an entry to the Wiki!

maillist : ntg-context@ntg.nl / http://www.ntg.nl/mailman/listinfo/ntg-context
webpage  : http://www.pragma-ade.nl / http://tex.aanhet.net
archive  : http://foundry.supelec.fr/projects/contextrev/
wiki     : http://contextgarden.net
___________________________________________________________________________________

Re: TeX in \xmlsetentity and DTDs in DOCTYPEs

[Thomas A. Schmitz]

On 01/18/2016 10:26 PM, Hans Hagen wrote:
> i already feared that ...
>
...
>
> i still wonder what happens in tuc files (roundtrip mess)
>
> beta coming


Yep, this seems to be a somewhat difficult area. Was just preparing an 
example when yours arrived. The latest beta seems to be doing the right 
thing. FYI: my example was an entity "linebreak" which I wanted to 
induce a linebreak (duh!) in the pdf but not when I serve the file via 
xsl...

Thanks for looking into it!

Thomas
___________________________________________________________________________________
If your question is of interest to others as well, please add an entry to the Wiki!

maillist : ntg-context@ntg.nl / http://www.ntg.nl/mailman/listinfo/ntg-context
webpage  : http://www.pragma-ade.nl / http://tex.aanhet.net
archive  : http://foundry.supelec.fr/projects/contextrev/
wiki     : http://contextgarden.net
___________________________________________________________________________________

Re: TeX in \xmlsetentity and DTDs in DOCTYPEs

[Kate F]

On 18 January 2016 at 21:16, Hans Hagen <pragma@wxs.nl> wrote:
> On 1/18/2016 9:49 PM, Kate F wrote:
>>
>> On 18 January 2016 at 19:13, Hans Hagen <pragma@wxs.nl> wrote:
>>>
>>> On 1/18/2016 5:22 PM, Kate F wrote:
>>>>
>>>>
>>>> On 18 January 2016 at 13:30, Thomas A. Schmitz
>>>> <thomas.schmitz@uni-bonn.de> wrote:
>>>>>
>>>>>
>>>>> On 01/17/2016 07:24 PM, Hans Hagen wrote:
>>>>>>
>>>>>>
>>>>>>
>>>>>> it should work in the in beta now
>>>>>
>>>>>
>>>>>
>>>>>
>>>>> Hi Hans,
>>>>>
>>>>> now I have a problem :-) What should take precedence if an entity is
>>>>> both
>>>>> defined in the dtd and as a \xmltexentity? The way I see it, the
>>>>> latter:
>>>>> e.g., in the DTD, I might declare something for use in a browser but
>>>>> require
>>>>> a different solution when typesetting with ConTeXt. The latest and
>>>>> greatest
>>>>> now takes my DTD definitions instead of the \xmltexentities, which did
>>>>> not
>>>>> happen before. Is that an unwanted side effect or the new default?
>>>>>
>>>>
>>>> Ah, there's a bug:
>>>>
>>>>       <!ENTITY i.opt "<option>-i</option>">
>>>>
>>>> This should produce an <option> node in the DOM tree, just as if you'd
>>>> typed that out where the entity is used. Currently ConTeXt takes that
>>>> as literal text, as if you'd typed "&lt;option&gt;-i&lt;option/&gt;"
>>>>
>>>> Often I wish XML weren't so complex...
>>>
>>>
>>>
>>> are you sure? i've never seen that
>>>
>>> Hans
>>
>>
>> Yep!
>>
>> These are called "internal parsed entities". "Parsed" requires that
>> any tags *inside* the entity must be balanced, unlike in SGML
>> entities.
>>
>> Sorry I can't find a clear explanation in the XML spec; it's a pretty
>> confusing document.
>> But here's some random person's slide illustrating an example:
>> http://images.slideplayer.com/23/6622270/slides/slide_47.jpg
>>
>> libxml2 deals with these correctly, which is what I've been using
>> (xsltproc and friends) for my documents which use them. I generally
>> trust libxml2 to get things right.
>>
>> I use these entities to centralise often-repeated fragments between
>> documents, kind of like how you might use a primitive macro in TeX.
>>
>> So for example in one external DTD I have some general things:
>>
>>      <!ENTITY macro.arg  "<replaceable>macro</replaceable>">
>>      <!ENTITY equal.lit      "<literal>=</literal>">
>>
>> And then in one specific document's internal entities, something which
>> uses them:
>>
>>          <!ENTITY D2.opt
>>
>> "<option>-D</option>&macro.arg;&equal.lit;<replaceable>defn</replaceable>">
>>
>> Then if I change my mind about how I want to mark up "=", for example,
>> I only have one place to change it. This makes life with XML a little
>> bit less painful.
>
>
> well, i've learned not to trust all these docs on the web too much and
> applications can do what they want (and thereby even influence standards)
>
> xml pocket reference:
>
> - parsed entity: replacement text that can be referenced
>
>   internal: literal string to be injected (then the example
>             shows only text and entities
>
> in your example you use a (decimal) character entity ... the link you give
> says that you cannot use & and % as part of the entities value so that would
> mean your example is wrong
>

Indeed I did not mean that somebody's presentation slides are normative.

This is normative for XML 1.0:
https://www.w3.org/TR/1998/REC-xml-19980210#wf-entities

Which has the following productions:

    extParsedEnt ::= TextDecl? content
    content ::= (element | CharData | Reference | CDSect | PI | Comment)*

So you can see that both elements (by "element") and entity references
(by "Reference") are permitted in the grammar. The latter includes:

    Reference ::= EntityRef | CharRef
    EntityRef::='&' Name ';'

Per this, all my examples are correct.

O'Reilly XML in a Nutshell has an example:
http://docstore.mik.ua/orelly/xml/xmlnut/ch03_04.htm


> of course we can consider an option to parse the entity as xml
>
> (we can consider < as a trigger for parsing thereby kin dof automatically
> adapting)
>

I do not think this is a good idea - per the XML spec, these entities
should always be taken as well-formed fragments of XML. So treating
them otherwise would be incorrect.

Thanks,

-- 
Kate
___________________________________________________________________________________
If your question is of interest to others as well, please add an entry to the Wiki!

maillist : ntg-context@ntg.nl / http://www.ntg.nl/mailman/listinfo/ntg-context
webpage  : http://www.pragma-ade.nl / http://tex.aanhet.net
archive  : http://foundry.supelec.fr/projects/contextrev/
wiki     : http://contextgarden.net
___________________________________________________________________________________

Re: TeX in \xmlsetentity and DTDs in DOCTYPEs

[Kate F]

On 19 January 2016 at 02:16, Kate F <kate@elide.org> wrote:
> On 18 January 2016 at 21:16, Hans Hagen <pragma@wxs.nl> wrote:
>> On 1/18/2016 9:49 PM, Kate F wrote:
>>>
>>> On 18 January 2016 at 19:13, Hans Hagen <pragma@wxs.nl> wrote:
>>>>
>>>> On 1/18/2016 5:22 PM, Kate F wrote:
>>>>>
>>>>>
>>>>> On 18 January 2016 at 13:30, Thomas A. Schmitz
>>>>> <thomas.schmitz@uni-bonn.de> wrote:
>>>>>>
>>>>>>
>>>>>> On 01/17/2016 07:24 PM, Hans Hagen wrote:
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> it should work in the in beta now
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>> Hi Hans,
>>>>>>
>>>>>> now I have a problem :-) What should take precedence if an entity is
>>>>>> both
>>>>>> defined in the dtd and as a \xmltexentity? The way I see it, the
>>>>>> latter:
>>>>>> e.g., in the DTD, I might declare something for use in a browser but
>>>>>> require
>>>>>> a different solution when typesetting with ConTeXt. The latest and
>>>>>> greatest
>>>>>> now takes my DTD definitions instead of the \xmltexentities, which did
>>>>>> not
>>>>>> happen before. Is that an unwanted side effect or the new default?
>>>>>>
>>>>>
>>>>> Ah, there's a bug:
>>>>>
>>>>>       <!ENTITY i.opt "<option>-i</option>">
>>>>>
>>>>> This should produce an <option> node in the DOM tree, just as if you'd
>>>>> typed that out where the entity is used. Currently ConTeXt takes that
>>>>> as literal text, as if you'd typed "&lt;option&gt;-i&lt;option/&gt;"
>>>>>
>>>>> Often I wish XML weren't so complex...
>>>>
>>>>
>>>>
>>>> are you sure? i've never seen that
>>>>
>>>> Hans
>>>
>>>
>>> Yep!
>>>
>>> These are called "internal parsed entities". "Parsed" requires that
>>> any tags *inside* the entity must be balanced, unlike in SGML
>>> entities.
>>>
>>> Sorry I can't find a clear explanation in the XML spec; it's a pretty
>>> confusing document.
>>> But here's some random person's slide illustrating an example:
>>> http://images.slideplayer.com/23/6622270/slides/slide_47.jpg
>>>
>>> libxml2 deals with these correctly, which is what I've been using
>>> (xsltproc and friends) for my documents which use them. I generally
>>> trust libxml2 to get things right.
>>>
>>> I use these entities to centralise often-repeated fragments between
>>> documents, kind of like how you might use a primitive macro in TeX.
>>>
>>> So for example in one external DTD I have some general things:
>>>
>>>      <!ENTITY macro.arg  "<replaceable>macro</replaceable>">
>>>      <!ENTITY equal.lit      "<literal>=</literal>">
>>>
>>> And then in one specific document's internal entities, something which
>>> uses them:
>>>
>>>          <!ENTITY D2.opt
>>>
>>> "<option>-D</option>&macro.arg;&equal.lit;<replaceable>defn</replaceable>">
>>>
>>> Then if I change my mind about how I want to mark up "=", for example,
>>> I only have one place to change it. This makes life with XML a little
>>> bit less painful.
>>
>>
>> well, i've learned not to trust all these docs on the web too much and
>> applications can do what they want (and thereby even influence standards)
>>
>> xml pocket reference:
>>
>> - parsed entity: replacement text that can be referenced
>>
>>   internal: literal string to be injected (then the example
>>             shows only text and entities
>>
>> in your example you use a (decimal) character entity ... the link you give
>> says that you cannot use & and % as part of the entities value so that would
>> mean your example is wrong
>>
>
> Indeed I did not mean that somebody's presentation slides are normative.
>
> This is normative for XML 1.0:
> https://www.w3.org/TR/1998/REC-xml-19980210#wf-entities
>
> Which has the following productions:
>
>     extParsedEnt ::= TextDecl? content
>     content ::= (element | CharData | Reference | CDSect | PI | Comment)*
>
> So you can see that both elements (by "element") and entity references
> (by "Reference") are permitted in the grammar. The latter includes:
>
>     Reference ::= EntityRef | CharRef
>     EntityRef::='&' Name ';'

Sorry, I pointed to an outdated version of the XML 1.0 spec there.

The current version (fifth revision) has the grammar written slightly
differently, but still permits both well-formed <xyz>...</xyz> tags
and &xyz; entities inside entity declarations:

https://www.w3.org/TR/xml/#sec-entity-decl

    [70]    EntityDecl  ::= GEDecl | PEDecl
    [72]    PEDecl      ::= '<!ENTITY' S '%' S Name S PEDef S? '>'
    [74]    PEDef       ::= EntityValue | ExternalID
    [9]     EntityValue ::= '"' ([^%&"] | PEReference | Reference)* '"'
                         |  "'" ([^%&'] | PEReference | Reference)* "'"
    [67]    Reference   ::= EntityRef | CharRef
    [68]    EntityRef   ::= '&' Name ';'

Thus &xyz; is permitted by productions applying ultimately to
EntityRef, and <xyz>...</xyz> is permitted by productions applying
through the * closure of EntityValue's [^%&"] text (where [^...] means
"not").


> Per this, all my examples are correct.
>
> O'Reilly XML in a Nutshell has an example:
> http://docstore.mik.ua/orelly/xml/xmlnut/ch03_04.htm
>
>
>> of course we can consider an option to parse the entity as xml
>>
>> (we can consider < as a trigger for parsing thereby kin dof automatically
>> adapting)
>>
>
> I do not think this is a good idea - per the XML spec, these entities
> should always be taken as well-formed fragments of XML. So treating
> them otherwise would be incorrect.
>
> Thanks,
>
> --
> Kate
___________________________________________________________________________________
If your question is of interest to others as well, please add an entry to the Wiki!

maillist : ntg-context@ntg.nl / http://www.ntg.nl/mailman/listinfo/ntg-context
webpage  : http://www.pragma-ade.nl / http://tex.aanhet.net
archive  : http://foundry.supelec.fr/projects/contextrev/
wiki     : http://contextgarden.net
___________________________________________________________________________________

Re: TeX in \xmlsetentity and DTDs in DOCTYPEs

[Hans Hagen]

On 1/18/2016 10:45 PM, Thomas A. Schmitz wrote:
> On 01/18/2016 10:26 PM, Hans Hagen wrote:
>> i already feared that ...
>>
> ...
>>
>> i still wonder what happens in tuc files (roundtrip mess)
>>
>> beta coming
>
>
> Yep, this seems to be a somewhat difficult area. Was just preparing an
> example when yours arrived. The latest beta seems to be doing the right
> thing. FYI: my example was an entity "linebreak" which I wanted to
> induce a linebreak (duh!) in the pdf but not when I serve the file via
> xsl...

aha. grep for "injector" in the test suite

-----------------------------------------------------------------
                                           Hans Hagen | PRAGMA ADE
               Ridderstraat 27 | 8061 GH Hasselt | The Netherlands
       tel: 038 477 53 69 | www.pragma-ade.com | www.pragma-pod.nl
-----------------------------------------------------------------
___________________________________________________________________________________
If your question is of interest to others as well, please add an entry to the Wiki!

maillist : ntg-context@ntg.nl / http://www.ntg.nl/mailman/listinfo/ntg-context
webpage  : http://www.pragma-ade.nl / http://tex.aanhet.net
archive  : http://foundry.supelec.fr/projects/contextrev/
wiki     : http://contextgarden.net
___________________________________________________________________________________