BUG: % is comment in btx fields

BUG: % is comment in btx fields

[Henri Menke]

Dear list,

The percent character in btx field is taken as a comment characters
which leads to the rest of the field being discarded.  In the MWE
below, the URL is cut off at the % sign and everything after it is
lost.

Cheers, Henri


---


\startbuffer[references]
@misc{kraft,
  author = "Kraft, Nathan",
  title = "Mr Kraft's Virtual Filing Cabinet",
  year = "2015",
  url = "http://mrkraft.wikispaces.com/Mr+Kraft%27s+Virtual+Filing+Cabi
net"
}
\stopbuffer

\usebtxdataset[references.buffer]
\usebtxdefinitions[apa]

\starttext

\nocite[kraft]
\placelistofpublications

\stoptext
___________________________________________________________________________________
If your question is of interest to others as well, please add an entry to the Wiki!

maillist : ntg-context@ntg.nl / http://www.ntg.nl/mailman/listinfo/ntg-context
webpage  : http://www.pragma-ade.nl / http://context.aanhet.net
archive  : https://bitbucket.org/phg/context-mirror/commits/
wiki     : http://contextgarden.net
___________________________________________________________________________________

Re: BUG: % is comment in btx fields

[Martin Schröder]

2017-05-08 23:44 GMT+02:00 Henri Menke <henrimenke@gmail.com>:
> The percent character in btx field is taken as a comment characters
> which leads to the rest of the field being discarded.  In the MWE
> below, the URL is cut off at the % sign and everything after it is
> lost.

See also https://tex.stackexchange.com/q/368730/5763

Best
   Martin
___________________________________________________________________________________
If your question is of interest to others as well, please add an entry to the Wiki!

maillist : ntg-context@ntg.nl / http://www.ntg.nl/mailman/listinfo/ntg-context
webpage  : http://www.pragma-ade.nl / http://context.aanhet.net
archive  : https://bitbucket.org/phg/context-mirror/commits/
wiki     : http://contextgarden.net
___________________________________________________________________________________

Re: BUG: % is comment in btx fields

[Alan Braslau]

Of course, you have to escape the percent character if you do not want
it to be interpreted as a comment... however, in the example below, the
URL does not get rendered properly, nor does using \letterpercent, so
there is something that we need to fix in our treatment of URLs.

\startbuffer[references]

@misc{kraft,
  author = "Kraft, Nathan",
  title = "Mr Kraft's Virtual Filing Cabinet",
  year = "2015",
  url = "http://mrkraft.wikispaces.com/Mr+Kraft\%27s+Virtual+Filing+Cabinet"
}

\stopbuffer

\usebtxdefinitions[apa]
\usebtxdataset[references.buffer]

\starttext

\nocite[kraft]

\placelistofpublications

\showbtxdatasetcompleteness

\stoptext



On Tue, 9 May 2017 09:44:01 +1200
Henri Menke <henrimenke@gmail.com> wrote:

> Dear list,
> 
> The percent character in btx field is taken as a comment characters
> which leads to the rest of the field being discarded.  In the MWE
> below, the URL is cut off at the % sign and everything after it is
> lost.
> 
> Cheers, Henri
> 
> 
> ---
> 
> 
> \startbuffer[references]
> @misc{kraft,
>   author = "Kraft, Nathan",
>   title = "Mr Kraft's Virtual Filing Cabinet",
>   year = "2015",
>   url =
> "http://mrkraft.wikispaces.com/Mr+Kraft%27s+Virtual+Filing+Cabi net"
> }
> \stopbuffer
> 
> \usebtxdataset[references.buffer]
> \usebtxdefinitions[apa]
> 
> \starttext
> 
> \nocite[kraft]
> \placelistofpublications
> 
> \stoptext
> ___________________________________________________________________________________
> If your question is of interest to others as well, please add an
> entry to the Wiki!
> 
> maillist : ntg-context@ntg.nl /
> http://www.ntg.nl/mailman/listinfo/ntg-context webpage  :
> http://www.pragma-ade.nl / http://context.aanhet.net archive  :
> https://bitbucket.org/phg/context-mirror/commits/ wiki     :
> http://contextgarden.net
> ___________________________________________________________________________________


-- 
Alan Braslau
CEA DSM-IRAMIS-SPEC
CNRS UMR 3680
Orme des Merisiers
91191 Gif-sur-Yvette cedex FRANCE
tel: +33 1 69 08 73 15
fax: +33 1 69 08 87 86
mailto:alan.braslau@cea.fr
___________________________________________________________________________________
If your question is of interest to others as well, please add an entry to the Wiki!

maillist : ntg-context@ntg.nl / http://www.ntg.nl/mailman/listinfo/ntg-context
webpage  : http://www.pragma-ade.nl / http://context.aanhet.net
archive  : https://bitbucket.org/phg/context-mirror/commits/
wiki     : http://contextgarden.net
___________________________________________________________________________________

Re: BUG: % is comment in btx fields

[Henri Menke]

On Mon, 2017-05-08 at 16:27 -0600, Alan Braslau wrote:
> Of course, you have to escape the percent character if you do not
> want
> it to be interpreted as a comment... however, in the example below,
> the
> URL does not get rendered properly, nor does using \letterpercent, so
> there is something that we need to fix in our treatment of URLs.

That is easily fixed by making \hyphenatedurl aware of \% being
\letterpercent.


\appendtoks
  \let\%\letterpercent
\to \everyhyphenatedurl


However, I wonder whether the treatment of % in bib files is correct,
as regular BibTeX does treat the % verbatim.  Because the following
LaTeX example works just fine, with unescaped %.  In my opinion this
behaviour should be adopted by ConTeXt.


\begin{filecontents*}{\jobname.bib}
@misc{kraft,
  author = "Kraft, Nathan",
  title = "Mr Kraft's Virtual Filing Cabinet",
  year = "2015",
  url = "http://mrkraft.wikispaces.com/Mr+Kraft%27s+Virtual+Filing+Cabi
net"
}
\end{filecontents*}

\documentclass{article}
\usepackage{apacite,url}
\begin{document}

\nocite{kraft}
\bibliographystyle{apacite}
\bibliography{\jobname.bib}

\end{document}

> 
> \startbuffer[references]
> 
> @misc{kraft,
>   author = "Kraft, Nathan",
>   title = "Mr Kraft's Virtual Filing Cabinet",
>   year = "2015",
>   url = "http://mrkraft.wikispaces.com/Mr+Kraft\%27s+Virtual+Filing+C
> abinet"
> }
> 
> \stopbuffer
> 
> \usebtxdefinitions[apa]
> \usebtxdataset[references.buffer]
> 
> \starttext
> 
> \nocite[kraft]
> 
> \placelistofpublications
> 
> \showbtxdatasetcompleteness
> 
> \stoptext
> 
> 
> 
> On Tue, 9 May 2017 09:44:01 +1200
> Henri Menke <henrimenke@gmail.com> wrote:
> 
> > 
> > Dear list,
> > 
> > The percent character in btx field is taken as a comment characters
> > which leads to the rest of the field being discarded.  In the MWE
> > below, the URL is cut off at the % sign and everything after it is
> > lost.
> > 
> > Cheers, Henri
> > 
> > 
> > ---
> > 
> > 
> > \startbuffer[references]
> > @misc{kraft,
> >   author = "Kraft, Nathan",
> >   title = "Mr Kraft's Virtual Filing Cabinet",
> >   year = "2015",
> >   url =
> > "http://mrkraft.wikispaces.com/Mr+Kraft%27s+Virtual+Filing+Cabi
> > net"
> > }
> > \stopbuffer
> > 
> > \usebtxdataset[references.buffer]
> > \usebtxdefinitions[apa]
> > 
> > \starttext
> > 
> > \nocite[kraft]
> > \placelistofpublications
> > 
> > \stoptext
> > ___________________________________________________________________
> > ________________
> > If your question is of interest to others as well, please add an
> > entry to the Wiki!
> > 
> > maillist : ntg-context@ntg.nl /
> > http://www.ntg.nl/mailman/listinfo/ntg-context webpage  :
> > http://www.pragma-ade.nl / http://context.aanhet.net archive  :
> > https://bitbucket.org/phg/context-mirror/commits/ wiki     :
> > http://contextgarden.net
> > ___________________________________________________________________
> > ________________
> 
___________________________________________________________________________________
If your question is of interest to others as well, please add an entry to the Wiki!

maillist : ntg-context@ntg.nl / http://www.ntg.nl/mailman/listinfo/ntg-context
webpage  : http://www.pragma-ade.nl / http://context.aanhet.net
archive  : https://bitbucket.org/phg/context-mirror/commits/
wiki     : http://contextgarden.net
___________________________________________________________________________________

Re: BUG: % is comment in btx fields

[Alan Braslau]

On Tue, 9 May 2017 11:27:51 +1200
Henri Menke <henrimenke@gmail.com> wrote:

> However, I wonder whether the treatment of % in bib files is correct,
> as regular BibTeX does treat the % verbatim.  Because the following
> LaTeX example works just fine, with unescaped %.  In my opinion this
> behaviour should be adopted by ConTeXt.

From BibTeXing:

7. For Scribe compatibility, the database files allow an @COMMENT
command; it’s not really needed because BibTEX allows in the database
files any comment that’s not within an entry. If you want to comment
out an entry, simply remove the ‘@’ character preceding the entry type.

...

14. LATEX’s comment character ‘%’ is not a comment character in the
database files.




From Tame the BeaST:

New entries always start with @. Anything outside the “argument” of a
“command” starting with an @ is considered as a comment. This gives an
easy way to comment a given entry: just remove the initial @. As usual
when a language allows comments, don’t hesitate to use them so that you
have a clean, ordered, and easy-to-maintain database. Conversely,
anything starting with an @ is considered as being a new entry.
----
There is a special entry type named @comment. The main use of such an
entry type is to comment a large part of the bibliography easily, since
anything outside an entry is already a comment, and commenting out one
entry may be achieved by just removing its initial @.


So bibtex has a funny way of dealing with comments, and do we really
care about Scribe? We made the choice NOT to remain *strictly* bibtex
compatible, after all bibtex is limited to ASCII 7-bit!

The % comment is *universal* in TeX, Metapost, etc. so we chose to
respect this, much stronger in my opinion, convention.

Alan




-- 
Alan Braslau
CEA DSM-IRAMIS-SPEC
CNRS UMR 3680
Orme des Merisiers
91191 Gif-sur-Yvette cedex FRANCE
tel: +33 1 69 08 73 15
fax: +33 1 69 08 87 86
mailto:alan.braslau@cea.fr
___________________________________________________________________________________
If your question is of interest to others as well, please add an entry to the Wiki!

maillist : ntg-context@ntg.nl / http://www.ntg.nl/mailman/listinfo/ntg-context
webpage  : http://www.pragma-ade.nl / http://context.aanhet.net
archive  : https://bitbucket.org/phg/context-mirror/commits/
wiki     : http://contextgarden.net
___________________________________________________________________________________

Re: BUG: % is comment in btx fields

[Rik Kabel]

[-- Attachment #1.1: Type: text/plain, Size: 1366 bytes --]

On 2017-05-08 17:44, Henri Menke wrote:
> Dear list,
>
> The percent character in btx field is taken as a comment characters
> which leads to the rest of the field being discarded.  In the MWE
> below, the URL is cut off at the % sign and everything after it is
> lost.
>
> Cheers, Henri
>
>
> ---
>
>
> \startbuffer[references]
> @misc{kraft,
>    author = "Kraft, Nathan",
>    title = "Mr Kraft's Virtual Filing Cabinet",
>    year = "2015",
>    url = "http://mrkraft.wikispaces.com/Mr+Kraft%27s+Virtual+Filing+Cabi
> net"
> }
> \stopbuffer
>
> \usebtxdataset[references.buffer]
> \usebtxdefinitions[apa]
>
> \starttext
>
> \nocite[kraft]
> \placelistofpublications
>
> \stoptext
The bibtex entry in this example is provided in a buffer. The buffer 
mechanism does not handle characters any differently than other text in 
the file, and % in a ConTeXt buffer has to be escaped if it does not 
signify a comment. If you specify an external (.bib or .bibtex) file to 
supply the data,  % does not need to be escaped. So, this part is simple 
user error.

(With

    \starttext
    \startbuffer
       abc%def

       ghi\%jkl
    \stopbuffer
    \getbuffer
    \stoptext

you do not get %def in the output. Why do you expect different behavior 
if the buffer holds a bibtex entry?)

The way hyphenatedurl handles things may well be worthy of improvement.

-- 
Rik

[-- Attachment #1.2: Type: text/html, Size: 1956 bytes --]

[-- Attachment #2: Type: text/plain, Size: 492 bytes --]

___________________________________________________________________________________
If your question is of interest to others as well, please add an entry to the Wiki!

maillist : ntg-context@ntg.nl / http://www.ntg.nl/mailman/listinfo/ntg-context
webpage  : http://www.pragma-ade.nl / http://context.aanhet.net
archive  : https://bitbucket.org/phg/context-mirror/commits/
wiki     : http://contextgarden.net
___________________________________________________________________________________

Re: BUG: % is comment in btx fields

[Henri Menke]

On Mon, 2017-05-08 at 20:15 -0400, Rik Kabel wrote:
> On 2017-05-08 17:44, Henri Menke wrote:
> > Dear list,
> > 
> > The percent character in btx field is taken as a comment characters
> > which leads to the rest of the field being discarded.  In the MWE
> > below, the URL is cut off at the % sign and everything after it is
> > lost.
> > 
> > Cheers, Henri
> > 
> > 
> > ---
> > 
> > 
> > \startbuffer[references]
> > @misc{kraft,
> >   author = "Kraft, Nathan",
> >   title = "Mr Kraft's Virtual Filing Cabinet",
> >   year = "2015",
> >   url = "http://mrkraft.wikispaces.com/Mr+Kraft%27s+Virtual+Filing+
> > Cabi
> > net"
> > }
> > \stopbuffer
> > 
> > \usebtxdataset[references.buffer]
> > \usebtxdefinitions[apa]
> > 
> > \starttext
> > 
> > \nocite[kraft]
> > \placelistofpublications
> > 
> > \stoptext
>  The bibtex entry in this example is provided in a buffer. The buffer
> mechanism does not handle characters any differently than other text
> in the file, and % in a ConTeXt buffer has to be escaped if it does
> not signify a comment.

That is simply not true!  Content inside ConTeXt buffers is as verbatim
as reading an external file.  Try \typebuffer instead of \getbuffer
your example below and observe that the comment sign is perfectly
preserved.

> If you specify an external (.bib or .bibtex) file to supply the
> data,  % does not need to be escaped. So, this part is simple user
> error.
> (With
> \starttext
> \startbuffer
>   abc%def
> 
>   ghi\%jkl
> \stopbuffer
> \getbuffer
> \stoptext
> you do not get %def in the output. Why do you expect different
> behavior if the buffer holds a bibtex entry?)
> 
> The way hyphenatedurl handles things may well be worthy of
> improvement.
> 
> -- 
> Rik
> _____________________________________________________________________
> ______________
> If your question is of interest to others as well, please add an
> entry to the Wiki!
> 
> maillist : ntg-context@ntg.nl / http://www.ntg.nl/mailman/listinfo/nt
> g-context
> webpage  : http://www.pragma-ade.nl / http://context.aanhet.net
> archive  : https://bitbucket.org/phg/context-mirror/commits/
> wiki     : http://contextgarden.net
> _____________________________________________________________________
> ______________
___________________________________________________________________________________
If your question is of interest to others as well, please add an entry to the Wiki!

maillist : ntg-context@ntg.nl / http://www.ntg.nl/mailman/listinfo/ntg-context
webpage  : http://www.pragma-ade.nl / http://context.aanhet.net
archive  : https://bitbucket.org/phg/context-mirror/commits/
wiki     : http://contextgarden.net
___________________________________________________________________________________

Re: BUG: % is comment in btx fields

[Rik Kabel]

On 2017-05-08 21:40, Henri Menke wrote:
>
> That is simply not true!  Content inside ConTeXt buffers is as verbatim
> as reading an external file.  Try \typebuffer instead of \getbuffer
> your example below and observe that the comment sign is perfectly
> preserved.
>
I am sorry. You are, of course, correct, and I use both, so should know 
better.


-- 
Rik
___________________________________________________________________________________
If your question is of interest to others as well, please add an entry to the Wiki!

maillist : ntg-context@ntg.nl / http://www.ntg.nl/mailman/listinfo/ntg-context
webpage  : http://www.pragma-ade.nl / http://context.aanhet.net
archive  : https://bitbucket.org/phg/context-mirror/commits/
wiki     : http://contextgarden.net
___________________________________________________________________________________

Re: BUG: % is comment in btx fields

[Ulrike Fischer]

Am Mon, 8 May 2017 17:56:08 -0600 schrieb Alan Braslau:

> So bibtex has a funny way of dealing with comments, and do we really
> care about Scribe?

biber handles comments differently than bibtex, see e.g.
https://tex.stackexchange.com/questions/261261/are-comments-discouraged-in-a-bibtex-file

As a thumbrule one can sum the rule up that *inside* field text
comment chars should be input according the tex rule for the field
*type*. That means that normal textfields should use \%, while url's
and other verbatim-like field can use %.



> We made the choice NOT to remain *strictly* bibtex
> compatible, after all bibtex is limited to ASCII 7-bit!

That's not true bibtex has no real problem with 8-bit-encodings, and
works even quite often quite good with utf8 (when you don't try to
use at places where it can be broken into parts.




-- 
Ulrike Fischer 
http://www.troubleshooting-tex.de/

___________________________________________________________________________________
If your question is of interest to others as well, please add an entry to the Wiki!

maillist : ntg-context@ntg.nl / http://www.ntg.nl/mailman/listinfo/ntg-context
webpage  : http://www.pragma-ade.nl / http://context.aanhet.net
archive  : https://bitbucket.org/phg/context-mirror/commits/
wiki     : http://contextgarden.net
___________________________________________________________________________________