Transliterated and original titles/names in citations

Transliterated and original titles/names in citations

[BPJ]

Does anyone know how to handle transliterated titles and names in 
citations, when you want to include both the transliteration and the 
original? Does CSL have any fields for that?

TIA,

/bpj

Re: Transliterated and original titles/names in citations

[Paulo Ney de Souza]

[-- Attachment #1: Type: text/plain, Size: 1481 bytes --]

I do the simple thing which is to use the existent fields and include the
transliteration in between brackets, after the main text...

I made an attempt to separate transliteration from main text a few years
back, but the complications of language were too much to take on --
transliterations have two languages: original and target.

PN

On Wed, Sep 4, 2019, 9:33 AM BPJ <melroch-Re5JQEeQqe8AvxtiuMwx3w@public.gmane.org> wrote:

> Does anyone know how to handle transliterated titles and names in
> citations, when you want to include both the transliteration and the
> original? Does CSL have any fields for that?
>
> TIA,
>
> /bpj
>
> --
> You received this message because you are subscribed to the Google Groups
> "pandoc-discuss" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to pandoc-discuss+unsubscribe-/JYPxA39Uh5TLH3MbocFF+G/Ez6ZCGd0@public.gmane.org
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/pandoc-discuss/0c05fcec-fbb7-aed6-c1ec-e84610bcdd96%40gmail.com
> .
>

-- 
You received this message because you are subscribed to the Google Groups "pandoc-discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email to pandoc-discuss+unsubscribe-/JYPxA39Uh5TLH3MbocFF+G/Ez6ZCGd0@public.gmane.org
To view this discussion on the web visit https://groups.google.com/d/msgid/pandoc-discuss/CAFVhNZODynBBYTfxUk9Gg4y7QGfGCFd926rsc3%3D_89D8p5bPZw%40mail.gmail.com.

[-- Attachment #2: Type: text/html, Size: 2473 bytes --]

Re: Transliterated and original titles/names in citations

[Denis Maier]

[-- Attachment #1.1: Type: text/plain, Size: 1152 bytes --]

Hi,

CSL currently has no fields for transliteration, so you should just include 
the transliteration in the regular fields. Having said that, there is a CSL 
variant called CSL-M (https://citeproc-js.readthedocs.io/en/latest/csl-m/) 
that, among other things, offers special support for multilingual demands. 
Currently, CSL-M is only supported by citeproc-js, so you can use those 
features with the Zotero fork Juris-M, and also with upstream Zotero 
(Zotero will complain about styles using these features, but they will work 
nevertheless). Pandoc-citeproc does not understand CSL-M, but citeproc-rs, 
a new citeproc under development (https://github.com/cormacrelf/citeproc-rs), 
might bring those features also to pandoc.

Best,
Denis

-- 
You received this message because you are subscribed to the Google Groups "pandoc-discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email to pandoc-discuss+unsubscribe-/JYPxA39Uh5TLH3MbocFF+G/Ez6ZCGd0@public.gmane.org
To view this discussion on the web visit https://groups.google.com/d/msgid/pandoc-discuss/16b1ae8c-559d-4576-aeea-d9cb661fbe66%40googlegroups.com.

[-- Attachment #1.2: Type: text/html, Size: 1639 bytes --]

Re: Transliterated and original titles/names in citations

[Joseph Reagle]

Hello Benct Philip, 
On 9/4/19 3:33 AM, BPJ wrote:
> Does anyone know how to handle transliterated titles and names in
> citations, when you want to include both the transliteration and the
> original? Does CSL have any fields for that?

CSL allows for "original-title." None of the sheets I use support it.
CMS17.14.99 recommends putting the alternative in square brackets, but there's lots of leeway.

Re: Transliterated and original titles/names in citations

[BPJ]

[-- Attachment #1: Type: text/plain, Size: 1681 bytes --]

Thanks everyone, especially Denis for the interesting information. Not what
I had hoped for but kind of what I had expected. Usability with
Pandoc→LaTeX is a must for me. I guess I could write my own code to locate
references, look up the data, invoke something which produces a
bibliography in HTML, then massage that HTML into something Pandoc will
convert into LaTeX appropriately. Sounds like a lot of work though —
although not as much as implementing a whole citation processor!

Another question is whether pandoc-citeproc will ignore unknown fields or
throw an error? If the former I can have custom fields with the separate
original and transliterated fields, as I need for my searching purposes and
automatically update the combined version in the official fields. If the
latter I'll need to produce a special version for Pandoc's consumption with
the custom fields removed, which complicates things considerably.

Den ons 4 sep. 2019 09:33BPJ <melroch-Re5JQEeQqe8AvxtiuMwx3w@public.gmane.org> skrev:

> Does anyone know how to handle transliterated titles and names in
> citations, when you want to include both the transliteration and the
> original? Does CSL have any fields for that?
>
> TIA,
>
> /bpj
>

-- 
You received this message because you are subscribed to the Google Groups "pandoc-discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email to pandoc-discuss+unsubscribe-/JYPxA39Uh5TLH3MbocFF+G/Ez6ZCGd0@public.gmane.org
To view this discussion on the web visit https://groups.google.com/d/msgid/pandoc-discuss/CADAJKhAd8iPOD%3D-gaZn_7-03t07HjSTq0SiH9Bq%3DHJ%3DymtSU1A%40mail.gmail.com.

[-- Attachment #2: Type: text/html, Size: 2385 bytes --]

Re: Transliterated and original titles/names in citations

[John MacFarlane]

I believe pandoc-citeproc will ignore the unknown fields, but
you can try it.

BPJ <melroch-Re5JQEeQqe8AvxtiuMwx3w@public.gmane.org> writes:

> Thanks everyone, especially Denis for the interesting information. Not what
> I had hoped for but kind of what I had expected. Usability with
> Pandoc→LaTeX is a must for me. I guess I could write my own code to locate
> references, look up the data, invoke something which produces a
> bibliography in HTML, then massage that HTML into something Pandoc will
> convert into LaTeX appropriately. Sounds like a lot of work though —
> although not as much as implementing a whole citation processor!
>
> Another question is whether pandoc-citeproc will ignore unknown fields or
> throw an error? If the former I can have custom fields with the separate
> original and transliterated fields, as I need for my searching purposes and
> automatically update the combined version in the official fields. If the
> latter I'll need to produce a special version for Pandoc's consumption with
> the custom fields removed, which complicates things considerably.
>
> Den ons 4 sep. 2019 09:33BPJ <melroch-Re5JQEeQqe8AvxtiuMwx3w@public.gmane.org> skrev:
>
>> Does anyone know how to handle transliterated titles and names in
>> citations, when you want to include both the transliteration and the
>> original? Does CSL have any fields for that?
>>
>> TIA,
>>
>> /bpj
>>
>
> -- 
> You received this message because you are subscribed to the Google Groups "pandoc-discuss" group.
> To unsubscribe from this group and stop receiving emails from it, send an email to pandoc-discuss+unsubscribe-/JYPxA39Uh5TLH3MbocFF+G/Ez6ZCGd0@public.gmane.org
> To view this discussion on the web visit https://groups.google.com/d/msgid/pandoc-discuss/CADAJKhAd8iPOD%3D-gaZn_7-03t07HjSTq0SiH9Bq%3DHJ%3DymtSU1A%40mail.gmail.com.

-- 
You received this message because you are subscribed to the Google Groups "pandoc-discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email to pandoc-discuss+unsubscribe-/JYPxA39Uh5TLH3MbocFF+G/Ez6ZCGd0@public.gmane.org
To view this discussion on the web visit https://groups.google.com/d/msgid/pandoc-discuss/yh480kh85pcf5d.fsf%40johnmacfarlane.net.

Re: Transliterated and original titles/names in citations

[Denis Maier]

[-- Attachment #1.1: Type: text/plain, Size: 3298 bytes --]

1. I am almost sure that Pandoc will just ignore unknown fields, at least 
that's what happened (unfortunately and to my disappointment) the last time 
I have tried to use non-standard fields in a style.

2. If you don't want to hardcode the transliteration into one of the 
existing fields, the easiest solution might be to use something along the 
lines of this script.

```
import json
import argparse

parser = argparse.ArgumentParser(description='Merge title and subtitle 
fields in CSL JSON with custom separator')

parser.add_argument('-f', '--from', action="store", dest="from_file")
parser.add_argument('-t', '--to', action="store", dest="to_file")
parser.add_argument('-s', '--separator', action="store", dest="separator", 
default=": ")

args = parser.parse_args()

# Load JSON File

with open(args.from_file, 'r') as from_file:
    references = json.load(from_file)

# Merge Title and Subtitle Fields

for reference in references:
    try:
        reference['title'] = reference['title'] + args.separator + 
reference['subtitle']
        reference.pop('subtitle', None)
    except:
        pass

# Write results back to File

with open(args.to_file, "w") as to_file:
    json.dump(references, to_file, indent=2,ensure_ascii=False)

```

This will merge title and subtitle fields, but you can of course adapt this 
to merge `title` and `title:transliteration`. (You'll need to you have your 
reference data available in CSL JSON.)

Best,
Denis

Am Freitag, 6. September 2019 21:49:16 UTC+2 schrieb BP:
>
> Thanks everyone, especially Denis for the interesting information. Not 
> what I had hoped for but kind of what I had expected. Usability with 
> Pandoc→LaTeX is a must for me. I guess I could write my own code to locate 
> references, look up the data, invoke something which produces a 
> bibliography in HTML, then massage that HTML into something Pandoc will 
> convert into LaTeX appropriately. Sounds like a lot of work though — 
> although not as much as implementing a whole citation processor!
>
> Another question is whether pandoc-citeproc will ignore unknown fields or 
> throw an error? If the former I can have custom fields with the separate 
> original and transliterated fields, as I need for my searching purposes and 
> automatically update the combined version in the official fields. If the 
> latter I'll need to produce a special version for Pandoc's consumption with 
> the custom fields removed, which complicates things considerably.
>
> Den ons 4 sep. 2019 09:33BPJ <mel...-Re5JQEeQqe8AvxtiuMwx3w@public.gmane.org <javascript:>> skrev:
>
>> Does anyone know how to handle transliterated titles and names in 
>> citations, when you want to include both the transliteration and the 
>> original? Does CSL have any fields for that?
>>
>> TIA,
>>
>> /bpj
>>
>

-- 
You received this message because you are subscribed to the Google Groups "pandoc-discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email to pandoc-discuss+unsubscribe-/JYPxA39Uh5TLH3MbocFF+G/Ez6ZCGd0@public.gmane.org
To view this discussion on the web visit https://groups.google.com/d/msgid/pandoc-discuss/38609be6-0edc-4755-b2e7-6448795c907c%40googlegroups.com.

[-- Attachment #1.2: Type: text/html, Size: 4755 bytes --]

Re: Transliterated and original titles/names in citations

[BPJ]

[-- Attachment #1: Type: text/plain, Size: 4204 bytes --]

Yes, something like that, although mine is written in Perl and uses
YAML::PP to parse both YAML and JSON which works just fine. :-)

Den lör 7 sep. 2019 00:05Denis Maier <maier.de-Re5JQEeQqe8AvxtiuMwx3w@public.gmane.org> skrev:

> 1. I am almost sure that Pandoc will just ignore unknown fields, at least
> that's what happened (unfortunately and to my disappointment) the last time
> I have tried to use non-standard fields in a style.
>
> 2. If you don't want to hardcode the transliteration into one of the
> existing fields, the easiest solution might be to use something along the
> lines of this script.
>
> ```
> import json
> import argparse
>
> parser = argparse.ArgumentParser(description='Merge title and subtitle
> fields in CSL JSON with custom separator')
>
> parser.add_argument('-f', '--from', action="store", dest="from_file")
> parser.add_argument('-t', '--to', action="store", dest="to_file")
> parser.add_argument('-s', '--separator', action="store", dest="separator",
> default=": ")
>
> args = parser.parse_args()
>
> # Load JSON File
>
> with open(args.from_file, 'r') as from_file:
>     references = json.load(from_file)
>
> # Merge Title and Subtitle Fields
>
> for reference in references:
>     try:
>         reference['title'] = reference['title'] + args.separator +
> reference['subtitle']
>         reference.pop('subtitle', None)
>     except:
>         pass
>
> # Write results back to File
>
> with open(args.to_file, "w") as to_file:
>     json.dump(references, to_file, indent=2,ensure_ascii=False)
>
> ```
>
> This will merge title and subtitle fields, but you can of course adapt
> this to merge `title` and `title:transliteration`. (You'll need to you have
> your reference data available in CSL JSON.)
>
> Best,
> Denis
>
> Am Freitag, 6. September 2019 21:49:16 UTC+2 schrieb BP:
>>
>> Thanks everyone, especially Denis for the interesting information. Not
>> what I had hoped for but kind of what I had expected. Usability with
>> Pandoc→LaTeX is a must for me. I guess I could write my own code to locate
>> references, look up the data, invoke something which produces a
>> bibliography in HTML, then massage that HTML into something Pandoc will
>> convert into LaTeX appropriately. Sounds like a lot of work though —
>> although not as much as implementing a whole citation processor!
>>
>> Another question is whether pandoc-citeproc will ignore unknown fields or
>> throw an error? If the former I can have custom fields with the separate
>> original and transliterated fields, as I need for my searching purposes and
>> automatically update the combined version in the official fields. If the
>> latter I'll need to produce a special version for Pandoc's consumption with
>> the custom fields removed, which complicates things considerably.
>>
>> Den ons 4 sep. 2019 09:33BPJ <mel...-Re5JQEeQqe8AvxtiuMwx3w@public.gmane.org> skrev:
>>
>>> Does anyone know how to handle transliterated titles and names in
>>> citations, when you want to include both the transliteration and the
>>> original? Does CSL have any fields for that?
>>>
>>> TIA,
>>>
>>> /bpj
>>>
>> --
> You received this message because you are subscribed to the Google Groups
> "pandoc-discuss" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to pandoc-discuss+unsubscribe-/JYPxA39Uh5TLH3MbocFF+G/Ez6ZCGd0@public.gmane.org
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/pandoc-discuss/38609be6-0edc-4755-b2e7-6448795c907c%40googlegroups.com
> <https://groups.google.com/d/msgid/pandoc-discuss/38609be6-0edc-4755-b2e7-6448795c907c%40googlegroups.com?utm_medium=email&utm_source=footer>
> .
>

-- 
You received this message because you are subscribed to the Google Groups "pandoc-discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email to pandoc-discuss+unsubscribe-/JYPxA39Uh5TLH3MbocFF+G/Ez6ZCGd0@public.gmane.org
To view this discussion on the web visit https://groups.google.com/d/msgid/pandoc-discuss/CADAJKhD1Yk6diC3RVC4SnSyEx9avVOto0nX3ZzbOXqxmmzDHHA%40mail.gmail.com.

[-- Attachment #2: Type: text/html, Size: 5837 bytes --]

Re: Transliterated and original titles/names in citations

[Maria Shinoto]

I have the same problem with different writing systems, translations, and transliterations in citations. 

To me the problem turned out that there is not one solution that will work with all journals etc. My conclusion was to store the literature privately as I need it, then export the data into Bib(La)tex data (or CSL, which I did not use yet) that will create a standard bibliography as requested by the journal. 

E.g. do some journals request an English translation as title and the original text in brackets just behind the title. In that case, I export my private fields "title-transl-en" + " [" + "title-orig-jap" + "]" into the title field of a Bibtex database for just this article. In other cases, I would use just "title-transl-en" in the title field and a comment at the end of the citation "in Japanese", etc. -- So I am free with my own data, and I am in command of my export scripts (to CSL, Bibtex etc.), the exported text file then works as if it was ordinary Western bibliography. 

A bit different approach, until now it worked well for me. 

Maria




> Am 07.09.2019 um 04:49 schrieb BPJ <melroch-Re5JQEeQqe8AvxtiuMwx3w@public.gmane.org>:
> 
> Thanks everyone, especially Denis for the interesting information. Not what I had hoped for but kind of what I had expected. Usability with Pandoc→LaTeX is a must for me. I guess I could write my own code to locate references, look up the data, invoke something which produces a bibliography in HTML, then massage that HTML into something Pandoc will convert into LaTeX appropriately. Sounds like a lot of work though — although not as much as implementing a whole citation processor!
> 
> Another question is whether pandoc-citeproc will ignore unknown fields or throw an error? If the former I can have custom fields with the separate original and transliterated fields, as I need for my searching purposes and automatically update the combined version in the official fields. If the latter I'll need to produce a special version for Pandoc's consumption with the custom fields removed, which complicates things considerably.
> 
> Den ons 4 sep. 2019 09:33BPJ <melroch-Re5JQEeQqe8AvxtiuMwx3w@public.gmane.org> skrev:
> Does anyone know how to handle transliterated titles and names in 
> citations, when you want to include both the transliteration and the 
> original? Does CSL have any fields for that?
> 
> TIA,
> 
> /bpj
> 
> -- 
> You received this message because you are subscribed to the Google Groups "pandoc-discuss" group.
> To unsubscribe from this group and stop receiving emails from it, send an email to pandoc-discuss+unsubscribe-/JYPxA39Uh5TLH3MbocFF+G/Ez6ZCGd0@public.gmane.org
> To view this discussion on the web visit https://groups.google.com/d/msgid/pandoc-discuss/CADAJKhAd8iPOD%3D-gaZn_7-03t07HjSTq0SiH9Bq%3DHJ%3DymtSU1A%40mail.gmail.com.

-- 
You received this message because you are subscribed to the Google Groups "pandoc-discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email to pandoc-discuss+unsubscribe-/JYPxA39Uh5TLH3MbocFF+G/Ez6ZCGd0@public.gmane.org
To view this discussion on the web visit https://groups.google.com/d/msgid/pandoc-discuss/A4E70922-943A-49D3-9D9A-F5AC38E025AA%40zaw.uni-heidelberg.de.

Re: Transliterated and original titles/names in citations

[Denis Maier]

[-- Attachment #1: Type: text/plain, Size: 141 bytes --]

What kind of script do you use for that? 

By the way, if your target is biblatex you could use buyer's toolmode to do the transformation...

Re: Transliterated and original titles/names in citations

[Maria Shinoto]

I just wrote my own exports from a personal BibLatex database in JabRef, as if I wanted to create a bibliography style. Just, that the result is a BibTex database that can then be exported to a bibliography with standard scripts and still contains all information that a normal BibTex database would not contain. 

What is "buyer's toolmode"?


> Am 07.09.2019 um 14:45 schrieb Denis Maier <maier.de-Re5JQEeQqe8AvxtiuMwx3w@public.gmane.org>:
> 
> What kind of script do you use for that? 
> 
> By the way, if your target is biblatex you could use buyer's toolmode to do the transformation...
> 
> -- 
> You received this message because you are subscribed to the Google Groups "pandoc-discuss" group.
> To unsubscribe from this group and stop receiving emails from it, send an email to pandoc-discuss+unsubscribe-/JYPxA39Uh5TLH3MbocFF+G/Ez6ZCGd0@public.gmane.org
> To view this discussion on the web visit https://groups.google.com/d/msgid/pandoc-discuss/f17993e8-dddc-4f10-a905-dc84fe6a5782%40googlegroups.com.

-- 
You received this message because you are subscribed to the Google Groups "pandoc-discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email to pandoc-discuss+unsubscribe-/JYPxA39Uh5TLH3MbocFF+G/Ez6ZCGd0@public.gmane.org
To view this discussion on the web visit https://groups.google.com/d/msgid/pandoc-discuss/F0FB7353-D963-4F54-BF29-F9477F194044%40zaw.uni-heidelberg.de.

Re: Transliterated and original titles/names in citations

[Denis Maier]

[-- Attachment #1: Type: text/plain, Size: 70 bytes --]

I wanred to write: biber's toolmode. (Autocorrection got in the way.)

Re: Transliterated and original titles/names in citations

[Maria Shinoto]

That sounds good, a lot has changed since I set up my little system. Will check out the biber tools. 

> Am 08.09.2019 um 14:09 schrieb Denis Maier <maier.de-Re5JQEeQqe8AvxtiuMwx3w@public.gmane.org>:
> 
> I wanred to write: biber's toolmode. (Autocorrection got in the way.)
> 
> -- 
> You received this message because you are subscribed to the Google Groups "pandoc-discuss" group.
> To unsubscribe from this group and stop receiving emails from it, send an email to pandoc-discuss+unsubscribe-/JYPxA39Uh5TLH3MbocFF+G/Ez6ZCGd0@public.gmane.org
> To view this discussion on the web visit https://groups.google.com/d/msgid/pandoc-discuss/e4843c0c-7cf5-43fc-8653-83ddc075f954%40googlegroups.com.

Re: Transliterated and original titles/names in citations

[BPJ]

[-- Attachment #1: Type: text/plain, Size: 2097 bytes --]

I just realized two things which make matters much worse:

1.  Not all publications accept the same transliteration schemes. Just by
surveying one author's references to his own works in one bibliography I
find that his surname, Яблонский, can be transliterated in five different
ways (although two predominate)! So I'll need both a `transliterated title`
field and a field `transliterated authors` field with (in each item) a
mapping of alternative transliterations. Even Icelandic needs to be
transliterated sometimes, e.g. Þórður becoming Thórdur (with data loss!)

2. Sorting. Latin letters like _č, š,  ž_ need to sort as _c, s, z_ and
probably _Þ_ must sometimes sort like _Th_ and sometimes after _z_! This
needs sometimes tailored locale dependent sorting! Accented letters can
ideally be handled by entering things in NFC and hoping that sort
algorithms ignore combining marks, but then e.g. in Scandinavian languages
_ö_ sorts not as _o_ but at the end of the alphabet (ideally _þ, æ, ø, å,
ä, ö_ go at the end of the alphabet in that order, but often _æ/ä, ø/ö_ are
conflated either before or after _å_!). Anyway it seems CSL has no
customizable sort key field. I know how to handle these things myself with
[Unicode::Collate][] but that at least means some postprocessing of as yet
unknown complexity.

Den ons 4 sep. 2019 09:33BPJ <melroch-Re5JQEeQqe8AvxtiuMwx3w@public.gmane.org> skrev:

> Does anyone know how to handle transliterated titles and names in
> citations, when you want to include both the transliteration and the
> original? Does CSL have any fields for that?
>
> TIA,
>
> /bpj
>

-- 
You received this message because you are subscribed to the Google Groups "pandoc-discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email to pandoc-discuss+unsubscribe-/JYPxA39Uh5TLH3MbocFF+G/Ez6ZCGd0@public.gmane.org
To view this discussion on the web visit https://groups.google.com/d/msgid/pandoc-discuss/CADAJKhButR28wyxanhAEYJnjmnRtBQ8OraU0aZ86BHRmkm1SjA%40mail.gmail.com.

[-- Attachment #2: Type: text/html, Size: 2803 bytes --]

Re: Transliterated and original titles/names in citations

[BPJ]

[-- Attachment #1: Type: text/plain, Size: 2313 bytes --]

Oops, forgot the link:

https://metacpan.org/pod/Unicode::Collate

Den lör 7 sep. 2019 12:05BPJ <melroch-Re5JQEeQqe8AvxtiuMwx3w@public.gmane.org> skrev:

> I just realized two things which make matters much worse:
>
> 1.  Not all publications accept the same transliteration schemes. Just by
> surveying one author's references to his own works in one bibliography I
> find that his surname, Яблонский, can be transliterated in five different
> ways (although two predominate)! So I'll need both a `transliterated title`
> field and a field `transliterated authors` field with (in each item) a
> mapping of alternative transliterations. Even Icelandic needs to be
> transliterated sometimes, e.g. Þórður becoming Thórdur (with data loss!)
>
> 2. Sorting. Latin letters like _č, š,  ž_ need to sort as _c, s, z_ and
> probably _Þ_ must sometimes sort like _Th_ and sometimes after _z_! This
> needs sometimes tailored locale dependent sorting! Accented letters can
> ideally be handled by entering things in NFC and hoping that sort
> algorithms ignore combining marks, but then e.g. in Scandinavian languages
> _ö_ sorts not as _o_ but at the end of the alphabet (ideally _þ, æ, ø, å,
> ä, ö_ go at the end of the alphabet in that order, but often _æ/ä, ø/ö_ are
> conflated either before or after _å_!). Anyway it seems CSL has no
> customizable sort key field. I know how to handle these things myself with
> [Unicode::Collate][] but that at least means some postprocessing of as yet
> unknown complexity.
>
> Den ons 4 sep. 2019 09:33BPJ <melroch-Re5JQEeQqe8AvxtiuMwx3w@public.gmane.org> skrev:
>
>> Does anyone know how to handle transliterated titles and names in
>> citations, when you want to include both the transliteration and the
>> original? Does CSL have any fields for that?
>>
>> TIA,
>>
>> /bpj
>>
>

-- 
You received this message because you are subscribed to the Google Groups "pandoc-discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email to pandoc-discuss+unsubscribe-/JYPxA39Uh5TLH3MbocFF+G/Ez6ZCGd0@public.gmane.org
To view this discussion on the web visit https://groups.google.com/d/msgid/pandoc-discuss/CADAJKhDm8bibSjJfrM6W69qM_j1N9tPHEgRwaic4bZmrsB1CVw%40mail.gmail.com.

[-- Attachment #2: Type: text/html, Size: 3356 bytes --]

Re: Transliterated and original titles/names in citations

['Nick Bart' via pandoc-discuss]

[-- Attachment #1: Type: text/plain, Size: 4470 bytes --]

On sorting/collation, from https://pandoc.org/installing.html (note that you’ll have to compile pandoc-citeproc yourself; it’s not contained in the precompiled packages):

> By default pandoc-citeproc uses the "i;unicode-casemap" method to sort
> bibliography entries (RFC 5051). If you would like to use the
> locale-sensitive unicode collation algorithm instead, specify the
> `unicode_collation` flag:

>     cabal install pandoc-citeproc -funicode_collation

> Note that this requires the text-icu library, which in turn depends on
> the C library icu4c. Installation directions vary by platform. Here is
> how it might work on macOS with Homebrew:

>     brew install icu4c
>     cabal install --extra-lib-dirs=/usr/local/Cellar/icu4c/51.1/lib \
>     --extra-include-dirs=/usr/local/Cellar/icu4c/51.1/include \
>     -funicode_collation text-icu pandoc-citeproc

Some of these paths on longer seem to be accurate, and I’m using stack rather than cabal, so my current incantation on macOS is:

```
  stack install pandoc-citeproc \
    --flag "pandoc-citeproc:unicode_collation" \
    --extra-lib-dirs=/usr/local/opt/icu4c/lib \
    --extra-include-dirs=/usr/local/opt/icu4c/include
```

I’ve been using this for quite a while now, and the resulting collation in, e.g., Danish, German, Spanish, seems to be absolutely correct.

‐‐‐‐‐‐‐ Original Message ‐‐‐‐‐‐‐
On Saturday, September 7, 2019 10:10 AM, BPJ <melroch-Re5JQEeQqe8AvxtiuMwx3w@public.gmane.org> wrote:

> Oops, forgot the link:
>
> https://metacpan.org/pod/Unicode::Collate
>
> Den lör 7 sep. 2019 12:05BPJ <melroch-Re5JQEeQqe8AvxtiuMwx3w@public.gmane.org> skrev:
>
>> I just realized two things which make matters much worse:
>>
>> 1.  Not all publications accept the same transliteration schemes. Just by surveying one author's references to his own works in one bibliography I find that his surname, Яблонский, can be transliterated in five different ways (although two predominate)! So I'll need both a `transliterated title` field and a field `transliterated authors` field with (in each item) a mapping of alternative transliterations. Even Icelandic needs to be transliterated sometimes, e.g. Þórður becoming Thórdur (with data loss!)
>>
>> 2. Sorting. Latin letters like _č, š,  ž_ need to sort as _c, s, z_ and probably _Þ_ must sometimes sort like _Th_ and sometimes after _z_! This needs sometimes tailored locale dependent sorting! Accented letters can ideally be handled by entering things in NFC and hoping that sort algorithms ignore combining marks, but then e.g. in Scandinavian languages _ö_ sorts not as _o_ but at the end of the alphabet (ideally _þ, æ, ø, å, ä, ö_ go at the end of the alphabet in that order, but often _æ/ä, ø/ö_ are conflated either before or after _å_!). Anyway it seems CSL has no customizable sort key field. I know how to handle these things myself with [Unicode::Collate][] but that at least means some postprocessing of as yet unknown complexity.
>>
>> Den ons 4 sep. 2019 09:33BPJ <melroch-Re5JQEeQqe8AvxtiuMwx3w@public.gmane.org> skrev:
>>
>>> Does anyone know how to handle transliterated titles and names in
>>> citations, when you want to include both the transliteration and the
>>> original? Does CSL have any fields for that?
>>>
>>> TIA,
>>>
>>> /bpj
>
> --
> You received this message because you are subscribed to the Google Groups "pandoc-discuss" group.
> To unsubscribe from this group and stop receiving emails from it, send an email to pandoc-discuss+unsubscribe-/JYPxA39Uh5TLH3MbocFF+G/Ez6ZCGd0@public.gmane.org
> To view this discussion on the web visit [https://groups.google.com/d/msgid/pandoc-discuss/CADAJKhDm8bibSjJfrM6W69qM_j1N9tPHEgRwaic4bZmrsB1CVw%40mail.gmail.com](https://groups.google.com/d/msgid/pandoc-discuss/CADAJKhDm8bibSjJfrM6W69qM_j1N9tPHEgRwaic4bZmrsB1CVw%40mail.gmail.com?utm_medium=email&utm_source=footer).

-- 
You received this message because you are subscribed to the Google Groups "pandoc-discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email to pandoc-discuss+unsubscribe-/JYPxA39Uh5TLH3MbocFF+G/Ez6ZCGd0@public.gmane.org
To view this discussion on the web visit https://groups.google.com/d/msgid/pandoc-discuss/fIV6s9kzbISHgHoeDhgEEA9GgrDoyN6BgQZqqDi3ZqPGCKYi77R0CfaRAzDizlcsyySuTW-hUJ09r490q5YnNTuTK0A-Ag8aRrD7Fca9gfM%3D%40protonmail.com.

[-- Attachment #2: Type: text/html, Size: 6584 bytes --]

Re: Transliterated and original titles/names in citations

[Denis Maier]

[-- Attachment #1.1: Type: text/plain, Size: 2578 bytes --]

Regarding 1. you can perhaps look into how Juris-M deals with language 
variants: 
https://juris-m.readthedocs.io/en/latest/dev-sync-simplification.html
Looks quite solid. Basically it looks like this:
field--language-variant
title--he-alalc97 => title, Hebrew, transliterated according to the Library 
of Congress rules.

Best,
Denis

Am Samstag, 7. September 2019 12:05:48 UTC+2 schrieb BP:
>
> I just realized two things which make matters much worse:
>
> 1.  Not all publications accept the same transliteration schemes. Just by 
> surveying one author's references to his own works in one bibliography I 
> find that his surname, Яблонский, can be transliterated in five different 
> ways (although two predominate)! So I'll need both a `transliterated title` 
> field and a field `transliterated authors` field with (in each item) a 
> mapping of alternative transliterations. Even Icelandic needs to be 
> transliterated sometimes, e.g. Þórður becoming Thórdur (with data loss!)
>
> 2. Sorting. Latin letters like _č, š,  ž_ need to sort as _c, s, z_ and 
> probably _Þ_ must sometimes sort like _Th_ and sometimes after _z_! This 
> needs sometimes tailored locale dependent sorting! Accented letters can 
> ideally be handled by entering things in NFC and hoping that sort 
> algorithms ignore combining marks, but then e.g. in Scandinavian languages 
> _ö_ sorts not as _o_ but at the end of the alphabet (ideally _þ, æ, ø, å, 
> ä, ö_ go at the end of the alphabet in that order, but often _æ/ä, ø/ö_ are 
> conflated either before or after _å_!). Anyway it seems CSL has no 
> customizable sort key field. I know how to handle these things myself with 
> [Unicode::Collate][] but that at least means some postprocessing of as yet 
> unknown complexity.
>
> Den ons 4 sep. 2019 09:33BPJ <mel...-Re5JQEeQqe8AvxtiuMwx3w@public.gmane.org <javascript:>> skrev:
>
>> Does anyone know how to handle transliterated titles and names in 
>> citations, when you want to include both the transliteration and the 
>> original? Does CSL have any fields for that?
>>
>> TIA,
>>
>> /bpj
>>
>

-- 
You received this message because you are subscribed to the Google Groups "pandoc-discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email to pandoc-discuss+unsubscribe-/JYPxA39Uh5TLH3MbocFF+G/Ez6ZCGd0@public.gmane.org
To view this discussion on the web visit https://groups.google.com/d/msgid/pandoc-discuss/bb1e7b0b-d02e-4059-9842-46e7d29004ec%40googlegroups.com.

[-- Attachment #1.2: Type: text/html, Size: 3617 bytes --]

Re: Transliterated and original titles/names in citations

[Maria Shinoto]

This exactly my scenario. I have transscriptions from Japanese into Chinese or Western Scripts (several transcriptive systems), and of course, sorting is a problem. But until now I could solve all this with a Biblatex database that is tailored to my own needs with very many extra fields -- I do not fill them all but only if needed -- that I export with different fields into a standard Bibtex database that works for a certain publication, and with Biber, all sorting etc. worked just as needed. 

Unfortunately, at the moment this is not my workflow, but I will return to that this year again and hope to be able to explain this in more detail as well as perhaps set up some rules that work beyond my special needs in East Asia. But I am sure you could solve your problem with this approach easily and certainly with more expertise than I do.  

> Am 07.09.2019 um 19:05 schrieb BPJ <melroch-Re5JQEeQqe8AvxtiuMwx3w@public.gmane.org>:
> 
> I just realized two things which make matters much worse:
> 
> 1.  Not all publications accept the same transliteration schemes. Just by surveying one author's references to his own works in one bibliography I find that his surname, Яблонский, can be transliterated in five different ways (although two predominate)! So I'll need both a `transliterated title` field and a field `transliterated authors` field with (in each item) a mapping of alternative transliterations. Even Icelandic needs to be transliterated sometimes, e.g. Þórður becoming Thórdur (with data loss!)
> 
> 2. Sorting. Latin letters like _č, š,  ž_ need to sort as _c, s, z_ and probably _Þ_ must sometimes sort like _Th_ and sometimes after _z_! This needs sometimes tailored locale dependent sorting! Accented letters can ideally be handled by entering things in NFC and hoping that sort algorithms ignore combining marks, but then e.g. in Scandinavian languages _ö_ sorts not as _o_ but at the end of the alphabet (ideally _þ, æ, ø, å, ä, ö_ go at the end of the alphabet in that order, but often _æ/ä, ø/ö_ are conflated either before or after _å_!). Anyway it seems CSL has no customizable sort key field. I know how to handle these things myself with [Unicode::Collate][] but that at least means some postprocessing of as yet unknown complexity.
> 
> Den ons 4 sep. 2019 09:33BPJ <melroch-Re5JQEeQqe8AvxtiuMwx3w@public.gmane.org> skrev:
> Does anyone know how to handle transliterated titles and names in 
> citations, when you want to include both the transliteration and the 
> original? Does CSL have any fields for that?
> 
> TIA,
> 
> /bpj
> 
> -- 
> You received this message because you are subscribed to the Google Groups "pandoc-discuss" group.
> To unsubscribe from this group and stop receiving emails from it, send an email to pandoc-discuss+unsubscribe-/JYPxA39Uh5TLH3MbocFF+G/Ez6ZCGd0@public.gmane.org
> To view this discussion on the web visit https://groups.google.com/d/msgid/pandoc-discuss/CADAJKhButR28wyxanhAEYJnjmnRtBQ8OraU0aZ86BHRmkm1SjA%40mail.gmail.com.

-- 
You received this message because you are subscribed to the Google Groups "pandoc-discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email to pandoc-discuss+unsubscribe-/JYPxA39Uh5TLH3MbocFF+G/Ez6ZCGd0@public.gmane.org
To view this discussion on the web visit https://groups.google.com/d/msgid/pandoc-discuss/2DF40D83-A1D5-48BD-972C-2D7CF8127038%40zaw.uni-heidelberg.de.