Bibliographic Databases

Bibliographic Databases

[Robin Kirkham]

Dear all,

I want to set up a shared bibliographic reference database for my  
research group, and I'm looking at software like refbase http:// 
refbase.sourceforge.net/ or refdb http://refdb.sourceforge.net/ to  
replace the somewhat random collection of personal BibTeX .bib files  
we have.

Does anyone have any experience or advice to offer in using such  
things, and hooking them up to ConTeXt?  Most of these systems will  
of course emit a .bib file which will obviously work, but will any  
emit the .bbl so I can forget about BibTeX?  Will luatex one day  
connect to a bib database and fetch the details of a cited  
reference?  Is there a ConTeXt "approved" way forward for this sort  
of thing?

Thanks in advance, Robin
___________________________________________________________________________________
If your question is of interest to others as well, please add an entry to the Wiki!

maillist : ntg-context@ntg.nl / http://www.ntg.nl/mailman/listinfo/ntg-context
webpage  : http://www.pragma-ade.nl / http://tex.aanhet.net
archive  : https://foundry.supelec.fr/projects/contextrev/
wiki     : http://contextgarden.net
___________________________________________________________________________________

Re: Bibliographic Databases

[Andreas Wagner]

Hello Robin, hello list,

* Robin Kirkham wrote on Apr/18/2008:
> I want to set up a shared bibliographic reference database for my  research group, and I'm looking at 
> software like refbase http://refbase.sourceforge.net/ or refdb http://refdb.sourceforge.net/ to  replace 
> the somewhat random collection of personal BibTeX .bib files  we have.
>
> Does anyone have any experience or advice to offer in using such  things, and hooking them up to ConTeXt?  
> Most of these systems will  of course emit a .bib file which will obviously work, but will any  emit the 
> .bbl so I can forget about BibTeX?  Will luatex one day  connect to a bib database and fetch the details 
> of a cited  reference?  Is there a ConTeXt "approved" way forward for this sort  of thing?

Not that I am able to help you very much further, but I am right now in a similar situation. I am in the 
process of setting up a db with wikindx http://wikindx.sf.net/ The main developer is obviously not working 
with any TeX flavour, but there is some Bibtex im-/export (which also concerns only .bib files) that I am 
ATM fiddling with, but I thought I might add that one to the list of databases you mentioned.

Cheers,
Andreas
___________________________________________________________________________________
If your question is of interest to others as well, please add an entry to the Wiki!

maillist : ntg-context@ntg.nl / http://www.ntg.nl/mailman/listinfo/ntg-context
webpage  : http://www.pragma-ade.nl / http://tex.aanhet.net
archive  : https://foundry.supelec.fr/projects/contextrev/
wiki     : http://contextgarden.net
___________________________________________________________________________________

Re: Bibliographic Databases

[Taco Hoekwater]

Robin Kirkham wrote:
> Does anyone have any experience or advice to offer in using such  
> things, and hooking them up to ConTeXt?  Most of these systems will  
> of course emit a .bib file which will obviously work, but will any  

Please have a look at the biblographic module. It comes with its own
bst files that convert bib to a private data format.

   http://modules.contextgarden.net/bib

Currently, the bibliographic module only supports .bib files, but:
if you want to be ready for the future, find a reference manager
that can export to MODS:

   http://www.loc.gov/standards/mods/

Best wishes,
Taco
___________________________________________________________________________________
If your question is of interest to others as well, please add an entry to the Wiki!

maillist : ntg-context@ntg.nl / http://www.ntg.nl/mailman/listinfo/ntg-context
webpage  : http://www.pragma-ade.nl / http://tex.aanhet.net
archive  : https://foundry.supelec.fr/projects/contextrev/
wiki     : http://contextgarden.net
___________________________________________________________________________________

Re: Bibliographic Databases

[Wolfgang Schuster]

On Fri, Apr 18, 2008 at 10:08 AM, Taco Hoekwater <taco@elvenkind.com> wrote:
>
>
> Robin Kirkham wrote:
> > Does anyone have any experience or advice to offer in using such
> > things, and hooking them up to ConTeXt?  Most of these systems will
> > of course emit a .bib file which will obviously work, but will any
>
> Please have a look at the biblographic module. It comes with its own
> bst files that convert bib to a private data format.
>
>   http://modules.contextgarden.net/bib
>
> Currently, the bibliographic module only supports .bib files, but:
> if you want to be ready for the future, find a reference manager
> that can export to MODS:
>
>   http://www.loc.gov/standards/mods/

Or even better write a new bib module with xml in MODS format as database :-)

Wolfgang
___________________________________________________________________________________
If your question is of interest to others as well, please add an entry to the Wiki!

maillist : ntg-context@ntg.nl / http://www.ntg.nl/mailman/listinfo/ntg-context
webpage  : http://www.pragma-ade.nl / http://tex.aanhet.net
archive  : https://foundry.supelec.fr/projects/contextrev/
wiki     : http://contextgarden.net
___________________________________________________________________________________

Re: Bibliographic Databases

[Andreas Wagner]

Hello Taco, hello list,

* Taco Hoekwater wrote on Apr/18/2008:
> if you want to be ready for the future, find a reference manager that can export to MODS:
>
>   http://www.loc.gov/standards/mods/

Just out of curiosity: What are your reasons for preferring this over TEI:

http://www.tei-c.org/release/doc/tei-p5-doc/html/CO.html#COBI

(I have only by your mail learnt about MODS and am currently working through it, so please excuse me if that 
question doesn't even make sense at all.)

Thanks,
Andreas
___________________________________________________________________________________
If your question is of interest to others as well, please add an entry to the Wiki!

maillist : ntg-context@ntg.nl / http://www.ntg.nl/mailman/listinfo/ntg-context
webpage  : http://www.pragma-ade.nl / http://tex.aanhet.net
archive  : https://foundry.supelec.fr/projects/contextrev/
wiki     : http://contextgarden.net
___________________________________________________________________________________

Re: Bibliographic Databases

[Taco Hoekwater]

Hi,

Andreas Wagner wrote:
> Just out of curiosity: What are your reasons for preferring this over TEI:

MODS was a logical choice mostly my background (scientific publishers
=> MARC databases => MODS), and that BruceD'Arcus liked it.  Btw,
his blog is full of bibliographic articles, if you are interested:

   http://community.muohio.edu/blogs/darcusb/

(but it looks like he has switched over to RDF now)

I am not really set to any particular xml format, and there are
more mainstream choices (risx comes to mind).

But the few times I've had to work with TEI stuff I found that you
can easily get much more than you bargained for. Bibliographic data
is not easy on its own, and a format that allows (almost promotes)
extra tags to be embedded also is not helping at all.

Look at this:

   http://www.tei-c.org/release/doc/tei-p5-doc/html/ref-author.html

Just the 'core' module is already pretty complex, but 'namesdates'
and 'linking' are definately also required for a useful bibliographic
database.

The nice, consise examples in the TEI docs are misleading because

   <author>Lucy Allen Paton</author>

is useless, more specifics are needed. We need at least this:

   <author>
     <persName>
       <forename>Lucy</forename>
       <forename>Allen</forename>
       <surname>Paton</surname>
     </persName>
   </author>

But with the use of <persName>, there are suddenly a gazillion
ways an author can encode the same name  (and it does not
preclude any of the other ways to encode a name).

   http://www.tei-c.org/release/doc/tei-p5-doc/html/ND.html#NDPER

Etc. etc. Imagine having to support that in a simple context module.

Cheers, Taco


___________________________________________________________________________________
If your question is of interest to others as well, please add an entry to the Wiki!

maillist : ntg-context@ntg.nl / http://www.ntg.nl/mailman/listinfo/ntg-context
webpage  : http://www.pragma-ade.nl / http://tex.aanhet.net
archive  : https://foundry.supelec.fr/projects/contextrev/
wiki     : http://contextgarden.net
___________________________________________________________________________________

Re: Bibliographic Databases

[Andreas Wagner]

Hello Taco, hello list,

* Taco Hoekwater wrote on Apr/18/2008:
> Andreas Wagner wrote:
>> Just out of curiosity: What are your reasons for preferring this over TEI:
>
> MODS was a logical choice mostly my background (scientific publishers 
> => MARC databases => MODS), and that BruceD'Arcus liked it.  Btw, 
> his blog is full of bibliographic articles, if you are interested:
>
[...and then some...]

Thank you really a lot, Taco. I have been more or less learning about these things for a few weeks and you 
have indeed provided me with a couple of very good resources and concerns.

Cheers,
Andreas
___________________________________________________________________________________
If your question is of interest to others as well, please add an entry to the Wiki!

maillist : ntg-context@ntg.nl / http://www.ntg.nl/mailman/listinfo/ntg-context
webpage  : http://www.pragma-ade.nl / http://tex.aanhet.net
archive  : https://foundry.supelec.fr/projects/contextrev/
wiki     : http://contextgarden.net
___________________________________________________________________________________

Re: Bibliographic Databases

[George N. White III]

On Thu, Apr 17, 2008 at 11:19 AM, Robin Kirkham <robin.kirkham@csiro.au> wrote:

> Dear all,
>
>  I want to set up a shared bibliographic reference database for my
>  research group, and I'm looking at software like refbase http://
>  refbase.sourceforge.net/ or refdb http://refdb.sourceforge.net/ to
>  replace the somewhat random collection of personal BibTeX .bib files
>  we have.
>
>  Does anyone have any experience or advice to offer in using such
>  things, and hooking them up to ConTeXt?  Most of these systems will
>  of course emit a .bib file which will obviously work, but will any
>  emit the .bbl so I can forget about BibTeX?  Will luatex one day
>  connect to a bib database and fetch the details of a cited
>  reference?  Is there a ConTeXt "approved" way forward for this sort
>  of thing?

I can tell you a few things that don't work!  In our lab we have both
TeX and Word users.  Many of them had been using a DOS package
called papyrus, using a special markup that could be translated
to tex (.bbl) files.  Nothing we found was really satisfactory, so
we bought EndNote, which could import from papyrus via "refer"
format and can export to "almost bibtex".  One problem is that
EndNote uses unicode, so we end up with è, etc. that must
be translated for some user's versions of bibtex.  The database
now has a nearly infinite variety of different quote marks:
`a`, 'a', ``a'', "a", etc. depending on how the entry was made
(many are pasted from online or pdf sources).

EndNote is really designed for individual users, although sold
in bulk.  If 2 people open the same database on a shared
drive they end up with a corrupt database.

In my view, a bibliographic database needs to store each
reference in the "source" or original format, whether bibtex,
refer, or one of the newer forms, and provide translators
and version tracking, so each file can have forks for different
uses (e.g., ascii vs unicode char. sets) and edits can be
preserved for the next user.  In practice, people just dump
selected refs to a bib file, make the .bbl file, and fix problems
there, so fixes rarely make it back to the master database.
If they did, we would still have accents and quote marks
being switched back and forth depending on who last used
the entry.


-- 
George N. White III <aa056@chebucto.ns.ca>
Head of St. Margarets Bay, Nova Scotia
___________________________________________________________________________________
If your question is of interest to others as well, please add an entry to the Wiki!

maillist : ntg-context@ntg.nl / http://www.ntg.nl/mailman/listinfo/ntg-context
webpage  : http://www.pragma-ade.nl / http://tex.aanhet.net
archive  : https://foundry.supelec.fr/projects/contextrev/
wiki     : http://contextgarden.net
___________________________________________________________________________________

Re: Bibliographic Databases

[Bruce D'Arcus]

Taco Hoekwater <taco <at> elvenkind.com> writes:

> 
> Hi,
> 
> Andreas Wagner wrote:
> > Just out of curiosity: What are your reasons for preferring this over TEI:
> 
> MODS was a logical choice mostly my background (scientific publishers
> => MARC databases => MODS), and that BruceD'Arcus liked it.  Btw,
> his blog is full of bibliographic articles, if you are interested:
> 
>    http://community.muohio.edu/blogs/darcusb/
> 
> (but it looks like he has switched over to RDF now)

Yes, but ...
 
> I am not really set to any particular xml format, and there are
> more mainstream choices (risx comes to mind).

... I'd say for the design of something like mbib v2 I'd advocate an internal
model that abstracts away from any particular more concrete representation. So
think in terms of maybe a standard input driver, but leave room for easy
development of others. 

There's some work going on a Python version of my citeproc effort, for example,
and he's planning input drivers for MODS, RDF, BibTeX, etc.

<http://xbiblio.svn.sourceforge.net/viewvc/xbiblio/citeproc-py/citeproc/>

This makes is easy for someone to write another input driver for some SQL model.
 
> But the few times I've had to work with TEI stuff I found that you
> can easily get much more than you bargained for. Bibliographic data
> is not easy on its own, and a format that allows (almost promotes)
> extra tags to be embedded also is not helping at all.
> 
> Look at this:
> 
>    http://www.tei-c.org/release/doc/tei-p5-doc/html/ref-author.html
> 
> Just the 'core' module is already pretty complex, but 'namesdates'
> and 'linking' are definately also required for a useful bibliographic
> database.
> 
> The nice, consise examples in the TEI docs are misleading because
> 
>    <author>Lucy Allen Paton</author>
> 
> is useless, more specifics are needed. We need at least this:
> 
>    <author>
>      <persName>
>        <forename>Lucy</forename>
>        <forename>Allen</forename>
>        <surname>Paton</surname>
>      </persName>
>    </author>
> 
> But with the use of <persName>, there are suddenly a gazillion
> ways an author can encode the same name  (and it does not
> preclude any of the other ways to encode a name).
> 
>    http://www.tei-c.org/release/doc/tei-p5-doc/html/ND.html#NDPER
> 
> Etc. etc. Imagine having to support that in a simple context module.

In the XML citation style language I designed [1] (which *could* serve as the
basis for that "internal model" I mention above), there's an implicit notion
that any name can have both a sort form and a display form, and that they may
(but in contexts like Eastern Europe or Asia often don't) differ. 

This makes things in many ways both simpler, and more general (works for
organizations, as well as is more international-friendly than traditional
first/last). You just handle the details you note above in the input drive code.

Bruce

[1]
<http://xbiblio.svn.sourceforge.net/viewvc/xbiblio/csl/schema/trunk/csl.rnc?view=markup>

___________________________________________________________________________________
If your question is of interest to others as well, please add an entry to the Wiki!

maillist : ntg-context@ntg.nl / http://www.ntg.nl/mailman/listinfo/ntg-context
webpage  : http://www.pragma-ade.nl / http://tex.aanhet.net
archive  : https://foundry.supelec.fr/projects/contextrev/
wiki     : http://contextgarden.net
___________________________________________________________________________________

Re: Bibliographic Databases

[Bruce D\'Arcus]

Taco Hoekwater <taco <at> elvenkind.com> writes:

[snip]

> But the few times I've had to work with TEI stuff I found that you
> can easily get much more than you bargained for. Bibliographic data
> is not easy on its own, and a format that allows (almost promotes)
> extra tags to be embedded also is not helping at all.

... MODS has some of these issues too. Consider these are both valid:

<name type="personal">
  <namePart>Jane Doe</namePart>
  <role>
    <roleTerm type="text">creator</roleTerm>
  </role>
</name>

<name type="personal">
  <namePart type="given">Jane</namePart>
  <namePart type="family">Doe</namePart>
  <role>
    <roleTerm type="text">creator</roleTerm>
  </role>
</name>

So in many formats there's a balance between flexibility and
brevity/predictability. 

FWIW, I've just settled on RDF for my own data needs between it provides the
formal rigor of relational databases (that XML per se lacks), but much more
flexibility. 

But as I said in the previous note, I don't think the data format has to matter
that much to formatting software (at its core that is).

Bruce

___________________________________________________________________________________
If your question is of interest to others as well, please add an entry to the Wiki!

maillist : ntg-context@ntg.nl / http://www.ntg.nl/mailman/listinfo/ntg-context
webpage  : http://www.pragma-ade.nl / http://tex.aanhet.net
archive  : https://foundry.supelec.fr/projects/contextrev/
wiki     : http://contextgarden.net
___________________________________________________________________________________

Re: Bibliographic Databases

[Dietrich Rordorf / MDPI]

Hi All,

Why not try an existing, online tool.? There are severals, most of them 
support export to various formats, e.g. BibTeX, RIS, EndNote, etc.. Some 
of them have a Word plug-in, too.

http://www.connotea.org/
Free online reference management tool for scientists
Nature Publishing Group

http://www.citeulike.org/
Free online reference management tool for scientists
Oversity Ldt.

http://www.bibsonomy.org/
Free online reference management tool for scientists
University of Kassel

http://www.zotero.org/
Free online reference management tool for scientists
WordPress

http://www.2collab.com/
Free online group collaboration tool for scientists
Elsevier B.V.


Otherwaise, for Desktop solutions see a comparision:


Reference Management Software Comparison:
http://en.wikipedia.org/wiki/Comparison_of_reference_management_software
http://mahbub.wordpress.com/2007/03/04/comparison-of-free-bibliographic-managers/

Dietrich 


--
Mr. Dietrich Rordorf
MDPI Center
Matthaeusstrasse 11
CH-4057 Basel
Switzerland

E-mail: rordorf@mdpi.org
Skype:  d.rordorf

Tel. +41 61 683 77 34 (office)
Tel. +41 76 561 41 83 (mobile)
Fax  +41 61 302 89 18 

 


>
> ------------------------------------------------------------------------
>
> Subject:
> Re: [NTG-context] Bibliographic Databases
> From:
> Robin Kirkham <robin.kirkham@csiro.au>
> Date:
> Sun, 20 Apr 2008 19:13:28 +1000
> To:
> ntg-context@ntg.nl
>
> To:
> ntg-context@ntg.nl
>
>
> On 20 April 2008, "George N. White III" <gnwiii@gmail.com> wrote:
>
>> On Thu, Apr 17, 2008 at 11:19 AM, Robin Kirkham 
>> <robin.kirkham@csiro.au> wrote:
>>
>>>  I want to set up a shared bibliographic reference database for my
>>>  research group, [...]
>>
>> I can tell you a few things that don't work!  In our lab we have both
>> TeX and Word users.  Many of them had been using a DOS package
>> called papyrus, using a special markup that could be translated
>> to tex (.bbl) files.  Nothing we found was really satisfactory, so
>> we bought EndNote, which could import from papyrus via "refer"
>> format and can export to "almost bibtex".  One problem is that
>> EndNote uses unicode, so we end up with è, etc. that must
>> be translated for some user's versions of bibtex.  The database
>> now has a nearly infinite variety of different quote marks:
>> `a`, 'a', ``a'', "a", etc. depending on how the entry was made
>> (many are pasted from online or pdf sources).
>>
>> EndNote is really designed for individual users, although sold
>> in bulk.  If 2 people open the same database on a shared
>> drive they end up with a corrupt database.
>>
>> In my view, a bibliographic database needs to store each
>> reference in the "source" or original format, whether bibtex,
>> refer, or one of the newer forms, and provide translators
>> and version tracking, so each file can have forks for different
>> uses (e.g., ascii vs unicode char. sets) and edits can be
>> preserved for the next user.  In practice, people just dump
>> selected refs to a bib file, make the .bbl file, and fix problems
>> there, so fixes rarely make it back to the master database.
>> If they did, we would still have accents and quote marks
>> being switched back and forth depending on who last used
>> the entry.
>
>
> Thanks George. EndNote is I believe the corporately-approved solution 
> here, and similar disasters occur when people try and share its data 
> files. For this reason I don't call these sorts of personal-level 
> programs "databases" (any more than I'll call a .bib file a database).
>
> The TeX folk don't fare much better. Multiple personal .bib files are 
> common, often with duplicate references but different citation keys, 
> leading to rather variable results depending on who run LaTeX/ConTeXt 
> on the file. Inconsistency in data entry is also a problem, although 
> for us, accents and quotes don't seem to too big a problem. 
> (`Authorless', i.e., corporate author documents, like data sheets, 
> seems to be more of an issue).
>
> For this reason I'm looking for a proper SQL database solution like 
> refdb or refbase (or maybe wikindx, thanks Andreas) with a web 
> front-end that will hopefully enforce somewhat more consistent data 
> entry, and maybe even auto-generate citation keys. Taco, your remarks 
> regarding interchange formats are valuable (both refdb and refbase 
> support MODS XML output).
>
> Robin
> ------------------------------------------------------------------------
>
> ___________________________________________________________________________________
> If your question is of interest to others as well, please add an entry to the Wiki!
>
> maillist : ntg-context@ntg.nl / http://www.ntg.nl/mailman/listinfo/ntg-context
> webpage  : http://www.pragma-ade.nl / http://tex.aanhet.net
> archive  : https://foundry.supelec.fr/projects/contextrev/
> wiki     : http://contextgarden.net
> ___________________________________________________________________________________
>   
___________________________________________________________________________________
If your question is of interest to others as well, please add an entry to the Wiki!

maillist : ntg-context@ntg.nl / http://www.ntg.nl/mailman/listinfo/ntg-context
webpage  : http://www.pragma-ade.nl / http://tex.aanhet.net
archive  : https://foundry.supelec.fr/projects/contextrev/
wiki     : http://contextgarden.net
___________________________________________________________________________________

Re: Bibliographic Databases

[Robin Kirkham]

On 20 April 2008, "George N. White III" <gnwiii@gmail.com> wrote:

> On Thu, Apr 17, 2008 at 11:19 AM, Robin Kirkham  
> <robin.kirkham@csiro.au> wrote:
>
>>  I want to set up a shared bibliographic reference database for my
>>  research group, [...]
>
> I can tell you a few things that don't work!  In our lab we have both
> TeX and Word users.  Many of them had been using a DOS package
> called papyrus, using a special markup that could be translated
> to tex (.bbl) files.  Nothing we found was really satisfactory, so
> we bought EndNote, which could import from papyrus via "refer"
> format and can export to "almost bibtex".  One problem is that
> EndNote uses unicode, so we end up with è, etc. that must
> be translated for some user's versions of bibtex.  The database
> now has a nearly infinite variety of different quote marks:
> `a`, 'a', ``a'', "a", etc. depending on how the entry was made
> (many are pasted from online or pdf sources).
>
> EndNote is really designed for individual users, although sold
> in bulk.  If 2 people open the same database on a shared
> drive they end up with a corrupt database.
>
> In my view, a bibliographic database needs to store each
> reference in the "source" or original format, whether bibtex,
> refer, or one of the newer forms, and provide translators
> and version tracking, so each file can have forks for different
> uses (e.g., ascii vs unicode char. sets) and edits can be
> preserved for the next user.  In practice, people just dump
> selected refs to a bib file, make the .bbl file, and fix problems
> there, so fixes rarely make it back to the master database.
> If they did, we would still have accents and quote marks
> being switched back and forth depending on who last used
> the entry.


Thanks George. EndNote is I believe the corporately-approved solution  
here, and similar disasters occur when people try and share its data  
files. For this reason I don't call these sorts of personal-level  
programs "databases" (any more than I'll call a .bib file a database).

The TeX folk don't fare much better. Multiple personal .bib files are  
common, often with duplicate references but different citation keys,  
leading to rather variable results depending on who run LaTeX/ConTeXt  
on the file. Inconsistency in data entry is also a problem, although  
for us, accents and quotes don't seem to too big a problem.  
(`Authorless', i.e., corporate author documents, like data sheets,  
seems to be more of an issue).

For this reason I'm looking for a proper SQL database solution like  
refdb or refbase (or maybe wikindx, thanks Andreas) with a web front- 
end that will hopefully enforce somewhat more consistent data entry,  
and maybe even auto-generate citation keys. Taco, your remarks  
regarding interchange formats are valuable (both refdb and refbase  
support MODS XML output).

Robin
___________________________________________________________________________________
If your question is of interest to others as well, please add an entry to the Wiki!

maillist : ntg-context@ntg.nl / http://www.ntg.nl/mailman/listinfo/ntg-context
webpage  : http://www.pragma-ade.nl / http://tex.aanhet.net
archive  : https://foundry.supelec.fr/projects/contextrev/
wiki     : http://contextgarden.net
___________________________________________________________________________________