CSL styles from zotero.org

CSL styles from zotero.org

[John MacFarlane]

I fetched the svn repository at https://www.zotero.org/svn/csl/
and tried to use some of the styles there, but got parsing errors. There seem
to be some differences in format between these files and the ones we've been
testing with. Have there been changes in the csl spec?

Are the files in the csl repository more recent or less
recent than the ones citeproc-hs can handle?

If less recent, how do we get versions that citeproc-hs
can handle?

If more recent, shouldn't citeproc-hs be modified to handle
them?


John

Re: CSL styles from zotero.org

[Andrea Rossato]

On Tue, Nov 23, 2010 at 09:43:58PM -0800, John MacFarlane wrote:
> I fetched the svn repository at https://www.zotero.org/svn/csl/
> and tried to use some of the styles there, but got parsing errors. There seem
> to be some differences in format between these files and the ones we've been
> testing with. Have there been changes in the csl spec?
> 
> Are the files in the csl repository more recent or less
> recent than the ones citeproc-hs can handle?
> 
> If less recent, how do we get versions that citeproc-hs
> can handle?
> 
> If more recent, shouldn't citeproc-hs be modified to handle
> them?


For the time being the Zotero repository only hosts CSL-0.8 styles,
while citeproc-hs requires CSL-1.0 styles. Those styles will be
upgraded to CSL-1.0 as soon as zotero will complete the switch to
citeproc-js.

You can update styles by using the utility you can find here:

http://bitbucket.org/bdarcus/csl-utils/

I'm still a bit puzzled on how citeproc should handle the problem:
probably citeproc-hs should produce an error saying that the style
version is not supported. But the style version can be checked after
parsing it, while old styles will not be parsed.

If compiled with HXT citeprc could just use the xsl stylesheet to
update the style.

Anyway this is something to fix before a release.

Andrea


> 
> 
> John
> 
> -- 
> You received this message because you are subscribed to the Google Groups "pandoc-discuss" group.
> To post to this group, send email to pandoc-discuss-/JYPxA39Uh5TLH3MbocFF+G/Ez6ZCGd0@public.gmane.org
> To unsubscribe from this group, send email to pandoc-discuss+unsubscribe-/JYPxA39Uh5TLH3MbocFF+G/Ez6ZCGd0@public.gmane.org
> For more options, visit this group at http://groups.google.com/group/pandoc-discuss?hl=en.
> 

Re: CSL styles from zotero.org

[John MacFarlane]

+++ Andrea Rossato [Nov 24 10 09:25 ]:
> On Tue, Nov 23, 2010 at 09:43:58PM -0800, John MacFarlane wrote:
> > I fetched the svn repository at https://www.zotero.org/svn/csl/
> > and tried to use some of the styles there, but got parsing errors. There seem
> > to be some differences in format between these files and the ones we've been
> > testing with. Have there been changes in the csl spec?
> > 
> > Are the files in the csl repository more recent or less
> > recent than the ones citeproc-hs can handle?
> > 
> > If less recent, how do we get versions that citeproc-hs
> > can handle?
> > 
> > If more recent, shouldn't citeproc-hs be modified to handle
> > them?
> 
> 
> For the time being the Zotero repository only hosts CSL-0.8 styles,
> while citeproc-hs requires CSL-1.0 styles. Those styles will be
> upgraded to CSL-1.0 as soon as zotero will complete the switch to
> citeproc-js.
> 
> You can update styles by using the utility you can find here:
> 
> http://bitbucket.org/bdarcus/csl-utils/

Is there a repository with updated versions of the styles? It would be good
to be able to point people to something like that, rather than asking them to
convert the old styles.

> I'm still a bit puzzled on how citeproc should handle the problem:
> probably citeproc-hs should produce an error saying that the style
> version is not supported. But the style version can be checked after
> parsing it, while old styles will not be parsed.

Yes, I think an error message is crucial, as this will be a common issue
until zotero is updated. Presumably there's some kind of "signature" that all
the 0.8 styles will have? So, if unpickleXML fails, maybe you could check the
xml for some 0.8-specific features.

> If compiled with HXT citeprc could just use the xsl stylesheet to
> update the style.
> 
> Anyway this is something to fix before a release.
> 
> Andrea
> 
> 
> > 
> > 
> > John
> > 
> > -- 
> > You received this message because you are subscribed to the Google Groups "pandoc-discuss" group.
> > To post to this group, send email to pandoc-discuss-/JYPxA39Uh5TLH3MbocFF+G/Ez6ZCGd0@public.gmane.org
> > To unsubscribe from this group, send email to pandoc-discuss+unsubscribe-/JYPxA39Uh5TLH3MbocFF+G/Ez6ZCGd0@public.gmane.org
> > For more options, visit this group at http://groups.google.com/group/pandoc-discuss?hl=en.
> > 
> 
> -- 
> You received this message because you are subscribed to the Google Groups "pandoc-discuss" group.
> To post to this group, send email to pandoc-discuss-/JYPxA39Uh5TLH3MbocFF+G/Ez6ZCGd0@public.gmane.org
> To unsubscribe from this group, send email to pandoc-discuss+unsubscribe-/JYPxA39Uh5TLH3MbocFF+G/Ez6ZCGd0@public.gmane.org
> For more options, visit this group at http://groups.google.com/group/pandoc-discuss?hl=en.
> 

Re: CSL styles from zotero.org

[Bruce]

On Nov 24, 10:44 am, John MacFarlane <fiddlosop...-Re5JQEeQqe8AvxtiuMwx3w@public.gmane.org> wrote:
> +++ Andrea Rossato [Nov 24 10 09:25 ]:
>
> > On Tue, Nov 23, 2010 at 09:43:58PM -0800, John MacFarlane wrote:
> > > I fetched the svn repository athttps://www.zotero.org/svn/csl/
> > > and tried to use some of the styles there, but got parsing errors. There seem
> > > to be some differences in format between these files and the ones we've been
> > > testing with. Have there been changes in the csl spec?
>
> > > Are the files in the csl repository more recent or less
> > > recent than the ones citeproc-hs can handle?
>
> > > If less recent, how do we get versions that citeproc-hs
> > > can handle?
>
> > > If more recent, shouldn't citeproc-hs be modified to handle
> > > them?
>
> > For the time being the Zotero repository only hosts CSL-0.8 styles,
> > while citeproc-hs requires CSL-1.0 styles. Those styles will be
> > upgraded to CSL-1.0 as soon as zotero will complete the switch to
> > citeproc-js.
>
> > You can update styles by using the utility you can find here:
>
> >http://bitbucket.org/bdarcus/csl-utils/
>
> Is there a repository with updated versions of the styles? It would be good
> to be able to point people to something like that, rather than asking them to
> convert the old styles.

Let me run this by the CSL community. Things have been a bit slow
moving on this for two reasons:

First, moving Mendeley and Zotero to 1.0, which have a very large
number of collective users. It's not a huge deal for them because they
can run the XSLT transform in the client apps.

Second, we've been talking for a long while about a full blown web app
for style editing, creation and distribution. But the latest idea we'd
chatted about would use git and github as the mechanism for this, so
perhaps we ought to just move 1.0 to a github repo now.

More later ...

Bruce

> > I'm still a bit puzzled on how citeproc should handle the problem:
> > probably citeproc-hs should produce an error saying that the style
> > version is not supported. But the style version can be checked after
> > parsing it, while old styles will not be parsed.
>
> Yes, I think an error message is crucial, as this will be a common issue
> until zotero is updated. Presumably there's some kind of "signature" that all
> the 0.8 styles will have? So, if unpickleXML fails, maybe you could check the
> xml for some 0.8-specific features.

1.0 styles have a version attribute with a value of "1.0" on the root,
which 0.8 styles do not. So yes, it's easy to check.

Bruce

-- 
You received this message because you are subscribed to the Google Groups "pandoc-discuss" group.
To post to this group, send email to pandoc-discuss-/JYPxA39Uh5TLH3MbocFF+G/Ez6ZCGd0@public.gmane.org
To unsubscribe from this group, send email to pandoc-discuss+unsubscribe@googlegroups.com.
For more options, visit this group at http://groups.google.com/group/pandoc-discuss?hl=en.

Re: CSL styles from zotero.org

[John MacFarlane]

Just following up on this:

Bruce: I'd like to be able to mention a repository of CSL 1.0 styles
in the pandoc README.  Is anyone planning to set up such a thing?
(If needed, I could host on github.)  I'd rather be able to point to
a repository than explain how to convert 0.8 styles to 1.0...

Andrea: Did you ever add code to give an informative error message
if people try to use CSL 0.8 styles?

John

+++ Bruce [Nov 24 10 09:35 ]:
> 
> 
> On Nov 24, 10:44 am, John MacFarlane <fiddlosop...-Re5JQEeQqe8AvxtiuMwx3w@public.gmane.org> wrote:
> > +++ Andrea Rossato [Nov 24 10 09:25 ]:
> >
> > > On Tue, Nov 23, 2010 at 09:43:58PM -0800, John MacFarlane wrote:
> > > > I fetched the svn repository athttps://www.zotero.org/svn/csl/
> > > > and tried to use some of the styles there, but got parsing errors. There seem
> > > > to be some differences in format between these files and the ones we've been
> > > > testing with. Have there been changes in the csl spec?
> >
> > > > Are the files in the csl repository more recent or less
> > > > recent than the ones citeproc-hs can handle?
> >
> > > > If less recent, how do we get versions that citeproc-hs
> > > > can handle?
> >
> > > > If more recent, shouldn't citeproc-hs be modified to handle
> > > > them?
> >
> > > For the time being the Zotero repository only hosts CSL-0.8 styles,
> > > while citeproc-hs requires CSL-1.0 styles. Those styles will be
> > > upgraded to CSL-1.0 as soon as zotero will complete the switch to
> > > citeproc-js.
> >
> > > You can update styles by using the utility you can find here:
> >
> > >http://bitbucket.org/bdarcus/csl-utils/
> >
> > Is there a repository with updated versions of the styles? It would be good
> > to be able to point people to something like that, rather than asking them to
> > convert the old styles.
> 
> Let me run this by the CSL community. Things have been a bit slow
> moving on this for two reasons:
> 
> First, moving Mendeley and Zotero to 1.0, which have a very large
> number of collective users. It's not a huge deal for them because they
> can run the XSLT transform in the client apps.
> 
> Second, we've been talking for a long while about a full blown web app
> for style editing, creation and distribution. But the latest idea we'd
> chatted about would use git and github as the mechanism for this, so
> perhaps we ought to just move 1.0 to a github repo now.
> 
> More later ...
> 
> Bruce
> 
> > > I'm still a bit puzzled on how citeproc should handle the problem:
> > > probably citeproc-hs should produce an error saying that the style
> > > version is not supported. But the style version can be checked after
> > > parsing it, while old styles will not be parsed.
> >
> > Yes, I think an error message is crucial, as this will be a common issue
> > until zotero is updated. Presumably there's some kind of "signature" that all
> > the 0.8 styles will have? So, if unpickleXML fails, maybe you could check the
> > xml for some 0.8-specific features.
> 
> 1.0 styles have a version attribute with a value of "1.0" on the root,
> which 0.8 styles do not. So yes, it's easy to check.
> 
> Bruce
> 
> -- 
> You received this message because you are subscribed to the Google Groups "pandoc-discuss" group.
> To post to this group, send email to pandoc-discuss-/JYPxA39Uh5TLH3MbocFF+G/Ez6ZCGd0@public.gmane.org
> To unsubscribe from this group, send email to pandoc-discuss+unsubscribe@googlegroups.com.
> For more options, visit this group at http://groups.google.com/group/pandoc-discuss?hl=en.
> 

-- 
You received this message because you are subscribed to the Google Groups "pandoc-discuss" group.
To post to this group, send email to pandoc-discuss-/JYPxA39Uh5TLH3MbocFF+G/Ez6ZCGd0@public.gmane.org
To unsubscribe from this group, send email to pandoc-discuss+unsubscribe@googlegroups.com.
For more options, visit this group at http://groups.google.com/group/pandoc-discuss?hl=en.

Re: Re: CSL styles from zotero.org

[Andrea Rossato]

On Sun, Dec 05, 2010 at 12:45:58PM -0800, John MacFarlane wrote:
> Just following up on this:
> 
> Bruce: I'd like to be able to mention a repository of CSL 1.0 styles
> in the pandoc README.  Is anyone planning to set up such a thing?
> (If needed, I could host on github.)  I'd rather be able to point to
> a repository than explain how to convert 0.8 styles to 1.0...
> 
> Andrea: Did you ever add code to give an informative error message
> if people try to use CSL 0.8 styles?

Yes, this is the error message:

pandoc: the parsed style seems to be CSL-0.8 while citeproc-hs only supports CSL 1.0.
Styles may be updated by using the csl-utils. For more information see:
http://citationstyles.org/downloads/upgrade-notes.html#updating-csl-0-8-styles

Andrea

Re: CSL styles from zotero.org

[Bruce]

On Dec 5, 3:45 pm, John MacFarlane <fiddlosop...-Re5JQEeQqe8AvxtiuMwx3w@public.gmane.org> wrote:
> Just following up on this:
>
> Bruce: I'd like to be able to mention a repository of CSL 1.0 styles
> in the pandoc README.  Is anyone planning to set up such a thing?
> (If needed, I could host on github.)  I'd rather be able to point to
> a repository than explain how to convert 0.8 styles to 1.0...
>
> Andrea: Did you ever add code to give an informative error message
> if people try to use CSL 0.8 styles?


As you know, I'm working on it. When do you need this?

Bruce

-- 
You received this message because you are subscribed to the Google Groups "pandoc-discuss" group.
To post to this group, send email to pandoc-discuss-/JYPxA39Uh5TLH3MbocFF+G/Ez6ZCGd0@public.gmane.org
To unsubscribe from this group, send email to pandoc-discuss+unsubscribe@googlegroups.com.
For more options, visit this group at http://groups.google.com/group/pandoc-discuss?hl=en.

Re: CSL styles from zotero.org

[John MacFarlane]

+++ Bruce [Dec 07 10 13:24 ]:
> 
> 
> On Dec 5, 3:45 pm, John MacFarlane <fiddlosop...-Re5JQEeQqe8AvxtiuMwx3w@public.gmane.org> wrote:
> > Just following up on this:
> >
> > Bruce: I'd like to be able to mention a repository of CSL 1.0 styles
> > in the pandoc README.  Is anyone planning to set up such a thing?
> > (If needed, I could host on github.)  I'd rather be able to point to
> > a repository than explain how to convert 0.8 styles to 1.0...
> >
> > Andrea: Did you ever add code to give an informative error message
> > if people try to use CSL 0.8 styles?
> 
> 
> As you know, I'm working on it. When do you need this?

I'd like to have it before releasing the next version of pandoc, which will
have citeproc support. I'm not sure exactly when that will be. We're getting
fairly close, though. I'm not sure what Andrea's target date is for releasing
citeproc-hs, but pandoc would have to come after that (and pandoc-types before
it).

There are a few other changes I'd like to make in pandoc before a
release.  So, just guessing, I'd say:  2-4 weeks.

John

-- 
You received this message because you are subscribed to the Google Groups "pandoc-discuss" group.
To post to this group, send email to pandoc-discuss-/JYPxA39Uh5TLH3MbocFF+G/Ez6ZCGd0@public.gmane.org
To unsubscribe from this group, send email to pandoc-discuss+unsubscribe@googlegroups.com.
For more options, visit this group at http://groups.google.com/group/pandoc-discuss?hl=en.

Re: CSL styles from zotero.org

[Paul R]

John, Andrea,

please forgive me for being slightly off topic, but I'm trying to build pandoc git 
version, with little succes as citeproc-hs won't compile. I don't have much time to 
investiguate, but it looks like citeproc-hs dep "hxt >= 8.1" will be resolved as of today 
as hxt-9.*, which has stripped out the relax-ng module to the hxt-relaxng package. So 
citeproc-hs can't find what it needs.


src/Text/CSL/Parser.hs:32:17:
     Could not find module `Text.XML.HXT.RelaxNG':
       Use -v to see a list of the files searched for.
cabal: Error: some packages failed to install:
citeproc-hs-0.2 failed during the building phase




-- 
Paul

Re: Re: CSL styles from zotero.org

[Andrea Rossato]

On Thu, Dec 09, 2010 at 11:28:31AM +0100, Paul R wrote:
> John, Andrea,
> 
> please forgive me for being slightly off topic, but I'm trying to
> build pandoc git version, with little succes as citeproc-hs won't
> compile. I don't have much time to investiguate, but it looks like
> citeproc-hs dep "hxt >= 8.1" will be resolved as of today as
> hxt-9.*, which has stripped out the relax-ng module to the
> hxt-relaxng package. So citeproc-hs can't find what it needs.
> 
> 
> src/Text/CSL/Parser.hs:32:17:
>     Could not find module `Text.XML.HXT.RelaxNG':
>       Use -v to see a list of the files searched for.
> cabal: Error: some packages failed to install:
> citeproc-hs-0.2 failed during the building phase
> 

citeproc-hs-0.2 will not work with the pandoc git version. You need to
grab the darcs version. In this version the HXT dependency has been
entirely removed.

But first you need to get pandoc-types:

    git clone git://github.com/jgm/pandoc-types.git
    cd pandoc-types

and then:

    cabal update
    cabal install

or:

    runhaskell Setup.hs configure
    runhaskell Setup.hs build
    runhaskell Setup.hs install

Then you get the darcs version of citeproc-hs:

    darcs get http://code.haskell.org/citeproc-hs/
    cd citeproc-hs
    cabal install

And then you can build pandoc.

Hope this helps.

Andrea

Re: Re: CSL styles from zotero.org

[Andrea Rossato]

On Tue, Dec 07, 2010 at 02:30:33PM -0800, John MacFarlane wrote:
> +++ Bruce [Dec 07 10 13:24 ]:
> > 
> > 
> > On Dec 5, 3:45 pm, John MacFarlane <fiddlosop...-Re5JQEeQqe8AvxtiuMwx3w@public.gmane.org> wrote:
> > > Just following up on this:
> > >
> > > Bruce: I'd like to be able to mention a repository of CSL 1.0 styles
> > > in the pandoc README.  Is anyone planning to set up such a thing?
> > > (If needed, I could host on github.)  I'd rather be able to point to
> > > a repository than explain how to convert 0.8 styles to 1.0...
> > >
> > > Andrea: Did you ever add code to give an informative error message
> > > if people try to use CSL 0.8 styles?
> > 
> > 
> > As you know, I'm working on it. When do you need this?
> 
> I'd like to have it before releasing the next version of pandoc, which will
> have citeproc support. I'm not sure exactly when that will be. We're getting
> fairly close, though. I'm not sure what Andrea's target date is for releasing
> citeproc-hs, but pandoc would have to come after that (and pandoc-types before
> it).
> 
> There are a few other changes I'd like to make in pandoc before a
> release.  So, just guessing, I'd say:  2-4 weeks.

I'm almost ready for a release. I'd really like to make a 0.3 release
before the end of the month: it would be cool if CSL-1.0 could have
two independent implementation in 2010.

Andrea

-- 
You received this message because you are subscribed to the Google Groups "pandoc-discuss" group.
To post to this group, send email to pandoc-discuss-/JYPxA39Uh5TLH3MbocFF+G/Ez6ZCGd0@public.gmane.org
To unsubscribe from this group, send email to pandoc-discuss+unsubscribe@googlegroups.com.
For more options, visit this group at http://groups.google.com/group/pandoc-discuss?hl=en.

Re: CSL styles from zotero.org

[Paul R]

> Hope this helps.

Thank you Adrea, I was underway.

In citeproc-hs : Bibutils.hs, there is :

   #ifdef USE_BIBUTILS
   (...)
   import System.FilePath ( takeExtension, (</>) )
   (...)
   #endif

Then, slightly below, there is :

   getExt :: String -> String
   getExt = takeExtension . map toLower



So that won't build. I'll take the shortest way, which is probably to use the bibutils
flag.



-- 
Paul

Re: Re: CSL styles from zotero.org

[Andrea Rossato]

On Thu, Dec 09, 2010 at 12:27:38PM +0100, Paul R wrote:
> >Hope this helps.
> 
> Thank you Adrea, I was underway.
> 
> In citeproc-hs : Bibutils.hs, there is :
> 
>   #ifdef USE_BIBUTILS
>   (...)
>   import System.FilePath ( takeExtension, (</>) )
>   (...)
>   #endif
> 
> Then, slightly below, there is :
> 
>   getExt :: String -> String
>   getExt = takeExtension . map toLower
> 

Thanks for reporting this. I'll fix it.

> So that won't build. I'll take the shortest way, which is probably to use the bibutils
> flag.

bibutils dependency can be removed but it is used by default: it
should be portable and very easy to install (cabal should take care of
it transparently).

Andrea

Re: CSL styles from zotero.org

[Paul R]

> bibutils dependency can be removed but it is used by default: it
> should be portable and very easy to install (cabal should take care of
> it transparently).

I see. Cabal didn't set it here, I don't know why.


-- 
Paul

Re: CSL styles from zotero.org

[John MacFarlane]

+++ Andrea Rossato [Dec 09 10 12:17 ]:
> On Tue, Dec 07, 2010 at 02:30:33PM -0800, John MacFarlane wrote:
> > +++ Bruce [Dec 07 10 13:24 ]:
> > > 
> > > 
> > > On Dec 5, 3:45 pm, John MacFarlane <fiddlosop...-Re5JQEeQqe8AvxtiuMwx3w@public.gmane.org> wrote:
> > > > Just following up on this:
> > > >
> > > > Bruce: I'd like to be able to mention a repository of CSL 1.0 styles
> > > > in the pandoc README.  Is anyone planning to set up such a thing?
> > > > (If needed, I could host on github.)  I'd rather be able to point to
> > > > a repository than explain how to convert 0.8 styles to 1.0...
> > > >
> > > > Andrea: Did you ever add code to give an informative error message
> > > > if people try to use CSL 0.8 styles?
> > > 
> > > 
> > > As you know, I'm working on it. When do you need this?
> > 
> > I'd like to have it before releasing the next version of pandoc, which will
> > have citeproc support. I'm not sure exactly when that will be. We're getting
> > fairly close, though. I'm not sure what Andrea's target date is for releasing
> > citeproc-hs, but pandoc would have to come after that (and pandoc-types before
> > it).
> > 
> > There are a few other changes I'd like to make in pandoc before a
> > release.  So, just guessing, I'd say:  2-4 weeks.
> 
> I'm almost ready for a release. I'd really like to make a 0.3 release
> before the end of the month: it would be cool if CSL-1.0 could have
> two independent implementation in 2010.

Great!  Let me know when you're ready, so I can release pandoc-types
first.

John

-- 
You received this message because you are subscribed to the Google Groups "pandoc-discuss" group.
To post to this group, send email to pandoc-discuss-/JYPxA39Uh5TLH3MbocFF+G/Ez6ZCGd0@public.gmane.org
To unsubscribe from this group, send email to pandoc-discuss+unsubscribe@googlegroups.com.
For more options, visit this group at http://groups.google.com/group/pandoc-discuss?hl=en.