[ANN] Camomile 0.7.3

[ANN] Camomile 0.7.3

[Yoriyuki Yamagata]

[-- Attachment #1: Type: text/plain, Size: 1590 bytes --]

I'm pleased to announce Camomile 0.7.3, a new version of Camomile, a
comprehensive Unicode library for OCaml.
This is a bug fix release.  It fixes the  following bugs and Camomile now
works on Windows.
    - Aliases of character encodings containing ":" are removed,
    to support Windows platform.
    - Buffering bugs in CharEncoding and OOChannel modules.
    - Tree-merging bugs of ISet and IMap.
    - Locale data are properly loaded by binary channels. (Windows related)
    - "make depend" properly generates .depend file.
    - cpp is no longer required to build from the distribution.
    The license documentation for locales/*.txt files is added.
(locales/license.html)

  The package is tested on Windows (MinGW-port of OCaml 3.11.0  + Cygwin on
Vista SP1) and Linux (Ubuntu 8.04 + godi version of OCaml 3.11.2).

  You can download the package from
https://sourceforge.net/projects/camomile/

  For more information on Camomile, see the project web page
http://camomile.sourceforge.net/.

  I would appreciate your comments or/and opinions.  In particular, I'd like
to know whether you can successfully operate Camomile in your platform.  I
have complaints on Mac and MinGW environments, and although I believe that
the problems are fixed, I'm too lazy to find spare Mac around and test the
package :-)  Also, I would like to hear about a success ( /failure ) story
of Camomile.  Do you use Camomile?  What for?  This is important since I
have to convince my boss to allow me to invest some spare time to Camomile
project.

Regards,
-- 
Yoriyuki Yamagata
yoriyuki.y@gmail.com

[-- Attachment #2: Type: text/html, Size: 2160 bytes --]

Re: [Caml-list] [ANN] Camomile 0.7.3

[Romain Beauxis]

	Hi !

Le mercredi 12 mai 2010 07:13:17, Yoriyuki Yamagata a écrit :
>  Also, I would like to hear about a success ( /failure ) story
> of Camomile.  Do you use Camomile?  What for?  This is important since I
> have to convince my boss to allow me to invest some spare time to Camomile
> project.

Thanks for your work in camomile !

We have been using and enjoying it with liquidsoap [1] for probably almost 3 
years now. We are using camomile to convert implicitely the various metadata 
tags into unicode.

I am personally impressed by how good the module works. As far as I can tell 
we have had no issues with character encoding since we are using camomile !

Romain

[1]: http://savonet.sf.net/

Re: [Caml-list] [ANN] Camomile 0.7.3

[Romain Beauxis]

	Hi !

Le mercredi 12 mai 2010 07:13:17, Yoriyuki Yamagata a écrit :
>  Also, I would like to hear about a success ( /failure ) story
> of Camomile.  Do you use Camomile?  What for?  This is important since I
> have to convince my boss to allow me to invest some spare time to Camomile
> project.

Thanks for your work in camomile !

We have been using and enjoying it with liquidsoap [1] for probably almost 3 
years now. We are using camomile to convert implicitely the various metadata 
tags into unicode.

I am personally impressed by how good the module works. As far as I can tell 
we have had no issues with character encoding since we are using camomile !

Romain

[1]: http://savonet.sf.net/

Re: [ANN] Camomile 0.7.3

[Sylvain Le Gall]

On 12-05-2010, Yoriyuki Yamagata <yoriyuki.y@gmail.com> wrote:
>
>   I would appreciate your comments or/and opinions.  In particular, I'd like
> to know whether you can successfully operate Camomile in your platform.  I
> have complaints on Mac and MinGW environments, and although I believe that
> the problems are fixed, I'm too lazy to find spare Mac around and test the
> package :-)  Also, I would like to hear about a success ( /failure ) story
> of Camomile.  Do you use Camomile?  What for?  This is important since I
> have to convince my boss to allow me to invest some spare time to Camomile
> project.
>

I use camomile as the default implementation for ocaml-gettext:
http://forge.ocamlcore.org/projects/ocaml-gettext

Basically I do encoding conversion with it. Projects of mine built with
ocaml-gettext use camomile as well.

Regards
Sylvain 

ps: as asked on the BTS, a way to be able to relocate the .mar file
would be something great -- this actually prevents me to distribute my
projects built with camomile. (relocate -> remove the hardcoded path)

Re: [Caml-list] Re: [ANN] Camomile 0.7.3

[Yoriyuki Yamagata]

[-- Attachment #1: Type: text/plain, Size: 591 bytes --]

2010/5/13 Sylvain Le Gall <sylvain@le-gall.net>

>
> ps: as asked on the BTS, a way to be able to relocate the .mar file
> would be something great -- this actually prevents me to distribute my
> projects built with camomile. (relocate -> remove the hardcoded path)
>
>
Hmm...  you should be able to relocate .mar file by providing the location
of .mar files through the functor CamomileLibrary.Main.Make.  If you do not
link CamomileLibrary.Default, Camomile won't load the data from the
hardcoded path.  If it does, then  I regards it as a bug.

-- 
Yoriyuki Yamagata
yoriyuki.y@gmail.com

[-- Attachment #2: Type: text/html, Size: 999 bytes --]

Re: [ANN] Camomile 0.7.3

[Sylvain Le Gall]

On 13-05-2010, Yoriyuki Yamagata <yoriyuki.y@gmail.com> wrote:
> 2010/5/13 Sylvain Le Gall <sylvain@le-gall.net>
>
>>
>> ps: as asked on the BTS, a way to be able to relocate the .mar file
>> would be something great -- this actually prevents me to distribute my
>> projects built with camomile. (relocate -> remove the hardcoded path)
>>
>>
> Hmm...  you should be able to relocate .mar file by providing the location
> of .mar files through the functor CamomileLibrary.Main.Make.  If you do not
> link CamomileLibrary.Default, Camomile won't load the data from the
> hardcoded path.  If it does, then  I regards it as a bug.
>

Will try, thanks.

By the way (and for your boss), I consider camomile as a first class
OCaml library. Your work on this is really important.

What do you need to prove what I said? Other OSS products using camomile? 

Regards
Sylvain Le Gall

Re: [Caml-list] Re: [ANN] Camomile 0.7.3

[Yoriyuki Yamagata]

[-- Attachment #1: Type: text/plain, Size: 755 bytes --]

2010/5/13 Sylvain Le Gall <sylvain@le-gall.net>

> On 13-05-2010, Yoriyuki Yamagata <yoriyuki.y@gmail.com> wrote:
> > 2010/5/13 Sylvain Le Gall <sylvain@le-gall.net>By the way (and for your
> boss), I consider camomile as a first class
> OCaml library. Your work on this is really important.
>
> What do you need to prove what I said? Other OSS products using camomile?
>
>
- successful projects which use Camomile.  Big bonus If they are backed by
industry.  Whether they are OSS or not does not matter much.
- download statistics.  Is there a way to see download statistics of Debian
package?
- the number of OS to which Camomile is ported.  I know there are Debian,
FreeBSD and ALTLinux package.  Any other?

-- 
Yoriyuki Yamagata
yoriyuki.y@gmail.com

[-- Attachment #2: Type: text/html, Size: 1271 bytes --]

Re: [Caml-list] Re: [ANN] Camomile 0.7.3

[Paolo Donadeo]

Of course, it's worth to remember that Camomile is heavily used in Batteries.


-- 
Paolo

Re: [Caml-list] Re: [ANN] Camomile 0.7.3

[Dmitry Bely]

How "heavy-weight" is Camomile? I was a bit scared with the size of
its distribution. Currently I use under Windows the following my own
simple Unicode-support module (implemented via
WideCharToMultiByte/MultiByteToWideChar Win32 API functions). Maybe
it's time to switch to Camomile?

(** Wide char/string operations. *)

type codePage =
    CP_ACP (* ANSI code page *)
  | CP_MACCP (* Macintosh code page *)
  | CP_OEMCP (* OEM code page *)
  | CP_SYMBOL (* Symbol code page (42) *)
  | CP_THREAD_ACP (* The current thread's ANSI code page *)
  | CP_UTF7 (* Translate using UTF-7 *)
  | CP_UTF8 (* Translate using UTF-8 *)
  | CP_NUMERIC of int

module Wchar:
sig
(*IF-OCAML*)
  type t = private int
(*ENDIF-OCAML*)
(*F#
  type t = char
F#*)

  val compare: t -> t -> int
  val of_int: int -> t
  val to_int: t -> int
  val encode: codePage -> t -> Bytes.bytes
  val decode: codePage -> Bytes.bytes -> t
end

module Wstring:
sig
(*IF-OCAML*)
  type t = private int array
(*ENDIF-OCAML*)
(*F#
  type t = string
F#*)

  val compare: t -> t -> int
  val length : t -> int
  val get: t -> int -> Wchar.t
  val make: int -> Wchar.t -> t
  val append: t -> t -> t
  val uppercase: t -> t
  val lowercase: t -> t

  val of_array: Wchar.t array -> t
  val to_array: t -> Wchar.t array
  val of_int_array: int array -> t
  val to_int_array: t -> int array
  val of_list: Wchar.t list -> t
  val to_list: t -> Wchar.t list

  val empty: t
(*IF-OCAML*)
  external pack_to_string: t -> string = "w32_unicode_pack_to_string"
    (** convert to packed 2-byte UCS2 string, stored as Caml string *)
  external unpack_from_string: string -> t = "w32_unicode_unpack_from_string"
    (** inverse function for pack_to_string *)
(*ENDIF-OCAML*)
  val encode: codePage -> t -> Bytes.bytes
  val decode: codePage -> Bytes.bytes -> t
end

type wchar = Wchar.t
type wstring = Wstring.t

(**/**)

- Dmitry Bely

Re: [Caml-list] Re: [ANN] Camomile 0.7.3

[Yoriyuki Yamagata]

[-- Attachment #1: Type: text/plain, Size: 2276 bytes --]

Hi, Bely.

2010/5/14 Dmitry Bely <dmitry.bely@gmail.com>

> How "heavy-weight" is Camomile? I was a bit scared with the size of
> its distribution. Currently I use under Windows the following my own
> simple Unicode-support module (implemented via
> WideCharToMultiByte/MultiByteToWideChar Win32 API functions). Maybe
> it's time to switch to Camomile?
>
>
The size of the package is due to mapping tables of character encoding and
localization data.  they occupy several mega bytes on the disk but it is
nothing by today's standard.  If you still care, you can delete any .mar
files in charmaps, locales, mappings directory.  (Deleting source files in
these directory is not recommended, since it could cause a failure of
compilation.)  If you delete such files, related encoding and locales do not
function, but other functionality is intact.

As for memory consumption, character mapping tables and localization data
are loaded to the memory when they are required.  So if you do not use them,
they are dead on the disk.  After being loaded to the memory, they are
retained by weak hashtable, therefore, when they are no longer used, GC
releases them.

Some modules load data from database directory during initialization.  They
persists the end of the execution, but these data are usually small (mostly
some kilobytes in the marshaled form, only allkey.mar which used in uCol
module is >100k bytes (*)).  You can avoid loading them by not linking these
modules.

Finally, the programming style to use Camomile would be a bit heavy, since
Camomile uses functors a lot.  In fact, the library itself is a functor
which takes modules containing initialization parameters.

Heaviness aside, Camomile does not provide OS integration, which might be a
problem for you.  I tried POSIX integration but abandoned it by the reason
which I no longer remember :-)  But, you can use UTF16 module to access wch
array, since Windows wch is a 16-bit integer.
(UTF16.t is 16-bit integer array implemented in bigarray module.

(*) I noticed allkey.mar can be retained by weak pointer as well (currently,
it is held in Lazy.t).  I will fix it in the next release, maybe.

Feel free to ask further questions.  I would be grad to answer them.

Best,
-- 
Yoriyuki Yamagata
yoriyuki.y@gmail.com

[-- Attachment #2: Type: text/html, Size: 2776 bytes --]

Re: [ANN] Camomile 0.7.3

[Sylvain Le Gall]

In gmane.comp.lang.caml.inria, you wrote:
> 2010/5/14 Dmitry Bely <dmitry.bely@gmail.com>
>
>> How "heavy-weight" is Camomile? I was a bit scared with the size of
>> its distribution. Currently I use under Windows the following my own
>> simple Unicode-support module (implemented via
>> WideCharToMultiByte/MultiByteToWideChar Win32 API functions). Maybe
>> it's time to switch to Camomile?
>>
>>
> The size of the package is due to mapping tables of character encoding and
> localization data.  they occupy several mega bytes on the disk but it is
> nothing by today's standard.  If you still care, you can delete any .mar
> files in charmaps, locales, mappings directory.  (Deleting source files in
> these directory is not recommended, since it could cause a failure of
> compilation.)  If you delete such files, related encoding and locales do not
> function, but other functionality is intact.
>

I have just compared libcamomile-ocaml-data and locales-all packages.
The two packages contain almost the same thing:
http://packages.debian.org/sid/libcamomile-ocaml-data
http://packages.debian.org/sid/locales-all

The size is almost the same (3MB vs 5MB) but the uncompressed size is
not at all the same:
- camomile: 24MB
- locales-all (2 steps uncompression): 99MB

At least it means that camomile data is a reasonable size for this kind
of data on Linux system. On Windows system, I think these figures are
almost the same but data are part of the base windows installation.

Now, a question/suggestion/feature: is it possible to store .mar file
as .mar.gz? This would help to reduce occupation by a factor of 5...

Regards
Sylvain Le Gall

Re: [ANN] Camomile 0.7.3

[Yoriyuki Yamagata]

[-- Attachment #1: Type: text/plain, Size: 1100 bytes --]

2010/5/14 Sylvain Le Gall <sylvain@le-gall.net>

> I have just compared libcamomile-ocaml-data and locales-all packages.
> The two packages contain almost the same thing:
> http://packages.debian.org/sid/libcamomile-ocaml-data
> http://packages.debian.org/sid/locales-all
>
>
Character mapping tables are same to that of libc, but locale data of
Camomile are derived from IBM ICU.  Moreover, Camomile's data are not
updated several years.  So, I'm not sure that this is a fair comparison.


> Now, a question/suggestion/feature: is it possible to store .mar file
> as .mar.gz? This would help to reduce occupation by a factor of 5...
>
>
I guess it is possible through camlzip library.  But storage is cheap.
Increasing dependency and complexity to just save several mega bytes is not
an attractive idea for me.  Rather, I'm interested succinct data structures
(data structures with information theoretically optimal space usage).  In
this way, we can save not only disk space but memory as well, and perhaps
gain performance improvement as well.

Regards,
-- 
Yoriyuki Yamagata
yoriyuki.y@gmail.com

[-- Attachment #2: Type: text/html, Size: 1820 bytes --]

Re: [ANN] Camomile 0.7.3

[Sylvain Le Gall]

On 14-05-2010, Yoriyuki Yamagata <yoriyuki.y@gmail.com> wrote:
> 2010/5/13 Sylvain Le Gall <sylvain@le-gall.net>
>
>> On 13-05-2010, Yoriyuki Yamagata <yoriyuki.y@gmail.com> wrote:
>> > 2010/5/13 Sylvain Le Gall <sylvain@le-gall.net>By the way (and for your
>> boss), I consider camomile as a first class
>> OCaml library. Your work on this is really important.
>>
>> What do you need to prove what I said? Other OSS products using camomile?
>>
>>
> - download statistics.  Is there a way to see download statistics of Debian
> package?

http://qa.debian.org/popcon.php?package=camomile

But there is a lot of uncertainty around these figures.

I take into account the "vote" field which reflects regular users of
camomile: 25. (comparable figures: janest-core: 10; ocaml-fileutils: 11;
ocaml-gettext: 12; ocaml-nox: 479; extlib: 138).

There are ~75k voters but we estimate the number of installed computer at
around 500k (min) to 1M (max). So you could consider that there are
between 160 to 330 regular users of camomile... Of course this is an
approximation, there are many unknown figures.

Concerning other OSes, there is also Ubuntu, but this is basically
exactly the same package as Debian.

What are the statistics of sourceforge downloads?

Regards
Sylvain Le Gall

Re: [Caml-list] Re: [ANN] Camomile 0.7.3

[Yoriyuki Yamagata]

[-- Attachment #1: Type: text/plain, Size: 1427 bytes --]

2010/5/14 Sylvain Le Gall <sylvain@le-gall.net>

> http://qa.debian.org/popcon.php?package=camomile
>
> But there is a lot of uncertainty around these figures.
>
> I take into account the "vote" field which reflects regular users of
> camomile: 25. (comparable figures: janest-core: 10; ocaml-fileutils: 11;
> ocaml-gettext: 12; ocaml-nox: 479; extlib: 138).
>
> There are ~75k voters but we estimate the number of installed computer at
> around 500k (min) to 1M (max). So you could consider that there are
> between 160 to 330 regular users of camomile... Of course this is an
> approximation, there are many unknown figures.
>
> Concerning other OSes, there is also Ubuntu, but this is basically
> exactly the same package as Debian.
>
> What are the statistics of sourceforge downloads?
>
>
Thanks for data.

In comparison, data of m17n-lib (a multilingualization framework which
happen to be developed by m17n/i18n team of my institute.) is here.

http://qa.debian.org/popcon.php?package=m17n-lib

The runtime's votes are 537 but votes of m17n-lib-dev (34) is roughly same
popularity to Camomile.

About sourceforge download, 0.7.1 is downloaded 3269 times since 2006-08-20,
and 0.7.2 (which is identical to 0.7.1 except the license) is downloaded
1017 times.

So maybe we can conclude that Camomile is installed several thousand of
machines.  It might impress my boss.

Regards,
-- 
Yoriyuki Yamagata
yoriyuki.y@gmail.com

[-- Attachment #2: Type: text/html, Size: 2032 bytes --]

Re: [Caml-list] Re: [ANN] Camomile 0.7.3

[Florent Monnier]

> - the number of OS to which Camomile is ported.  I know there are Debian,
> FreeBSD and ALTLinux package.  Any other?

Mandriva :
http://svn.mandriva.com/cgi-bin/viewvc.cgi/packages/cooker/ocaml-
camomile/current/
http://maint.mandriva.com/viewpkg.php?pkg=14157

Fedora :
http://cvs.fedoraproject.org/viewvc/devel/ocaml-camomile/

seems to be around in Gentoo too :
http://packages.gentoo.org/package/dev-ml/camomile

-- 

Re: [Caml-list] Re: [ANN] Camomile 0.7.3

[Yoriyuki Yamagata]

A kind of Eureka moment comes to me....

2010年5月13日10:19 Yoriyuki Yamagata <yoriyuki.y@gmail.com>:
> 2010/5/13 Sylvain Le Gall <sylvain@le-gall.net>
> of .mar files through the functor CamomileLibrary.Main.Make.  If you do not
> link CamomileLibrary.Default, Camomile won't load the data from the
> hardcoded path.  If it does, then  I regards it as a bug.

As usual, I'm totally wrong.  I finally get it and yes, if you link
CamomileLibrary then all initialization code run and try to load the
data from the hardcoded location.

I will revise name-space so that this problem does not happen.

-- 
Yoriyuki　Yamagata
yoriyuki.y@gmail.com

Re: [Caml-list] [ANN] Camomile 0.7.3

[forum]

Le 12 mai 2010 à 14:13, Yoriyuki Yamagata a écrit :

> I'm pleased to announce Camomile 0.7.3, a new version of Camomile, a comprehensive Unicode library for OCaml.
> This is a bug fix release.  It fixes the  following bugs and Camomile now works on Windows.
>     - Aliases of character encodings containing ":" are removed,
>     to support Windows platform.
>     - Buffering bugs in CharEncoding and OOChannel modules.
>     - Tree-merging bugs of ISet and IMap.
>     - Locale data are properly loaded by binary channels. (Windows related)
>     - "make depend" properly generates .depend file.
>     - cpp is no longer required to build from the distribution.
>     The license documentation for locales/*.txt files is added. (locales/license.html)
> 
>   The package is tested on Windows (MinGW-port of OCaml 3.11.0  + Cygwin on Vista SP1) and Linux (Ubuntu 8.04 + godi version of OCaml 3.11.2).
> 
>   You can download the package from https://sourceforge.net/projects/camomile/
> 
>   For more information on Camomile, see the project web page http://camomile.sourceforge.net/.  
> 
>   I would appreciate your comments or/and opinions.  In particular, I'd like to know whether you can successfully operate Camomile in your platform.  I have complaints on Mac and MinGW environments, and although I believe that the problems are fixed, I'm too lazy to find spare Mac around and test the package :-)  Also, I would like to hear about a success ( /failure ) story of Camomile.  Do you use Camomile?  What for?  This is important since I have to convince my boss to allow me to invest some spare time to Camomile project.

Camomile is used in Barista [1] to encode / decode / modify UTF8 strings of Java classes.

My opinion of Camomile is absolutely positive:
 - installation is a breeze;
 - API is complete yet straightforward;
 - it meets a need of the OCaml ecosystem.

Not yet updated to 0.7.3 but 0.7.2 used to work like a charm on both MacOS X and Fedora.


Regards,

Xavier Clerc

[1] library (part of OCaml-Java) to load / manipulate / save Java ".class" files - http://barista.x9c.fr/