OPLP: Ocaml APache Log Parser

OPLP: Ocaml APache Log Parser

[Chris Yocum]

[-- Attachment #1: Type: text/plain, Size: 328 bytes --]

Dear Everyone,

I just wanted to let Ocaml'ers know that I am currently working on an
apache combined log parser.  It is still in very early stages of
development but patches/comments/feature requests/bug reports are very
welcome.  The code lives on github (http://github.com/cyocum/OPLP).

All the best,
Chris Yocum


[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 262 bytes --]

Re: [Caml-list] OPLP: Ocaml APache Log Parser

[oliver]

On Mon, Oct 11, 2010 at 02:45:49PM +0100, Chris Yocum wrote:
> Dear Everyone,
> 
> I just wanted to let Ocaml'ers know that I am currently working on an
> apache combined log parser.  It is still in very early stages of
> development but patches/comments/feature requests/bug reports are very
> welcome.  The code lives on github (http://github.com/cyocum/OPLP).
[...]

Aha... long ago I had done something similar, and it had an sql-like
language for using it.

The annoyance (and advantage ;)) with the apache logfiles is, that there
are many "standard" formats, and you also can design your own output format.

So it would make sense (I didn't do that) to parse the apache config files,
and create a logparser from that information...

Do you have planned to do this, or do you want to stick with some of the
most used formats?

Ciao,
   Oliver

Re: [Caml-list] OPLP: Ocaml APache Log Parser

[Chris Yocum]

[-- Attachment #1: Type: text/plain, Size: 1642 bytes --]

Hi Oliver,

For now, I was just concentrating on the "combined" format that is used
as an example on the apache site.  I will probably branch out into
parsing the log format itself later once I get more experience with it.

Also, does anyone have a smallish block of apache log files (<30k) that
I can use for testing?

All the best,
Chris Yocum

On 11/10/10 14:54, oliver@first.in-berlin.de wrote:
> On Mon, Oct 11, 2010 at 02:45:49PM +0100, Chris Yocum wrote:
>> Dear Everyone,
>>
>> I just wanted to let Ocaml'ers know that I am currently working on an
>> apache combined log parser.  It is still in very early stages of
>> development but patches/comments/feature requests/bug reports are very
>> welcome.  The code lives on github (http://github.com/cyocum/OPLP).
> [...]
> 
> Aha... long ago I had done something similar, and it had an sql-like
> language for using it.
> 
> The annoyance (and advantage ;)) with the apache logfiles is, that there
> are many "standard" formats, and you also can design your own output format.
> 
> So it would make sense (I didn't do that) to parse the apache config files,
> and create a logparser from that information...
> 
> Do you have planned to do this, or do you want to stick with some of the
> most used formats?
> 
> Ciao,
>    Oliver
> 
> _______________________________________________
> Caml-list mailing list. Subscription management:
> http://yquem.inria.fr/cgi-bin/mailman/listinfo/caml-list
> Archives: http://caml.inria.fr
> Beginner's list: http://groups.yahoo.com/group/ocaml_beginners
> Bug reports: http://caml.inria.fr/bin/caml-bugs


[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 262 bytes --]

Re: [Caml-list] OPLP: Ocaml APache Log Parser

[Mihamina Rakotomandimby]

> cyocum@gmail.com :
> I just wanted to let Ocaml'ers know that I am currently working on an
> apache combined log parser.

I would like to encourage you registering your project to the forge:
http://forge.ocamlcore.org/

Using github is not a problem, but there is an effort to make OCaml big
& small project visible. Let's also encourage it.

-- 

       Architecte Informatique chez Blueline/Gulfsat:
    Administration Systeme, Recherche & Developpement
                                    +261 34 56 000 19

Re: [Caml-list] OPLP: Ocaml APache Log Parser

[Chris Yocum]

[-- Attachment #1: Type: text/plain, Size: 478 bytes --]

Cool.  I will do so presently.

Thanks!,
Chris

On 12/10/10 07:48, Mihamina Rakotomandimby wrote:
>> cyocum@gmail.com :
>> I just wanted to let Ocaml'ers know that I am currently working on an
>> apache combined log parser.
> 
> I would like to encourage you registering your project to the forge:
> http://forge.ocamlcore.org/
> 
> Using github is not a problem, but there is an effort to make OCaml big
> & small project visible. Let's also encourage it.
> 


[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 262 bytes --]

Re: [Caml-list] OPLP: Ocaml APache Log Parser

[Richard Jones]

On Mon, Oct 11, 2010 at 02:45:49PM +0100, Chris Yocum wrote:
> Dear Everyone,
> 
> I just wanted to let Ocaml'ers know that I am currently working on an
> apache combined log parser.  It is still in very early stages of
> development but patches/comments/feature requests/bug reports are very
> welcome.  The code lives on github (http://github.com/cyocum/OPLP).

These projects are not maintained any more but may be of
interest:

http://merjis.com/developers/weblogs
http://merjis.com/developers/hostip

Rich.

-- 
Richard Jones
Red Hat

Re: [Caml-list] OPLP: Ocaml APache Log Parser

[Chris Yocum]

[-- Attachment #1: Type: text/plain, Size: 629 bytes --]

Hi Rich,

Thanks!

Chris

On 13/10/10 11:37, Richard Jones wrote:
> On Mon, Oct 11, 2010 at 02:45:49PM +0100, Chris Yocum wrote:
>> Dear Everyone,
>>
>> I just wanted to let Ocaml'ers know that I am currently working on an
>> apache combined log parser.  It is still in very early stages of
>> development but patches/comments/feature requests/bug reports are very
>> welcome.  The code lives on github (http://github.com/cyocum/OPLP).
> 
> These projects are not maintained any more but may be of
> interest:
> 
> http://merjis.com/developers/weblogs
> http://merjis.com/developers/hostip
> 
> Rich.
> 


[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 262 bytes --]

Re: [Caml-list] OPLP: Ocaml APache Log Parser

[Gaius Hammond]

On 11 Oct 2010, at 14:54, oliver@first.in-berlin.de wrote:

> On Mon, Oct 11, 2010 at 02:45:49PM +0100, Chris Yocum wrote:
>> Dear Everyone,
>>
>> I just wanted to let Ocaml'ers know that I am currently working on an
>> apache combined log parser.  It is still in very early stages of
>> development but patches/comments/feature requests/bug reports are  
>> very
>> welcome.  The code lives on github (http://github.com/cyocum/OPLP).
> [...]
>
> Aha... long ago I had done something similar, and it had an sql-like
> language for using it.




I do something like this, but using OCaml to load a logfile into  
SQLite, and analyze it with "real" SQL.



http://gaiustech.wordpress.com/2010/09/12/analyzing-logfiles-1/



Probably my technique using sscanf wouldn't be fast enough for  
something as big as an Apache log that might have a million lines.



Cheers,




G

Re: [Caml-list] OPLP: Ocaml APache Log Parser

[oliver]

On Mon, Oct 11, 2010 at 03:54:22PM +0200, oliver@first.in-berlin.de wrote:
> On Mon, Oct 11, 2010 at 02:45:49PM +0100, Chris Yocum wrote:
> > Dear Everyone,
> > 
> > I just wanted to let Ocaml'ers know that I am currently working on an
> > apache combined log parser.  It is still in very early stages of
> > development but patches/comments/feature requests/bug reports are very
> > welcome.  The code lives on github (http://github.com/cyocum/OPLP).
> [...]
> 
> Aha... long ago I had done something similar, and it had an sql-like
> language for using it.
[...]


Here is the URL for that tool:

  http://www.first.in-berlin.de/software/tools/apalogretrieve/



Ciao,
   Oliver