How to monitor a specific file for changes

How to monitor a specific file for changes

[Orlin Grigorov]

[-- Attachment #1: Type: text/plain, Size: 639 bytes --]

Hi.

This time I've done my research, but still I cannot find a good answer.
Basically, I need to make a little ocaml process, which runs in the
background and monitors a specific file for changes.   When a change occurs,
I want it to back-up a copy of the version of the file at that moment.
So, what I need is to set some signal or event, which will happen every time
that file is changed.

Can you guys help me do it?

Thank you in advance!

Orlin

PS. If in any way I'm misusing this list (writing too much, or out of
topic), please tell me, because I just don't know it.  The least I want is
to spam the list with improper questions.

[-- Attachment #2: Type: text/html, Size: 732 bytes --]

Re: [Caml-list] How to monitor a specific file for changes

[Robert Fischer]

[-- Attachment #1: Type: text/html, Size: 1951 bytes --]

Re: How to monitor a specific file for changes

[Sylvain Le Gall]

On 22-10-2007, Orlin Grigorov <ogrigorov@gmail.com> wrote:
> This time I've done my research, but still I cannot find a good answer.
> Basically, I need to make a little ocaml process, which runs in the
> background and monitors a specific file for changes.   When a change occurs,
> I want it to back-up a copy of the version of the file at that moment.
> So, what I need is to set some signal or event, which will happen every time
> that file is changed.
>
> Can you guys help me do it?
>

The thing you describe should be handled at OS level (at least this is
kind of notification are bound to OS). This means, you don't have high
level functions that are multi-OS.

If you use linux 2.6, take a look at ocaml-inotify, which helps you to
wait for specific event on some file descriptors (also work for
directories):

http://tab.snarc.org/projects/ocaml_inotify/

If you use older linux version, solaris, macos or windows, i have no
answer (but i am sure this kind of things already exists).

Regards,
Sylvain Le Gall

Re: [Caml-list] How to monitor a specific file for changes

[Andrew Gacek]

The OMake build system (omake.metaprl.org) is written in OCaml and has
support for monitoring the file system for changes (at least on Win32
and Linux). You might take a look at the source code for that system
and see what they use.

-Andrew

On 10/22/07, Orlin Grigorov <ogrigorov@gmail.com> wrote:
> Hi.
>
> This time I've done my research, but still I cannot find a good answer.
> Basically, I need to make a little ocaml process, which runs in the
> background and monitors a specific file for changes.   When a change occurs,
> I want it to back-up a copy of the version of the file at that moment.
> So, what I need is to set some signal or event, which will happen every time
> that file is changed.
>
> Can you guys help me do it?
>
> Thank you in advance!
>
> Orlin
>
> PS. If in any way I'm misusing this list (writing too much, or out of
> topic), please tell me, because I just don't know it.  The least I want is
> to spam the list with improper questions.
>
> _______________________________________________
> Caml-list mailing list. Subscription management:
> http://yquem.inria.fr/cgi-bin/mailman/listinfo/caml-list
> Archives: http://caml.inria.fr
> Beginner's list:
> http://groups.yahoo.com/group/ocaml_beginners
> Bug reports: http://caml.inria.fr/bin/caml-bugs
>
>

Re: [Caml-list] How to monitor a specific file for changes

[Erik de Castro Lopo]

Orlin Grigorov wrote:

> Hi.
> 
> This time I've done my research, but still I cannot find a good answer.
> Basically, I need to make a little ocaml process, which runs in the
> background and monitors a specific file for changes.   When a change occurs,
> I want it to back-up a copy of the version of the file at that moment.
> So, what I need is to set some signal or event, which will happen every time
> that file is changed.

If you are on Linux/Unix, have a look at Inotify. The Ocaml bindings are
here:

    http://tab.snarc.org/projects/ocaml_inotify/

HTH,
Erik
-- 
-----------------------------------------------------------------
Erik de Castro Lopo
-----------------------------------------------------------------
"Well behaved women rarely make history." -- Dorothy Parker

Re: [Caml-list] How to monitor a specific file for changes

[Orlin Grigorov]

[-- Attachment #1: Type: text/plain, Size: 2048 bytes --]

Well, I'm developing my application in Linux, but after it's done, it will
be used mostly on Mac OS.  So far, on regular basis, I test everything on my
Mac to make sure it's working.

I guess I'll have to find another way then.   One idea I have is to poll the
size of the file every certain amount of time, and if it's
different---there's my event.   Same applies to "date/time last modified".

Is there something I can use successfully on both Linux and Mac about that?

Just to mention that the file is a text file, and it doesn't change very
often.  I mean, it's not constantly changing, so I'm not concerned with
performance and resource issues.

Oh, here's another idea---every certain amount of time, I just compare my
last saved version of the file to the current one---if different, there's my
event.   What do you think?

O.

On 10/22/07, Erik de Castro Lopo <mle+ocaml@mega-nerd.com> wrote:
>
> Orlin Grigorov wrote:
>
> > Hi.
> >
> > This time I've done my research, but still I cannot find a good answer.
> > Basically, I need to make a little ocaml process, which runs in the
> > background and monitors a specific file for changes.   When a change
> occurs,
> > I want it to back-up a copy of the version of the file at that moment.
> > So, what I need is to set some signal or event, which will happen every
> time
> > that file is changed.
>
> If you are on Linux/Unix, have a look at Inotify. The Ocaml bindings are
> here:
>
>     http://tab.snarc.org/projects/ocaml_inotify/
>
> HTH,
> Erik
> --
> -----------------------------------------------------------------
> Erik de Castro Lopo
> -----------------------------------------------------------------
> "Well behaved women rarely make history." -- Dorothy Parker
>
> _______________________________________________
> Caml-list mailing list. Subscription management:
> http://yquem.inria.fr/cgi-bin/mailman/listinfo/caml-list
> Archives: http://caml.inria.fr
> Beginner's list: http://groups.yahoo.com/group/ocaml_beginners
> Bug reports: http://caml.inria.fr/bin/caml-bugs
>

[-- Attachment #2: Type: text/html, Size: 2847 bytes --]

Re: [Caml-list] How to monitor a specific file for changes

[Erik de Castro Lopo]

Orlin Grigorov wrote:

> Oh, here's another idea---every certain amount of time, I just compare my
> last saved version of the file to the current one---if different, there's my
> event.   What do you think?

I think you need to drop OSX and move to a better OS :-).

Erik
-- 
-----------------------------------------------------------------
Erik de Castro Lopo
-----------------------------------------------------------------
"I saw `cout' being shifted "Hello world" times to the left
and stopped right there." -- Steve Gonedes

Re: [Caml-list] How to monitor a specific file for changes

[Jake Donham]

On 10/22/07, Orlin Grigorov <ogrigorov@gmail.com> wrote:
> Is there something I can use successfully on both Linux and Mac about that?

There is a multiplatform library in OMake that uses whatever mechanism
is available. I don't know what it does on Mac; I think it falls back
to polling  on platforms where nothing else is available.

Jake

Re: [Caml-list] How to monitor a specific file for changes

[Orlin Grigorov]

[-- Attachment #1: Type: text/plain, Size: 1026 bytes --]

I completely agree about the OSX ;)   But I'm a grad student, and my
supervisor is quite fond of Macs.  And since he's paying, I need to make
sure he's happy.   The application I'm developing is in support to my
Master's thesis, and it would be a disaster if he can't play with it on his
Macs :(

On 10/22/07, Jake Donham <jake.donham@skydeck.com> wrote:
>
> On 10/22/07, Orlin Grigorov <ogrigorov@gmail.com> wrote:
> > Is there something I can use successfully on both Linux and Mac about
> that?
>
> There is a multiplatform library in OMake that uses whatever mechanism
> is available. I don't know what it does on Mac; I think it falls back
> to polling  on platforms where nothing else is available.
>
> Jake
>
> _______________________________________________
> Caml-list mailing list. Subscription management:
> http://yquem.inria.fr/cgi-bin/mailman/listinfo/caml-list
> Archives: http://caml.inria.fr
> Beginner's list: http://groups.yahoo.com/group/ocaml_beginners
> Bug reports: http://caml.inria.fr/bin/caml-bugs
>

[-- Attachment #2: Type: text/html, Size: 1654 bytes --]

Re: [Caml-list] How to monitor a specific file for changes

[Karl Zilles]

Orlin Grigorov wrote:
> Oh, here's another idea---every certain amount of time, I just compare 
> my last saved version of the file to the current one---if different, 
> there's my event.   What do you think?

Just stat it.  You can stat it every second and never notice any 
performance issue.  In fact you can stat a whole directory of files 
every second and never notice.  It'll work cross platform, and it's easy:

let modtime file =
         let filedata = Unix.stat file in
         filedata.Unix.st_mtime


In a loop, compare the modtime to the previous modtime, then
Unix.sleep 1;

Re: [Caml-list] How to monitor a specific file for changes

[Orlin Grigorov]

[-- Attachment #1: Type: text/plain, Size: 765 bytes --]

Using "Unix.stat" works perfectly for my purpose!   On both platforms,
hurray!    THANK YOU!

On 10/22/07, Karl Zilles <zilles@1969web.com> wrote:
>
> Orlin Grigorov wrote:
> > Oh, here's another idea---every certain amount of time, I just compare
> > my last saved version of the file to the current one---if different,
> > there's my event.   What do you think?
>
> Just stat it.  You can stat it every second and never notice any
> performance issue.  In fact you can stat a whole directory of files
> every second and never notice.  It'll work cross platform, and it's easy:
>
> let modtime file =
>          let filedata = Unix.stat file in
>          filedata.Unix.st_mtime
>
>
> In a loop, compare the modtime to the previous modtime, then
> Unix.sleep 1;
>

[-- Attachment #2: Type: text/html, Size: 1239 bytes --]

Re: [Caml-list] How to monitor a specific file for changes

[Dave Benjamin]

Orlin Grigorov wrote:
> Using "Unix.stat" works perfectly for my purpose!   On both platforms,
> hurray!    THANK YOU!

You may wish to consider "Unix.LargeFile.stat" if there is the 
possibility that some of these files may be large, since st_size 
contains the file size in bytes. Unix.st_size is an int, whereas 
Unix.LargeFile.st_size is an int64.

Dave

Re: [Caml-list] How to monitor a specific file for changes

[Orlin Grigorov]

[-- Attachment #1: Type: text/plain, Size: 627 bytes --]

Thanks Dave.  Actually, the files are not large, these are output text files
from another program, and I can't imagine a situation where the byte size
can exceed a 32 bit integer.

Best,

Orlin

On 10/23/07, Dave Benjamin <dave@ramenlabs.com> wrote:
>
> Orlin Grigorov wrote:
> > Using "Unix.stat" works perfectly for my purpose!   On both platforms,
> > hurray!    THANK YOU!
>
> You may wish to consider "Unix.LargeFile.stat" if there is the
> possibility that some of these files may be large, since st_size
> contains the file size in bytes. Unix.st_size is an int, whereas
> Unix.LargeFile.st_size is an int64.
>
> Dave
>

[-- Attachment #2: Type: text/html, Size: 989 bytes --]

Re: [Caml-list] How to monitor a specific file for changes

[Richard Jones]

On Mon, Oct 22, 2007 at 12:42:28PM -0700, Karl Zilles wrote:
> Orlin Grigorov wrote:
> >Oh, here's another idea---every certain amount of time, I just compare 
> >my last saved version of the file to the current one---if different, 
> >there's my event.   What do you think?
> 
> Just stat it.  You can stat it every second and never notice any 
> performance issue.  In fact you can stat a whole directory of files 
> every second and never notice.  It'll work cross platform, and it's easy:

/me weeps.

Please don't do this.  Processes which regularly poll are a bane for
people who are working to reduce the power consumption of laptops and
servers.

Rich.

-- 
Richard Jones
Red Hat

Re: [Caml-list] How to monitor a specific file for changes

[Karl Zilles]

Richard Jones wrote:
> /me weeps.
> 
> Please don't do this.  Processes which regularly poll are a bane for
> people who are working to reduce the power consumption of laptops and
> servers.

While your heart is in the right place, I think your priorities are 
wrong on this one.  They guy is writing a thesis project, not a 
commercial application.

Let him get it coded.  I'm sure he's got more to worry about than saving 
a few watts here and there.  Or perhaps, if you feel strongly about it, 
you can contribute some tested code that works on Mac OS X and linux 
platforms.

Re: How to monitor a specific file for changes

[Sylvain Le Gall]

Hello,

On 24-10-2007, Karl Zilles <zilles@1969web.com> wrote:
> Richard Jones wrote:
>> /me weeps.
>> 
>> Please don't do this.  Processes which regularly poll are a bane for
>> people who are working to reduce the power consumption of laptops and
>> servers.
>
> While your heart is in the right place, I think your priorities are 
> wrong on this one.  They guy is writing a thesis project, not a 
> commercial application.
>
> Let him get it coded.  I'm sure he's got more to worry about than saving 
> a few watts here and there.  Or perhaps, if you feel strongly about it, 
> you can contribute some tested code that works on Mac OS X and linux 
> platforms.
>

I strongly disagree with your remark, it is not a save "a few watts here
and there". This kind of modification can make your program 100% faster
than a "poll every seconds". Moreover, knowing how to program
in a "reactive" way is something that most people tend to ignore but
which is really a good thing to learn (and most of the time it changes
the architecture of your program).

OFf course, for the sake of "quick and dirty" programming, you can just
ignore everything and go directly to the easy solution... The only
problem is that the "quick and dirty" initial programming that should
have been rewritten, never get rewritten and you end up with  something
really not efficient "in production".

Regards,
Sylvain Le Gall

Re : [Caml-list] How to monitor a specific file for changes

[Adrien]

As far as I remember MacOSX has a better support for monitoring file
system changes than Linux. In fact I remember reading a paper stating
it was unacceptable linux had no proper support for this while mac osx
has it.

You should probably have a look at the following page :
http://www.osxbook.com/software/fslogger/
Describing spotlight : "An in-kernel notification mechanism that can
inform user-space "subscribers" of file system changes as they
happen."
And about fslogger : "fslogger is a user-space program that subscribes
to the same file system event notification mechanism as the Spotlight
metadata server."

You'll probably need to embed C code in ocaml.

As for cross-platform solutions, hum, good luck...


 ---

Adrien Nader

Re: [Caml-list] How to monitor a specific file for changes

[Gordon Henriksen]

On Oct 22, 2007, at 15:03, Orlin Grigorov wrote:

> This time I've done my research, but still I cannot find a good  
> answer.   Basically, I need to make a little ocaml process, which  
> runs in the background and monitors a specific file for changes.    
> When a change occurs, I want it to back-up a copy of the version of  
> the file at that moment.    So, what I need is to set some signal  
> or event, which will happen every time that file is changed.

Orlin,

If you wish to be cross-platform, you'll likely need to devise an  
abstract interface to smooth over the variances between platforms.  
You've plenty of Linux pointers already.

On Windows, use FindFirstChangeNotification[1].

On Mac OS X and BSD, use kqueue[2]. kqueue was introduced in Tiger,  
and is one of the supporting technologies for Spotlight. To support  
Panther and earlier, you'll need to weak link[3] with kqueue. If  
kqueue is unavailable or not supported on the filesystem in question,  
you can fall back to polling modification times.

Polling modification times is simple and highly portable; many Java  
programs (for instance, Eclipse) support file monitoring using this  
technique. It may be worth considering using it exclusively, at least  
to get up and running.

— Gordon


[1] http://msdn2.microsoft.com/en-us/library/aa364417.aspx
[2] http://developer.apple.com/documentation/Darwin/Reference/ 
ManPages/man2/kqueue.2.html
[3] http://developer.apple.com/technotes/tn2002/tn2064.html

Re: [Caml-list] How to monitor a specific file for changes

[Richard Jones]

On Mon, Oct 22, 2007 at 03:03:35PM -0400, Orlin Grigorov wrote:
> This time I've done my research, but still I cannot find a good answer.
> Basically, I need to make a little ocaml process, which runs in the
> background and monitors a specific file for changes.   When a change occurs,
> I want it to back-up a copy of the version of the file at that moment.
> So, what I need is to set some signal or event, which will happen every time
> that file is changed.
> 
> Can you guys help me do it?

As well as the others mentioned, there is also Gamin which is a GNOME
API designed for this.  Originally there was a thing called FAM which
SGI invented, however it required a system-wide daemon.  Gamin is
FAM-compatible at the API level and uses the appropriate kernel calls
for the current operating system.

  http://www.gnome.org/~veillard/gamin/
  http://www.gnome.org/~veillard/gamin/differences.html

Although if current operating system != Linux, then YMMV.  Apparently
there are backends for Mac OS X, BSD & Solaris.

Gamin has an OCaml binding:

  http://svn.metaprl.org/viewvc/mojave/omake/src/clib/omake_notify.c?view=log&pathrev=1326

(well, really it's a FAM binding, but the APIs are supposed to be
compatible).

None of this works over NFS unfortunately.

Rich.

-- 
Richard Jones
Red Hat