From: Alex Young Subject: [OxLUG] Watching files and delurking Date: Sat, 26 Jun 2004 16:14:27 +0000 =2D----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 Hi all, I've been lurking for a while, and figured I might ask if anyone knows the= =20 answer to what should be a fairly simple question (I think...): What is the Right Way to watch a file for modifications, and call something= =20 when it changes? I want to be able to do something like: $ watch-file interesting-file -e 'md5sum interesting-file >> hashlog' & for example, and there doeesn't seem to be an obvious method. Have I misse= d=20 something? I've knocked up a script to check modification times every seco= nd=20 or so, but it's not particularly elegant, and it doesn't seem to work on al= l=20 volumes for some reason. I'd be grateful for any clues... By the way, I was sorely tempted by the idea of punting, but I wasn't aroun= d=20 that day... Any plans to do it again? See You Later, =2D --=20 Alex Young From: Jon Masters Subject: Re: [OxLUG] Watching files and delurking Date: Sun, 27 Jun 2004 11:40:44 +0100 -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 Alex Young wrote: | What is the Right Way to watch a file for modifications, and call something | when it changes? I want to be able to do something like: You can sit there stating the file for a while (can someone tell us the correct VFS callback he should use instead please?). As of Linux 2.4.x releases it is possible to use the dnotify callbacks to get a notification of when a directory changes - see dnotify(1). | By the way, I was sorely tempted by the idea of punting, but I wasn't around | that day... Any plans to do it again? I would hope so. Cheers, Jon. From: drkjam Subject: Re: [OxLUG] Watching files and delurking Date: Tue, 29 Jun 2004 18:20:01 +0100 -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 Hi Alex, I think what you want to do isn't outside the realms of what any good stdio implementation should be able to handle. Below is an example of how to do a single threaded tail in Perl. Should be easily portable to C as required. Regards, David Moss Perl source for ptail.pl :- #!/usr/bin/perl -w use strict; use IO::File; $| = 1; # Turn off buffering on stdout. # Follow a file forever until interrupted. sub tail { ~ my $filename = shift; ~ my $callback = shift; ~ my $interval = shift || 3; # Default to a 3 second delay between file polls. ~ my $fh = new IO::File( $filename ) or die("Error: Failed to open $filename : $!\n"); ~ for(;;) ~ { ~ my $curr_pos = 0; ~ for( $curr_pos = tell($fh); my $data = <$fh>; $curr_pos = tell($fh) ) ~ { ~ # Invoke callback as new data found appears. ~ &$callback( $data ); ~ } ~ sleep( $interval ); ~ seek( $fh, $curr_pos, 0 ); ~ # Check if file was deleted. ~ if( (stat($fh))[3] == 0) ~ { ~ return; ~ } ~ } } # Do something useful with the data. sub callback { ~ my $data = shift; ~ chomp( $data ); # Strip newlines. ~ return if $data eq ''; # Skip blank lines. ~ print "Read: '$data'\n"; } &tail( "./file.log", \&callback, 1 ); # === EOF === Jon Masters wrote: | Alex Young wrote: | | | What is the Right Way to watch a file for modifications, and call | something | when it changes? I want to be able to do something | like: | | You can sit there stating the file for a while (can someone tell us | the correct VFS callback he should use instead please?). As of | Linux 2.4.x releases it is possible to use the dnotify callbacks to | get a notification of when a directory changes - see dnotify(1). | | | By the way, I was sorely tempted by the idea of punting, but I | wasn't around | that day... Any plans to do it again? | | I would hope so. | | Cheers, | | Jon. From: Jon Masters Subject: Re: [OxLUG] Watching files and delurking Date: Tue, 29 Jun 2004 23:39:12 +0100 -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 drkjam wrote: | I think what you want to do isn't outside the realms of what any good | stdio implementation should be able to handle. Very true however this requires that one open the file in question. Jon. From: Alex Young Subject: Re: [OxLUG] Watching files and delurking Date: Wed, 30 Jun 2004 12:54:59 +0000 =2D----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 On Tuesday 29 June 2004 17:20, drkjam wrote: > Hi Alex, > > I think what you want to do isn't outside the realms of what any good > stdio implementation should be able to handle. Below is an example of > how to do a single threaded tail in Perl. Should be easily portable to > C as required. Thanks! Actually that's not a million miles away from what I was thinking.= I=20 tried something very similar in Python, with moderate success. I'll give=20 yours a try next. Out of interest, is there any particularly good reason why the kernel can't= =20 give callback hooks to specific file changes in general? I know there's th= e=20 dnotify call, but why is it so limited? I haven't delved deep enough into= =20 the kernel to know what API (surely that should be KPI?) it uses internally. On the other hand, after some more google, the Right Way would seem to be t= o=20 use libfam - #include , use SGI::FAM, or import _fam, depending on=20 one's own particular fetish. I found enlightenment at=20 http://www.devchannel.org/devtoolschannel/04/05/13/2146252.shtml. See You Later, =2D --=20 Alex Young From: Matthew Kirkwood Subject: Re: [OxLUG] Watching files and delurking Date: Wed, 30 Jun 2004 15:50:53 +0100 (BST) On Wed, 30 Jun 2004, Alex Young wrote: > Out of interest, is there any particularly good reason why the kernel > can't give callback hooks to specific file changes in general? I know > there's the dnotify call, but why is it so limited? I haven't delved > deep enough into the kernel to know what API (surely that should be > KPI?) it uses internally. There's no particular reason, and it could fairly easily be added, once you refine what kinds of "change" are of interest. That's probably a non-trivial task: Notifying on every write(2) syscall would probably be too high overhead, and if you can trust all the apps whose activity you're interested in, you could probably arrange that they notify you explicitly. Matthew. From: Chris Wareham Subject: Re: [OxLUG] Watching files and delurking Date: Wed, 30 Jun 2004 16:13:03 +0100 Matthew Kirkwood wrote: > On Wed, 30 Jun 2004, Alex Young wrote: > > >>Out of interest, is there any particularly good reason why the kernel >>can't give callback hooks to specific file changes in general? I know >>there's the dnotify call, but why is it so limited? I haven't delved >>deep enough into the kernel to know what API (surely that should be >>KPI?) it uses internally. > > > There's no particular reason, and it could fairly easily > be added, once you refine what kinds of "change" are of > interest. > > That's probably a non-trivial task: > > Notifying on every write(2) syscall would probably be too > high overhead, and if you can trust all the apps whose > activity you're interested in, you could probably arrange > that they notify you explicitly. > > Matthew. > Isn't this an ideal task for FAM (File Alteration Monitor). I haven't used it myself, but it's been around for quite a while, and is very portable. If you've got a prepackaged GNOME desktop installed, then you've most likely got FAM installed as Nautilus uses it. http://oss.sgi.com/projects/fam/ Chris -- Privacy and Confidentiality Notice This message (including any attachments) is strictly confidential and intended solely for the person or organisation to which it is addressed. It may contain privileged and confidential information and if you are not an intended recipient, you must not copy, distribute or take any action in reliance on it. If you have received this message in error, please notify us as soon as possible and delete it and any attached files from your system. The contents of this transmission are the view of the sender and do not necessarily reflect those of FKI Logistex Limited. FKI Logistex Limited has taken all reasonable precautions to ensure that any attachments to this email do not carry software viruses. However we cannot accept any responsibility for any damage sustained as a result of software viruses and would advise you to carry out you own virus checks before opening any attachment.