QFileSystemWatcher vs KDirWatch

Is the problem in KDirWatch is too serious to be usable?
Because,
I found strange bug in QFileSysWatcher, especially for
fileChanged().

In my simple test program, when I edit a file that is being
monitored,
fileChanged() signal is emitted multiples of times. When the
size of
file is small (up to few kilobytes) it is ok, but when
filesize becomes
larger(300kb), same signal is emitted 20~30 times.
It caused application hanging because QTextStream should
parse
that big fie 20~30 times again and again

I sent qt mailing-list subscribe mail. But don't try to use
filesyswatcher
for a moment, unless we found what's more stable between
KDirWatch or QFileSysWatcher..

_______________________________________________
KDevelop-devel mailing list
KDevelop-develkdevelop.org
https://barney.cs.uni-potsdam.de/mailman/listinf
o/kdevelop-devel

On 02.06.07 04:56:11, dukju ahn wrote:
> Is the problem in KDirWatch is too serious to be
usable? Because,
> I found strange bug in QFileSysWatcher, especially for
fileChanged().
> 
> In my simple test program, when I edit a file that is
being monitored,
> fileChanged() signal is emitted multiples of times.
When the size of
> file is small (up to few kilobytes) it is ok, but when
filesize becomes
> larger(300kb), same signal is emitted 20~30 times.
> It caused application hanging because QTextStream
should parse
> that big fie 20~30 times again and again

That doesn't sound good, but I can't reproduce this here, so
until
otherwise found I think this is a system-specific problem.

Whats weird is that doing things like

echo foo >>test.txt doesn't trigger the signal...

> I sent qt mailing-list subscribe mail. But don't try to
use filesyswatcher
> for a moment, unless we found what's more stable
between
> KDirWatch or QFileSysWatcher..

As I tried to explain: KDirWatch uses questionable ways of
getting
notified for changes. We just recently received a bugreport
for
KDevelop3 from somebody who can't/want to switch away from
fam. This
lets KDevelop hog the CPU  on his system. So unless somebody
else can
reproduce the QFSWatcher problem, I don't think we shouldn't
use it.

Andreas

-- 
You have a reputation for being thoroughly reliable and
trustworthy.
A pity that it's totally undeserved.

_______________________________________________
KDevelop-devel mailing list
KDevelop-develkdevelop.org
https://barney.cs.uni-potsdam.de/mailman/listinf
o/kdevelop-devel

On 02.06.07 12:57:43, Andreas Pakulat wrote:
> On 02.06.07 04:56:11, dukju ahn wrote:
> > Is the problem in KDirWatch is too serious to be
usable? Because,
> > I found strange bug in QFileSysWatcher, especially
for fileChanged().
> > 
> > In my simple test program, when I edit a file that
is being monitored,
> > fileChanged() signal is emitted multiples of
times. When the size of
> > file is small (up to few kilobytes) it is ok, but
when filesize becomes
> > larger(300kb), same signal is emitted 20~30
times.
> > It caused application hanging because QTextStream
should parse
> > that big fie 20~30 times again and again
> 
> That doesn't sound good, but I can't reproduce this
here, so until
> otherwise found I think this is a system-specific
problem.
> 
> Whats weird is that doing things like
> 
> echo foo >>test.txt doesn't trigger the
signal...

I was kind of wrong, it does emit 2 times when you use
something like

cat foo >test.txt

but thats fine, because the file is first cleared and then
filled with
content again. Unless your kernel lacks inotify support you
should see
the same. I think the kernel sends a changed signal through
inotify
every time a write to the file happens.

Andreas

-- 
Keep it short for pithy sake.

_______________________________________________
KDevelop-devel mailing list
KDevelop-develkdevelop.org
https://barney.cs.uni-potsdam.de/mailman/listinf
o/kdevelop-devel

On Saturday 02 June 2007, Andreas Pakulat wrote:
> As I tried to explain: KDirWatch uses questionable ways
of getting
> notified for changes. We just recently received a
bugreport for
> KDevelop3 from somebody who can't/want to switch away
from fam.

I'm lost here, what are the KDirWatch problems? Were they
presented on 
kde-core-devel?

Andras

-- 
Quanta Plus developer - http://quanta.kdewebdev.o
rg
K Desktop Environment - http://www.kde.org

_______________________________________________
KDevelop-devel mailing list
KDevelop-develkdevelop.org
https://barney.cs.uni-potsdam.de/mailman/listinf
o/kdevelop-devel

On 02.06.07 20:27:13, Andras Mantia wrote:
> On Saturday 02 June 2007, Andreas Pakulat wrote:
> > As I tried to explain: KDirWatch uses questionable
ways of getting
> > notified for changes. We just recently received a
bugreport for
> > KDevelop3 from somebody who can't/want to switch
away from fam.
> 
> I'm lost here, what are the KDirWatch problems? Were
they presented on 
> kde-core-devel?

The most serious problem is that it uses a broken
notification system on
*nix (fam/gamin). Another problem is that various things are
not
implemented and some things are missing (for example
stopping watching
of individual files). 

Yes I asked (quite some while ago) what the plans are for
KDirWatch on
k-c-d, and IIRC the answer is: unless somebody steps up to
work on it it
will stay as is.

Andreas

-- 
You will visit the Dung Pits of Glive soon.

_______________________________________________
KDevelop-devel mailing list
KDevelop-develkdevelop.org
https://barney.cs.uni-potsdam.de/mailman/listinf
o/kdevelop-devel

On Saturday 02 June 2007, Andreas Pakulat wrote:
> On 02.06.07 20:27:13, Andras Mantia wrote:
> > On Saturday 02 June 2007, Andreas Pakulat wrote:
> > > As I tried to explain: KDirWatch uses
questionable ways of
> > > getting notified for changes. We just
recently received a
> > > bugreport for KDevelop3 from somebody who
can't/want to switch
> > > away from fam.
> >
> > I'm lost here, what are the KDirWatch problems?
Were they presented
> > on kde-core-devel?
>
> The most serious problem is that it uses a broken
notification system
> on *nix (fam/gamin). 
Depends. It is a configure switch for kdelibs. 
It uses:
- fam/gamin: if it was enabled and it is present
- kernel (inotify, maybe dnotify)
- polling

What does the Qt class use if not the ones above?	

> Another problem is that various things are not 
> implemented and some things are missing (for example
stopping
> watching of individual files).

KDirWatch::removeFile and KDirWatch::addFile works for me
just fine in 
KDE3.

> Yes I asked (quite some while ago) what the plans are
for KDirWatch
> on k-c-d, and IIRC the answer is: unless somebody steps
up to work on
> it it will stay as is.

Probably because it works for most cases and there was no
need to change 
it.

Andras

-- 
Quanta Plus developer - http://quanta.kdewebdev.o
rg
K Desktop Environment - http://www.kde.org

_______________________________________________
KDevelop-devel mailing list
KDevelop-develkdevelop.org
https://barney.cs.uni-potsdam.de/mailman/listinf
o/kdevelop-devel

On 02.06.07 21:47:13, Andras Mantia wrote:
> On Saturday 02 June 2007, Andreas Pakulat wrote:
> > On 02.06.07 20:27:13, Andras Mantia wrote:
> > > On Saturday 02 June 2007, Andreas Pakulat
wrote:
> > > > As I tried to explain: KDirWatch uses
questionable ways of
> > > > getting notified for changes. We just
recently received a
> > > > bugreport for KDevelop3 from somebody
who can't/want to switch
> > > > away from fam.
> > >
> > > I'm lost here, what are the KDirWatch
problems? Were they presented
> > > on kde-core-devel?
> >
> > The most serious problem is that it uses a broken
notification system
> > on *nix (fam/gamin). 
> Depends. It is a configure switch for kdelibs. 

Right, but distros use fam always and I doubt we can make
them change my
mind. They didn't change their mind until now.

> It uses:
> - fam/gamin: if it was enabled and it is present
> - kernel (inotify, maybe dnotify)
> - polling
> 
> What does the Qt class use if not the ones above?	

It just doesn't use fam/gamin. It uses inotify if available,
else it
polls.

> > Another problem is that various things are not 
> > implemented and some things are missing (for
example stopping
> > watching of individual files).
> 
> KDirWatch::removeFile and KDirWatch::addFile works for
me just fine in 
> KDE3.

Yes, but its a workaround for a broken implementation.

> > Yes I asked (quite some while ago) what the plans
are for KDirWatch
> > on k-c-d, and IIRC the answer is: unless somebody
steps up to work on
> > it it will stay as is.
> 
> Probably because it works for most cases and there was
no need to change 
> it.

I can't seem to find the mails again (except those from
k-c-d titles
qfilesystemwatcher vs. kdirwatch). You can find some
information in
those. 

So even if the above is not all that serious there are two
really
serious problems:

a) No Maintainer and seemingly few people that know the
code
b) Not platform independent, that is it only works on *nix
(maybe MacOS)
without polling. 

Andreas, who still thinks a wrapper around QFSWatcher is
better than
KDirWatch.

-- 
You have no real enemies.

_______________________________________________
KDevelop-devel mailing list
KDevelop-develkdevelop.org
https://barney.cs.uni-potsdam.de/mailman/listinf
o/kdevelop-devel

On Saturday 02 June 2007, Andreas Pakulat wrote:
> Right, but distros use fam always and I doubt we can
make them change
> my mind. They didn't change their mind until now.

You can always shut down the fam deamon.

> It just doesn't use fam/gamin. It uses inotify if
available, else it
> polls.

Which probably results is a very bad performance on NFS
mounted file 
systems.

> I can't seem to find the mails again (except those from
k-c-d titles
> qfilesystemwatcher vs. kdirwatch). You can find some
information in
> those.

I was also searching my "private" k-c-d folder and
couldn't find 
anything, but some problems from 2004 regarding kdirwatch.

> So even if the above is not all that serious there are
two really
> serious problems:
>
> a) No Maintainer and seemingly few people that know the
code
> b) Not platform independent, that is it only works on
*nix (maybe
> MacOS) without polling.

The first can be true, but for the second, does the qt class
use other 
method instead of polling on Win/Mac?

> Andreas, who still thinks a wrapper around QFSWatcher
is better than
> KDirWatch.

I think you just have to present your idea and the code on
the core 
devel list. 

Andras

-- 
Quanta Plus developer - http://quanta.kdewebdev.o
rg
K Desktop Environment - http://www.kde.org

_______________________________________________
KDevelop-devel mailing list
KDevelop-develkdevelop.org
https://barney.cs.uni-potsdam.de/mailman/listinf
o/kdevelop-devel

On 02.06.07 23:23:23, Andras Mantia wrote:
> On Saturday 02 June 2007, Andreas Pakulat wrote:
> > Right, but distros use fam always and I doubt we
can make them change
> > my mind. They didn't change their mind until now.
> 
> You can always shut down the fam deamon.

Sure, but that only works for fam, not for gamin.

> > It just doesn't use fam/gamin. It uses inotify if
available, else it
> > polls.
> 
> Which probably results is a very bad performance on NFS
mounted file 
> systems.

I think not only on NFS.

> > So even if the above is not all that serious there
are two really
> > serious problems:
> >
> > a) No Maintainer and seemingly few people that
know the code
> > b) Not platform independent, that is it only works
on *nix (maybe
> > MacOS) without polling.
> 
> The first can be true, but for the second, does the qt
class use other 
> method instead of polling on Win/Mac?

I didn't check Mac, but on Win it uses Win32 API to get
notified. And
something that also came up either in the thread I mentioned
or sometime
else on kde-devel: QFSWatcher actually works on OpenBSD
while
KDirWatcher doesn't work there (except by using polling),
because QFSW
uses kqueue on that platform.

Looking at the source dir I can't really see whats used on
mac, unless
its also inotify or kqueue.

Andreas

-- 
An avocado-tone refrigerator would look good on your
resume.

_______________________________________________
KDevelop-devel mailing list
KDevelop-develkdevelop.org
https://barney.cs.uni-potsdam.de/mailman/listinf
o/kdevelop-devel