Removing some functionality from custommakefile support

Hi,

I'm "pissed off" enough now to discuss this: I'd
like to remove the
automatic updating of custommakefile projects. There's a
dirwatcher in
place that watches all subdirectories of a project for
changes. Each
time a change happens there are certain things done

a) for each source file in that dir a stat is done to check
for its
existence (using QFileInfo::exists()), if it doesn't exist
its
afterwards removed from the filelist and also from the
blacklist

b) files that are not in the filelist and not in the
blacklist, but fit
into the wildcards defined as project files are added to an
internal
list. Also new dirs are added to the dirwatcher (which needs
another
stat to find out wether the path is actually a dir or not).

These two things make the Kdevelop ui hang as soon a file is
changed.
The amount of hanging depends on how many files are changed.
Using the
just created qmake_parser_experiment branch:

adding 10 files to the source dir causes a hang of a couple
of seconds
each run of kdev-pg (which alters files in the builddir,
which is inside
the project dir) causes a similar hang for a few seconds.

I've noticed similar problems when running kdevelop on kdev4
sources.
Also while KDevelop hangs I see quite large amounts of CPU
power spent
waiting for I/O (using top and block_dump). We even have one
or two
bugreports about this.

I'd like to remove this dirwatching and the logic that
updates the
filelist. This has the drawback of having to completely
re-populate a
filelist if one doesn't want to manually add new files
(think of adding
various files in different dirs). Currently a "Add New
Files To Project"
option exists which uses the list built during the watching.
Also a
cleaning of the stored filelist is then only done during
project
opening.

Are there objections against removing this, including the
menu entry (I
think removing user-visible strings is ok)?

Andreas

-- 
It's all in the mind, ya know.

_______________________________________________
KDevelop-devel mailing list
KDevelop-develkdevelop.org
https://barney.cs.uni-potsdam.de/mailman/listinf
o/kdevelop-devel

2007/7/21, Andreas Pakulat <apakugmx.de>:
> Hi,
>
> I'm "pissed off" enough now to discuss this:
I'd like to remove the
> automatic updating of custommakefile projects. There's
a dirwatcher in
> place that watches all subdirectories of a project for
changes. Each
> time a change happens there are certain things done
>
> a) for each source file in that dir a stat is done to
check for its
> existence (using QFileInfo::exists()), if it doesn't
exist its
> afterwards removed from the filelist and also from the
blacklist

No. rather than doing stat for each source file, stat is
done only for
the changed directory (the directory that contains
added/deleted file)
itself. Thus although 10 files are created/deleted in
certain parent
directory, existance check is done only for once, and
fileCreated()
signal is emitted only once.

> b) files that are not in the filelist and not in the
blacklist, but fit
> into the wildcards defined as project files are added
to an internal
> list. Also new dirs are added to the dirwatcher (which
needs another
> stat to find out wether the path is actually a dir or
not).

Are you saying kdevelop 3.4?

> These two things make the Kdevelop ui hang as soon a
file is changed.
> The amount of hanging depends on how many files are
changed. Using the
> just created qmake_parser_experiment branch:
>
> adding 10 files to the source dir causes a hang of a
couple of seconds
> each run of kdev-pg (which alters files in the
builddir, which is inside
> the project dir) causes a similar hang for a few
seconds.
>
> I've noticed similar problems when running kdevelop on
kdev4 sources.
> Also while KDevelop hangs I see quite large amounts of
CPU power spent
> waiting for I/O (using top and block_dump). We even
have one or two
> bugreports about this.
>
> I'd like to remove this dirwatching and the logic that
updates the
> filelist. This has the drawback of having to completely
re-populate a
> filelist if one doesn't want to manually add new files
(think of adding
> various files in different dirs). Currently a "Add
New Files To Project"
> option exists which uses the list built during the
watching. Also a
> cleaning of the stored filelist is then only done
during project
> opening.
>
> Are there objections against removing this, including
the menu entry (I
> think removing user-visible strings is ok)?

I'm not sure whether you are talking kdev4 or kdev3.4

_______________________________________________
KDevelop-devel mailing list
KDevelop-develkdevelop.org
https://barney.cs.uni-potsdam.de/mailman/listinf
o/kdevelop-devel

On 22.07.07 02:04:05, dukju ahn wrote:
> 2007/7/21, Andreas Pakulat <apakugmx.de>:
> > Hi,
> >
> > I'm "pissed off" enough now to discuss
this: I'd like to remove the
> > automatic updating of custommakefile projects.
There's a dirwatcher in
> > place that watches all subdirectories of a project
for changes. Each
> > time a change happens there are certain things
done
> >
> > a) for each source file in that dir a stat is done
to check for its
> > existence (using QFileInfo::exists()), if it
doesn't exist its
> > afterwards removed from the filelist and also from
the blacklist
> 
> No. rather than doing stat for each source file, stat
is done only for
> the changed directory (the directory that contains
added/deleted file)
> itself. Thus although 10 files are created/deleted in
certain parent
> directory, existance check is done only for once, and
fileCreated()
> signal is emitted only once.

Ooops, I guess I should've mentioned that its KDev3.4 

> > b) files that are not in the filelist and not in
the blacklist, but fit
> > into the wildcards defined as project files are
added to an internal
> > list. Also new dirs are added to the dirwatcher
(which needs another
> > stat to find out wether the path is actually a dir
or not).
> 
> Are you saying kdevelop 3.4?

Yeap.

Andreas

-- 
Don't plan any hasty moves.  You'll be evicted soon anyway.

_______________________________________________
KDevelop-devel mailing list
KDevelop-develkdevelop.org
https://barney.cs.uni-potsdam.de/mailman/listinf
o/kdevelop-devel

On Saturday 21 July 2007 18:52, Andreas Pakulat wrote:
> Hi,
>
> I'm "pissed off" enough now to discuss this:
I'd like to remove the
> automatic updating of custommakefile projects. There's
a dirwatcher in
> place that watches all subdirectories of a project for
changes. Each
> time a change happens there are certain things done
>
> a) for each source file in that dir a stat is done to
check for its
> existence (using QFileInfo::exists()), if it doesn't
exist its
> afterwards removed from the filelist and also from the
blacklist

I also noticed probably the same and I think too it's not
ideal.
When doing rm myfile.cpp; cvs up myfile.cpp to restore a
file to the original 
cvs state kdevelop notices that it doesn't exist anymore,
removes it from its 
internal list of project files and doesn't seem to add it
again, which has 
the effect that Quick open project files works worse the
longer kdevelop is 
running (in days).
So if I understood correctly, I'm for your change.

Alex

_______________________________________________
KDevelop-devel mailing list
KDevelop-develkdevelop.org
https://barney.cs.uni-potsdam.de/mailman/listinf
o/kdevelop-devel

On 22.07.07 10:33:34, Alexander Neundorf wrote:
> On Saturday 21 July 2007 18:52, Andreas Pakulat wrote:
> > Hi,
> >
> > I'm "pissed off" enough now to discuss
this: I'd like to remove the
> > automatic updating of custommakefile projects.
There's a dirwatcher in
> > place that watches all subdirectories of a project
for changes. Each
> > time a change happens there are certain things
done
> >
> > a) for each source file in that dir a stat is done
to check for its
> > existence (using QFileInfo::exists()), if it
doesn't exist its
> > afterwards removed from the filelist and also from
the blacklist
> 
> I also noticed probably the same and I think too it's
not ideal.
> When doing rm myfile.cpp; cvs up myfile.cpp to restore
a file to the original 
> cvs state kdevelop notices that it doesn't exist
anymore, removes it from its 
> internal list of project files and doesn't seem to add
it again, which has 
> the effect that Quick open project files works worse
the longer kdevelop is 
> running (in days).

Well, thats what Add new files to project is for. The custom
makefile
project stores newly added files in an internal list and
presents them
for inclusion when you choose the menu item (from the
Project menu).

Another option is to use a non-broken version control system
;)

Andreas 

-- 
You will become rich and famous unless you don't.

_______________________________________________
KDevelop-devel mailing list
KDevelop-develkdevelop.org
https://barney.cs.uni-potsdam.de/mailman/listinf
o/kdevelop-devel

On Sunday 22 July 2007 11:26, Andreas Pakulat wrote:
> On 22.07.07 10:33:34, Alexander Neundorf wrote:
> > On Saturday 21 July 2007 18:52, Andreas Pakulat
wrote:
> > > Hi,
> > >
> > > I'm "pissed off" enough now to
discuss this: I'd like to remove the
> > > automatic updating of custommakefile
projects. There's a dirwatcher in
> > > place that watches all subdirectories of a
project for changes. Each
> > > time a change happens there are certain
things done
> > >
> > > a) for each source file in that dir a stat is
done to check for its
> > > existence (using QFileInfo::exists()), if it
doesn't exist its
> > > afterwards removed from the filelist and also
from the blacklist
> >
> > I also noticed probably the same and I think too
it's not ideal.
> > When doing rm myfile.cpp; cvs up myfile.cpp to
restore a file to the
> > original cvs state kdevelop notices that it
doesn't exist anymore,
> > removes it from its internal list of project files
and doesn't seem to
> > add it again, which has the effect that Quick open
project files works
> > worse the longer kdevelop is running (in days).
>
> Well, thats what Add new files to project is for. The
custom makefile


I think you misunderstood. I have a file which is part of
the project, then 
delete it and get it again from cvs. Then kdevelop will have
removed it from 
the project files for this session.

Alex

_______________________________________________
KDevelop-devel mailing list
KDevelop-develkdevelop.org
https://barney.cs.uni-potsdam.de/mailman/listinf
o/kdevelop-devel

On 22.07.07 18:55:23, Alexander Neundorf wrote:
> On Sunday 22 July 2007 11:26, Andreas Pakulat wrote:
> > On 22.07.07 10:33:34, Alexander Neundorf wrote:
> > > On Saturday 21 July 2007 18:52, Andreas
Pakulat wrote:
> > > > Hi,
> > > >
> > > > I'm "pissed off" enough now to
discuss this: I'd like to remove the
> > > > automatic updating of custommakefile
projects. There's a dirwatcher in
> > > > place that watches all subdirectories of
a project for changes. Each
> > > > time a change happens there are certain
things done
> > > >
> > > > a) for each source file in that dir a
stat is done to check for its
> > > > existence (using QFileInfo::exists()),
if it doesn't exist its
> > > > afterwards removed from the filelist and
also from the blacklist
> > >
> > > I also noticed probably the same and I think
too it's not ideal.
> > > When doing rm myfile.cpp; cvs up myfile.cpp
to restore a file to the
> > > original cvs state kdevelop notices that it
doesn't exist anymore,
> > > removes it from its internal list of project
files and doesn't seem to
> > > add it again, which has the effect that Quick
open project files works
> > > worse the longer kdevelop is running (in
days).
> >
> > Well, thats what Add new files to project is for.
The custom makefile
> 
> 
> I think you misunderstood. I have a file which is part
of the project, then 
> delete it and get it again from cvs. Then kdevelop will
have removed it from 
> the project files for this session.

Right, because it removes files automatically when they're
removed from
disk. However it also monitors the directories for newly
added files and
keeps a list of those. However its not sensible to
automatically add
these to the project list so there's a menu option which
presents you
with any "new" files (i.e. files that weren't in
the projectlist when
kdevelop started or were removed and re-added) and you can
select which
ones to add to the project and which don't get added.

The problem is that the dirwatching and the amount of code
afterwards
(because from the dir watching I don't get more information
than: hey
there happened something in here) is pretty expensive. So
thats what
will be removed, which of course will "fix" your
problem as the file
won't get removed unless you close and re-open the project
after
removing and before cvs up.

Andreas

-- 
Write yourself a threatening letter and pen a defiant
reply.

_______________________________________________
KDevelop-devel mailing list
KDevelop-develkdevelop.org
https://barney.cs.uni-potsdam.de/mailman/listinf
o/kdevelop-devel

On 7/23/07, Andreas Pakulat <apakugmx.de> wrote:
> On 22.07.07 18:55:23, Alexander Neundorf wrote:
> > On Sunday 22 July 2007 11:26, Andreas Pakulat
wrote:
> > > On 22.07.07 10:33:34, Alexander Neundorf
wrote:
> > > > On Saturday 21 July 2007 18:52, Andreas
Pakulat wrote:
> > > > > Hi,
> > > > >
> > > > > I'm "pissed off" enough
now to discuss this: I'd like to remove the
> > > > > automatic updating of
custommakefile projects. There's a dirwatcher in
> > > > > place that watches all
subdirectories of a project for changes. Each
> > > > > time a change happens there are
certain things done
> > > > >
> > > > > a) for each source file in that dir
a stat is done to check for its
> > > > > existence (using
QFileInfo::exists()), if it doesn't exist its
> > > > > afterwards removed from the
filelist and also from the blacklist
> > > >
> > > > I also noticed probably the same and I
think too it's not ideal.
> > > > When doing rm myfile.cpp; cvs up
myfile.cpp to restore a file to the
> > > > original cvs state kdevelop notices that
it doesn't exist anymore,
> > > > removes it from its internal list of
project files and doesn't seem to
> > > > add it again, which has the effect that
Quick open project files works
> > > > worse the longer kdevelop is running (in
days).
> > >
> > > Well, thats what Add new files to project is
for. The custom makefile
> >
> >
> > I think you misunderstood. I have a file which is
part of the project, then
> > delete it and get it again from cvs. Then kdevelop
will have removed it from
> > the project files for this session.
>
> Right, because it removes files automatically when
they're removed from
> disk. However it also monitors the directories for
newly added files and
> keeps a list of those. However its not sensible to
automatically add
> these to the project list so there's a menu option
which presents you
> with any "new" files (i.e. files that weren't
in the projectlist when
> kdevelop started or were removed and re-added) and you
can select which
> ones to add to the project and which don't get added.
>
> The problem is that the dirwatching and the amount of
code afterwards
> (because from the dir watching I don't get more
information than: hey
> there happened something in here) is pretty expensive.
So thats what
> will be removed, which of course will "fix"
your problem as the file
> won't get removed unless you close and re-open the
project after
> removing and before cvs up.
>
> Andreas
>

I'm all for removing the automatic updating. It was a nice
idea but it
seems to have created more problems than it's worth. There
are enough
cases where this will break down badly that removing it is
probably
the best option for now.

// jens

_______________________________________________
KDevelop-devel mailing list
KDevelop-develkdevelop.org
https://barney.cs.uni-potsdam.de/mailman/listinf
o/kdevelop-devel

Could we also remove the functionality that asks to add new
files to the
project when it is being opened?  This is uber annoying. ;)

Kris Wong

_______________________________________________
KDevelop-devel mailing list
KDevelop-develkdevelop.org
https://barney.cs.uni-potsdam.de/mailman/listinf
o/kdevelop-devel

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1


On Jul 22, 2007, at 9:33 AM, Alexander Neundorf wrote:

> On Saturday 21 July 2007 18:52, Andreas Pakulat wrote:
>> Hi,
>>
>> I'm "pissed off" enough now to discuss
this: I'd like to remove the
>> automatic updating of custommakefile projects.
There's a  
>> dirwatcher in
>> place that watches all subdirectories of a project
for changes. Each
>> time a change happens there are certain things
done
>>
>> a) for each source file in that dir a stat is done
to check for its
>> existence (using QFileInfo::exists()), if it
doesn't exist its
>> afterwards removed from the filelist and also from
the blacklist
>
> I also noticed probably the same and I think too it's
not ideal.
> When doing rm myfile.cpp; cvs up myfile.cpp to restore
a file to  
> the original
> cvs state kdevelop notices that it doesn't exist
anymore, removes  
> it from its
> internal list of project files and doesn't seem to add
it again,  
> which has
> the effect that Quick open project files works worse
the longer  
> kdevelop is
> running (in days).
> So if I understood correctly, I'm for your change.
>
> Alex
>

run cvs up -C myfile.cpp instead 
- --
Matt


-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.6 (Darwin)

iD8DBQFGqCJWA6Vv5rghv0cRAtRvAKCZsq5j75bee2zAPMmWCDA4iodIwQCe
PGkX
/9KXTPB1NnMCvR4PzDIktE0=
=eydF
-----END PGP SIGNATURE-----

_______________________________________________
KDevelop-devel mailing list
KDevelop-develkdevelop.org
https://barney.cs.uni-potsdam.de/mailman/listinf
o/kdevelop-devel