Project parsing and loading

Why ImportProjectJob which is called in
KDevelop::Project::open()
parses _every_ contents of full project tree at openning
time?
IMHO, it is waste because not all user doesn't develop
every
part of one project. For example, I don't go over
languages,
sublime, editor etc.

It would be more efficient to parse just toplevel items,
and
only when the user clicks dome directory, we can  just
parse
that dir. It tooks 3 seconds to open kdepimlibs in
custommake
manager, and I even gave up to open kdelibs.

Also, was there any reason for ImportProjectJob not to use
QThread internally??

_______________________________________________
KDevelop-devel mailing list
KDevelop-develkdevelop.org
https://barney.cs.uni-potsdam.de/mailman/listinf
o/kdevelop-devel

On Saturday 26 May 2007 09:13, dukju ahn wrote:
> Why ImportProjectJob which is called in
KDevelop::Project::open()
> parses _every_ contents of full project tree at
openning time?
> IMHO, it is waste because not all user doesn't develop
every
> part of one project. For example, I don't go over
languages,
> sublime, editor etc.
>

ImportProjectJob drives the build system manager, which
means that we need to 
have every directory parsed when the project is opened.

> It would be more efficient to parse just toplevel
items, and
> only when the user clicks dome directory, we can  just
parse
> that dir. It tooks 3 seconds to open kdepimlibs in
custommake
> manager, and I even gave up to open kdelibs.
>

Then fix either the custom make manager or the generic
project manager 
(whichever one is used) to be faster. 3 seconds is an
arbitrary number. Run 
callgrind on it and provide the dump instead.

> Also, was there any reason for ImportProjectJob not to
use
> QThread internally??
>

You want to manage that extra complexity? It's already
fairly asynchronous as 
it is since it uses the KJob framework already.
-- 
Matt

_______________________________________________
KDevelop-devel mailing list
KDevelop-develkdevelop.org
https://barney.cs.uni-potsdam.de/mailman/listinf
o/kdevelop-devel

2007/5/26, Matt Rogers <mattrkde.org>:
> On Saturday 26 May 2007 09:13, dukju ahn wrote:
> > Why ImportProjectJob which is called in
KDevelop::Project::open()
> > parses _every_ contents of full project tree at
openning time?
> > IMHO, it is waste because not all user doesn't
develop every
> > part of one project. For example, I don't go over
languages,
> > sublime, editor etc.
> >
>
> ImportProjectJob drives the build system manager, which
means that we need to
> have every directory parsed when the project is
opened.

Yes it uses IProjectFileManager::parse(). But why does this
necessarily
mean that every directory should be parsed at openning? Can
you
be more clear?

> > Also, was there any reason for ImportProjectJob
not to use
> > QThread internally??
> >
>
> You want to manage that extra complexity? It's already
fairly asynchronous as
> it is since it uses the KJob framework already.

Simply using KJob doesn't mean that it is asynchronous. KJob
doesn't
provide asynchronous facilities automatically. Rather
Kjob::start() expect
asynchronous call such as QThread::start(),
KProcess::start(), or some
interprocess calling.
As a consequence of current implementation, GUI blocks when
loading a huge project.
Anyway, I think I should use QThread in importjob near
future.

_______________________________________________
KDevelop-devel mailing list
KDevelop-develkdevelop.org
https://barney.cs.uni-potsdam.de/mailman/listinf
o/kdevelop-devel

On 26.05.07 12:44:01, dukju ahn wrote:
> 2007/5/26, Matt Rogers <mattrkde.org>:
> > On Saturday 26 May 2007 09:13, dukju ahn wrote:
> > > Why ImportProjectJob which is called in
KDevelop::Project::open()
> > > parses _every_ contents of full project tree
at openning time?
> > > IMHO, it is waste because not all user
doesn't develop every
> > > part of one project. For example, I don't go
over languages,
> > > sublime, editor etc.
> >
> > ImportProjectJob drives the build system manager,
which means that we need to
> > have every directory parsed when the project is
opened.
> 
> Yes it uses IProjectFileManager::parse(). But why does
this necessarily
> mean that every directory should be parsed at openning?
Can you
> be more clear?

I think this is kind of a relict from a time where the plan
was to have
importers and editors for the buildsystems. The importer
would simply
import a given dir (recursively) into kdevelop as a project,
while the
editor provides means to edit the buildsystem. However
nowadays we don't
have that distinction anymore, we have managers. One thing
to note
though is that you don't know (until you parse them) which
parts of the
project depend on each other and wether you need to have
some infos from

foo/bar/buildsystemfile

to properly parse

bar/buildsystemfile

So I think we should still parse the whole project on
opening, but see
below.

> > > Also, was there any reason for
ImportProjectJob not to use
> > > QThread internally??
> > >
> >
> > You want to manage that extra complexity? It's
already fairly asynchronous as
> > it is since it uses the KJob framework already.
> 
> Simply using KJob doesn't mean that it is asynchronous.
KJob doesn't
> provide asynchronous facilities automatically. Rather
Kjob::start() expect
> asynchronous call such as QThread::start(),
KProcess::start(), or some
> interprocess calling.
> As a consequence of current implementation, GUI blocks
when
> loading a huge project.
> Anyway, I think I should use QThread in importjob near
future.

Please look into wether ThreadWeaver::Job provides help in
implementing
this. Its used for the background parser and does a really
good job
there. You're even easily able to create dependecies between
the various
folders, so for a project like

foo/bar
foo/buz
bar/boo
bar/doo

You could parse foo and bar on parallel but define that
parsing of foo/bar
and foo/buz depends on having finished bar/.

Also doing a threaded implementation here makes the above
problem "go
away" automatically, and the items would just pop up in
the tree after
some time.

Andreas

-- 
Today is National Existential Ennui Awareness Day.

_______________________________________________
KDevelop-devel mailing list
KDevelop-develkdevelop.org
https://barney.cs.uni-potsdam.de/mailman/listinf
o/kdevelop-devel