Making Access to project configuration threadsafe

Hi,

I'd like to ask our thread-experts for some ideas on how we
could make
accesses to the project configuration threadsafe.

Currently the project configuration object can be obtained
by

virtual KSharedConfig::Ptr projectConfiguration() const =
0;

which (afaik) isn't even safe to read from in multiple
threads. We need
to stick with KConfig* or KSharedKConfig* (or ::Ptr) because
there's
some logic involved when creating the KConfig object from
our two
project config files.

One way I can see is having a static helper function that
creates a
KConfig object from two given files which can be called from
any thread.
The downside here is that plugins get access to either the
(possibly
remote) project files or temporary copies of them and they
can do
changes which would be overwritten on project closing. With
a
KSharedConfig this doesn't happen.

Andreas

-- 
Your nature demands love and your happiness depends on it.

_______________________________________________
KDevelop-devel mailing list
KDevelop-develkdevelop.org
https://barney.cs.uni-potsdam.de/mailman/listinf
o/kdevelop-devel

I haven't look at the API for this at all, but I'll chime in
with some
knowledge on threading and concurrency.

1. Reading from an object is never a issue in a threaded
environment.
Writing to an object is where concurrency issues arise.
2. Depending on the nature of how and when the configuration
will be
accessed for reading and writing, there are different ways
to handle
this.  Usually the more assumptions that can be made, the
simpler the
implementation can become.

Here are two cover-all-bases approaches for handling this:

1. The project only gives out and accepts deep copies of
the
configuration.  It will have one mutex which will be locked
for long
enough to deep copy into another configuration object (in
the case of a
read), or from another configuration object (in the case of
a write).
The down side to this is obvious, it is inefficient.  The
upside are
that it is a simple implementation and not prone to errors.
Depending on
the size of the configuration object and the amount of
accesses, it may
be sufficient.

2. The project has two methods, one which locks a
configuration mutex
and returns a pointer, and one that unlocks it.  It then
becomes the
responsibility of the caller to lock and unlock
appropriately.  This is
obviously more efficient, and more prone to errors and
incorrect usage
scenarios.

I may be able to come up with some better solutions if I
have the time
to look into this further.

Kris Wong

_______________________________________________
KDevelop-devel mailing list
KDevelop-develkdevelop.org
https://barney.cs.uni-potsdam.de/mailman/listinf
o/kdevelop-devel

On 29.05.07 13:26:57, Kris Wong wrote:
> I haven't look at the API for this at all, but I'll
chime in with some
> knowledge on threading and concurrency.
> 
> 1. Reading from an object is never a issue in a
threaded environment.
> Writing to an object is where concurrency issues
arise.

I'm not sure that reading really is a non-issue, keep in
mind that we're
dealing with implicitly shared kconfig objects here. At
least thats the
impression that I got from the answers on kde-devel.

> 2. Depending on the nature of how and when the
configuration will be
> accessed for reading and writing, there are different
ways to handle
> this.  Usually the more assumptions that can be made,
the simpler the
> implementation can become.

Writing to the configuration will only be done in the GUI
thread, after
doing that the KSharedConfig::Ptr object in Project would
get updated to
reflect these changes (that last part is not implemented
yet, IIRC). The
writing will happen on a separate KConfig object that is
created inside
the KControl Modules which is created from the temp files,
those need to
be re-read once the configuration was applied/ok'ed.

> Here are two cover-all-bases approaches for handling
this:
> 
> 1. The project only gives out and accepts deep copies
of the
> configuration.  It will have one mutex which will be
locked for long
> enough to deep copy into another configuration object
(in the case of a
> read), or from another configuration object (in the
case of a write).
> The down side to this is obvious, it is inefficient. 
The upside are
> that it is a simple implementation and not prone to
errors. Depending on
> the size of the configuration object and the amount of
accesses, it may
> be sufficient.

As the KSharedConfig pointer is supposed to be short-lived
and KConfig
is meant to be accessed often (pretty fast in KDevelop3) I
think this
could suffice.

> 2. The project has two methods, one which locks a
configuration mutex
> and returns a pointer, and one that unlocks it.  It
then becomes the
> responsibility of the caller to lock and unlock
appropriately.  This is
> obviously more efficient, and more prone to errors and
incorrect usage
> scenarios.

Thats exactly what I'd like to avoid, for exactly the
reasons you
mention.

Andreas

-- 
Communicate!  It can't make things any worse.

_______________________________________________
KDevelop-devel mailing list
KDevelop-develkdevelop.org
https://barney.cs.uni-potsdam.de/mailman/listinf
o/kdevelop-devel

> I'm not sure that reading really is a non-issue, keep
in mind that
> we're dealing with implicitly shared kconfig objects
here. At least
> thats the impression that I got from the answers on
kde-devel.

Let me rephrase, reading is only an issue when the
possibility exists
for a simultaneous write.  i.e., multiple simultaneous reads
w/o the
possibility of a write would not require any sort of
locking.  In the
scenario, the possibility of a write exists, so reading is
an issue.

>> 1. The project only gives out and accepts deep
copies of the
>> configuration.  It will have one mutex which will
be locked for long
>> enough to deep copy into another configuration
object (in the case of
a
>> read), or from another configuration object (in the
case of a write).
>> The down side to this is obvious, it is
inefficient.  The upside are
>> that it is a simple implementation and not prone to
errors. Depending
on
>> the size of the configuration object and the amount
of accesses, it
may
>> be sufficient.

> As the KSharedConfig pointer is supposed to be
short-lived and
> Kconfig is meant to be accessed often (pretty fast in
KDevelop3) I
> think this could suffice.

Actually, I think a good solution would not be to deep copy
the entire
config object, but rather 1 particular group.  Since each
particular
piece of code is usually only going to be concerned with the
options in
one particular group, the config could be locked in order to
deep just
that group.  This would save a lot of overhead in not
copying a bunch of
stuff that we're not concerned about ATM.

Kris Wong

_______________________________________________
KDevelop-devel mailing list
KDevelop-develkdevelop.org
https://barney.cs.uni-potsdam.de/mailman/listinf
o/kdevelop-devel

On Wednesday 30 May 2007 02:31, Andreas Pakulat wrote:
> On 30.05.07 00:01:20, Matt Rogers wrote:
> > On Tuesday 29 May 2007 11:54, Andreas Pakulat
wrote:
> > > Hi,
> > >
> > > I'd like to ask our thread-experts for some
ideas on how we could make
> > > accesses to the project configuration
threadsafe.
> > >
> > > Currently the project configuration object
can be obtained by
> > >
> > > virtual KSharedConfig::Ptr
projectConfiguration() const = 0;
> > >
> > > which (afaik) isn't even safe to read from in
multiple threads. We need
> > > to stick with KConfig* or KSharedKConfig* (or
::Ptr) because there's
> > > some logic involved when creating the KConfig
object from our two
> > > project config files.
> > >
> > > One way I can see is having a static helper
function that creates a
> > > KConfig object from two given files which can
be called from any
> > > thread. The downside here is that plugins get
access to either the
> > > (possibly remote) project files or temporary
copies of them and they
> > > can do changes which would be overwritten on
project closing. With a
> > > KSharedConfig this doesn't happen.
> >
> > Is it possible for us to move on to other things
instead of dwelling on
> > something that already works, although it's a bit
slow? Speeding up the
> > project parsing is a noble goal to have. However,
I think our efforts are
> > best focused on making other things work and then
go back to make other
> > things work better.
>
> This is not about project opening beeing threadded, my
request is to
> make access to the project configuration thread safe.
Which is pretty
> important as this might happen in various parts of
KDevelop (getting the
> project config in a different thread).
>
> I personally don't care too much for project opening
time, its a few
> seconds at the moment for the projects I open
(including kdevelop).
>
> Andreas

ah, ok. i misunderstood. We do need to make access to the
project 
configuration threadsafe.
-- 
Matt

_______________________________________________
KDevelop-devel mailing list
KDevelop-develkdevelop.org
https://barney.cs.uni-potsdam.de/mailman/listinf
o/kdevelop-devel