List Info

Thread: Proposal for transactional concurrency in Eiffel.




Proposal for transactional concurrency in Eiffel.
country flaguser name
United States
2007-04-05 15:18:26

This is something I've been looking at for the last day or so. I was
really looking for a concurrency solution that solves issues of
concurrency from a language standpoint and it seems like SCOOP is
currently at a standstill research-wise.

I'm not an Eiffel compiler writer so I don't know exactly how every
piece can be implemented but it seems doable. The proposal relies on
transactional memory, probably implemented as software transactional
memory right now.

If this proposal is feasible and desirable, it would need to be
formalized by someone (not me) in to some language rules.

I'm crossing my fingers for a language solution to concurrency!

--------------------------------------------------
A proposal for transaction based concurrency in Eiffel by Colin LeMahieu,

Rationale:
The un-computability of determining if a program based on locks will
deadlock in general necessitates using alternative forms of concurrency.

Abstract:
The Eiffel language already supports the basic idea of transactions in
features. A feature either completes in its entirety or an exception
is raised. The only shortfall of the current implementation is that
any side effects made before the feature completes are not aborted in
the transactional sense. If we adapt the semantics of a feature to
include the idea of a transactional feature, we will be able to
implement concurrency through transactions in the Eiffel language cleanly.

Behavior:
-A feature can be a transaction
-All feature calls are asynchronous
-Transactional features guarantee atomicity and isolation across the
execution of the feature
-When a transactional feature calls another transactional feature, the
transactions are composed.
-The compiler and runtime determine the scheduling either through
compiler analysis, runtime statistics, or runtime transactions.
-When a transactional feature is aborted it is automatically retried
until it is not aborted.
-External features without abort and commit semantics are executed
when all transactions have been committed or aborted and executed
serially with respect to the system, this is ensured by the runtime.

Syntax:
Use the 'transaction' keyword to define a feature as a transaction

feature
obligatory_transfer_example(account, amount) is
transaction
withdraw(account, amount)
deposit(account, amount)
end

withdraw(account, amount: INTEGER) is
transaction
account.balance.withdraw(amount)
end

deposit(account, amount: INTEGER) is
transaction
account.balance.deposit(amount)
end

Use the 'external', 'abort', and 'commit' keywords to define a
transactional external feature. 'External' defines the external
transaction functionality. 'Abort' defines functionality to roll back
any state changed in 'external'. 'commit' defines functionality to
commit the transaction of 'external'.

feature external_feat is
external
""
abort
""
commit
""
end

Design by contract:
-Preconditions and postconditions retain ECMA semantics.
-Features with preconditions and postconditions can only be called
from a feature that is a transaction as a validation constraint.
-If a transactional feature 'f' wants to call feature 'g' with a
precondition 'a', 'f' can check 'a' before calling 'g' and is assured
that 'a' will be the same once 'g' is being executed by the
transactional nature of 'f'; 'a' will either remain valid or 'f' will
be aborted and retried.
-Postcondition wait semantics as introduced with SCOOP do not need to
be used as the compiler or runtime will determine the execution
schedule of a program.
-Class invariants determine the consistency of transactions and are
checked every time a transaction is committed and after it is aborted

Deferred features:
Deferred features can be redefined to be either a transaction or not.
If the feature must be transactional, the designer should encapsulate
the call to the deferred feature within a transaction feature.

Agents:
Agent semantics will not change.

Exceptions:
The feature .last_exception is obsolete
The ANY class contains a new feature last_exceptions and is a
collection of exception objects that have been thrown within the
context of the current feature.
All features within a feature must be executed and either succeed or
generate an exception before the enclosing feature can move to the
retry clause propagate an exception.
If a feature doesn't handle all exceptions, its transaction is
failed(aborted and not retried) with an exception of type ROUTINE_FAILURE
When a lower priority feature has a transactional dependency on a
higher priority feature that fails, it will fail with an exception of
type TRANSACTION_FAILURE which is a subtype of ROUTINE_FAILURE

Priority:
If two calls, 'g' and 'h' within a feature 'f' are in conflict,
priority is assigned by order of program text and the feature
appearing later in the program text will be aborted and retried.
If a looping construct is in conflict with earlier iterations of its
loop, priority is assigned to earlier iterations of the loop and later
iterations will be aborted.
If a recursive call is in conflict with itself, priority is assigned
to the top level nesting and the deeper nesting calls will be aborted.

Thread library:
The thread library is obsolete. All classes should be marked obsolete.

Threads:
Explicitly creating threads is redundant. Legacy external features
that create threads must block until the threads are complete. Code
inheriting from THREAD in the EiffelBase threading library and
effecting execute will continue to work simply by using new feature
call semantics; the library will need to be changed to allow the new
runtime to schedule the feature's execution.

Mutexes:
Code using mutexes will continue to work, however they should be
replaced with transactional code as soon as possible to avoid deadlocks.

Legacy:
This language change is fully backward compatible with serial code,
however, performance benefits would only be seen when using code that
does not reference external features that don't have transaction
semantics. Building transactions over code that references legacy
externals can cause runtime routine failures; the overlying solution
would be to eliminate legacy external features.

External clauses without abort and commit clauses are called 'legacy
externals' and are marked as obsolete by compilers supporting legacy
externals.

Systems that allow legacy external features, without abort and commit
clauses, cannot be executed within transactions. The compiler could
detect many of these through system analysis however the runtime will
need to make explicit checks against legacy external calls to make
sure they do not exist within the context of a transaction for things
such as agents or inheritance substitution where a feature is
redefined to be implemented by a legacy external.

Legacy external features are executed by stalling transaction
execution, making sure all transactions have committed or aborted,
making sure nothing else is executing in the system, and running the
legacy external.

We're essentially forcing legacy external features to be transactions
by making sure no other transactions are currently in progress and
there is nothing executing in the system.

Notes:
Things that are seemingly serial, such as writing text to the screen
in a certain order, are actually dependency relationships;
specifically appending to the screen. If there are 2 calls writing to
the screen, the second call depends on the result of the first call
and the compiler or runtime will either optimize this and have the
second call wait, or the second call will abort at runtime until the
first call completes. These two could actually run in parallel in the
case where the first call writes nothing to the screen.

Unlike database transactions, durability is not guaranteed with
feature transactions, memory is inherently volatile in the face of
system errors.

It's impossible for a transactional feature to reliably act on a
legacy external feature. During the execution of a transactional
feature, any of the actions may be aborted and retried and since
legacy external features cannot be aborted, there is no reliable way
to undo that action.

The asynchronous semantics of feature calls gives the interesting
effect of the programmer not being aware of the order in which feature
calls are executed. This allows the compiler or runtime to determine
the serialization schedule of the program and allows for a level of
parallelism not easily thought of by a human.

Previous work focused on an idea called 'compensation' which has no
real foundation in transactions. Things that are not transactional as
currently implemented such as writing files, sending network data,
etc, will need classes encapsulating a transaction around that write
or send. This may be a type of journaling at the file access or
network layer.

A feature that encounters exceptions tries to execute all code paths
in order to gain a consistent view of the exceptions caused. If this
wasn't done, the exception reported as causing the routine to fail
might vary depending on the execution plan.

It might be better to use a keyword to define a feature as
non-transactional instead of using a keyword to define a feature as
transactional.

__._,_.___
.

__,_._,___
RE: Proposal for transactional concurrency in Eiffel.
country flaguser name
United States
2007-04-06 05:07:51

Hi Colin,

Do you mind writing this on our wiki site http://eiffelsoftware.origo.ethz.ch
under the ECMA category? Also I could not see if it would work for queries?

Thanks,
Manu

> -----Original Message-----
> From: eiffel_software%40yahoogroups.com">eiffel_softwareyahoogroups.com
> [mailto: eiffel_software%40yahoogroups.com">eiffel_softwareyahoogroups.com] On Behalf Of colinlema
> Sent: Thursday, April 05, 2007 1:18 PM
> To: eiffel_software%40yahoogroups.com">eiffel_softwareyahoogroups.com
> Subject: [eiffel_software] Proposal for transactional
> concurrency in Eiffel.
>
> This is something I've been looking at for the last day or so. I was
> really looking for a concurrency solution that solves issues of
> concurrency from a language standpoint and it seems like SCOOP is
> currently at a standstill research-wise.
>
> I'm not an Eiffel compiler writer so I don't know exactly how every
> piece can be implemented but it seems doable. The proposal relies on
> transactional memory, probably implemented as software transactional
> memory right now.
>;
> If this proposal is feasible and desirable, it would need to be
> formalized by someone (not me) in to some language rules.
>
> I'm crossing my fingers for a language solution to concurrency!
...<;Skipped>;

__._,_.___
.

__,_._,___
Re: Proposal for transactional concurrency in Eiffel.
country flaguser name
United States
2007-04-07 16:27:11

> I was
> really looking for a concurrency solution that solves issues of
> concurrency from a language standpoint and it seems like SCOOP is
> currently at a standstill research-wise.
>

4 years ago Eiflex (www.eiflex.com) needed to provide distributed
transactional Eiffel objects (multithreaded, multiprocess and multi
box) for a project at the Chicago Board of Trade, and since there was
no language solution, Eiflex developed its own libary which hides the
distributed transactional Eiffel objects from the complexities of the
primitive THREAD libary and protects data in a similar fashion to SCOOP.

> The proposal relies on
> transactional memory, probably implemented as software transactional
> memory right now.

Eiflex has a class in its libary - PERSISTENT_ITEM - which acts as a
holder for all atomic transactional data. Eiflex processes have
transaction logs, and checkpoints from which to roll forward in
the event of failure.

> --------------------------------------------------
> A proposal for transaction based concurrency in Eiffel by Colin
LeMahieu,......
>

SCOOP is a brilliantly simple concept, but has taken a long time to
get to where it is now. I fully support
your desire to add transactional behaviour to it (or something
similar). Perhaps such a proposal might spark the enthusiasm one of
Bertrand's research students to explore the possibilities.

Regards Gordon Jones

__._,_.___
.

__,_._,___
Re: Proposal for transactional concurrency in Eiffel.
country flaguser name
Switzerland
2007-04-08 11:19:01

colinlema wrote:
&gt; it seems like SCOOP is
> currently at a standstill research-wise.

Where did you get this impression? The project is in full swing; Piotr
Nienaltowski's thesis, defended in February, made considerable
improvements to SCOOP; it will be on the Web soon. There's a whole set
of recent publications by him and other people. The implementation
currently works through a preprocessor but will be integrated into the
compiler.

-- BM

__._,_.___
.

__,_._,___
Re: Proposal for transactional concurrency in Eiffel.
country flaguser name
United States
2007-04-10 11:14:47

http://eiffelsoftware.origo.ethz.ch/index.php/Transactions

I guess this isn't generating as much discussion as I was hoping. I'm
not sure if this is because lack of clarity on my part or a flaw in
the idea.

Any feedback, positive or negative, would be appreciated.

--- In eiffel_software%40yahoogroups.com">eiffel_softwareyahoogroups.com, "colinlema" <clemahieu...> wrote:
&gt;
> This is something I've been looking at for the last day or so. I was
> really looking for a concurrency solution that solves issues of
> concurrency from a language standpoint and it seems like SCOOP is
> currently at a standstill research-wise.
>
> I'm not an Eiffel compiler writer so I don't know exactly how every
&gt; piece can be implemented but it seems doable. The proposal relies on
> transactional memory, probably implemented as software transactional
> memory right now.
>;
> If this proposal is feasible and desirable, it would need to be
> formalized by someone (not me) in to some language rules.
&gt;
> I'm crossing my fingers for a language solution to concurrency!

__._,_.___
.

__,_._,___
Re: Re: Proposal for transactional concurrency in Eiffel.
country flaguser name
United States
2007-04-10 23:14:41

Hey Colin, here is some feedback from me, a novice Eiffel guy. I read
this email and your article on Origo with much interest. In light of
all the new multi-core processors, distributed computing, etc.
anything to promote OO multithreaded programming would be a positive
thing. But here are some questions and suggestions:

1- On the eiffelsoftware.origo.ethz.ch site your new article is the
only search hit for SCOOP so perhaps you could link somewhere to
define SCOOP - not for me but for newbies. perhaps you could also add
the response from Bertand about the recent positive news for SCOOP?

2- Concurrency vs Transactions - When reading your article i didn't
understand how it follows that the Thread library would be obsoleted
by using transactions. Does the notion of transactions somehow
encompass the notion of concurrency? Or are you suggesting to by
default add transactions to concurrency in eiffel? i guess i just
don't understand the context of your article. From the response from
Gordon i gather that real-world transactional systems might also
require distributed/concurrent environments, and that example makes
sense.

3- I found this doc
<http://archive.eiffel.com/doc/manuals/technology/concurrency/short/page.html>
which explains the notion concurrency from an Eiffel perspective -
succinctly i think- although doesn't explicitly mention Transactions
at all. It also ends with a promo for SCOOP. it also might give some
possible answers (plural) to your analogy
manual memory management:garbage collection :: locks : ?

Alex

On 4/10/07, colinlema < clemahieu%40gmail.com">clemahieugmail.com> wrote:
&gt; http://eiffelsoftware.origo.ethz.ch/index.php/Transactions
>
> I guess this isn't generating as much discussion as I was hoping. I'm
> not sure if this is because lack of clarity on my part or a flaw in
> the idea.
&gt;
> Any feedback, positive or negative, would be appreciated.
>

__._,_.___
.

__,_._,___
Re: Proposal for transactional concurrency in Eiffel.
country flaguser name
Portugal
2007-04-11 09:14:47

Dear Colin,

You might be interested in my CORDIE's article and presentation:
http://www.ieeta.pt/~mos/pubs/intra-object-concurrency-2006.pdf
http://www.ieeta.pt/~mos/pubs/cordie06-mos-presentation.pdf

and in this draft article:

http://www.ieeta.pt/~mos/pubs/inter-object-reservation-2006-draft.pdf

(I advise you also to read the other CORDIE's articles and also Tim Harris
and Keir Fraser's proposal for a lightweight transactions atomic instruction
for Java:

inproceedings{949340,
author = {Tim Harris and Keir Fraser},
title = {Language support for lightweight transactions},
booktitle = {OOPSLA '03: Proceedings of the 18th annual ACM SIGPLAN
conference on Object-oriented programming, systems,
languages, and applications},
year = ,
isbn = {1-58113-712-5},
pages = ,
location = {Anaheim, California, USA},
doi = {http://doi.acm.org/10.1145/949305.949340},
publisher = {ACM Press},
})

--

I have several critiques to your proposal.

1. First, if I understood correctly your approach, you are
attempting to move the concurrency language mechanisms
away from objects into transactional routines. I think this
is an incorrect approach. In a shared memory inter-processor
approach (as transactions) concurrency problems arise from
concurrent objects (ones that might be used by more than one
processor), which makes all of their features (not just the
ones internally or externally annotated as a "transaction&quot;)
part of the problem (and, of course, a necessary part of
possible solutions).

Once an object is used within a transaction, that object
becomes a concurrent object, so all its possible uses
are required to be protected from concurrent problems
(meaning that, if the object is used concurrently
outside a declared transaction feature you might
have safety problems).

So, my first critique is that, in my opinion, you are
annotating the wrong program entities: routines instead
of typed program entities (which store object references).

2. Secondly, it is not clear to me how you can avoid
the wait condition attached to concurrent assertions
(assertions involving concurrent objects).
The wait behavior attached to those assertions is
not negotiable. It is the only behavior that is 100%
meaningful and safe.

An assertion expresses a correctness condition
to be *always* verified in that point of the program.
If the corresponding boolean expression uses a
concurrent object, it might not be the sole
responsibility of the testing processor program to
ensure it holds a true value.
The consequence of this semantics is that, if concurrent
assertions are not wait conditions, its value might
be true or false (depending on the uncontrollable behavior
of other processors), making those assertions useless
and not meaningful (race conditions).

3. My third critique is that I think you are over-specifying
the semantics of concurrent language mechanisms.
You take the operational approach of attaching an
specific implementation behavior to the language mechanism
(Java threads also share this bad property).
A much (much) better approach is an axiomatic one:
the concurrent mechanism simply expresses the desired
behavior (for instance: atomicy), leaving to the
compiler the detail of its realization.
The approach I have been studying to this problem
(for quite some years) uses abstract synchronization.
The (intra-)synchronization scheme of a concurrent
object could be a simple mutual exclusion (monitor)
mechanism, a readers-writer lock, a concurrent readers-writer,
a lock-free scheme (not tested), mixed schemes,
or a software transactional one (also not tested), depending
only on the automatic realizability of the chosen scheme
by the compiling system.
(It is very interesting to realize that most of the
so called inheritance anomalies attached to
concurrent mechanisms are the result of operational
approaches to synchronization).

If one chooses to use transactions as a synchronization
scheme, we should not forget that its automatic
realization by the compiling system requires
that it can only be safely applied to repeatable
features (those in which one can rollback to
a stable state without any visible external
side-effects). Input/output features (as those
you use in the examples) are not repeatable,
so they should not be allowed inside transactions.
In the database world, transactions work perfectly
because databases handle explicit data, so there
is no routine repeatable problem. However,
object-oriented programs deal with implicit data
(thought Abstract Data Type partial implementations),
so the problem is not as simple (although I agree
that it is feasible in many situations).
Another (not as important) problem is that transactions
are much heavier than simple exclusion or readers-writer
locks (why should we impose the use of a power drill
to open a hole in the sand?).
(Processors trying long transaction routines may also
be delayed, up to its "starvation";, due to smaller
very frequent transactions done by other processor on
shared concurrent objects).

4. It is not clear to me how processors are created
in your proposal. You say that all feature calls
are asynchronous. Does that mean that processors
are created on feature call (any feature call)?
Or only when a transaction feature is called?
If it's the latter case, how do you solve the problem
of ensuring the feature's contract to the caller
(postcondition immediately after feature invocation)?

5. Finally, your proposal has also another over-specification
problem. A class, built strictly for sequential programs
(for example: an ARRAY[T]), cannot be directly used, in a
safe way, to instantiate concurrent objects.
This brings reusability and extensibility problems
to concurrent programs (SCOOP has the same problem,
because of its approach for inter-object synchronization
through formal separate routine arguments).

Best regards,

-miguel

--
Miguel Oliveira e Silva
mos at det.ua.pt - http://www.ieeta.pt/~mos
DETI-IEETA, Universidade de Aveiro, PORTUGAL

__._,_.___
.

__,_._,___
Re: Proposal for transactional concurrency in Eiffel.
country flaguser name
United States
2007-04-11 12:48:29

Hi all,

--- In eiffel_software%40yahoogroups.com">eiffel_softwareyahoogroups.com, "colinlema" <clemahieu...> wrote:
&gt;
> This is something I've been looking at for the last day or so. I was
> really looking for a concurrency solution that solves issues of
> concurrency from a language standpoint and it seems like SCOOP is
> currently at a standstill research-wise.

I am aware of the existing work on STM and am a verification researcher.

While I find STM interesting, I am not convinced that it makes a
system any easier to program, use, or reason about. Nor have I seen
any real systems developed to support STM-style programming (the
single-threaded Haskell work does not count in my book), nor any even
moderately-sized case studies in the use of such.

That being said, introducing the notion in the Eiffel context would
indeed be interesting, as we get to combine quality specifications and
reasoning with a language with a "smallish" semantics. I.e., if
someone were to add support for this to EiffelStudio I would certainly
play with it and read the papers.

Your proposal sounded rational on first-blush, but I suspect there is
an enormous amount of subtlety that will not be obvious until you push
down one more level. We have certainly found this to be the case when
reasoning about concurrent Java programs because of semantics imposed
less by the language, but more by modern processor architectures, from
which Eiffel cannot escape.

Joe
---
Joseph Kiniry
School of Computer Science and Informatics
UCD Dublin
http://secure.ucd.ie/
http://srg.cs.ucd.ie/

__._,_.___
.

__,_._,___
[1-8]

about | contact  Other archives ( Real Estate discussion Medical topics )