List Info

Thread: Re: Improving MCL storage. Brought to u by Ben!




Re: Improving MCL storage. Brought to u by Ben!
user name
2007-09-16 07:40:49
Hi again,

I wanted to add that we could modify a little bit the  
FileAccessService adding the mentioned FileTransaction to
it. This  
way, the moment one invokes getPersistentFile, the
FileAccessService  
will look up for remaining "..tmpXXXXXX.original"
and  
"...tmpXXXXXX.transactionbackup" files and perform
a cleanup. The  
cleanup procedure would be the following:


1) If exist FILENAME + FILENAME.tmpXXXX.transactionbackup 
-> DISCARD  
FILENAME.tmpXXXX.transactionbackup
2) If exist FILENAME.tmpXXXX.original +  
FILENAME.tmpXXXX.transactionbackup -> RENAME  
FILENAME.tmpXXXX.transactionbackup to FILENAME and DISCARD 

FILENAME.tmpXXXX.original
3) If exist FILENAME + FILENAME.tmpXXXX.original ->
DISCARD  
FILENAME.tmpXXXX.original
4) In all other cases keep FILENAME.

WDYT?

Alex


Le 16 sept. 07 à 14:30, Alexander Pelov a écrit :

> Hi everyone,
>
> the question in hand is very interesting - what to do
when an error  
> occurs while replacing an existing (and potentially
important)  
> file. Maybe we can create a sort of a generic API that
can emulate  
> transactional behavior on top of a general FS. I am
thinking about  
> something like this (if we are following Java
Transaction API  
> http://java.sun.com
/products/jta/):
>
> File f = new File("/test.xml");
> Transaction trans;
> try {
> 	trans = FileTransaction.beginTransaction(f);
> 	// some actions with f
> 	trans.commit();
> }
> catch(Throwable e) {
> 	trans.rollback();
> 	throw e;
> }
>
> Following JTA would require that the originating file
is copied  
> when beginTransaction is called. On commit() the backup
file should  
> be deleted, whereas on rollback() the backup file
should replace  
> the invalidated "f". Here, there are some
potential issues, but I  
> think that they can be addressed relatively easily. The
default  
> behavior would be to automatically rollback a
transaction. This  
> way, one may encapsulate all kinds of file operations
in  
> transactions and in case of unexpected exceptions,
kernel panics,  
> premature application termination, etc. data corruption
would be  
> kept to minimum.
>
>
>
> A simpler approach would be to write all changes to a
secondary  
> file, and once done - replace the original. An example
would be like:
>
> File f = new File("/test.xml");
> Transaction trans;
> try {
> 	trans = FileTransaction.beginTransaction(f);
> 	File newFile = trans.getProtectedFile();
> 	// write to newFile and potentially read from f
> 	trans.commit();
> }
> catch(Throwable e) {
> 	trans.rollback();
> 	throw e;
> }
>
> An example behavior of the different functions could
be:
>
> getProtectedFile():
> create a file
"/test.xml.tmp03942940192.transactionbackup"
>
> commit():
> 1) Rename "/test.xml" to
"/test.xml.tmp01293093.original"
> 2) Rename
"/test.xml.tmp03942940192.transactionbackup" to
"/text.xml"
> 3) Delete "/test.xml.tmp01293093.original"
>
> Of course, one would always have to access a file in a
transaction  
> (or at least call a function that would search for
eventual  
> "....original" and
".....transactionbackup" files and perform  
> appropriate actions.
>
> What do you think ?
>
> Alex
>
>
>
>
>
> Le 16 sept. 07 à 13:39, Benoit Pradelle a écrit :
>
>> I Emil,
>>
>> Emil Ivov a écrit :
>>> Hi Ben,
>>>
>>>> I'm deleting the backup file only to
simplify the work when SC  
>>>> restarts:
>>>> If a backup is present, we use it because
it means that the last  
>>>> write didn't finish correctly, else we do
the normal job.
>>>> If the backup file is never deleted it is
more complex to  
>>>> determine if the last write was successful:
maybe the original  
>>>> file is correct and contains the
modification, maybe the  
>>>> shutdown occurred just before the write or
worst, maybe the  
>>>> write is only partial. Then, when SC
restarts, it become very  
>>>> difficult to determine what file to use.
>>>>
>>>
>>> I am not sure I get this. The way that things
are currently  
>>> implemented
>>> we have:
>>>
>>> 1) Create original; 2) Create copy; 3) Delete
copy
>>>
>>> After a normal execution of the sequence we
would end-up with  
>>> only the
>>> original and no copy because it would be
deleted.
>>>
>>> Then let's say that the program exits
unexpectedly and the  
>>> process gets
>>> interrupted in step 1. Step 2 would never
happen since the  
>>> program died
>>> and the backup file from a previous execution
of the sequence  
>>> would have
>>> been deleted. So next time we start SIP
Communicator we'd still be
>>> unable to reload the contact list since there
won't be any backup.
>>>
>>> If you don't delete the copy however everything
falls into place.  
>>> If the
>>> process is cut in Step 1, you would have the
backup from the  
>>> previous run.
>>>
>>> If a shutdown happens normally in this case you
would indeed have an
>>> unnecessary backup but this won't be a problem.
I don't think  
>>> this makes
>>> it difficult to determine which file to use.
You always load the  
>>> primary
>>> file and only use the backup if loading the
first one has failed.
>>>
>>> Does this make any sense?
>>>
>>
>> In fact the algorithm is a little bit different
than what you  
>> described:
>> 1) copy the original contact list
>> 2) modify the contact list
>> 3) delete the backup file
>>
>> this way, if a backup file is present it contains
an older but  
>> valid contactlist.
>>
>> However, while I'm writting these lines I'm
thinking at a possible  
>> error case: if the stop occurs during the copy of
the original  
>> contact list, the backup will be loaded at the next
SC start even  
>> if it isn't correct. So it seems that I still got
some work on it 
>>
>>>
>>>> I also would like to complete your
explications by saying that  
>>>> this little patch is only a first step
because this problem also  
>>>> occur sometimes with the
sip-comunicator.xml file and perhaps  
>>>> others so it'll probably be a new plugin or
service able to  
>>>> handle secured write to important files.
>>>>
>>>
>>> Cool! Glad to know that this is also in your
plans! After we have  
>>> tested
>>> this in the meta contact list we could actually
try to bring it  
>>> out in a
>>> service. WDYT?
>>>
>>
>> It's exactly what I was thinking 
>>
>> Cheers,
>> Ben
>>
>>
------------------------------------------------------------
---------
>> To unsubscribe, e-mail: dev-unsubscribesip-communicator.dev.java.net
>> For additional commands, e-mail: dev-helpsip- 
>> communicator.dev.java.net
>>
>
>
------------------------------------------------------------
---------
> To unsubscribe, e-mail: dev-unsubscribesip-communicator.dev.java.net
> For additional commands, e-mail: dev-helpsip- 
> communicator.dev.java.net
>

------------------------------------------------------------
---------
To unsubscribe, e-mail: dev-unsubscribesip-communicator.dev.java.net
For additional commands, e-mail: dev-helpsip-communicator.dev.java.net


Re: Improving MCL storage. Brought to u by Ben!
user name
2007-09-16 13:24:22
Hi alexander, all,

Alexander Pelov a écrit :
> Hi again,
>
> I wanted to add that we could modify a little bit the 
> FileAccessService adding the mentioned FileTransaction
to it. This 
> way, the moment one invokes getPersistentFile, the
FileAccessService 
> will look up for remaining
"..tmpXXXXXX.original" and 
> "...tmpXXXXXX.transactionbackup" files and
perform a cleanup. The 
> cleanup procedure would be the following:
>
>
> 1) If exist FILENAME +
FILENAME.tmpXXXX.transactionbackup  -> DISCARD 
> FILENAME.tmpXXXX.transactionbackup
> 2) If exist FILENAME.tmpXXXX.original + 
> FILENAME.tmpXXXX.transactionbackup -> RENAME 
> FILENAME.tmpXXXX.transactionbackup to FILENAME and
DISCARD 
> FILENAME.tmpXXXX.original
> 3) If exist FILENAME + FILENAME.tmpXXXX.original ->
DISCARD 
> FILENAME.tmpXXXX.original
> 4) In all other cases keep FILENAME.
>
> WDYT?
>
> Alex
>
>
> Le 16 sept. 07 à 14:30, Alexander Pelov a écrit :
>
>> Hi everyone,
>>
>> the question in hand is very interesting - what to
do when an error 
>> occurs while replacing an existing (and potentially
important) file. 
>> Maybe we can create a sort of a generic API that
can emulate 
>> transactional behavior on top of a general FS. I am
thinking about 
>> something like this (if we are following Java
Transaction API 
>> http://java.sun.com
/products/jta/):
>>
>> File f = new File("/test.xml");
>> Transaction trans;
>> try {
>>     trans = FileTransaction.beginTransaction(f);
>>     // some actions with f
>>     trans.commit();
>> }
>> catch(Throwable e) {
>>     trans.rollback();
>>     throw e;
>> }
>>
>> Following JTA would require that the originating
file is copied when 
>> beginTransaction is called. On commit() the backup
file should be 
>> deleted, whereas on rollback() the backup file
should replace the 
>> invalidated "f". Here, there are some
potential issues, but I think 
>> that they can be addressed relatively easily. The
default behavior 
>> would be to automatically rollback a transaction.
This way, one may 
>> encapsulate all kinds of file operations in
transactions and in case 
>> of unexpected exceptions, kernel panics, premature
application 
>> termination, etc. data corruption would be kept to
minimum.
>>
>>
>>
>> A simpler approach would be to write all changes to
a secondary file, 
>> and once done - replace the original. An example
would be like:
>>
>> File f = new File("/test.xml");
>> Transaction trans;
>> try {
>>     trans = FileTransaction.beginTransaction(f);
>>     File newFile = trans.getProtectedFile();
>>     // write to newFile and potentially read from
f
>>     trans.commit();
>> }
>> catch(Throwable e) {
>>     trans.rollback();
>>     throw e;
>> }
>>
>> An example behavior of the different functions
could be:
>>
>> getProtectedFile():
>> create a file
"/test.xml.tmp03942940192.transactionbackup"
>>
>> commit():
>> 1) Rename "/test.xml" to
"/test.xml.tmp01293093.original"
>> 2) Rename
"/test.xml.tmp03942940192.transactionbackup" to
"/text.xml"
>> 3) Delete
"/test.xml.tmp01293093.original"
>>
>> Of course, one would always have to access a file
in a transaction 
>> (or at least call a function that would search for
eventual 
>> "....original" and
".....transactionbackup" files and perform 
>> appropriate actions.
>>
>> What do you think ?
>>
>> Alex
>>
>>
>>

Basically it's what I was thinking at but  instead of using
a 
transaction model I was  thinking at a SecuredFile which
extends File 
and by overloading basic I/O operations of the File class we
could use 
these SecuredFiles really more easily and without any large

modifications. However, it's just an idea for the moment, I
don't really 
know if it is really possible.

The main drawback of this solution is that sometimes a lot
of little 
write are done in an operation and it'll be difficult to
determine when 
all these little write end. I was thinking on something like
a buffer 
for all the operations and a writing thread which
periodically commit 
the modifications. It's probably less efficient but this may
avoid large 
code modifications.

I don't know what we should prefer ? I don't really know how
many class 
will use these secured files and what they'll do with it...

Cheers,
Ben

------------------------------------------------------------
---------
To unsubscribe, e-mail: dev-unsubscribesip-communicator.dev.java.net
For additional commands, e-mail: dev-helpsip-communicator.dev.java.net


[1-2]

about | contact  Other archives ( Real Estate discussion Medical topics )