ResourceStore and its saving/flush strategies for writing out configuration to disk

Hello all,

i have a question regarding the ResourceStore on how changes
to the Jigsaw configuration are saved/flushed to disk.
Assuming the following Scenario:

The Jigsaw server is running and the computer crashes, so
that Jigsaw has no time to shutdown anymore.
I'm currently examine this Scenario, because this happens a
few days ago, not to me, but i'm used to examine this case 
and shall find out, whether it is possible that Jigsaw has
corrupted the files, because it wasn't able to shutdown 
properly, while the computer crashed, or if it's not
possible that Jigsaw corrupts the configuration files and
isn't 
able to recover from this at the next startup.

After restarting the computer, Jigsaw wasn't able to startup
again aborting with this message:
(I replaced the host, path specific entries at the beginning
with < >, the rest of the message is untouched )

#loading properties from: <Path to
Jigsaw>/Jigsaw/Jigsaw/config/server.props
*** Warning : JigAdmin[2.2.5]: no logger specified, not
logging.
JigAdmin[2.2.5]: serving at http://<host>:9444/
FATAL ERROR in element null
Sys  : (internal)
Line : 1
Col  : 0
org.xml.sax.SAXParseException: no document element
         at com.jclark.xml.sax.Driver.parse(Driver.java)
         at
org.w3c.tools.resources.serialization.xml.XMLReader.parse(XM
LReader.java:226)
         at
org.w3c.tools.resources.serialization.xml.XMLReader.readAttr
ibuteHolders(XMLReader.java:259)
         at
org.w3c.tools.resources.serialization.xml.XMLSerializer.read
AttributeHolders(XMLSerializer.java:157)
         at
org.w3c.tools.resources.store.ResourceStoreManager.<init&
gt;(ResourceStoreManager.java:1442)
         at
org.w3c.jigsaw.http.httpd.initializeResourceSpace(httpd.java
:752)
         at
org.w3c.jigsaw.http.httpd.start(httpd.java:2072)
         at
org.w3c.jigsaw.daemon.ServerHandlerManager.<init>(Serv
erHandlerManager.java:280)
         at
org.w3c.jigsaw.daemon.ServerHandlerManager.main(ServerHandle
rManager.java:500)
         at org.w3c.jigsaw.Main.main(Main.java:15)
*** salvaging resource manager state...
Unknown error while starting https-server: For input string:
"1.tmp"
Exception in thread "main"
java.lang.NullPointerException
         at
org.w3c.jigsaw.http.httpd.checkpoint(httpd.java:828)
         at
org.w3c.jigsaw.http.httpd.shutdown(httpd.java:1516)
         at
org.w3c.jigsaw.daemon.ServerHandlerManager.<init>(Serv
erHandlerManager.java:287)
         at
org.w3c.jigsaw.daemon.ServerHandlerManager.main(ServerHandle
rManager.java:500)
         at org.w3c.jigsaw.Main.main(Main.java:15)


In order to understand why this happend, it would be great,
if you could say, which of the following "Jigsaws
saving 
Policies" are correct or if it is handled different.

These strategies comes into my mind:
  - Does Jigsaw loads the configuration, once at the
startup, and further doesn't touch the configuration files
for
    writing while running, so it only writes the
configuration back from memory to disk at shutdown?

  - Or does Jigsaw additionally writes out configuration
changes to disk periodically.

  - Does Jigsaw flush changes, made in the AdminGUI,
immediately to disk after commiting and saving them or are
the
    changes just altered in memory and get written out at
shutdown or some periodically save procedure?


Thanks in advance for any help,
	Christian

On Thu, 28 Jun 2007, Christian Weber wrote:

>
> i have a question regarding the ResourceStore on how
changes to the Jigsaw 
> configuration are saved/flushed to disk.
> Assuming the following Scenario:
>
> The Jigsaw server is running and the computer crashes,
so that Jigsaw has no 
> time to shutdown anymore.
> I'm currently examine this Scenario, because this
happens a few days ago, not 
> to me, but i'm used to examine this case and shall find
out, whether it is 
> possible that Jigsaw has corrupted the files, because
it wasn't able to 
> shutdown properly, while the computer crashed, or if
it's not possible that 
> Jigsaw corrupts the configuration files and isn't able
to recover from this 
> at the next startup.


> In order to understand why this happend, it would be
great, if you could say, 
> which of the following "Jigsaws saving
Policies" are correct or if it is 
> handled different.
>
> These strategies comes into my mind:
> - Does Jigsaw loads the configuration, once at the
startup, and further 
> doesn't touch the configuration files for
>   writing while running, so it only writes the
configuration back from 
> memory to disk at shutdown?
>
> - Or does Jigsaw additionally writes out configuration
changes to disk 
> periodically.
>
> - Does Jigsaw flush changes, made in the AdminGUI,
immediately to disk after 
> commiting and saving them or are the
>   changes just altered in memory and get written out at
shutdown or some 
> periodically save procedure?


Jigsaw's configuration is cached partially (or completely)
read from the 
disk to the memory. You can tune the number of stores
(roughly 
corresponding to the number of directories in store a time
t), to make it 
larger. (org.w3c.jigsaw.loadedstore=..)

When the internal store cache is full, and a new one is read
and the LRU 
one is flushed. Only if a change happened in the
configuration, the 
flushed store will effectively be serialized to the disk.
When something is serialized, it is saved first in a temp
file, then the 
temp file replace the old one.
The same happens ehwn the server is shut down (using the
shutdownhook).

When something is edited using the GUI, you need to click on
'Save' to 
flush the changes to the disk, otherwise it will be done
only during 
shutdown, or if the impacted store is flushed.

You can tune also the limit where the stores are no longer
flushed 
automatically (as they would be too big to save/load and
impact the server 
performances... Typically over 1000 entries per store).
Hope this answers your questions.
Cheers,

-- 
Baroula que barouleras, au tiéu toujou t'entourneras.

         ~~Yves