Archtypes SQL Storage

Hello List,

I looked at the SQLStorage package that is part of
Archtypes, but found out that Archetypes objects stored
through SQLStorage still have a presence in the ZODB, which
means it's not a solution if you totally want to get rid of
Data.fs bloat. (It is also the policy and requirement in our
company to have everything in outsourced in the MySQLDB and
I can't change this)

I am looking for a way of using the Archetype Content Object
only as the View (presentation layer) for easy rendering of
the forms and mapping them directly to a relational database
without any interference of the ZODB.

Is it possible to modify an Archetype that it looks up the
content items completly in the relational database and not
first in the zope DB (that maps to the rel. database like in
SQLStorage)?
I don't see a reason why one should store the same object 2
times in the ZODB and MySQL.

Any help and hint will be appreciated, Thank you very much!

Regards, Dennis
-- 
GMX DSL-Flatrate 0,- Euro* - Überall, wo DSL verfügbar ist!
NEU: Jetzt bis zu 16.000 kBit/s! http://www.gmx.net/de/go
/dsl

------------------------------------------------------------
-------------
Take Surveys. Earn Cash. Influence the Future of IT
Join SourceForge.net's Techsay panel and you'll get the
chance to share your
opinions on IT & business topics through brief surveys
-- and earn cash
http://www.techsay.com/default.
php?page=join.php&p=sourceforge&CID=DEVDEV
_______________________________________________
Archetypes-devel mailing list
Archetypes-devellists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/archet
ypes-devel

On Tue, Oct 10, 2006 at 05:31:49PM +0200, Dennis Schulz
wrote:
| Hello List,
| 
| I looked at the SQLStorage package that is part of
Archtypes, but found out that Archetypes objects stored
through SQLStorage still have a presence in the ZODB, which
means it's not a solution if you totally want to get rid of
Data.fs bloat. (It is also the policy and requirement in our
company to have everything in outsourced in the MySQLDB and
I can't change this)
| 
| I am looking for a way of using the Archetype Content
Object only as the View (presentation layer) for easy
rendering of the forms and mapping them directly to a
relational database without any interference of the ZODB.
| 
| Is it possible to modify an Archetype that it looks up the
content items completly in the relational database and not
first in the zope DB (that maps to the rel. database like in
SQLStorage)?
| I don't see a reason why one should store the same object
2 times in the ZODB and MySQL.
| 
| Any help and hint will be appreciated, Thank you very
much!

It should be possible yes. The problem is that no-one did it
before so
the effort to get it to work is unknown. With enough time it
is
certainly doable.

-- 
Sidnei da Silva
Enfold Systems                http://enfoldsystems.com
Fax +1 832 201 8856     Office +1 713 942 2377 Ext 214

------------------------------------------------------------
-------------
Take Surveys. Earn Cash. Influence the Future of IT
Join SourceForge.net's Techsay panel and you'll get the
chance to share your
opinions on IT & business topics through brief surveys
-- and earn cash
http://www.techsay.com/default.
php?page=join.php&p=sourceforge&CID=DEVDEV
_______________________________________________
Archetypes-devel mailing list
Archetypes-devellists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/archet
ypes-devel

> I am looking for a way of using the Archetype Content
Object only as
> the View (presentation layer) for easy rendering of the
forms and
> mapping them directly to a relational database without
any interference
> of the ZODB.
> 
> Is it possible to modify an Archetype that it looks up
the content items
> completly in the relational database and not first in
the zope DB (that
> maps to the rel. database like in SQLStorage)?
> I don't see a reason why one should store the same
object 2 times in the ZODB and MySQL.

I'm doing this using zope3/Five formlib to build the view
and SQLAlchemy 
to manage the interaction with the database (PostgreSQL in
my case). 
It's thin and fast, which is exactly what I needed.

My first attempt used Archetypes. That became excruciatingly
slow when 
we loaded all of the data (many thousands of records).
Archetypes is 
great for a broad range of Plone apps, but when you're
dealing with lots 
of dynamic data, Archetypes is a bad fit.

My second attempt used a blend of Archetypes and zope3 that
I borrowed 
from plone_schemas. That was better, but it had lots of
problems. I had 
to patch around many of the features that Archetypes offers.
In the end, 
I was doing more patch-around than application building.

I finally bit the bullet and went for the thinnest layer I
could build 
on top of SQLAlchemy. (Thanks for pushing me that way,
rafrombrc!) It's 
still a work in progress, but so far it's very close to 20
times faster 
than the Archetypes version. (And I have far more of it
implemented now 
than I did in Archetypes.)

I intend to donate a stripped-down version of it back to the
community 
-- as an example app, at least. The learning curve is
outrageous, but 
the final code is stunningly simple. My problem in figuring
it out is 
the dearth of working examples. Unfortunately, I won't have
time to 
produce that example code until the end of the month. (I
have to deliver 
this app, first!)

Anyway, I'd recommend looking at zope3's formlib and views
as a 
potential solution.

--Bill / bitranch


------------------------------------------------------------
-------------
Take Surveys. Earn Cash. Influence the Future of IT
Join SourceForge.net's Techsay panel and you'll get the
chance to share your
opinions on IT & business topics through brief surveys
-- and earn cash
http://www.techsay.com/default.
php?page=join.php&p=sourceforge&CID=DEVDEV
_______________________________________________
Archetypes-devel mailing list
Archetypes-devellists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/archet
ypes-devel

> It should be possible yes. The problem is that no-one
did it before so
> the effort to get it to work is unknown. With enough
time it is
> certainly doable.

I also think it is doable, but I am relativly new to plone
and unfortunately don't have the time now 

But I am wondering how the other companies manage their long
time and shard data storage.
I think it is not a problem from out of space. It is quite a
common and trivial szenario for a company to have long time
stored data outsourced in a RDBMS and yet having the desire
to get everything beautifully mangaged by an (maybe content
type like) object, something similar to a EJB in J2EE, that
offers automatic form generation.

Or are all of you using the ZODB for long time and
application shared storage and dont feel the need to
outsource at all?

Well, I am quite new to Plone and I feel there is something
missing... but I don't know. Just can't believe that there
is no product out there that supports this.


-------- Original-Nachricht --------
Datum: Tue, 10 Oct 2006 13:01:33 -0300
Von: Sidnei da Silva <sidneiawkly.org>
An: Dennis Schulz <D.Schulz81gmx.net>
Betreff: Re: [Archetypes-devel] Archtypes SQL Storage

> On Tue, Oct 10, 2006 at 05:31:49PM +0200, Dennis Schulz
wrote:
> | Hello List,
> | 
> | I looked at the SQLStorage package that is part of
Archtypes, but found
> out that Archetypes objects stored through SQLStorage
still have a presence
> in the ZODB, which means it's not a solution if you
totally want to get
> rid of Data.fs bloat. (It is also the policy and
requirement in our company
> to have everything in outsourced in the MySQLDB and I
can't change this)
> | 
> | I am looking for a way of using the Archetype Content
Object only as the
> View (presentation layer) for easy rendering of the
forms and mapping them
> directly to a relational database without any
interference of the ZODB.
> | 
> | Is it possible to modify an Archetype that it looks
up the content items
> completly in the relational database and not first in
the zope DB (that
> maps to the rel. database like in SQLStorage)?
> | I don't see a reason why one should store the same
object 2 times in the
> ZODB and MySQL.
> | 
> | Any help and hint will be appreciated, Thank you very
much!
> 

> 
> -- 
> Sidnei da Silva
> Enfold Systems                http://enfoldsystems.com
> Fax +1 832 201 8856     Office +1 713 942 2377 Ext 214
> 
>
------------------------------------------------------------
-------------
> Take Surveys. Earn Cash. Influence the Future of IT
> Join SourceForge.net's Techsay panel and you'll get the
chance to share
> your
> opinions on IT & business topics through brief
surveys -- and earn cash
> http://www.techsay.com/default.
php?page=join.php&p=sourceforge&CID=DEVDEV
> _______________________________________________
> Archetypes-devel mailing list
> Archetypes-devellists.sourceforge.net
> https://lists.sourceforge.net/lists/listinfo/archet
ypes-devel

-- 
GMX DSL-Flatrate 0,- Euro* - Überall, wo DSL verfügbar ist!
NEU: Jetzt bis zu 16.000 kBit/s! http://www.gmx.net/de/go
/dsl

------------------------------------------------------------
-------------
Using Tomcat but need to do more? Need to support web
services, security?
Get stuff done quickly with pre-integrated technology to
make your job easier
Download IBM WebSphere Application Server v.1.0.1 based on
Apache Geronimo
http://sel.as-us.falkag.net/
sel?cmd=lnk&kid=120709&bid=263057&dat=121642
_______________________________________________
Archetypes-devel mailing list
Archetypes-devellists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/archet
ypes-devel

Dennis Schulz wrote:
>> It should be possible yes. The problem is that
no-one did it before
>> so the effort to get it to work is unknown. With
enough time it is 
>> certainly doable.
> 
> I also think it is doable, but I am relativly new to
plone and
> unfortunately don't have the time now 
> 
> But I am wondering how the other companies manage their
long time and
> shard data storage. I think it is not a problem from
out of space. It
> is quite a common and trivial szenario for a company to
have long
> time stored data outsourced in a RDBMS and yet having
the desire to
> get everything beautifully mangaged by an (maybe
content type like)
> object, something similar to a EJB in J2EE, that offers
automatic
> form generation.

Look out for the Relational Alchemist product. In general,
though, 
anything that will "magically" consume your
database with no 
development/configuration involved at all will likely be too
simplistic 
to be very useful.

I'd encourage you to look at Flay as well, though, which
promises some 
out-of-the-box solution.

Most people would store content in the ZODB (RDBMS' are not
very good at 
semi-structured content) and write whatever integration code
they'd need.

If you can handle SQL and HTML and a bit of learning (not
too much) 
looking at ZSQL methods + Zope Page Templates (see Zope book
on 
plope.org) would get you a long way I think.

> Or are all of you using the ZODB for long time and
application shared
> storage and dont feel the need to outsource at all?
> 
> Well, I am quite new to Plone and I feel there is
something
> missing... but I don't know. Just can't believe that
there is no
> product out there that supports this.

It's a fairly tall order to expect some product to
understand your 
organisation-specific database just-like-that, but the
solutions 
recommended above deserve a closer look. They may get you
close enough 
or even do exactly what you need.

Martin


------------------------------------------------------------
-------------
Using Tomcat but need to do more? Need to support web
services, security?
Get stuff done quickly with pre-integrated technology to
make your job easier
Download IBM WebSphere Application Server v.1.0.1 based on
Apache Geronimo
http://sel.as-us.falkag.net/
sel?cmd=lnk&kid=120709&bid=263057&dat=121642
_______________________________________________
Archetypes-devel mailing list
Archetypes-devellists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/archet
ypes-devel