Suggest we cache the SDO model somehow

OK, perhaps I am being a bit fanciful here, imagining a
plethora of
independent scripts running SDO in a hosted environment. But
there is
a potential data integrity problem if we allow a cache key
to be
overwritten in flight. I'm worried about the proposed
interface.

On Apr 3, 4:12 pm, "Matthew Peters"
<matthew.f.pet...googlemail.com>
wrote:
>
> Of course you have worried me now. If the keys are
chosen poorly - for
> example every call chooses key "my_first_key"
or suchlike - then that
> is a recipe for disaster. The key needs to correspond
to the set of
> types that are loaded into that DAS: the keys must be
chosen so that
> if the model is different, so is the key. I can't
enforce that, of
> course, all I can do is use it that way throughout the
SCA code.
>


--~--~---------~--~----~------------~-------~--~----~
You received this message because you are subscribed to the
Google Groups "phpsoa" group.
To post to this group, send email to phpsoagooglegroups.com
To unsubscribe from this group, send email to
phpsoa-unsubscribegooglegroups.com
For more options, visit this group at http://
groups.google.co.uk/group/phpsoa?hl=en
-~----------~----~----~----~------~----~------~--~---

Matthew, I've updated the BUZZARD branch to Tuscany revision
level
524004, which includes this enhancement. I haven't done
anything yet
about exploiting it, though.

On Apr 3, 4:40 pm, "Matthew Peters"
<matthew.f.pet...googlemail.com>
wrote:
> Nice to know. If and when that Tuscany change gets into
our codebase I
> can try that. It looks marginally safer than what we I
am proposing to
> do right now, which is to create the data factory once
then use it
> indefinitely.


--~--~---------~--~----~------------~-------~--~----~
You received this message because you are subscribed to the
Google Groups "phpsoa" group.
To post to this group, send email to phpsoagooglegroups.com
To unsubscribe from this group, send email to
phpsoa-unsubscribegooglegroups.com
For more options, visit this group at http://
groups.google.co.uk/group/phpsoa?hl=en
-~----------~----~----~----~------~----~------~--~---

Sorry all that I am really late in this conversation (only a
few weeks
behind). Been meaning to respond but always get distracted
by
something (think I got ADD or something) :/.
I think I had mentioned this before but I really think there
needs to
be 2 (at least) modes to this.
In my case, our company controls the servers, so I would
want to be
able to set my own key to be used for caching. If other
apps/departs
want to use my schemas I would share the key with them and
everyone
would use the same.

This really wont work in the shared hosting environment
though. I bet
in almost all those cases the hosting provider would not
want caching
enabled (otherwise not install the extension). They dont
want one
customer hogging memory here. In this case the per request
caching
might be applicable (if you are even considering this type
of
caching).

So in short, it would be nice for at the minimum of system
caching
(shared between processes) and an option to disable
completely.
Nice to haves would be option for per request or file based
caching
like ext.soap.

I know I just asked for everything here but at the minimum a
way to
disable caching on a server level (ini option) is definitely
needed.
Hope I am not too far behind and mentioning things already
implemented 

Rob


On 3 Apr, 11:12, "Matthew Peters"
<matthew.f.pet...googlemail.com>
wrote:
> Yes, exactly right. The scope is intended to be module
init to
> shutdown, and across all callers. I don't really know
how I could do
> otherwise - a per-request cache would be throwing away
all the
> benefit.
>
> Of course you have worried me now. If the keys are
chosen poorly - for
> example every call chooses key "my_first_key"
or suchlike - then that
> is a recipe for disaster. The key needs to correspond
to the set of
> types that are loaded into that DAS: the keys must be
chosen so that
> if the model is different, so is the key. I can't
enforce that, of
> course, all I can do is use it that way throughout the
SCA code.
>
> On Mar 23, 1:52 pm, "cem"
<Caroline.Mayn...gmail.com> wrote:
>
> > This discussion thread hasn't explicitly spelled
out the scope of the
> > cache, but I've been assuming that we're intending
it to endure from
> > module init to module shutdown. In other words,
depending on the SAPI,
> > the cache entries may be available to multiple
independent SCA
> > programs, sequentially and simultaneously.
>
> > Did I get this wrong? I can see how the interface
you describe would
> > work for a request-scoped or possibly
thread-scoped cache. But it
> > doesn't feel right to me to hand out the task of
key management to
> > uncoordinated applications. Imagine if everyone
used the same key
> > string that's in the example code snippet  Please put
me right.


--~--~---------~--~----~------------~-------~--~----~
You received this message because you are subscribed to the
Google Groups "phpsoa" group.
To post to this group, send email to phpsoagooglegroups.com
To unsubscribe from this group, send email to
phpsoa-unsubscribegooglegroups.com
For more options, visit this group at http://
groups.google.co.uk/group/phpsoa?hl=en
-~----------~----~----~----~------~----~------~--~---

Hi Rob, thank you for this. Once again you've thought of
things that I
haven't so your input is very valuable whenever it
comes...though
sorry if you've had to say it twice; I promise not to forget
this
time.

We (that's Graham and a couple of others) are going to get
together
here tomorrow to have a look at what I have done so far and
I will
make sure we talk through:

1. an ini option to turn off caching - a must-have for
shared hosting
2. per-request caching - I wasn't doing this yet but a
definite
possibility - just need to work out when it pays off
3. file-based caching

What I really want to know is .... what is the meaning of
the
smiley :/
I keep looking at it and keep seeing it as a frog with its
tongue
out 

Matthew

On Apr 10, 7:44 pm, "Rob" <rricha...ctindustries.net> wrote:
> Sorry all that I am really late in this conversation
(only a few weeks
> behind). Been meaning to respond but always get
distracted by
> something (think I got ADD or something) :/.
> I think I had mentioned this before but I really think
there needs to
> be 2 (at least) modes to this.
> In my case, our company controls the servers, so I
would want to be
> able to set my own key to be used for caching. If other
apps/departs
> want to use my schemas I would share the key with them
and everyone
> would use the same.
>
> This really wont work in the shared hosting environment
though. I bet
> in almost all those cases the hosting provider would
not want caching
> enabled (otherwise not install the extension). They
dont want one
> customer hogging memory here. In this case the per
request caching
> might be applicable (if you are even considering this
type of
> caching).
>
> So in short, it would be nice for at the minimum of
system caching
> (shared between processes) and an option to disable
completely.
> Nice to haves would be option for per request or file
based caching
> like ext.soap.
>
> I know I just asked for everything here but at the
minimum a way to
> disable caching on a server level (ini option) is
definitely needed.
> Hope I am not too far behind and mentioning things
already
> implemented 
>
> Rob
>
> On 3 Apr, 11:12, "Matthew Peters"
<matthew.f.pet...googlemail.com>
> wrote:
>
> > Yes, exactly right. The scope is intended to be
module init to
> > shutdown, and across all callers. I don't really
know how I could do
> > otherwise - a per-request cache would be throwing
away all the
> > benefit.
>
> > Of course you have worried me now. If the keys are
chosen poorly - for
> > example every call chooses key
"my_first_key" or suchlike - then that
> > is a recipe for disaster. The key needs to
correspond to the set of
> > types that are loaded into that DAS: the keys must
be chosen so that
> > if the model is different, so is the key. I can't
enforce that, of
> > course, all I can do is use it that way throughout
the SCA code.
>
> > On Mar 23, 1:52 pm, "cem"
<Caroline.Mayn...gmail.com> wrote:
>
> > > This discussion thread hasn't explicitly
spelled out the scope of the
> > > cache, but I've been assuming that we're
intending it to endure from
> > > module init to module shutdown. In other
words, depending on the SAPI,
> > > the cache entries may be available to
multiple independent SCA
> > > programs, sequentially and simultaneously.
>
> > > Did I get this wrong? I can see how the
interface you describe would
> > > work for a request-scoped or possibly
thread-scoped cache. But it
> > > doesn't feel right to me to hand out the task
of key management to
> > > uncoordinated applications. Imagine if
everyone used the same key
> > > string that's in the example code snippet

Please put me right.


--~--~---------~--~----~------------~-------~--~----~
You received this message because you are subscribed to the
Google Groups "phpsoa" group.
To post to this group, send email to phpsoagooglegroups.com
To unsubscribe from this group, send email to
phpsoa-unsubscribegooglegroups.com
For more options, visit this group at http://
groups.google.co.uk/group/phpsoa?hl=en
-~----------~----~----~----~------~----~------~--~---

On 16 Apr, 09:28, "Matthew Peters"
<matthew.f.pet...googlemail.com>
wrote:
> Hi Rob, thank you for this. Once again you've thought
of things that I
> haven't so your input is very valuable whenever it
comes...though
> sorry if you've had to say it twice; I promise not to
forget this
> time.

Not a big deal, was probably in one of the emails way back
when this
initially came up.

>
> We (that's Graham and a couple of others) are going to
get together
> here tomorrow to have a look at what I have done so far
and I will
> make sure we talk through:
>
> 1. an ini option to turn off caching - a must-have for
shared hosting
> 2. per-request caching - I wasn't doing this yet but a
definite
> possibility - just need to work out when it pays off
> 3. file-based caching

Disabling caching by default would probably be safest. Those
needing
to use it would have the capability to enable it.

> What I really want to know is .... what is the meaning
of the
> smiley :/
> I keep looking at it and keep seeing it as a frog with
its tongue
> out 

More along the lines of - Sorry for coming to the party so
late;
please don't hit me with the plastic utensils.

I had meant to follow up on some of your postings (almost a
month ago
now) and never got around to it.

Rob


--~--~---------~--~----~------------~-------~--~----~
You received this message because you are subscribed to the
Google Groups "phpsoa" group.
To post to this group, send email to phpsoagooglegroups.com
To unsubscribe from this group, send email to
phpsoa-unsubscribegooglegroups.com
For more options, visit this group at http://
groups.google.co.uk/group/phpsoa?hl=en
-~----------~----~----~----~------~----~------~--~---

I just checked in to DUNLIN a temporary change so that the
ebaysoap
binding uses the caching of the data factory. Specifically,
what I did
was:
1. enable the caching in the sdo extension when the special
two-
argument version of SDO_DAS_XML::create is used
2. make the table of cached data factories only one deep ie
only allow
one cached data factory
3. make the ebaysoap binding the only place in SCA where we
use the
two-argument create

This is a temporary state of affairs but probably how it
will go out
in the next release.

What I want to do long-term is to expose the table of cached
das's as
a proper PHP associative array which is a static member of
the SDO
class, then anyone can use it any way they see fit. The
logic to use
it would then look something like:

$key = // calculate this in some way based on the xsd or
xsds to be
used.
if (array_key_exists(SDO::cache($key)) {
  $das = SDO::cache[$key];
} else {
  $das = SDO_DAS_XML::create(<path to the xsd>);
  SDO::cache[$key] = $das;
}

Once the table of cached das's is in user space then people
can decide
about what to cache when, and when to invalidate entries and
so on.
Which is more or less what Ben suggested in the first
place...

Matthew


--~--~---------~--~----~------------~-------~--~----~
You received this message because you are subscribed to the
Google Groups "phpsoa" group.
To post to this group, send email to phpsoagooglegroups.com
To unsubscribe from this group, send email to
phpsoa-unsubscribegooglegroups.com
For more options, visit this group at http://
groups.google.co.uk/group/phpsoa?hl=en
-~----------~----~----~----~------~----~------~--~---

Hi Matthew, this looks good to me.  I'll give your changes a
go and
let you know how I get on.

The proposed caching approach looks nice and simple.  I
guess care
must be taken to make sure keys don't collide (perhaps some
recommended best practices on  creating good keys would
help).

One *real* comment: I think we should be getting into the
habit of
using "magic method"-style things whichappear on
the SDO interface
(and also our proxy interfaces) to reduce the risk of
clashes with the
business interfaces (in this case the properties of an SDO,
and in the
case of our proxies it would be the service methods).

Regards, Graham.

On 1 Jun, 16:38, Matthew Peters <matthew.f.pet...googlemail.com>
wrote:
> I just checked in to DUNLIN a temporary change so that
the ebaysoap
> binding uses the caching of the data factory.
Specifically, what I did
> was:
> 1. enable the caching in the sdo extension when the
special two-
> argument version of SDO_DAS_XML::create is used
> 2. make the table of cached data factories only one
deep ie only allow
> one cached data factory
> 3. make the ebaysoap binding the only place in SCA
where we use the
> two-argument create
>
> This is a temporary state of affairs but probably how
it will go out
> in the next release.
>
> What I want to do long-term is to expose the table of
cached das's as
> a proper PHP associative array which is a static member
of the SDO
> class, then anyone can use it any way they see fit. The
logic to use
> it would then look something like:
>
> $key = // calculate this in some way based on the xsd
or xsds to be
> used.
> if (array_key_exists(SDO::cache($key)) {
>   $das = SDO::cache[$key];} else {
>
>   $das = SDO_DAS_XML::create(<path to the xsd>);
>   SDO::cache[$key] = $das;
>
> }
>
> Once the table of cached das's is in user space then
people can decide
> about what to cache when, and when to invalidate
entries and so on.
> Which is more or less what Ben suggested in the first
place...
>
> Matthew


--~--~---------~--~----~------------~-------~--~----~
You received this message because you are subscribed to the
Google Groups "phpsoa" group.
To post to this group, send email to phpsoagooglegroups.com
To unsubscribe from this group, send email to
phpsoa-unsubscribegooglegroups.com
For more options, visit this group at http://
groups.google.co.uk/group/phpsoa?hl=en
-~----------~----~----~----~------~----~------~--~---

Hi Matthew,

I've just taken the latest code and it goes like something
off a
shovel  (IMO). 
The response time for a GetSearchResults is in the
region of 3 seconds, which is a > 50% improvement over
what I was
seeing without the caching.

Thanks!

Graham.

On 4 Jun, 11:22, Graham Charters <gchart...googlemail.com> wrote:
> Hi Matthew, this looks good to me.  I'll give your
changes a go and
> let you know how I get on.
>
> The proposed caching approach looks nice and simple.  I
guess care
> must be taken to make sure keys don't collide (perhaps
some
> recommended best practices on  creating good keys would
help).
>
> One *real* comment: I think we should be getting into
the habit of
> using "magic method"-style things whichappear
on the SDO interface
> (and also our proxy interfaces) to reduce the risk of
clashes with the
> business interfaces (in this case the properties of an
SDO, and in the
> case of our proxies it would be the service methods).
>
> Regards, Graham.
>
> On 1 Jun, 16:38, Matthew Peters
<matthew.f.pet...googlemail.com>
> wrote:> I just checked in to DUNLIN a temporary
change so that the ebaysoap
> > binding uses the caching of the data factory.
Specifically, what I did
> > was:
> > 1. enable the caching in the sdo extension when
the special two-
> > argument version of SDO_DAS_XML::create is used
> > 2. make the table of cached data factories only
one deep ie only allow
> > one cached data factory
> > 3. make the ebaysoap binding the only place in SCA
where we use the
> > two-argument create
>
> > This is a temporary state of affairs but probably
how it will go out
> > in the next release.
>
> > What I want to do long-term is to expose the table
of cached das's as
> > a proper PHP associative array which is a static
member of the SDO
> > class, then anyone can use it any way they see
fit. The logic to use
> > it would then look something like:
>
> > $key = // calculate this in some way based on the
xsd or xsds to be
> > used.
> > if (array_key_exists(SDO::cache($key)) {
> >   $das = SDO::cache[$key];} else {
>
> >   $das = SDO_DAS_XML::create(<path to the
xsd>);
> >   SDO::cache[$key] = $das;
>
> > }
>
> > Once the table of cached das's is in user space
then people can decide
> > about what to cache when, and when to invalidate
entries and so on.
> > Which is more or less what Ben suggested in the
first place...
>
> > Matthew


--~--~---------~--~----~------------~-------~--~----~
You received this message because you are subscribed to the
Google Groups "phpsoa" group.
To post to this group, send email to phpsoagooglegroups.com
To unsubscribe from this group, send email to
phpsoa-unsubscribegooglegroups.com
For more options, visit this group at http://
groups.google.co.uk/group/phpsoa?hl=en
-~----------~----~----~----~------~----~------~--~---

Graham Charters wrote:

> One *real* comment: I think we should be getting into
the habit of
> using "magic method"-style things which
appear on the SDO interface
> (and also our proxy interfaces) to reduce the risk of
clashes with the
> business interfaces (in this case the properties of an
SDO, and in the
> case of our proxies it would be the service methods).

Graham, I quite agree that the SCA proxies should not be
cluttered up 
with service methods, but I'm not quite sure why you
mentioned this in 
this particular thread - are you unhappy with the interface
Matthew 
proposed?


--~--~---------~--~----~------------~-------~--~----~
You received this message because you are subscribed to the
Google Groups "phpsoa" group.
To post to this group, send email to phpsoagooglegroups.com
To unsubscribe from this group, send email to
phpsoa-unsubscribegooglegroups.com
For more options, visit this group at http://
groups.google.co.uk/group/phpsoa?hl=en
-~----------~----~----~----~------~----~------~--~---

On 5 Jun, 10:36, Caroline Maynard <c...php.net> wrote:
> Graham Charters wrote:
> > One *real* comment: I think we should be getting
into the habit of
> > using "magic method"-style things which
appear on the SDO interface
> > (and also our proxy interfaces) to reduce the risk
of clashes with the
> > business interfaces (in this case the properties
of an SDO, and in the
> > case of our proxies it would be the service
methods).
>
> Graham, I quite agree that the SCA proxies should not
be cluttered up
> with service methods, but I'm not quite sure why you
mentioned this in
> this particular thread - are you unhappy with the
interface Matthew
> proposed?

I mentioned the proxies as one of the two areas where we
need to keep
the interface clean (the other was SDO).  I mentioned SDO
because the
"cache" appears to be on the SDO interface in
Matthew's example code
(unless I've misunderstood).


--~--~---------~--~----~------------~-------~--~----~
You received this message because you are subscribed to the
Google Groups "phpsoa" group.
To post to this group, send email to phpsoagooglegroups.com
To unsubscribe from this group, send email to
phpsoa-unsubscribegooglegroups.com
For more options, visit this group at http://
groups.google.co.uk/group/phpsoa?hl=en
-~----------~----~----~----~------~----~------~--~---