SOAP::Lite leaking memory

I have a fairly simple script, which creates a new
SOAP::Lite object (2 
actually, but thats not important), sends 3 calls (login,
dosomething, 
logout), and then SOAP::Lite object goes out of scope and
"should" be 
destroyed.  I was running some soak tests on this last night
(repeated 
calls) and was getting some relatively large memory leaks
(the script 
started off at about 20Mb, and rose to around 100Mb after
about 30 
minutes with a 1 second sleep between).

After poking about with Devel:;Symdump and Devel::Leak I'm
fairly 
confident that it is definately SOAP::Lite leaking the
memory from 
somewhere but before I start delving into the depths of
SOAP::Lite I 
wondered if anyone had come across this before and had
managed to 
fix/workaround it (in a worst case scenario I can always
fork off the 
SOAP::Lite process but I'd really rather not do that).


Gareth

On Thu, 2007-03-15 at 11:01 +0000, Gareth Harper wrote:
> I have a fairly simple script, which creates a new
SOAP::Lite object (2 
> actually, but thats not important), sends 3 calls
(login, dosomething, 
> logout), and then SOAP::Lite object goes out of scope
and "should" be 
> destroyed.  I was running some soak tests on this last
night (repeated 
> calls) and was getting some relatively large memory
leaks (the script 
> started off at about 20Mb, and rose to around 100Mb
after about 30 
> minutes with a 1 second sleep between).
> 

"going out of scope" is not necessarily going to
cause everything to go
away as SOAP::Lite dynamically creates methods in a variety
of
namespaces (I'm not going to check right now as the code
makes me
uneasy) and it is possible that it is closing over some of
the lexical
variables.

> After poking about with Devel:;Symdump and Devel::Leak
I'm fairly 
> confident that it is definately SOAP::Lite leaking the
memory from 
> somewhere but before I start delving into the depths of
SOAP::Lite I 
> wondered if anyone had come across this before and had
managed to 
> fix/workaround it (in a worst case scenario I can
always fork off the 
> SOAP::Lite process but I'd really rather not do that).

If you are using the "autoloaded" interface (i.e.
$soap->YourMethod(args) ) you might want to try the explicit
call (i.e.
$soap->call('MyMethod', args) ) as that is possibly
going to fiddle
with the symbol table less.

You might also want to test this with just re-using the
single
SOAP::Lite object as I seem to recall that most of the
horrible stuff
happens in the constructor.

/J

Jonathan Stowe wrote:

>"going out of scope" is not necessarily going
to cause everything to go
>away as SOAP::Lite dynamically creates methods in a
variety of
>namespaces (I'm not going to check right now as the code
makes me
>uneasy) and it is possible that it is closing over some
of the lexical
>variables.
>
>  
>

It's going out of scope as far as any of the code I have
written works 
(there's nothing clever in my code, it's a very basic
class), however 
I'm fairly certain that SOAP::Lite is creating some things
and caching them.

>If you are using the "autoloaded" interface
(i.e.
>$soap->YourMethod(args) ) you might want to
try the explicit call (i.e.
>$soap->call('MyMethod', args) ) as that is possibly
going to fiddle
>with the symbol table less.
>
>  
>
I'll check that out, but from what I was seeing in the
results from 
Devel::Leak a lot of the information in those variables is
the actual 
XML back and forth between my client and the server.  I've
just 
installed Devel::LeakTrace::Fast which supposedly lets me
know which 
line is allocating the variables.  if it's in the
constructor I might 
have to just bite the bullet and use a global.  I'd rather
not have to 
manually remove references to various objects internal to
SOAP::Lite.

>You might also want to test this with just re-using the
single
>SOAP::Lite object as I seem to recall that most of the
horrible stuff
>happens in the constructor.
>  
>
I can try this, though I'd rather not, as the module I'm
writing is to 
be dynamically loaded as required by one of our systems
(effectively 
it's a turing engine, but it's a little more complex and
specialised 
than that) as required.  Making the module when loaded keep
a global 
around with a SOAP object in it is feasable but
undesireable.  I did 
troll through the SOAP::Lite documentation in the hope of a
"don't cache 
objects" option just in case but that didn't appear. 
For testing 
purposes I'll create a global and see if that prevents the
memory leaks, 
if it does at least it gives me a somewhat nicer alternative
than 
forking off a process every time this function gets called.

On Thu, 2007-03-15 at 12:02 +0000, Gareth Harper wrote:
> Jonathan Stowe wrote:
> 
> >"going out of scope" is not necessarily
going to cause everything to go
> >away as SOAP::Lite dynamically creates methods in a
variety of
> >namespaces (I'm not going to check right now as the
code makes me
> >uneasy) and it is possible that it is closing over
some of the lexical
> >variables.
> >
> >  
> >
> 
> It's going out of scope as far as any of the code I
have written works 
> (there's nothing clever in my code, it's a very basic
class), however 
> I'm fairly certain that SOAP::Lite is creating some
things and caching them.
> 

Yes it is, it creates a load of methods dynamically directly
into
various namespaces under 'SOAP::Lite'. without examining the
code or
more probably instrumenting it in some way I would suspect
that these
subroutines are creating closures which are keeping
otherwise lexical
variables alive.

> >If you are using the "autoloaded"
interface (i.e.
> >$soap->YourMethod(args) ) you might want to
try the explicit call (i.e.
> >$soap->call('MyMethod', args) ) as that is possibly
going to fiddle
> >with the symbol table less.
> >
> >  
> >
> I'll check that out, but from what I was seeing in the
results from 
> Devel::Leak a lot of the information in those variables
is the actual 
> XML back and forth between my client and the server. 
I've just 
> installed Devel::LeakTrace::Fast which supposedly lets
me know which 
> line is allocating the variables.  if it's in the
constructor I might 
> have to just bite the bullet and use a global.  I'd
rather not have to 
> manually remove references to various objects internal
to SOAP::Lite.
> 
> >You might also want to test this with just re-using
the single
> >SOAP::Lite object as I seem to recall that most of
the horrible stuff
> >happens in the constructor.
> >  
> >
> I can try this, though I'd rather not, as the module
I'm writing is to 
> be dynamically loaded as required by one of our systems
(effectively 
> it's a turing engine, but it's a little more complex
and specialised 
> than that) as required.  Making the module when loaded
keep a global 
> around with a SOAP object in it is feasable but
undesireable.  I did 
> troll through the SOAP::Lite documentation in the hope
of a "don't cache 
> objects" option just in case but that didn't
appear.  For testing 
> purposes I'll create a global and see if that prevents
the memory leaks, 
> if it does at least it gives me a somewhat nicer
alternative than 
> forking off a process every time this function gets
called.
> 

If you could provide us with an example of the code that you
are testing
with then I'm sure we could move on from wild guesses to the
real cause
- mocking up the server proxy is fairly trivial.

/J