Memory Leak?

I've just begun experimenting with PyICU and I'm seeing
what appear to
be some pretty significant memory leaks, so much so that
I'm guessing
I may just be doing something very wrong.  I was wondering
if anyone
might have some suggestions as to what it is.

At the moment all of my tests have been under Windows (when
I get a
chance I plan on trying under Unix).

For example, a simple test using Locale() objects shows the
following
(using task manager to check memory usage):

          - - - - - - - - - - - - - - - - - - - - - - - - -
Python 2.4 (#60, Nov 30 2004, 11:49:19) [MSC v.1310 32 bit
(Intel)] on win32
Type "help", "copyright",
"credits" or "license" for more
information.
>>>
(Memory at 2740KB)

>>> from PyICU import Locale
>>> 
(Memory at 4436KB)

>>> for i in range(10000):
...   l = Locale('en_US')
...
>>> 
(Memory at 6272KB)

>>> for i in range(10000):
...   l = Locale('en_US')
...
>>> 
(Memory at 7920KB)
          - - - - - - - - - - - - - - - - - - - - - - - - -

The growth seems to continue indefinitely (e.g., it doesn't
seem to be
a fragmented working set that eventually stabilizes).  I
realize the
2.4 release isn't fully up to date, but since I also see
the behavior
with 2.3, I don't think it's an internal Python issue.

I've chosen Locale above, but it seems to happen with just
about
everything (and for example, 10000 NumberFormat classes
created with
createInstance will cause growth of about 13MB).

The above is from the pre-built PyICU-0.3 binaries along
with a
pre-built ICU-3.2 library.  But I've verified similar
behavior in the
following combinations when building PyICU from source (the
current
SVN head):

* PyICU-svn built against ICU-3.4, both compiled by VC7
(VC.NET 2003)
  and linked with Python 2.4.
* PyICU-svn built against ICU-3.4, using a pre-compiled ICU
library for VC7.
* PyICU-svn built against ICU-3.4, both compiled by VC6, and
linked against
  Python 2.3.

Note that in the last case, I needed to temporarily disable
the use of
the internal C API to create datetime types in common.cpp,
as well as
replace the use of the Py_RETURN_NONE macro in bases.i and
locales.i
since that to account for differences from Python 2.3 to
2.4.

Also, when building PyICU from source in these cases I have
tried both
SWIG 1.3.24 and 1.3.28.

Interestingly, swig 1.3.28 generates warnings during
compilation
(about the use of the -nodefault option) and runtime such
as:

Python 2.4 (#60, Nov 30 2004, 11:49:19) [MSC v.1310 32 bit
(Intel)] on win32
Type "help", "copyright",
"credits" or "license" for more
information.
>>> from PyICU import Locale
>>> l = Locale('en_US')
>>> l = Locale('en_US')
swig/python detected a memory leak of type 'icu::Locale
*', no destructor found.

This continues to happen even if I replace the
"-nodefault" option
with "-nodefaultctor" and
"-nodefaultdtor" as suggested by the swig
warning during compilation.  I haven't really tried to
evaluate why
PyICU is choosing to disable constructor creation.

Given the scale of this apparent leak, I have to imagine it
would have
been seen in other cases where it might be getting used, so
I'm hoping
I'm just doing something wrong, and would appreciate any
thoughts.

Thanks.

-- David Bolen

_______________________________________________
pyicu-dev mailing list
pyicu-devosafoundation.org
http://lists.osafoundation.org/cgi-bin/mailman/
listinfo/pyicu-dev

On Fri, 10 Mar 2006, David Bolen wrote:

> I've just begun experimenting with PyICU and I'm
seeing what appear to
> be some pretty significant memory leaks, so much so
that I'm guessing
> I may just be doing something very wrong.  I was
wondering if anyone
> might have some suggestions as to what it is.
>
>  ... snip ...
>
> Interestingly, swig 1.3.28 generates warnings during
compilation
> (about the use of the -nodefault option) and runtime
such as:
>
> Python 2.4 (#60, Nov 30 2004, 11:49:19) [MSC v.1310 32
bit (Intel)] on win32
> Type "help", "copyright",
"credits" or "license" for more
information.
>>>> from PyICU import Locale
>>>> l = Locale('en_US')
>>>> l = Locale('en_US')
> swig/python detected a memory leak of type
'icu::Locale *', no destructor found.

This is the clue, apparently no destructor code was
generated. Ugh.

> This continues to happen even if I replace the
"-nodefault" option
> with "-nodefaultctor" and
"-nodefaultdtor" as suggested by the swig
> warning during compilation.  I haven't really tried to
evaluate why
> PyICU is choosing to disable constructor creation.

Actually, I tried to replace -nodefault with -nodefaultctor
and rebuild all 
with swig 1.3.28 and the destructor code was generated and
these warnings 
didn't appear anymore.

> Given the scale of this apparent leak, I have to
imagine it would have
> been seen in other cases where it might be getting
used, so I'm hoping
> I'm just doing something wrong, and would appreciate
any thoughts.

Indeed. Apart from probably not rebuilding it correctly, you
didn't do 
anything wrong. I, however, did something completely wrong
by using -nodefault 
when what I really wanted was -nodefaultctor, that is, no
constructor code 
generation by default, only generate code for constructors
that I declare.

So, apparently, the fix to this is simple:

   - use -nodefaultctor instead of -nodefault
   - and use swig 1.3.28 since -nodefaultctor doesn't
appear to be supported
     with swig 1.3.24

Thank you for finding and reporting this.
This is indeed a bad one !

I haven't done any other testing of PyICU with swig 1.3.28
and would 
appreciate it if you could report back about how it is
working for you.
Every swig upgrade introduces a dose of unknowns...

I intend to make a new PyICU release in about two weeks or
so...

Thanks again !

Andi..


_______________________________________________
pyicu-dev mailing list
pyicu-devosafoundation.org
http://lists.osafoundation.org/cgi-bin/mailman/
listinfo/pyicu-dev