Process Size

Howdy,

I just built a new Bricolage install on 64-bit Linux with
SFTP,  
Imager, and LDAP support included.  We're loading only the
en-US  
localization and the UTC and America/Los_Angeles time zones.
 
Processes when they start up are about 260MB. Is this
normal? It  
seems a bit excessive to me.

What sort of process sizes are you getting on your build,
and on what  
architecture with what optional modules included?

Thanks,

David

"David E. Wheeler" <davidkineticode.com> writes:
> Howdy,
>
> I just built a new Bricolage install on 64-bit Linux
with SFTP,
> Imager, and LDAP support included.  We're loading only
the en-US
> localization and the UTC and America/Los_Angeles time
zones.
> Processes when they start up are about 260MB. Is this
normal? It
> seems a bit excessive to me.
>
> What sort of process sizes are you getting on your
build, and on what
> architecture with what optional modules included?

My Bricolage apache processes start at about 120MB on Linux
Intel
x86_64 and on AMD64. I use practically all modules except
SFTP, and a
german localization. They keep quite stable in size after a
while at
about 260MB.

On x86/Solaris10 they start at 50MB but seem to grow *much*
more
(feels like memleaking).

Steffen
-- 
Steffen Schwigon <schwigonwebit.de>
Dresden Perl Mongers <http://dresden-pm.org/>

Deutscher Perl-Workshop <http://www.perl-work
shop.de/>

"David E. Wheeler" <davidkineticode.com> writes:
> Howdy,
>
> I just built a new Bricolage install on 64-bit Linux
with SFTP,

BTW, how did you solve the typical SFTP install problems?
Did you
install its problematic dependencies (Crypt:H) from
CPAN or did you
use pre-packaged modules?

Steffen
-- 
Steffen Schwigon <schwigonwebit.de>
Dresden Perl Mongers <http://dresden-pm.org/>

Deutscher Perl-Workshop <http://www.perl-work
shop.de/>

On Wed, Jan 09, 2008 at 11:16:31PM -0500, David E. Wheeler
wrote:
> Howdy,
> 
> I just built a new Bricolage install on 64-bit Linux
with SFTP,  
> Imager, and LDAP support included.  We're loading only
the en-US  
> localization and the UTC and America/Los_Angeles time
zones.  
> Processes when they start up are about 260MB. Is this
normal? It  
> seems a bit excessive to me.
> 
> What sort of process sizes are you getting on your
build, and on what  
> architecture with what optional modules included?

Hey David:

1.10.3 on 32-bit NetBSD: 41724K

This is a vanilla install with no extra options enabled...
lang == eng-us, utf8, tz == utc.

1.10.2 on 64-bit FreeBSD: 136460K (avg)
has RELATED_MEDIA_UPLOAD, FTP Server, lang == eng-us, utf8,
tz == utc


hth,

> Thanks,
> 
> David
> 

-- 

Brad Harder,
Method Digital Logic
http://www.methodlogic.net

On Thu, Jan 10, 2008 at 07:12:52AM -0800, bharder wrote:
> On Wed, Jan 09, 2008 at 11:16:31PM -0500, David E.
Wheeler wrote:
> > Howdy,
> > 
> > I just built a new Bricolage install on 64-bit
Linux with SFTP,  
> > Imager, and LDAP support included.  We're loading
only the en-US  
> > localization and the UTC and America/Los_Angeles
time zones.  
> > Processes when they start up are about 260MB. Is
this normal? It  
> > seems a bit excessive to me.
> > 
> > What sort of process sizes are you getting on your
build, and on what  
> > architecture with what optional modules included?
> 
> Hey David:
> 
> 1.10.3 on 32-bit NetBSD: 41724K
> 
> This is a vanilla install with no extra options
enabled... lang == eng-us, utf8, tz == utc.
> 
> 1.10.2 on 64-bit FreeBSD: 136460K (avg)
> has RELATED_MEDIA_UPLOAD, FTP Server, lang == eng-us,
utf8, tz == utc

Hrmm -- David, to be clear, can you tell me what you
measured exactly? I just noticed what I measured was the
spawning (parent) process on the NetBSD box, and the avg of
the children on the FreeBSD box... the root process on the
FreeBSD box is listed as 16316K, fwiw...


> hth,
> 
> > Thanks,
> > 
> > David
> > 
> 
> -- 
> 
> Brad Harder,
> Method Digital Logic
> http://www.methodlogic.net




-- 

Brad Harder,
Method Digital Logic
http://www.methodlogic.net

On Jan 10, 2008, at 3:11 AM, Steffen Schwigon wrote:

> "David E. Wheeler" <davidkineticode.com> writes:
>> Howdy,
>>
>> I just built a new Bricolage install on 64-bit
Linux with SFTP,
>
> BTW, how did you solve the typical SFTP install
problems? Did you
> install its problematic dependencies (Crypt:H) from
CPAN or did you
> use pre-packaged modules?

I installed from CPAN. The test failures I ran into for
Crypt:H and  
Math::Pari seemed harmless, so I just installed them. There
have been  
no problems since.

Best,

David

On Jan 10, 2008, at 10:34 AM, bharder wrote:

>> 1.10.2 on 64-bit FreeBSD: 136460K (avg)
>> has RELATED_MEDIA_UPLOAD, FTP Server, lang ==
eng-us, utf8, tz == utc
>
> Hrmm -- David, to be clear, can you tell me what you
measured  
> exactly? I just noticed what I measured was the
spawning (parent)  
> process on the NetBSD box, and the avg of the children
on the  
> FreeBSD box... the root process on the FreeBSD box is
listed as  
> 16316K, fwiw...

I was going on what Apache::SizeLimit was reporting in the
error log,  
where it was killing processes after every request. So each
child  
would handle one request and die. Seemed stupid.

I just looked at the process table, where VSZ is 264488 and
RSS is  
65524. Looking at the ps docs, the latter is the real
number. So  
Bricolage is taking up 64MB of memory when it starts --
that's a  
*much* more reasonable number.

So I guess the question I have is, why does
Apache::SizeLimit rely on  
VSZ instead of RSS? And what is the "virtual memory
size," anyway?

Thanks,

David

On Fri, Jan 11, 2008 at 06:47:35PM -0500, David E. Wheeler
wrote:
> On Jan 10, 2008, at 10:34 AM, bharder wrote:
> 
> >>1.10.2 on 64-bit FreeBSD: 136460K (avg)
> >>has RELATED_MEDIA_UPLOAD, FTP Server, lang ==
eng-us, utf8, tz == utc
> >
> >Hrmm -- David, to be clear, can you tell me what
you measured  
> >exactly? I just noticed what I measured was the
spawning (parent)  
> >process on the NetBSD box, and the avg of the
children on the  
> >FreeBSD box... the root process on the FreeBSD box
is listed as  
> >16316K, fwiw...
> 
> I was going on what Apache::SizeLimit was reporting in
the error log,  
> where it was killing processes after every request. So
each child  
> would handle one request and die. Seemed stupid.
> 
> I just looked at the process table, where VSZ is 264488
and RSS is  
> 65524. Looking at the ps docs, the latter is the real
number. So  
> Bricolage is taking up 64MB of memory when it starts --
that's a  
> *much* more reasonable number.
> 
> So I guess the question I have is, why does
Apache::SizeLimit rely on  
> VSZ instead of RSS? And what is the "virtual
memory size," anyway?

see: http://www.tek-tips.com/viewthread.cfm?qid=1190960&
amp;page=1


> Thanks,
> 
> David

-- 

Brad Harder,
Method Digital Logic
http://www.methodlogic.net

On Fri, Jan 11, 2008 at 03:54:16PM -0800, bharder wrote:
> On Fri, Jan 11, 2008 at 06:47:35PM -0500, David E.
Wheeler wrote:
> > On Jan 10, 2008, at 10:34 AM, bharder wrote:
> > 
> > >>1.10.2 on 64-bit FreeBSD: 136460K (avg)
> > >>has RELATED_MEDIA_UPLOAD, FTP Server, lang
== eng-us, utf8, tz == utc
> > >
> > >Hrmm -- David, to be clear, can you tell me
what you measured  
> > >exactly? I just noticed what I measured was
the spawning (parent)  
> > >process on the NetBSD box, and the avg of the
children on the  
> > >FreeBSD box... the root process on the FreeBSD
box is listed as  
> > >16316K, fwiw...

I was using RSS from ps -- looking further into this though,
it looks like one can get more details in Linux via: cat
/proc/[pid]/maps

like so (for a bash shell on Debian):

bharderparis:~$ cat /proc/29781/maps
08048000-080e9000 r-xp 00000000 03:05 28146      /bin/bash
080e9000-080ef000 rw-p 000a0000 03:05 28146      /bin/bash
080ef000-081a6000 rwxp 00000000 00:00 0
40000000-40015000 r-xp 00000000 03:05 26145     
/lib/ld-2.3.6.so
40015000-40017000 rw-p 00014000 03:05 26145     
/lib/ld-2.3.6.so
40017000-40018000 rw-p 00000000 00:00 0
4001e000-40056000 r-xp 00000000 03:05 26255     
/lib/libncurses.so.5.5
40056000-4005e000 rw-p 00038000 03:05 26255     
/lib/libncurses.so.5.5
4005e000-4005f000 rw-p 00000000 00:00 0
4005f000-40061000 r-xp 00000000 03:05 26279     
/lib/libdl-2.3.6.so
40061000-40063000 rw-p 00001000 03:05 26279     
/lib/libdl-2.3.6.so
40063000-40174000 r-xp 00000000 03:05 26246     
/lib/libc-2.3.6.so
40174000-40179000 r--p 00110000 03:05 26246     
/lib/libc-2.3.6.so
40179000-4017b000 rw-p 00115000 03:05 26246     
/lib/libc-2.3.6.so
4017b000-4017f000 rw-p 00000000 00:00 0
4017f000-40185000 r-xp 00000000 03:05 26288     
/lib/libnss_compat-2.3.6.so
40185000-40187000 rw-p 00005000 03:05 26288     
/lib/libnss_compat-2.3.6.so
40187000-40198000 r-xp 00000000 03:05 26285     
/lib/libnsl-2.3.6.so
40198000-4019a000 rw-p 00010000 03:05 26285     
/lib/libnsl-2.3.6.so
4019a000-4019c000 rw-p 00000000 00:00 0
4019c000-401a4000 r-xp 00000000 03:05 26292     
/lib/libnss_nis-2.3.6.so
401a4000-401a6000 rw-p 00007000 03:05 26292     
/lib/libnss_nis-2.3.6.so
401a6000-401ae000 r-xp 00000000 03:05 26290     
/lib/libnss_files-2.3.6.so
401ae000-401b0000 rw-p 00007000 03:05 26290     
/lib/libnss_files-2.3.6.so
bfffb000-c0000000 rwxp ffffc000 00:00 0


For the case of shared text/static data, you may want to try
to measure it, and boost the SizeLimit accordingly.

> > I was going on what Apache::SizeLimit was
reporting in the error log,  
> > where it was killing processes after every
request. So each child  
> > would handle one request and die. Seemed stupid.

> > I just looked at the process table, where VSZ is
264488 and RSS is  
> > 65524. Looking at the ps docs, the latter is the
real number. So  
> > Bricolage is taking up 64MB of memory when it
starts -- that's a  
> > *much* more reasonable number.
> > 
> > So I guess the question I have is, why does
Apache::SizeLimit rely on  
> > VSZ instead of RSS? And what is the "virtual
memory size," anyway?
> 
> see: http://www.tek-tips.com/viewthread.cfm?qid=1190960&
amp;page=1
> 
> 
> > Thanks,
> > 
> > David
> 
> -- 
> 
> Brad Harder,
> Method Digital Logic
> http://www.methodlogic.net




-- 

Brad Harder,
Method Digital Logic
http://www.methodlogic.net