spam_scan failes with out of memory error

Hello everyone,

I have a email in my postfix queue which has the following
status:

host 127.0.0.1[127.0.0.1] said: 451 4.5.0 Error in
processing,
id=36819-05, spam_scan FAILED: Out of memory during
"large" request for
33558528 bytes, total sbrk() is 528226304 bytes at
/usr/local/sbin/amavisd line 3786, <GEN11> line
239016. (in reply to end
of DATA command)

Now the email in question 8.8Mb large when I check it in the
queue, in
the amavisd tmp dir the email.txt is 8.7Mb, which looks
allright. This
email is submitted 5 times (To different users, I guess the
sender never
heard of CC / BCC). 
When maia scans it it eats up all the memory in the server,
per process
it runs up to 500+ Mb. I run 10 amavisd children and my
server can't
handle that .

I tried lowering the $max_servers, I'm now at 2 processes
but I still
see the out of memory error. But the server does not run out
of memory.
When I check the contents of the email it exists out of a
small bit of
text and 5 attached images. I've checked the images and all
seem to be
normal images.

Amavisd debug logfile:
The only thin I see in the amavisd log about spam_scan is:
Oct 22 16:46:40 smtp2 amavis[64955]: (64955-04) spam_scan:
not wasting
time on SA, message longer than 262144 bytes: 681+9107930
Oct 22 16:46:40 smtp2 amavis[64955]: (64955-04) spam_scan:
hits=
Oct 22 16:46:40 smtp2 amavis[64955]: (64955-04)
prolong_timer after
spam_scan: remaining time = 476 s
Oct 22 16:46:40 smtp2 amavis[64955]: (64955-04)
prolong_timer after
spam_scan: remaining time = 476 s
Oct 22 16:46:40 smtp2 amavis[64955]: (64955-04) lookup =>
undef,
"XXXgmail.com" does not match
Oct 22 16:46:40 smtp2 amavis[64955]: (64955-04) Maia:
[recipient_is_local] Recipient XXXgmail.com is non-local
Oct 22 16:46:40 smtp2 amavis[64955]: (64955-04) Maia:
[autocreate_users]
Ignoring non-local recipient XXXgmail.com
Oct 22 16:46:40 smtp2 amavis[64955]: (64955-04) lookup =>
undef,
"XXXpion-overheidsjuristen.nl" does not match
Oct 22 16:46:40 smtp2 amavis[64955]: (64955-04) Maia:
[recipient_is_local] Recipient XXXpion-overheidsjuristen.nl
is
non-local
Oct 22 16:46:40 smtp2 amavis[64955]: (64955-04) Maia:
[autocreate_users]
Ignoring non-local recipient XXXpion-overheidsjuristen.nl
Oct 22 16:46:40 smtp2 amavis[64955]: (64955-04) lookup =>
undef,
"XXXwanadoo.nl" does not match
Oct 22 16:46:40 smtp2 amavis[64955]: (64955-04) Maia:
[recipient_is_local] Recipient XXXwanadoo.nl is non-local
Oct 22 16:46:40 smtp2 amavis[64955]: (64955-04) Maia:
[autocreate_users]
Ignoring non-local recipient XXXwanadoo.nl
Oct 22 16:47:07 smtp2 amavis[64955]: (64955-04) TROUBLE in
check_mail:
spam_scan FAILED: Out of memory during "large"
request for 33558528
bytes, total sbrk() is 529225728 bytes at
/usr/local/sbin/amavisd line
3786, <GEN11> line 240664.
Oct 22 16:47:07 smtp2 amavis[64955]: (64955-04) Maia:
[disconnect]
Disconnecting from SQL database
Oct sending SMTP response: "451 4.5.0 Error in
processing, id=64955-04,
spam_scan FAILED: Out of memory during "large"
request for 33558528
bytes, total sbrk() is 529225728 bytes at
/usr/local/sbin/amavisd line
3786, <GEN11> line 240664."22 16:47:07 smtp2
amavis[64955]: (64955-04) 
Oct 22 16:47:07 smtp2 amavis[64955]: (64955-04) PRESERVING
EVIDENCE in
/var/amavisd/tmp/amavis-20071022T164418-64955
Oct 22 16:47:07 smtp2 amavis[64955]: (64955-04) TIMING
[total 35265 ms]
- SMTP EHLO: 5 (0%), SMTP pre-MAIL: 4 (0%), lookup_sql: 15
(0%),
lookup_sql: 5 (0%), lookup_sql: 5 (0%), SMTP pre-DATA-flush:
3 (0%),
SMTP DATA: 2076 (6%), body_hash: 97 (0%), maia_connect: 5
(0%),
maia_read_system_config: 4 (0%), maia_get_mysql_size_limit:
1 (0%),
lookup_sql: 5 (0%), lookup_sql: 5 (0%), lookup_sql: 4 (0%),
mime_decode:
2738 (8%), get-file-type7: 66 (0%), decompose_part: 2 (0%),
parts_decode: 2 (0%), AV-scan-1: 2922 (8%), spam-wb-list: 21
(0%),
update_cache: 2 (0%), maia_autocreate_users: 6 (0%),
maia_delete_mail:
27269 (77%), maia_disconnect: 1 (0%), rundown: 4 (0%)
Oct 22 16:47:07 smtp2 amavis[64955]: (64955-04) ESMTP>
451 4.5.0 Error
in processing, id=64955-04, spam_scan FAILED: Out of memory
during
"large" request for 33558528 bytes, total sbrk()
is 529225728 bytes at
/usr/local/sbin/amavisd line 3786, <GEN11> line
240664.
Oct 22 16:47:07 smtp2 amavis[64955]: (64955-04)
prolong_timer after
reading SMTP command: remaining time = 0 s
Oct 22 16:47:07 smtp2 amavis[64955]: (64955-04) ESMTP<
QUITrn
Oct 22 16:47:07 smtp2 amavis[64955]: (64955-04) ESMTP>
221 2.0.0
[127.0.0.1] amavisd-maia closing transmission channel
Oct 22 16:47:07 smtp2 amavis[64955]: (64955-04) extra
modules loaded:
Crypt/Blowfish.pm, Crypt/CBC.pm,
Mail/SpamAssassin/Locales.pm,
Mail/SpamAssassin/Plugin/Bayes.pm,
Mail/SpamAssassin/Plugin/BodyEval.pm,
Mail/SpamAssassin/Plugin/Check.pm,
Mail/SpamAssassin/Plugin/DNSEval.pm,
Mail/SpamAssassin/Plugin/HTMLEval.pm,
Mail/SpamAssassin/Plugin/HTTPSMismatch.pm,
Mail/SpamAssassin/Plugin/HeaderEval.pm,
Mail/SpamAssassin/Plugin/ImageInfo.pm,
Mail/SpamAssassin/Plugin/MIMEEval.pm,
Mail/SpamAssassin/Plugin/RelayEval.pm,
Mail/SpamAssassin/Plugin/URIDetail.pm,
Mail/SpamAssassin/Plugin/URIEval.pm,
Mail/SpamAssassin/Plugin/VBounce.pm,
Mail/SpamAssassin/Plugin/WLBLEval.pm
Oct 22 16:47:07 smtp2 amavis[64955]: (64955-04) load: 35 %,
total idle
109.158 s, busy 59.834 s
Oct 22 16:47:07 smtp2 amavis[64955]: (64955-04) loaded base
policy bank
Oct 22 16:47:07 smtp2 amavis[64955]: (64955-04)
lookup_ip_acl:
key="127.0.0.1" matches "127.0.0.1",
result=1
Oct 22 16:47:07 smtp2 amavis[64955]: (64955-04)
prolong_timer after new
request - timer reset: remaining time = 480 s

Also Maia/amavisd doesn't clean these mails from the tmp
dir, it leaves
them behind. Now I never had these kind of troubles with a
plain
amavisd-new, does anyone has any idea what is happening, or
how can I
find out what is going on?

Thanks,

Floris Termorshuizen

*** Relevant Config ***

Server:	Dual Intel PIII 1.4 Ghz, 2 Gb memory.
Host OS: 	Freebsd 6.1
Postfix 2.4.5
Maia-Mailguard 1.0.2a
ClamAV 0.91
SpamAssassin version 3.2.3

Postfix Main.cf:
content_filter=smtp-amavis:[127.0.0.1]:10024
message_size_limit = 10000000

Postfix Master.cf
smtp-amavis unix - - n - 2 smtp
  -o smtp_data_done_timeout=2400
  -o smtp_send_xforward_command=yes
  -o disable_dns_lookups=yes
  -o max_use=12

Amavisd.conf
$max_servers = 2;
$sa_mail_body_size_limit = 256*1024;
$MAXLEVELS = 14;
$MAXFILES = 1500;
$MIN_EXPANSION_QUOTA =      100*1024;  # bytes  (default
undef, not
enforced)
$MAX_EXPANSION_QUOTA = 300*1024*1024;  # bytes  (default
undef, not
enforced)

Maia.conf
$default_max_size = 256*1024;

Mysql my.cnf:
max_allowed_packet = 10M
_______________________________________________
Maia-users mailing list
Maia-usersrenaissoft.com
http://www.renaissoft.com/mailman/listinfo/maia-users

spam_scan failes with out of memory error

Thread: spam_scan failes with out of memory error