X-Spam headers from another amavisd/SA

Is there a way to configure amavisd to completely ignore
X-Spam-*
headers that may have previously been applied by an earlier
pass through
amavisd on a different host?  I realize that this could be a
Very Bad
Thing, but here's my problem:

We send mail to several customer domains across private
connections that
do not traverse the internet.  For various security reasons,
many of
these customers require dedicated relays for their mail.

The problem lies with one of those private networks
(actually several of
them, but we'll focus on one.)  Let's call it bar.com.  All
outbound
mail from our domain (foo.com) is delivered to bar.com via a
private
connection, and some of the inbound mail from bar.com is
returned via
the same path.   No problem there.  But some divisions
within the super
maga corp bar.com don't route outbound mail back across this
private
line... they route mail across the internet.  In accordance
with the
agreements made with some divisions of bar.com, our outbound
mail to
their domain may not traverse any internet-connected relay.

This is only an issue with respect to amavis' penpal
support.  Inbound
replies sent in response to messages that went out across
the private
line do not benefit from the penpal score reduction because
the original
messages were never seen and logged by the internet-facing
relays.  

We use sql logging, so my solution is to insert a dummy
amavisd on one
of the common internal mail exchanger hosts and log all
traffic to a
common database.  This dummy amavisd has the virus scanners
and
spamassassin disabled, but it's destroying the X-Spam
headers for
inbound internet-originated mail that was scanned on its way
in by our
internet-facing amavisd/SA machines.  The original headers
are replaced
with this:

 X-Spam-Flag: NO
 X-Spam-Score: 0
 X-Spam-Level: 
 X-Spam-Status: No, score=x tests=[]

The dummy amavisd appears to be inserting these header lines
even if
they don't already exist.

The penpal feature is very important to us because this
particular
customer domain is very commonly forged by spammers and the
contents of
their legitimate communications are very sensitive to spam
filters
(their messages are often quite spam-like.)

So, how can I get amavisd to leave pre-existing X-Spam
headers alone AND
not insert them if they don't already exist?  Basically, to
completely
ignore these headers.

Here are what I believe are the relevant portions of the
amavisd.conf
file on the dummy server:

 bypass_virus_checks_maps = (1);
 bypass_spam_checks_maps  = (1);
 $bypass_decode_parts = 1;
 $mydomain = 'foo.com';
 local_domains_maps = ( ['.foo.com'] );
 mynetworks = qw( 127.0.0.0/8 192.168.2.0/23
192.168.4.0/24 );

 $sa_tag_level_deflt  = undef;
 $sa_tag2_level_deflt = undef;
 $sa_kill_level_deflt = undef;
 $sa_dsn_cutoff_level = undef;
 $sa_crediblefrom_dsn_cutoff_level = undef
 $final_spam_destiny       = D_PASS;

I've also tried arbitrarily high values (1000) for the
various sa_tag
settings with no effect.  Blanking mynetworks also had no
effect.
SpamAssassin isn't even installed on the machine with the
dummy amavisd.
We're using amavisd-new 2.6.0pre1.

Any thoughts?  I could probably do what I want by hacking on
the amavisd
script itself, but I'd rather not do that if I don't have
to.

On an unrelated note, has anyone thought of using the
penpals feature to
check inbound NDRs for a corresponding outbound message?  If
the message
ID contained within the inbound NDR doesn't match a valid
message ID
from the apparent sender, such messages could be dropped. 
Parsing the
inbound NDR for the message ID might be difficult, but I
don't think
that it would be impossible.

This e-mail transmission contains information that is
confidential and may be 
privileged. It is intended only for the addressee(s) named
above. If you receive 
this e-mail in error, please do not read, copy or
disseminate it in any manner. 
If you are not the intended recipient, any disclosure,
copying, distribution or 
use of the contents of this information is prohibited.
Please reply to the 
message immediately by informing the sender that the message
was misdirected. 
After replying, please erase it from your computer system.
Your assistance in 
correcting this error is appreciated.


------------------------------------------------------------
-------------
Check out the new SourceForge.net Marketplace.
It's the best place to buy or sell services for
just about anything Open Source.
http://ad.doubleclick.net/clk;164216
239;13503038;w?http://sf.net/marketplace
_______________________________________________
AMaViS-user mailing list
AMaViS-userlists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/amavis-user

AMaViS-FAQ:http://www.amav
is.org/amavis-faq.php3
AMaViS-HowTos:http://www.amavis.org/ho
wto/

On 3/26/08, Crosby, Scott F. wrote:
> Is there a way to configure amavisd to completely
ignore X-Spam-*
> headers that may have previously been applied by an
earlier pass through
> amavisd on a different host?  I realize that this could
be a Very Bad
> Thing, but here's my problem:
>
> We send mail to several customer domains across private
connections that
> do not traverse the internet.  For various security
reasons, many of
> these customers require dedicated relays for their
mail.
>
> The problem lies with one of those private networks
(actually several of
> them, but we'll focus on one.)  Let's call it bar.com. 
All outbound
> mail from our domain (foo.com) is delivered to bar.com
via a private
> connection, and some of the inbound mail from bar.com
is returned via
> the same path.   No problem there.  But some divisions
within the super
> maga corp bar.com don't route outbound mail back across
this private
> line... they route mail across the internet.  In
accordance with the
> agreements made with some divisions of bar.com, our
outbound mail to
> their domain may not traverse any internet-connected
relay.
>
> This is only an issue with respect to amavis' penpal
support.  Inbound
> replies sent in response to messages that went out
across the private
> line do not benefit from the penpal score reduction
because the original
> messages were never seen and logged by the
internet-facing relays.
>
> We use sql logging, so my solution is to insert a dummy
amavisd on one
> of the common internal mail exchanger hosts and log all
traffic to a
> common database.  This dummy amavisd has the virus
scanners and
> spamassassin disabled, but it's destroying the X-Spam
headers for
> inbound internet-originated mail that was scanned on
its way in by our
> internet-facing amavisd/SA machines.  The original
headers are replaced
> with this:
>
>  X-Spam-Flag: NO
>  X-Spam-Score: 0
>  X-Spam-Level:
>  X-Spam-Status: No, score=x tests=[]
>

Hmm, this says the score is zero, which indicates to me that
that
message was scanned (by what? - nothing? I dunno). How did
you
"disable" spam scanning? To disable spam scanning
try:

bypass_spam_checks_maps  = (1);

You may also desire:
bypass_virus_checks_maps = (1);
bypass_banned_checks_maps = (1);
bypass_header_checks_maps = (1);

Also, if you want to keep existing headers, add:
$remove_existing_spam_headers  = 0;

> The dummy amavisd appears to be inserting these header
lines even if
> they don't already exist.
>
> The penpal feature is very important to us because this
particular
> customer domain is very commonly forged by spammers and
the contents of
> their legitimate communications are very sensitive to
spam filters
> (their messages are often quite spam-like.)
>
> So, how can I get amavisd to leave pre-existing X-Spam
headers alone AND
> not insert them if they don't already exist? 
Basically, to completely
> ignore these headers.
>
> Here are what I believe are the relevant portions of
the amavisd.conf
> file on the dummy server:
>
>  bypass_virus_checks_maps = (1);
>  bypass_spam_checks_maps  = (1);
>  $bypass_decode_parts = 1;
>  $mydomain = 'foo.com';
>  local_domains_maps = ( ['.foo.com'] );

X-Spam headers are only inserted if the domain is considered
local. If
it makes no difference, try:

local_domains_maps = ( [''] );

but it may not matter if spam and virus scanning are
disabled.

>  mynetworks = qw( 127.0.0.0/8 192.168.2.0/23
192.168.4.0/24 );
>
>  $sa_tag_level_deflt  = undef;
>  $sa_tag2_level_deflt = undef;
>  $sa_kill_level_deflt = undef;
>  $sa_dsn_cutoff_level = undef;
>  $sa_crediblefrom_dsn_cutoff_level = undef
>  $final_spam_destiny       = D_PASS;

If spam scanning is truly disabled, none of this should
matter.

>
> I've also tried arbitrarily high values (1000) for the
various sa_tag
> settings with no effect.  Blanking mynetworks also had no effect.
> SpamAssassin isn't even installed on the machine with
the dummy amavisd.
> We're using amavisd-new 2.6.0pre1.
>
> Any thoughts?  I could probably do what I want by
hacking on the amavisd
> script itself, but I'd rather not do that if I don't
have to.
>
> On an unrelated note, has anyone thought of using the
penpals feature to
> check inbound NDRs for a corresponding outbound
message?  If the message
> ID contained within the inbound NDR doesn't match a
valid message ID
> from the apparent sender, such messages could be
dropped.  Parsing the
> inbound NDR for the message ID might be difficult, but
I don't think
> that it would be impossible.
>

-- 
Gary V

------------------------------------------------------------
-------------
Check out the new SourceForge.net Marketplace.
It's the best place to buy or sell services for
just about anything Open Source.
http://ad.doubleclick.net/clk;164216
239;13503038;w?http://sf.net/marketplace
_______________________________________________
AMaViS-user mailing list
AMaViS-userlists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/amavis-user

AMaViS-FAQ:http://www.amav
is.org/amavis-faq.php3
AMaViS-HowTos:http://www.amavis.org/ho
wto/

>> Hmm, this says the score is zero, which indicates
to me that that
>> message was scanned (by what? - nothing? I dunno).
How did you
>> "disable" spam scanning? To disable spam
scanning try:
>
> bypass_spam_checks_maps  = (1);
>
> You may also desire:
> bypass_virus_checks_maps = (1);
> bypass_banned_checks_maps = (1);
> bypass_header_checks_maps = (1);

Yes, this is the method I used to disable the various
scanners.  I did
not have bypass_header_checks_maps set earlier, but I've set
it now and
still see the following when I send a test message without a
valid Date
field:

X-Amavis-Alert: BAD HEADER SECTION, Missing required header
field:
"Date"

Shouldn't that also disappear if I truly have header tests
disabled?
This is a side issue, of course, but it is curious.
 
> Also, if you want to keep existing headers, add:
> $remove_existing_spam_headers  = 0;

Ah, that was part of what I was looking for.  Thanks!

> X-Spam headers are only inserted if the domain is
considered local. If
> it makes no difference, try:
>
> local_domains_maps = ( [''] );
> 
> but it may not matter if spam and virus scanning are
disabled.

I had tried "local_domains_maps = ();" without
luck.  Alas, your
suggestion also does not appear to have worked.

By the way, I'm running 2.6.0-rc1, not -pre1 as stated
earlier.



This e-mail transmission contains information that is
confidential and may be 
privileged. It is intended only for the addressee(s) named
above. If you receive 
this e-mail in error, please do not read, copy or
disseminate it in any manner. 
If you are not the intended recipient, any disclosure,
copying, distribution or 
use of the contents of this information is prohibited.
Please reply to the 
message immediately by informing the sender that the message
was misdirected. 
After replying, please erase it from your computer system.
Your assistance in 
correcting this error is appreciated.


------------------------------------------------------------
-------------
Check out the new SourceForge.net Marketplace.
It's the best place to buy or sell services for
just about anything Open Source.
http://ad.doubleclick.net/clk;164216
239;13503038;w?http://sf.net/marketplace
_______________________________________________
AMaViS-user mailing list
AMaViS-userlists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/amavis-user

AMaViS-FAQ:http://www.amav
is.org/amavis-faq.php3
AMaViS-HowTos:http://www.amavis.org/ho
wto/

> I had tried "local_domains_maps = ();" without
luck.  Alas, your
> suggestion also does not appear to have worked.
> 
> By the way, I'm running 2.6.0-rc1, not -pre1 as stated
earlier.

Ah, found it.  The policy defined in the sql policy table
for the
servers in the DMZ was overriding the settings in the config
file for
the internal 'dummy' amavisd.  I should be able to get
around that by
setting lookup_sql_dsn to undef and leaving storage_sql_dsn set.
 
Thanks for the help!  $remove_existing_spam_headers answered
the other
half of the issue, as well.

This e-mail transmission contains information that is
confidential and may be 
privileged. It is intended only for the addressee(s) named
above. If you receive 
this e-mail in error, please do not read, copy or
disseminate it in any manner. 
If you are not the intended recipient, any disclosure,
copying, distribution or 
use of the contents of this information is prohibited.
Please reply to the 
message immediately by informing the sender that the message
was misdirected. 
After replying, please erase it from your computer system.
Your assistance in 
correcting this error is appreciated.


------------------------------------------------------------
-------------
Check out the new SourceForge.net Marketplace.
It's the best place to buy or sell services for
just about anything Open Source.
http://ad.doubleclick.net/clk;164216
239;13503038;w?http://sf.net/marketplace
_______________________________________________
AMaViS-user mailing list
AMaViS-userlists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/amavis-user

AMaViS-FAQ:http://www.amav
is.org/amavis-faq.php3
AMaViS-HowTos:http://www.amavis.org/ho
wto/

Scott,

> > I had tried "local_domains_maps =
();" without luck.  Alas, your
> > suggestion also does not appear to have worked.
>
> Ah, found it.  The policy defined in the sql policy
table for the
> servers in the DMZ was overriding the settings in the
config file for
> the internal 'dummy' amavisd.  I should be able to get
around that by
> setting lookup_sql_dsn to undef and leaving storage_sql_dsn set.

Right. Thanks for tracking it down. On startup the log
should report:

  amavis[46211]: ANTI-VIRUS code      NOT loaded
  amavis[46211]: ANTI-SPAM code       NOT loaded
  amavis[46211]: ANTI-SPAM-SA code    NOT loaded
  amavis[46211]: Unpackers code       NOT loaded

Consider also:

  $bypass_decode_parts = 1;

which turns off also the first-step MIME decoding.

And if only SQL logging is needed (for other server's
penpals benefit),
the penpals checks by a dummy amavisd can also be turned
off:

  $penpals_bonus_score = 0;


> Thanks for the help!  $remove_existing_spam_headers
answered the other
> half of the issue, as well.

The value of $remove_existing_spam_headers is irrelevant if
anti-spam code
is not loaded, existing X-Spam* headers are not stripped if
replacements
are not intended to be provided.

> local_domains_maps = ( [''] );

Unnecessarily complicated, the local_domains_maps=()
suffices.


> But some divisions within the super maga corp bar.com
don't route
> outbound mail back across this private line... they
route mail
> across the internet. [...]
> This is only an issue with respect to amavis' penpal
support.

It might matter if they use SPF, or if you are DKIM-signing
their
outbound mail for them.

> We use sql logging, so my solution is to insert a dummy
amavisd on one
> of the common internal mail exchanger hosts and log all
traffic to a
> common database.

So you _are_ in position to get hold of such traffic, why
not just
re-route it to wherever you like.

  Mark

------------------------------------------------------------
-------------
Check out the new SourceForge.net Marketplace.
It's the best place to buy or sell services for
just about anything Open Source.
http://ad.doubleclick.net/clk;164216
239;13503038;w?http://sf.net/marketplace
_______________________________________________
AMaViS-user mailing list
AMaViS-userlists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/amavis-user

AMaViS-FAQ:http://www.amav
is.org/amavis-faq.php3
AMaViS-HowTos:http://www.amavis.org/ho
wto/