Phases & SecDefaultAction

Am 13.04.2007 um 13:43 schrieb Marc Stern:
> I'm not sure I understand clearly the usage of the
"phase" in the  
> "SecDefaultAction" directive.

You have to read the documentation very carefully. There are
a lot of  
hints/details that need to
be taken care of.

The following comments on your questions are mostly my short
 
experience mit mod_sec, if I am
wrong with any of it, I'd appreciate corrections.

> 1. If I define a phase in the
"SecDefaultAction", like
>     SecDefaultAction "phase: 
>
2,t:replaceNulls,t:compressWhitespace,t:lowercase,log,deny,s
tatus:500"
> Does it set the default actions for phase 2 - and so I
can do it  
> also for phase 1,3, 4 - or does it set the default to
parse the  
> rules only in phase 2 ?


ModSec-rules consist of a selector, an operator, and an
action. The  
operator is evaluated on the
properties chosen by the selector and in case the operator
returns  
true, the action is fired. Some
of the actions (as tranformations, for example) will be
processed  
before the execution of the operator.
The phase-action will ensure that the rule is only executed
in that  
phase.

When specifying a rule, the action can be left out, in which
case a  
default-action will be inherited.
This default-action is defined by SecDefaultAction. For
example:

    SecDefaultAction "phase:2,deny,status:500"
    ...

    SecRule ARGS:p1 ^!d+$

This will result in the rule to use the action
"phase:2,deny,status: 
500", thus in phase-2 the parameter
"p1" is checked to be not an integer, in which
case it will rejected  
with status-code 500.

There is however a special thing to take care of:
Specifying transformations will *add* these to the list of
transforms  
that are to be executed. Thus in
the following

     SecDefaultAction
"phase:2,deny,status:500,t:compressWhitespace"
     ...
     SecRule ARGS:p1 evil "t:lowercase"

the "lowercase" is an additional transformation
and prior the check  
of p1 against "evil" the transformations
compressWhitespace AND lowercase are executed.

> 3. Another - maybe stupid - question is: aren't the
rules  
> implicitly linked to a phase ?
> Headers should be parsed during phase 1, body during
phase 2, etc.
> So, why do we have to provide this parameter ? When
specifying a  
> rule against headers, it cannot be parsed during phase
3, for  
> instance.

Yes, I see this in the same way. However some things are not
as  
implicit as they look in the first glance.
For example ARGS:p1 checks parameter p1 in the QUERY_STRING
(header)  
as well as the payload (body). Hopefully
there will be a split of that into GET_ARGS and POST_ARGS
sometime as  
suggested by Christian Folini.

> 4. And last, but not least, I saw that
"SecDefaultAction" - and  
> also rules - do not work inside locations.
> How do we specify one rule to be parsed in one location
and not in  
> another one ?

The Location-directive is processed for a request after
phase:1 has  
finished and before phase:2. Thus you cannot
specify location-specific rules that are phase:1-rules or
overwrite  
phase-1 behaviours by the use of the
Location-directive.

However, you can circumvent this by using the skip-action:

SecDefaultAction "phase:1,deny,status:500"

SecRule REQUEST_URI !^/cgi-bin/FormMail.pl$ skip:1,pass
SecRule &HTTP_USER_AGENT !^1$
...


This should deny a request to /cgi-bin/FormMail.pl that has
no user- 
agent set. (Pseudo-notation, not checked)

Regards,
     Chris

------------------------------------------------------------
-------------
This SF.net email is sponsored by DB2 Express
Download DB2 Express C - the FREE version of DB2 express and
take
control of your XML. No limits. Just data. Click to get it
now.
http://sourcefor
ge.net/powerbar/db2/
_______________________________________________
mod-security-users mailing list
mod-security-userslists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/mod-
security-users

> -----Original Message-----
> From: mod-security-users-bounceslists.sourceforge.net
[mailto:mod-
> security-users-bounceslists.sourceforge.net] On
Behalf Of Christian
> Bockermann
> Sent: Saturday, April 14, 2007 10:52 AM
> To: Marc Stern
> Cc: Mod Security
> Subject: Re: [mod-security-users] Phases &
SecDefaultAction
> 
> Am 13.04.2007 um 13:43 schrieb Marc Stern:
> > I'm not sure I understand clearly the usage of the
"phase" in the
> > "SecDefaultAction" directive.
> 
> You have to read the documentation very carefully.
There are a lot of
> hints/details that need to
> be taken care of.
> 
> The following comments on your questions are mostly my
short
> experience mit mod_sec, if I am
> wrong with any of it, I'd appreciate corrections.
> 
> > 1. If I define a phase in the
"SecDefaultAction", like
> >     SecDefaultAction "phase:
> >
2,t:replaceNulls,t:compressWhitespace,t:lowercase,log,deny,s
tatus:500"
> > Does it set the default actions for phase 2 - and
so I can do it
> > also for phase 1,3, 4 - or does it set the default
to parse the
> > rules only in phase 2 ?
> 
> 
> ModSec-rules consist of a selector, an operator, and an
action. The
> operator is evaluated on the
> properties chosen by the selector and in case the
operator returns
> true, the action is fired. 
[Ryan Barnett] The only exception to this would be the
"validate"
operators which only trigger on false conditions.  For
example, with
validateByteRange you need to specify a whitelist of
allowed
characters.  If the operator sees a non-match/false return,
then it
takes action.  The validateSchema/validateDTD operators are
similar in
that they attempt to parse the data and they only trigger if
there is a
failure.

> Some
> of the actions (as tranformations, for example) will be
processed
> before the execution of the operator.
> The phase-action will ensure that the rule is only
executed in that
> phase.
> 
> When specifying a rule, the action can be left out, in
which case a
> default-action will be inherited.
> This default-action is defined by SecDefaultAction. For
example:
> 
>     SecDefaultAction
"phase:2,deny,status:500"
>     ...
> 
>     SecRule ARGS:p1 ^!d+$
> 
> This will result in the rule to use the action
"phase:2,deny,status:
> 500", thus in phase-2 the parameter
> "p1" is checked to be not an integer, in
which case it will rejected
> with status-code 500.
> 
> There is however a special thing to take care of:
> Specifying transformations will *add* these to the list
of transforms
> that are to be executed. Thus in
> the following
> 
>      SecDefaultAction
"phase:2,deny,status:500,t:compressWhitespace"
>      ...
>      SecRule ARGS:p1 evil "t:lowercase"
> 
> the "lowercase" is an additional
transformation and prior the check
> of p1 against "evil" the transformations
> compressWhitespace AND lowercase are executed.
[Ryan Barnett] This is true.  It does mention this in the
Reference
Manual however it probably needs to be more clearly defined.
 The
transformation functions are additive.  If you don't want to
apply the
inherited transformation functions (or if you want to start
from
scratch) you specify "t:none" or
"t:none,t:whatever,t:whateverelse..."

> 
> > 3. Another - maybe stupid - question is: aren't
the rules
> > implicitly linked to a phase ?
> > Headers should be parsed during phase 1, body
during phase 2, etc.
> > So, why do we have to provide this parameter ?
When specifying a
> > rule against headers, it cannot be parsed during
phase 3, for
> > instance.
> 
> Yes, I see this in the same way. However some things
are not as
> implicit as they look in the first glance.
> For example ARGS:p1 checks parameter p1 in the
QUERY_STRING (header)
> as well as the payload (body). Hopefully
> there will be a split of that into GET_ARGS and
POST_ARGS sometime as
> suggested by Christian Folini.
[Ryan Barnett] This is an important point where there can be
false
positives.  If you assign the improper phase, there is a
chance that
they Variable will be empty and thus not match a rule.  This
is not
because the data wasn't present but that it wasn't available
in the
phase specified.  I believe that we will be working on some
error
notices for improper phase specification that will alert
upon startup
(similar to what you might see from RegEx compilation
problems).

> 
> > 4. And last, but not least, I saw that
"SecDefaultAction" - and
> > also rules - do not work inside locations.
> > How do we specify one rule to be parsed in one
location and not in
> > another one ?
> 
> The Location-directive is processed for a request after
phase:1 has
> finished and before phase:2. Thus you cannot
> specify location-specific rules that are phase:1-rules
or overwrite
> phase-1 behaviours by the use of the
> Location-directive.
> 
> However, you can circumvent this by using the
skip-action:
> 
> SecDefaultAction "phase:1,deny,status:500"
> 
> SecRule REQUEST_URI !^/cgi-bin/FormMail.pl$
skip:1,pass
> SecRule &HTTP_USER_AGENT !^1$
> ...
> 
> 
> This should deny a request to /cgi-bin/FormMail.pl that
has no user-
> agent set. (Pseudo-notation, not checked)
[Ryan Barnett] While it may not be as pretty on the eyes,
you can
translate any Apache scope locations into ModSecurity
chained rulesets.
You would just need to take any existing
Location/LocationMatch Apache
directives and then use SecRules that use REQUEST_URI to
match the
Location data.  If you use chained SecRules, you can then
specify them
in phase:1 if desired.  So, to over-ride phase:1 rules for
specific
locations, you can either do what Chris is suggesting and
place them
before the rule that is causing the problem and use skip. 
Or,
alternatively, you could specify them within a custom rules
file and
then use SecRuleRemoveById to disable the rules that are
causing
problems.

------------------------------------------------------------
-------------
This SF.net email is sponsored by DB2 Express
Download DB2 Express C - the FREE version of DB2 express and
take
control of your XML. No limits. Just data. Click to get it
now.
http://sourcefor
ge.net/powerbar/db2/
_______________________________________________
mod-security-users mailing list
mod-security-userslists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/mod-
security-users

Am 17.04.2007 um 08:33 schrieb Marc Stern:

> Ryan Barnett wrote:
>>> -----Original Message----- From:
mod-security-users- 
>>> bounceslists.sourceforge.net [mailto:mod-
security-users- 
>>> bounceslists.sourceforge.net] On Behalf Of
Christian Bockermann  
>>> Sent: Saturday, April 14, 2007 10:52 AM To:
Marc Stern Cc: Mod  
>>> Security Subject: Re: [mod-security-users]
Phases &  
>>> SecDefaultAction Am 13.04.2007 um 13:43 schrieb
Marc Stern:
>>>> I'm not sure I understand clearly the usage
of the "phase" in  
>>>> the "SecDefaultAction"
directive.
>>> You have to read the documentation very
carefully. There are a  
>>> lot of hints/details that need to be taken care
of. The following  
>>> comments on your questions are mostly my short
experience mit  
>>> mod_sec, if I am wrong with any of it, I'd
appreciate corrections.
>>>> 1. If I define a phase in the
"SecDefaultAction", like  
>>>> SecDefaultAction
"phase:2,t:replaceNulls,..." Does it set the  
>>>> default actions for phase 2 - and so I can
do it also for phase  
>>>> 1,3, 4 - or does it set the default to
parse the rules only in  
>>>> phase 2 ?
>>> ModSec-rules consist of a selector, an
operator, and an action.  
>>> The operator is evaluated on the properties
chosen by the  
>>> selector and in case the operator returns true,
the action is fired.
>> [Ryan Barnett] The only exception to this would be
the "validate"  
>> operators which only trigger on false conditions.
For example,  
>> with validateByteRange you need to specify a
whitelist of allowed  
>> characters. If the operator sees a non-match/false
return, then it  
>> takes action. The validateSchema/validateDTD
operators are similar  
>> in that they attempt to parse the data and they
only trigger if  
>> there is a failure.
> So, I should define 4 rules, one for each phase, right
?
> Like
>   SecDefaultAction
>
"phase:1,deny,status:500,t:compressWhitespace,t:UrlDeco
de"
>   SecDefaultAction
"phase:2,deny,status:500,t:compressWhitespace"
>   SecDefaultAction "phase:3,..."
>   SecDefaultAction "phase:2,..."
>

No. SecDefaultAction defines the actions that will be
executed by all  
rules specified in *subsequent*
directives, if these rules do not overwrite the default
action- 
behaviour. For example:

   SecDefaultAction
"phase:1,deny,status:500:t:urlDecode"
   SecRule QUERYSTRING attack
   ...


   SecDefaultAction "phase:2,pass,log,auditlog"
   SecRule ARGS:p1 !d{1,5}


The first SecRule creates a rule that will check the
querystring for  
the word "attack".
This rule is placed in Phase-1 (since it did not specify a
phase by  
it's own and the
default action was set to phase:1). It will deny a request
that  
matches the querystring
with "attack" and send back a status of 500.

The 2nd SecRule creates another rule, but since the
DefaultAction was  
changed to
	"phase:2,deny,status:500"
this rule is placed in Phase-2 and will only log a request
if it's  
parameter p1 is not
an integer (does not match d+).


> Can I regroup several rules, like phase:2-3, or
something equivalent ?

You don't need to. A rule that is defined to be in phase-2
is  
evaluated in phase-2 and there
is no reason why it should be evaluated in phase-3 for a
second time...


>> [Ryan Barnett] While it may not be as pretty on the
eyes, you can  
>> translate any Apache scope locations into
ModSecurity chained  
>> rulesets. You would just need to take any existing
Location/ 
>> LocationMatch Apache directives and then use
SecRules that use  
>> REQUEST_URI to match the Location data. If you use
chained  
>> SecRules, you can then specify them in phase:1 if
desired. So, to  
>> over-ride phase:1 rules for specific locations, you
can either do  
>> what Chris is suggesting and place them before the
rule that is  
>> causing the problem and use skip. Or,
alternatively, you could  
>> specify them within a custom rules file and then
use  
>> SecRuleRemoveById to disable the rules that are
causing problems.
> So, we can't have something like:
>
> <Location /app1>
> include rules1.conf
> include rules2.conf
> include rules3.conf
> </Location>
>
> <Location /app2>
> include rules1.conf
> </Location>
>
> <Location /app3>
> include rules1.conf
> include rules2.conf
> </Location>

You can certainly do that (if apache allows inclusion within
Location- 
directives, but the
included files can only contain rules of phase > 1. This
is due to  
the following:

When apache receives a request it runs through its in-memory
config,  
applying all GLOBAL
settings to that request for checking (e.g.
access-restriction- 
configs, script-alias-configs,...)

mod_security's phase-1 is very early in that process. It is
processed  
BEFORE the settings
of Location-directives are applied to the request. Thus, all
phase-1  
rules are loaded from
the config and get checked against the request.

After all this has been done, apache advances and applies
all  
LOCATION-specific settings
to a request (Location/LocationMatch). This is when
modsecurity's  
Phase-2 starts. Be aware
that until now, the phase-1-rules already have been
executed. Thus,  
overwriting/removing a
phase-1-rule within a Location-match cannot work.

I hope this will clearify a little bit and I don't get
beaten by  
Brian for my improper/incomplete
description of apache's req-processing with modsec 

You should perhaps carefully read over this:

	http://www.modsecurity.org/documentation/modse
curity-apache/2.1.0/ 
html-multipage/04-processing-phases.html


Regards,
     Chris


------------------------------------------------------------
-------------
This SF.net email is sponsored by DB2 Express
Download DB2 Express C - the FREE version of DB2 express and
take
control of your XML. No limits. Just data. Click to get it
now.
http://sourcefor
ge.net/powerbar/db2/
_______________________________________________
mod-security-users mailing list
mod-security-userslists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/mod-
security-users

Thanks for all explanations.
One last question:
>> Can I regroup several rules, like phase:2-3, or
something equivalent ?
> You don't need to. A rule that is defined to be in
phase-2 is 
> evaluated in phase-2 and there
> is no reason why it should be evaluated in phase-3 for
a second time...
"phase:2-3" was a bad example, but what about
"phase:1-2" for ARGS ?


Btw, I read (several times) the doc, but it seems there are
a lot of 
(powerful) undocumented features.

Marc

------------------------------------------------------------
-------------
This SF.net email is sponsored by DB2 Express
Download DB2 Express C - the FREE version of DB2 express and
take
control of your XML. No limits. Just data. Click to get it
now.
http://sourcefor
ge.net/powerbar/db2/
_______________________________________________
mod-security-users mailing list
mod-security-userslists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/mod-
security-users

Am 17.04.2007 um 10:03 schrieb Marc Stern:
>>> Can I regroup several rules, like phase:2-3, or
something  
>>> equivalent ?
>> You don't need to. A rule that is defined to be in
phase-2 is  
>> evaluated in phase-2 and there
>> is no reason why it should be evaluated in phase-3
for a second  
>> time...
> "phase:2-3" was a bad example, but what about
"phase:1-2" for ARGS ?

Should work. It will check ARGS against the header-fields  
(QUERY_STRING-ARGS) in phase:1 and check
ARGS against BOTH (header- AND payload-fields) in phase:2

> Btw, I read (several times) the doc, but it seems there
are a lot  
> of (powerful) undocumented features.

Yes, that's a pity. There are a lot of traps one can step
into.

Regards,
    Chris



------------------------------------------------------------
-------------
This SF.net email is sponsored by DB2 Express
Download DB2 Express C - the FREE version of DB2 express and
take
control of your XML. No limits. Just data. Click to get it
now.
http://sourcefor
ge.net/powerbar/db2/
_______________________________________________
mod-security-users mailing list
mod-security-userslists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/mod-
security-users

I am not able to log to the console server when I just
upgraded from
apache-2.2.3 to apache-2.2.4 and modsecurity 1.9 to
modsecurity 2.1.  On
one of our servers

I am not getting access denied, everything is logging to the
appropriate
directory on the local server, but it isn't sending the
information to
the security console.  I am also getting an error on the
console under
administrative events.

Here is the error on the console page.

Failed processing RPC request: PreparedStatementCallback;
SQL [INSERT
INTO http_transactions ( sensorid, sensor_txid, webappid,
sessionid,
userid, data_type, data_source, data_offset, data_length,
data_hash,
create_time, tx_time, hostname, remote_addr, remote_host,
remote_port,
local_user, request_method, request_uri, query_string,
protocol,
request_content_length, request_content_type,
response_status,
response_content_type, referer, user_agent, tx_duration,
tx_request_duration, tx_processing_duration,
tx_response_duration,
is_valid, was_blocked, alert_severity, alert_message, keep )
VALUES ( ?,
?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?,
?, ?, ?, ?,
?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?)]; The resulting value is
outside the
range for the data type INTEGER.; nested exception is
org.apache.derby.impl.jdbc.EmbedSQLException: The resulting
value is
outside the range for the data type INTEGER.

I can only assume this is only coming from the first
modsecurity 2.1
server we have so far. Since all other servers are logging
as they
should be.


Here is the configuration of the 2.1 server regarding
logging.

SecAuditEngine RelevantOnly
SecAuditLogType Concurrent
SecAuditLogParts "ABCDEFGHZ"

SecAuditLogStorageDir "/var/log/modsecure/data/"
SecAuditLog
|/usr/local/apache2.2.4/bin/modsec-auditlog-collector.pl
/var/log/modsecure/data/ /var/log/modsecure/index"

What am I missing here?

------------------------------------------------------------
-------------
This SF.net email is sponsored by DB2 Express
Download DB2 Express C - the FREE version of DB2 express and
take
control of your XML. No limits. Just data. Click to get it
now.
http://sourcefor
ge.net/powerbar/db2/
_______________________________________________
mod-security-users mailing list
mod-security-userslists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/mod-
security-users

Christian Bockermann wrote:
>>>> Can I regroup several rules, like
"phase:1-2" for ARGS ?
> Should work. It will check ARGS against the
header-fields 
> (QUERY_STRING-ARGS) in phase:1 and check
> ARGS against BOTH (header- AND payload-fields) in
phase:2
Which syntax do I have to use ?
    "phase:1-2" does only work for phase 1
    "phase:1,phase:2" does only work for phase 2

If I don't specify any phase, it should work, but will it
parse ARGS 
also during phase 3 & 4 ?
It would be a pity to encode lines twice :-(

------------------------------------------------------------
-------------
This SF.net email is sponsored by DB2 Express
Download DB2 Express C - the FREE version of DB2 express and
take
control of your XML. No limits. Just data. Click to get it
now.
http://sourcefor
ge.net/powerbar/db2/
_______________________________________________
mod-security-users mailing list
mod-security-userslists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/mod-
security-users

To clarify my previous post, I tried several syntaxes to
parse both 
query/post args:
    1. SecRule ARGS "..."
    2. SecRule ARGS "..." "phase:1-2"
    3. SecRule ARGS "...""phase:1,
phase:2"
    4. SecRule ARGS "..." "phase:1"
        SecRule ARGS "..." "phase:2"

1 & 2 only work for query args.
3 only works for post args.
Only 4 works for both.


If I look to core rules, I see that they use, to block SQL
Injection, 
"SecDefaultAction log,pass,phase:2,..."
So, it seems they only check POST args ???

------------------------------------------------------------
-------------
This SF.net email is sponsored by DB2 Express
Download DB2 Express C - the FREE version of DB2 express and
take
control of your XML. No limits. Just data. Click to get it
now.
http://sourcefor
ge.net/powerbar/db2/
_______________________________________________
mod-security-users mailing list
mod-security-userslists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/mod-
security-users