Alerting on exe file download size

Now this is an excellent idea! Glad you brought it up. You
ask several
questions though, I'll split them into separate threads
here:


Martin Holste wrote:
> A colleague and I were discussing the value of a sig
that would match on
> an exe download of less than a given size.  At first we
thought we could
> use the dsize rule param to check for that, but we
quickly saw that it
> wouldn't work across packet boundaries, which is
exactly what we want. 
> It also seems that plenty of exe downloads don't fill
up the first
> payload packet even though more data follows, so we
can't make a sig
> that uses dsize to check for an exe of less than the
packet MTU.  It
> seems that the only way to identify the start and end
would be to search
> the stream for the actual PE header and footer.  We'd
like to start
> alerting on streams containing exe code with a payload
of anything under
> say, 50k.  We see trojan downloaders under 1k commonly.
 Any ideas on
> how we could match on a file size in a stream?  I can
think of two
> possibilities, though I'd like a discussion to flesh
them out. 
> 

An rfc compliant http server will report the size of the
binary in the
way in the return headers, like so:


HTTP/1.1 200 OK
Date: Wed, 07 Nov 2007 23:56:05 GMT
Server: Apache
Last-Modified: Wed, 07 Nov 2007 23:55:46 GMT
ETag: "24d26c-79f0-7cf3a480"
Accept-Ranges: bytes
Content-Length: 31216
Keep-Alive: timeout=15, max=99
Connection: Keep-Alive
Content-Type: application/x-msdos-program

That's a get request for a binary that the web server knows
is a binary
by filename. If it didn't identify it'd say plain/text for
the
content-type, but otherwise the same.

It's possible to get the size of the binary on the way from
the
content-length field. If this is being served by a
non-altered http
server that should be valid. In this case a 31k binary.

We could either pcre to look for a high number, or byte_test
it.
byte_test would be far more efficient.

So... if we set a flow bit from the get if it looks like a
binary then a
second rule to look for the file size it could work.

One flaw is that many of the servers serving spyware/malware
aren't full
blown http servers, and don't report the valid headers as
apache does.
So this certainly won't be fool-proof, but it'll be
interesting...

I'll do up a sig for this in a few minutes and we can test.
If anyone
has comments or caveats please speak up!

I'll split your other questions to a new thread, they
deserve separate
discussion.

Matt


--------------------------------------------
Matthew Jonkman
Bleeding Edge Threats
US Phone 765-429-0398
US Fax 312-264-0205
AUS Phone 61-42-4157-491
AUS Fax 61-29-4750-026
http://www.bleedingthr
eats.net
--------------------------------------------

PGP: http:/
/www.bleedingthreats.com/mattjonkman.asc


_______________________________________________
Bleeding-sigs mailing list
Bleeding-sigsbleedingthreats.net
http://lists.bleedingthreats.net/cgi-bin/
mailman/listinfo/bleeding-sigs

Martin Holste wrote:
> 
> The first would be to create signatures based on the
Air Force's
> Foremost program which will do file header and footer
carving.  Using
> those signatures to mark file headers and footers, you
could then use
> flowbits to record the start and end of an exe.  The
part I'm fuzzy on
> is how to calculate the distance between using native
Snort stream
> reassembly features.

I haven't played with foremost but hear good of it. It could
be an
interesting thing to use.

Header carving is a good thing to consider though. It'd be
heavy load
since the header can vary within limits, so we're talking
pcre. But with
proper qualification it is possible we could keep it under
control.

> 
> The other thing I can think of would be a dynamic
pre-processor for exe
> files.  That would be great, because then you could
build in all kinds
> of matches for various packers, etc.  It could also
incorporate code
> from Foremost.  The ClamAV spp is a start, but it
doesn't do stream
> processing and won't identify the beginning and end of
a file en-route. 
> An spp that could analyze files in-stream would be very
valuable, and I
> don't think it would overburden the sensor since it
wouldn't have to
> analyze the entire stream, just the part carved out
between the file
> header and footer.

The idea of a preproc I think is the best way for the long
term. That's
an excellent idea really. And I hope maybe we could recruit
Will and
Victor to help out making one up since they're pretty
familiar with the
concept from their work in the clamav preproc.

If we could have the preproc report on preset parameters
like size or
whether it appears to be packed with one of the malicious
packers, we
could then get an alert or a block with far less load than
trying to do
it in rules...

Anyone qualified to write something like that interested?

> 
> Has anyone heard if Snort 3.0 will have features like
this?
> 


As far as I know it doesn't have this exact thing on the
drawing board,
but it is more modular making this easier to make happen.
but 3.0's a
ways off yet before production level code is available.

matt

> 
> 
> 
> 
> 
> 
>
------------------------------------------------------------
------------
> 
> _______________________________________________
> Bleeding-sigs mailing list
> Bleeding-sigsbleedingthreats.net
> http://lists.bleedingthreats.net/cgi-bin/
mailman/listinfo/bleeding-sigs

-- 
--------------------------------------------
Matthew Jonkman
Bleeding Edge Threats
US Phone 765-429-0398
US Fax 312-264-0205
AUS Phone 61-42-4157-491
AUS Fax 61-29-4750-026
http://www.bleedingthr
eats.net
--------------------------------------------

PGP: http:/
/www.bleedingthreats.com/mattjonkman.asc


_______________________________________________
Bleeding-sigs mailing list
Bleeding-sigsbleedingthreats.net
http://lists.bleedingthreats.net/cgi-bin/
mailman/listinfo/bleeding-sigs

As you mention, sensing relative places within the stream as
a whole
isn't possible. It's an unfortunate flaw of Snort. Flowbits
goes some
way to helping mitigate that, but doesn't solve it
completely.

What would be ideal is a way to specify
distance/depth/offset in
relation to the stream and not the packet. Or maybe a way to
say we want
this rule to only be applied to the reassembled stream, or
just to
packets, or both. Something like flow:reassembled_only;

But that assumes that you also have the entire stream
reassembled and in
memory the entire time. That's a huge load problem of
course, as well as
a problem when running inline, etc.

I don't know if this will be or if it even IS solvable for
3.0. My guess
would be not, but maybe the SF guys can comment there.

Matt

Martin Holste wrote:
> A colleague and I were discussing the value of a sig
that would match on
> an exe download of less than a given size.  At first we
thought we could
> use the dsize rule param to check for that, but we
quickly saw that it
> wouldn't work across packet boundaries, which is
exactly what we want. 
> It also seems that plenty of exe downloads don't fill
up the first
> payload packet even though more data follows, so we
can't make a sig
> that uses dsize to check for an exe of less than the
packet MTU.  It
> seems that the only way to identify the start and end
would be to search
> the stream for the actual PE header and footer.  We'd
like to start
> alerting on streams containing exe code with a payload
of anything under
> say, 50k.  We see trojan downloaders under 1k commonly.
 Any ideas on
> how we could match on a file size in a stream?  I can
think of two
> possibilities, though I'd like a discussion to flesh
them out. 
> 
> The first would be to create signatures based on the
Air Force's
> Foremost program which will do file header and footer
carving.  Using
> those signatures to mark file headers and footers, you
could then use
> flowbits to record the start and end of an exe.  The
part I'm fuzzy on
> is how to calculate the distance between using native
Snort stream
> reassembly features.
> 
> The other thing I can think of would be a dynamic
pre-processor for exe
> files.  That would be great, because then you could
build in all kinds
> of matches for various packers, etc.  It could also
incorporate code
> from Foremost.  The ClamAV spp is a start, but it
doesn't do stream
> processing and won't identify the beginning and end of
a file en-route. 
> An spp that could analyze files in-stream would be very
valuable, and I
> don't think it would overburden the sensor since it
wouldn't have to
> analyze the entire stream, just the part carved out
between the file
> header and footer.
> 
> Has anyone heard if Snort 3.0 will have features like
this?
> 
> --Martin
> 
> 
> 
> 
> 
> 
> 
>
------------------------------------------------------------
------------
> 
> _______________________________________________
> Bleeding-sigs mailing list
> Bleeding-sigsbleedingthreats.net
> http://lists.bleedingthreats.net/cgi-bin/
mailman/listinfo/bleeding-sigs

-- 
--------------------------------------------
Matthew Jonkman
Bleeding Edge Threats
US Phone 765-429-0398
US Fax 312-264-0205
AUS Phone 61-42-4157-491
AUS Fax 61-29-4750-026
http://www.bleedingthr
eats.net
--------------------------------------------

PGP: http:/
/www.bleedingthreats.com/mattjonkman.asc


_______________________________________________
Bleeding-sigs mailing list
Bleeding-sigsbleedingthreats.net
http://lists.bleedingthreats.net/cgi-bin/
mailman/listinfo/bleeding-sigs

Matt Jonkman wrote:

> I'll do up a sig for this in a few minutes and we can
test. If anyone
> has comments or caveats please speak up!

GET with the Range: header.

a) might false this occasionally
b) if this signature works, it's not really hard to make
your non-rfc-compliant
malware "webserver" return the exe in bytes.. oh..
9688374-9995860.  I would
fully expect that as a countermeasure.

But otherwise, it's a nice little idea.



-- 
Erik Fichtner; Unix Ronin

Why is our 'democracy' looking more and more like a game of
Calvinball?


_______________________________________________
Bleeding-sigs mailing list
Bleeding-sigsbleedingthreats.net
http://lists.bleedingthreats.net/cgi-bin/
mailman/listinfo/bleeding-sigs

So, as discussed we can likely make a sig to pull the
reported size out
of the header. Byte_test I don't see is an option because we
don't know
how long the number will be. Could be 2 bytes, could be 10,
byte_test
can't deal with that. So we'll have to go pcre I think.
(Other ideas?
Please!)

We still need to figure out the size threshold, but lets
assume for now
a binary under 1 MB is suspicious. So we're looking for a
content-length
with less than a 7 digit number. 999,999 bytes or less.

pcre:"/x0dx0aContent-Length:
d{0,6}x0dx0a/";

That look sane to everyone? A rule like:

alert tcp $EXTERNAL_NET $HTTP_PORTS -> $HOME_NET any
(msg:"BLEEDING-EDGE
POLICY Binary Download Smaller than 1 MB -- Likely
Hostile";
flow:established,from_server;
flowbits:isset,BE.http.binary;
pcre:"/x0dx0aContent-Length:
d{0,6}x0dx0a/";
classtype:policy-violation; sid:2007671; rev:1;)

Next we need a way to prequalify the return as a likely
binary. We can
use the content-type as ONE qualifier, but that only works
if the
extension is something the web server KNOWS is a binary.
Otherwise it'll
be plain/text.

We can use flowbits to do this since we'll have more than
one trigger.

alert tcp $EXTERNAL_NET $HTTP_PORTS -> $HOME_NET any
(msg:"BLEEDING-EDGE
POLICY Likely Binary in HTTP by Type Flowbit";
flow:established,from_server;
flowbits:isnotset,BE.http.binary;
content:"HTTP/1"; depth:6; content:"|0d
0a|Content-Type: application/";
nocase; flowbits:noalert; flowbits:set,BE.http.binary;
classtype:not-suspicious; sid:2007670; rev:1;)

What else can we use to reasonably suspect the returning
content is a
binary?

Posting these momentarily, please test for me!

Matt

Martin Holste wrote:
> A colleague and I were discussing the value of a sig
that would match on
> an exe download of less than a given size.  At first we
thought we could
> use the dsize rule param to check for that, but we
quickly saw that it
> wouldn't work across packet boundaries, which is
exactly what we want. 
> It also seems that plenty of exe downloads don't fill
up the first
> payload packet even though more data follows, so we
can't make a sig
> that uses dsize to check for an exe of less than the
packet MTU.  It
> seems that the only way to identify the start and end
would be to search
> the stream for the actual PE header and footer.  We'd
like to start
> alerting on streams containing exe code with a payload
of anything under
> say, 50k.  We see trojan downloaders under 1k commonly.
 Any ideas on
> how we could match on a file size in a stream?  I can
think of two
> possibilities, though I'd like a discussion to flesh
them out. 
> 
> The first would be to create signatures based on the
Air Force's
> Foremost program which will do file header and footer
carving.  Using
> those signatures to mark file headers and footers, you
could then use
> flowbits to record the start and end of an exe.  The
part I'm fuzzy on
> is how to calculate the distance between using native
Snort stream
> reassembly features.
> 
> The other thing I can think of would be a dynamic
pre-processor for exe
> files.  That would be great, because then you could
build in all kinds
> of matches for various packers, etc.  It could also
incorporate code
> from Foremost.  The ClamAV spp is a start, but it
doesn't do stream
> processing and won't identify the beginning and end of
a file en-route. 
> An spp that could analyze files in-stream would be very
valuable, and I
> don't think it would overburden the sensor since it
wouldn't have to
> analyze the entire stream, just the part carved out
between the file
> header and footer.
> 
> Has anyone heard if Snort 3.0 will have features like
this?
> 
> --Martin
> 
> 
> 
> 
> 
> 
> 
>
------------------------------------------------------------
------------
> 
> _______________________________________________
> Bleeding-sigs mailing list
> Bleeding-sigsbleedingthreats.net
> http://lists.bleedingthreats.net/cgi-bin/
mailman/listinfo/bleeding-sigs

-- 
--------------------------------------------
Matthew Jonkman
Bleeding Edge Threats
US Phone 765-429-0398
US Fax 312-264-0205
AUS Phone 61-42-4157-491
AUS Fax 61-29-4750-026
http://www.bleedingthr
eats.net
--------------------------------------------

PGP: http:/
/www.bleedingthreats.com/mattjonkman.asc


_______________________________________________
Bleeding-sigs mailing list
Bleeding-sigsbleedingthreats.net
http://lists.bleedingthreats.net/cgi-bin/
mailman/listinfo/bleeding-sigs