List Info

Thread: benchmarking the web app scanners




benchmarking the web app scanners
user name
2006-01-21 01:47:41
Thomas, Thanks for you kind comments 

I was going to wait a little bit before I posted this (
http://owas
p.net/forums/428/ShowPost.aspx) information here
(webappsecsecurityfocus.com), but since the genie is now out of
the
bottle, here are a series of quick comments:

1) this tool has been sponsored by Foundstone, BUT (and it
is a bit
but) it is being released under the Owasp .Net Project and
an Open
Source Licence. So Kudos for Foundstone for doing this and I
hope they
get good exposure from it

2) The main objective of the tool is to create dynamic
websites based
on XML files which will 'map' to a database containing
hundreds of
different vulnerabilities (some simple to detect/exploit,
some harder)
covering multiple languages and web development
architectures (for
example navigation: Html, JavaScript, Flash, Java, etc...)

3) There are many ways this tool can be used, here are just
a couple
starting ideas:
    a) As a training tool since it allows the creation of
multiple
websites with multiple variations of vulnerabilities
    b) As a Web Application Honeypot (since we are able to
create
dynamic ( i.e. false) websites and track / monitor in
real-time all
requests made)
    c) As a test ground for newly discovered vulnerabilities
types and
its exploit vectors
    d) As a benchmark for Web Security Scanners

4) The Web Security Scanner benchmarking and testing is the
most
obvious short-term application for this tool, but I think
that as it
evolves the others will be proven to be as (if not more)
valuable

5) On the Web Security Scanner issue:

    a) My main hope is that the Web Security Scanner
Companies will
see this tool as an opportunity and work with the Owasp .Net
project
(and other groups that want to be involved) in a productive
and
constructive way.

    b) Although in the short term some Web Security Scanners
might
have some bad results (well, at least when compared with
what their
Marketing machine publishes  in the
medium term, as they adapt and
improve their scanning techniques, everybody will benefit

    c) One of the core objectives of the tool (when thinking
about
benchmarking Web Security Scanners) is to be able to create
real and
measurable metrics. For example:
        - Scanner X was able to detect 65% of the
vulnerabilities
where Scanner Y was able to detect 90%
        - Scanner X made 10000 to detect those 65% (over a
period of
16h) where scanner Y made 4000 request (over a period of
10h)
        - 20% of Scanner X results where false positives,
where
Scanner Y had 50% false positives
        - Scanner X was able to deal with Html and
JavaScript
navigation, Scanner Y was able to deal with Html, JavaScript
and
Flash, and both where NOT able to deal with Java based
navigation
systems
        - Scanner X is not able to go more than 40 levels
deep,
Scanner Y is able to go up at least 100 levels deep (if note
more)
        - etc, etc, etc.

    d) There will be two main types of tests that can be
done in the
short term:
        i) provive the links to all different types of
vulnerabilities
existent in the database, and see how many can the scanner
correctly
identify? and
        ii) when multiple types of website architectures and
navigation techniques are used, how many vulnerabilities is
the
scanner able to detect?

    e) In order to test (and further improve the tool) I
want to take
this opportunity to ask the Web Application Security
Scanners that
subscribe to this list (which I believe all do) to give the
Owasp .Net
project a temporary licence to they product so that we can
use it
during  development and during some basic benchmarking that
we might
do (and NO, I will not sign an NDA that doesn't allow me to
publish
the data collected, in fact I will not sign ANY NDA with ANY
web
application security scanner company)

    f) note that at the moment I (Dinis) have no plans to do
a full
benchmarking exercise since I don't have the time required,
but I know
of at least one group of experienced security consultants
which is
starting such project (and I will be supporting them). If
anybody else
is interested in doing a similar benchmarking project please
contact
me directly

6) Regarding how the tool works, here is a brief technical
description:

    There are two main components: A webserver (which can be
IIS or a
custom webserver) and a GUI application (written in C# 2.0).
The Gui
Application is responsible for handling all mappings (from
the virtual
requests to the actual pages on disk) and there is an
unmanaged C++
DLL loaded by both which implements a Shared Memory to send
and
receive data between them.

    The current version is hardcoded to IIS, although in the
code
there is support for using a custom .Net webserver. This IIS
version
uses an HttpHander to capture all requests and communicate
with the
GUI Application (called SiteGeneratorGUI). The previous
version used
C++ Detours to hook all sorts of functions in either IIS or
the Custom
Webserver (this worked ok, but ultimately I decided to use
IIS since
it was much more robust and scalable)

    The dynamic websites are defined by XML files like this
(which are
edited on the GUI Application using the WYSIWYG Altova
Authentic
Browser Object (SPS files created via Altova's StyleVision
application)):

     <?xml version="1.0"
encoding="utf-8" ?>
    <SiteGenerator name="SiteGenerator Demo"
xmlns:ipo="http://www.altova.com
/IPO" xmlns="
http://w
ww.xmlspy.com/schemas/orgchart"
xmlnssi=&q
uot;h
ttp://www.w3.org/2001/XMLSchema-instance">
         <site>
             <folder name="">
                <file
mappedTo="aspx/Default.aspx"
name="HelloWorld.aspx" />
                <folder name="htm" />
                <folder name="aspx">
                    <file
mappedTo="aspx/pages.htm"
name="pages.htm" />
                    <file
mappedTo="aspx/xss.aspx" name="
xss.aspx" />
                    <file
mappedTo="aspx/SqlInjection_Easy.aspx"
name="SqlInjection.aspx" />
                    <file
mappedTo="aspx/SqlInjection_Hard.aspx"
name=" SqlInjection2.aspx" />
                </folder>
                <folder name="flash">
                     <file
mappedTo="flash/cromas_xml.swf"
name="cromas_xml.swf" />
                     <file
mappedTo="flash/cromas_xml.htm"
name="menu.htm" />
                     <file
mappedTo="/flash/cromas_menu.xml"
name="cromas_menu.xml" />
                </folder>
             </folder>
        </site>
    </SiteGenerator>

    SiteGeneratorGUI.exe and IIS will map the virtual name
"HelloWorld.aspx" to the file on disk
"aspx/Default.aspx" . For
example:

        http://localhost/Hel
loWorld.aspx --> F:Owasp
SiteGeneratorSiteGenerator_ContentPagesaspxDefault.aspx

    So to create new websites all you need to do is to
create a new XML file

    And to create new vulnerabilities type, all you need to
create in
an Aspx page and map it to the xml file

7) the tool is still in Beta, so please be patient with it.
The code
is still a bit in mess, since there are multiple past
experiments in
there which I will need to clean up

8) What we need now is:
    a) comments on how the tool works
    b) new vulnerabilities (i.e. new Aspx pages)
    c) help in the development of the missing features
(namely an Xml
Metadata section to contain details about each
vulnerabilities (risk
rating, remediation, exploit difficulty level, etc...). I
also want to
add support for other types of languages and servers (at the
moment it
is limited to .Net Framework and IIS (I had ASP Classic in
one of the
earlier betas, but didn't include it in this one since it
was
implemented using the detours technique)). I have many more
ideas on
what is comming next, but first I want to make sure that
what this
version is fully functional and usable

I hope this make since (it's 1:30am here and I just wanted
to do a
quick brain dump with some ideas and pointers about
SiteGenerator)

Best regards

Dinis Cruz
Owasp .Net Project


On 20/01/06, thomas.joneshushmail.com < 
thomas.joneshushmail.com> wrote:
> This tool looks awesome. About time someone built this!
Has anyone
> got a plan in place to provide some independent,
objective results
> rather than the marketing hype I have to read from the
likes of
> eWeek?
>
> Dinis you rock !
>
>   http://owas
p.net/forums/428/ShowPost.aspx
>
>
>
> Concerned about your privacy? Instantly send FREE
secure email, no account required
>  http://www.hushmai
l.com/send?l=480
>
> Get the best prices on SSL certificates from Hushmail
> https://www.hushssl.com
?l=485
>
>
>
------------------------------------------------------------
-------------
> This List Sponsored by: Watchfire
>
> Watchfire's AppScan is the industry's first and leading
web application
> security testing suite, and the only solution to
provide comprehensive
> remediation tasks at every level of the application.
See for yourself.
> Download AppScan 6.0 today.
>
> https://www.watchfire.com/securearea/apps
cansix.aspx?id=701300000003Ssh
>
------------------------------------------------------------
--------------
>
>

------------------------------------------------------------
-------------
This List Sponsored by: Watchfire

Watchfire's AppScan is the industry's first and leading web
application 
security testing suite, and the only solution to provide
comprehensive 
remediation tasks at every level of the application. See for
yourself. 
Download AppScan 6.0 today.

https://www.watchfire.com/securearea/apps
cansix.aspx?id=701300000003Ssh
------------------------------------------------------------
--------------

[1]

about | contact  Other archives ( Real Estate discussion Medical topics )