Developing a user space library for filtering

Hi,

One of the core problems I see as people want to more and
more with firewall/NAT technology is integrate using it
into
their application (whatever that may be.)  As time goes by,
this problem is becoming more and more acute and perhaps
is doing us (those who develop said technologies) a
disservice
by making the "barrier to entry" too high.

Currently, to interact with filtering software inside the
kernel
requires developers to build their application against
whatever
specific version of the filtering software runs in the
kernel.  For
application developers, this is a PITA.  What they want to
see is
the equivalent of a libc for firewalls with functions that
have a
similar stability to the likes of "fopen",
"printf", etc.

And therein lies the problem.  Nothing currently exists, so
if you
engage in developing for any one particular firewall/NAT
product
then you wed yourself to using that product.  Not a great
place
to be if you're the 3rd party.

This problem/situation is one that I believe needs to be
improved.

Is anyone in the Linux netfilter community currently
engaged
on working on anything to address this problem?

If there isn't, can I ask if anyone would be interested in
being
involved in designing and implementing a new library where
the primary focus or target is going to be 3rd party
developers
that want to build applications on top of netfilter, etc? 
It is
important to understand that my focus isn't just on
netfilter or
iptables, but also includes ipf/ipfw/pf and potentially
others too
if people become interested.

The end goal isn't to build something that userland tools
would
be rewritten to use (although ostensible they could), just
to
provide what those on the outside need in order to build
apps
that are both portable and functional.

Darren

Hi Darren,

On 22.05.2007 00:27, Darren.ReedSun.COM wrote:
> 
> One of the core problems I see as people want to more
and
> more with firewall/NAT technology is integrate using it
into
> their application (whatever that may be.)  As time goes
by,
> this problem is becoming more and more acute and
perhaps
> is doing us (those who develop said technologies) a
disservice
> by making the "barrier to entry" too high.

Sorry if I'm being dense. Do you want to target firewall
frontends
or applications which have the desire to punch holes into
the
firewall?

> Currently, to interact with filtering software inside
the kernel
> requires developers to build their application against
whatever
> specific version of the filtering software runs in the
kernel.  For
> application developers, this is a PITA.  What they want
to see is
> the equivalent of a libc for firewalls with functions
that have a
> similar stability to the likes of "fopen",
"printf", etc.
> 
> And therein lies the problem.  Nothing currently
exists, so if you
> engage in developing for any one particular
firewall/NAT product
> then you wed yourself to using that product.  Not a
great place
> to be if you're the 3rd party.

Maybe you're looking for the firewalling side of UPnP?

Regards,
Carl-Daniel

Carl-Daniel Hailfinger wrote:

>Hi Darren,
>
>On 22.05.2007 00:27, Darren.ReedSun.COM wrote:
>  
>
>>One of the core problems I see as people want to
more and
>>more with firewall/NAT technology is integrate using
it into
>>their application (whatever that may be.)  As time
goes by,
>>this problem is becoming more and more acute and
perhaps
>>is doing us (those who develop said technologies) a
disservice
>>by making the "barrier to entry" too
high.
>>    
>>
>
>Sorry if I'm being dense. Do you want to target firewall
frontends
>or applications which have the desire to punch holes
into the
>firewall?
>  
>

Neither.  Applications that sit on top of firewall
software.

As an example, using squid on your Linux firewall/router in
transparent proxying mode requires that squid has some code
in it that knows how to talk to Linux in order to discover
the
original destination and change the outgoing connection
such that the original address is used again.  Doing that
requires specific knowledge of the API that
netfilter/iptables
uses.

Another example might be IDS software running on your Linux
firewall/router.  If that detects an attack, it should be
able to
talk to netfilter/iptables and do "something" to
mitigate it.

In both cases I'd like to see something developed such that
the 3rd party applications don't need to know what NAT or
firewall technology is being used.

Darren

On May 21 2007 15:52, Darren.ReedSun.COM wrote:
>> 
>> Sorry if I'm being dense. Do you want to target
firewall frontends
>> or applications which have the desire to punch
holes into the
>> firewall?
>
> Neither.  Applications that sit on top of firewall
software.
>
> As an example, using squid on your Linux
firewall/router in
> transparent proxying mode requires that squid has some
code
> in it that knows how to talk to Linux in order to
discover the
> original destination

As with most client-side transparent proxies, it uses the
Host: header.

> and change the outgoing connection
> such that the original address is used again.  Doing
that
> requires specific knowledge of the API that
netfilter/iptables
> uses.

	Jan
-- 

> > As an example, using squid on your Linux
firewall/router in
> > transparent proxying mode requires that squid has
some code
> > in it that knows how to talk to Linux in order to
discover the
> > original destination
> 
> As with most client-side transparent proxies, it uses
the Host: header.

And when the Host: header is missing, it uses the
SO_GET_ORIGINAL_DESTINATION
getsockopt call, a speciality of netfilter, as described by
the $OP above.

Anyway, regarding the original request, I don't think it is
sensible to
expect from netfilter developers to invent such a library,
especially
when the scope is desired to be abstracting from netfilter.
As an analogy,
it was neither Microsoft nor Unix vendors that invented the
Apache
Runtime Library, it was the Apache developers. Bringing back
that
analogy, it would be a task for the developers of a vastly
successful
high level firewall application running on all kinds of
basic firewalls.

Personally, I think that it would be bound to fail anyway,
because
different basic firewall structures are very different in
what and
how they operate. But that's just the fast opinion of
somebody who
has neither need nor vision for userlevel firewall
applications
and systems other than Linux/netfilter.

best regards
  Patrick

I worked a little bit with "Hogwash".
htt
p://hogwash.sourceforge.net/docs/overview.html

This was a mangling of IDS/Snort into layer 2 for real-time
filtering.

With a little help from my friends there I got an IPTables
"target"
so that IPTables could direct traffic into the
hogwash/snort.

I like it that you said "libc".

It would be nice to intercept traffic after stream
reassembly which was 
always a problem.

In that case what you're asking makes maybe more sense to
focus on "libc" 
itself and having an "intercept" there.

That way you could for example filter stream-reassembled
data coming into 
a web server, or not, depending on the "rules
engine" deciding to catch 
the traffic or not.

I always thought that would be the place to be, in the call
to get data, 
but hookable, like a library with a hook that checks to see
if it should 
be "scrubbing" incoming traffic on a particular
port.

"Slick" is the word.

Word.

;)

-AEF


On Mon, 21 May 2007, Darren.ReedSun.COM wrote:

> Carl-Daniel Hailfinger wrote:
>
>> Hi Darren,
>> 
>> On 22.05.2007 00:27, Darren.ReedSun.COM
wrote:
>> 
>>> One of the core problems I see as people want
to more and
>>> more with firewall/NAT technology is integrate
using it into
>>> their application (whatever that may be.)  As
time goes by,
>>> this problem is becoming more and more acute
and perhaps
>>> is doing us (those who develop said
technologies) a disservice
>>> by making the "barrier to entry" too
high.
>>> 
>> 
>> Sorry if I'm being dense. Do you want to target
firewall frontends
>> or applications which have the desire to punch
holes into the
>> firewall?
>> 
>
> Neither.  Applications that sit on top of firewall
software.
>
> As an example, using squid on your Linux
firewall/router in
> transparent proxying mode requires that squid has some
code
> in it that knows how to talk to Linux in order to
discover the
> original destination and change the outgoing
connection
> such that the original address is used again.  Doing
that
> requires specific knowledge of the API that
netfilter/iptables
> uses.
>
> Another example might be IDS software running on your
Linux
> firewall/router.  If that detects an attack, it should
be able to
> talk to netfilter/iptables and do "something"
to mitigate it.
>
> In both cases I'd like to see something developed such
that
> the 3rd party applications don't need to know what NAT
or
> firewall technology is being used.
>
> Darren
>
>
>

Patrick Schaaf wrote:

>...
>Anyway, regarding the original request, I don't think it
is sensible to
>expect from netfilter developers to invent such a
library, especially
>when the scope is desired to be abstracting from
netfilter.
>

At this point in time, I was looking for people who might be
interested
in helping design such an API.  In the end, what I'm hoping
for is to
have a common API delivered as part of OpenSolaris as well
as both
FreeBSD and NetBSD.  Given that it's still being drafted,
I'm opening
the door and asking if there is anyone from Linux who's
interested in
participating.  I should point out that I'm not interested
in requesting
anyone here write code that isn't [L]GPL'd.

...

> Bringing back that
>analogy, it would be a task for the developers of a
vastly successful
>high level firewall application running on all kinds of
basic firewalls.
>  
>

Why should they have to do that?  That requires:
1) they understand the API of every firewall they support
2) to track the API changes of every firewall they support
3) recompile/redeliver their software every time that API
changes

None of those 3 options are what I would call palatable.

Imagine if everytime a new glibc was delivered you needed
to
recompile all of your programs, from ls all the way through
to the
X server, or...

>Personally, I think that it would be bound to fail
anyway, because
>different basic firewall structures are very different
in what and
>how they operate. But that's just the fast opinion of
somebody who
>has neither need nor vision for userlevel firewall
applications
>and systems other than Linux/netfilter.
>  
>

The point of this is to understand that the low level/basic
structures of firewall software are quite different but the
"end goal" is the same.

For example, all firewalls let you say "block traffic
between
X & Y", it is just how that is done which is
different.  The
idea is to capture the high level goals (c.f. "block
..")
into an API that applications can use.

Darren

On Tue, May 22, 2007 at 01:50:20PM -0700, Darren.ReedSun.COM
wrote:
> Patrick Schaaf wrote:
> 
> >...
> >Anyway, regarding the original request, I don't
think it is sensible to
> >expect from netfilter developers to invent such a
library, especially
> >when the scope is desired to be abstracting from
netfilter.
> >
> 
> At this point in time, I was looking for people who
might be interested
> in helping design such an API.  In the end, what I'm
hoping for is to
> have a common API delivered as part of OpenSolaris as
well as both
> FreeBSD and NetBSD.  Given that it's still being
drafted, I'm opening
> the door and asking if there is anyone from Linux who's
interested in
> participating.  I should point out that I'm not
interested in requesting
> anyone here write code that isn't [L]GPL'd.

Actually the netfilter folks wrote an entire infrastructure
for just this
purpose.

netfilter is a generic infrastructure for firewall software
with a defined
kernel-user API and they're now writing many libraries on
top of that.

My software, iptstate, uses libnetfilter-conntrack, which is
built upon the
netfilter framework.

All this is not to be confused with iptables, which is
simply an implmentation
of netfilter coincidentally written by the same people who
write the netfilter
framework.

Or so I understand it.

> None of those 3 options are what I would call
palatable.
> 
> Imagine if everytime a new glibc was delivered you
needed to
> recompile all of your programs, from ls all the way
through to the
> X server, or...

Darren, you're correct, this is definitely needed. If IPF
and IPtables and
everyone else all used a common core kernel-userspace API,
with a standard
library on top of it, that would be awesome.

Netfilter brings a lot of of this to the table, but the
people involved in
writing the specs mostly worked on ipchains, and iptables,
so they may have
made linux-specific assumptions without realizing it - but
it was very much
purposed to be OS-agnostic.

-- 
Phil Dibowitz                             philipom.com
Open Source software and tech docs        Insanity Palace of
Metallica
http://www.phildev.net/  
                http://www.ipom.com/

"Never write it in C if you can do it in 'awk';
 Never do it in 'awk' if 'sed' can handle it;
 Never use 'sed' when 'tr' can do the job;
 Never invoke 'tr' when 'cat' is sufficient;
 Avoid using 'cat' whenever possible" -- Taylor's Laws
of Programming

tis 2007-05-22 klockan 14:14 -0700 skrev Phil Dibowitz:

> Darren, you're correct, this is definitely needed. If
IPF and IPtables and
> everyone else all used a common core kernel-userspace
API, with a standard
> library on top of it, that would be awesome.

As things is today I am not sure how this would work in a
general sense
for what is proposed. The way you structure rules is quite
different in
the different firewall implementations.

But sure, for things like connection tracking and related
events it's
surely doable. Also, writing a generic firewall ruleset
"compiler" which
can translate to ipf, iptables and a few others is doable,
but the
actual installed ruleset will need to be somewhat different
in both
structure and syntax in the different implementations, not
only syntax.

Having an API which says things like "Add a rule to
accept this kind of
traffic" is only possible cross the different firewall
implementations
if there also is a defined firewall ruleset structure
requirement
defining suitable places in the ruleset where this API can
make it's
modifications.

Regards
Henrik

Henrik Nordstrom wrote:

>tis 2007-05-22 klockan 14:14 -0700 skrev Phil Dibowitz:
>
>  
>
>>Darren, you're correct, this is definitely needed.
If IPF and IPtables and
>>everyone else all used a common core
kernel-userspace API, with a standard
>>library on top of it, that would be awesome.
>>    
>>
>
>As things is today I am not sure how this would work in
a general sense
>for what is proposed. The way you structure rules is
quite different in
>the different firewall implementations.
>
>But sure, for things like connection tracking and
related events it's
>surely doable. Also, writing a generic firewall ruleset
"compiler" which
>can translate to ipf, iptables and a few others is
doable, but the
>actual installed ruleset will need to be somewhat
different in both
>structure and syntax in the different implementations,
not only syntax.
>
>Having an API which says things like "Add a rule to
accept this kind of
>traffic" is only possible cross the different
firewall implementations
>if there also is a defined firewall ruleset structure
requirement
>defining suitable places in the ruleset where this API
can make it's
>modifications.
>  
>

Which means that coming up with a design that works won't
necessarily be a slam-dunk.

Darren