Array

From: Amin Azez <azezufomechanic.net>
Date: Fri, 30 Jun 2006 11:44:33 +0100

> Well, my src/linux-2.6.17.1/include/linux/config.h
(line 6) tries to
> include linux/autoconf.h, which is now part of the
O=dir build output.
> If it can't find my build autoconf.h then it falls
back to
> /usr/include/linux/autoconf.h which is:
> #error Invalid kernel header included in userspace
> 
> In case this much is bogus, here are the include
dependancies;
> 
> Extensions found: IPv4:CLUSTERIP IPv4:connbytes
IPv4:dccp IPv4:recent
> IPv4:string IPv6:ah IPv6:esp IPv6:frag IPv6:ipv6header
IPv6:hbh IPv6:dst
> IPv6:REJECT IPv6:rt
> cc -O2 -Wall -Wunused
-I/opt/KERNEL/linux-2.6.17.1//include -Iinclude/
> -DIPTABLES_VERSION=\"1.3.5-20060629\"
-D_UNKNOWN_KERNEL_POINTER_SIZE
> -fPIC -o extensions/libipt_ah_sh.o -c
extensions/libipt_ah.c
> In file included from
/opt/KERNEL/linux-2.6.17.1/include/linux/config.h:6,
> from
/opt/KERNEL/linux-2.6.17.1/include/linux/netfilter_ipv4.h:8,
> from
/opt/KERNEL/linux-2.6.17.1/include/linux/netfilter_ipv4/ip_t
ables.h:26,
> from include/libiptc/libiptc.h:6,
> from include/iptables.h:5,
> from extensions/libipt_ah.c:8:
> /usr/include/linux/autoconf.h:1:2: #error Invalid
kernel header included
> in usersp

I was seeing the recent git tree that the above problem was
fixed.

As you says, netfilter_ipv4.h in 2.6.17.x has unnecessary
'#include <config.h>' outside of __KERNEL__. But it
seemd to be deleted
in recent kernel as follows.

> commit 62c4f0a2d5a188f73a94f2cb8ea0dba3e7cf0a7f
> Author: David Woodhouse <dwmw2infradead.org>
> Date:   Wed Apr 26 12:56:16 2006 +0100
> 
>     Don't include linux/config.h from anywhere else in
include/
> 
>     Signed-off-by: David Woodhouse <dwmw2infradead.org> 

I don't think/know that this goes to stable tree, though.

-- Yasuyuki Kozakai

Yasuyuki KOZAKAI wrote:
> As you says, netfilter_ipv4.h in 2.6.17.x has
unnecessary
> '#include <config.h>' outside of __KERNEL__.
But it seemd to be deleted
> in recent kernel as follows.
> 
>> commit 62c4f0a2d5a188f73a94f2cb8ea0dba3e7cf0a7f
>> Author: David Woodhouse <dwmw2infradead.org>
>> Date:   Wed Apr 26 12:56:16 2006 +0100
>>
>>     Don't include linux/config.h from anywhere
else in include/
>>
>>     Signed-off-by: David Woodhouse <dwmw2infradead.org> 
> 
> I don't think/know that this goes to stable tree,
though.

Thanks; thats helpful and also solves a long-standing
puzzlement for me.
(Into my quilt stack it goes)

Next question relates to the port to xtables and friends,
and what I
need to do to convert iptables targets.

I notice a lot of things have moved to net/netfilter/xt_*.c

Is this an absolute requirement, or is there some
compatability system?

I'm porting the layer7 match first, currently when adding a
layer7 rule
I get (dmesg):
[71987.513531] ip_tables: layer7 match: invalid size 0 !=
8452

I know where the error is coming from I just wonder if I
have to do a
complete conversion now, or a quick fix and fuller
conversion later...

thanks

Sam

Yasuyuki KOZAKAI wrote:
> As you says, netfilter_ipv4.h in 2.6.17.x has
unnecessary
> '#include <config.h>' outside of __KERNEL__.
But it seemd to be deleted
> in recent kernel as follows.
> 
>> commit 62c4f0a2d5a188f73a94f2cb8ea0dba3e7cf0a7f
>> Author: David Woodhouse <dwmw2infradead.org>
>> Date:   Wed Apr 26 12:56:16 2006 +0100
>>
>>     Don't include linux/config.h from anywhere
else in include/
>>
>>     Signed-off-by: David Woodhouse <dwmw2infradead.org> 
> 
> I don't think/know that this goes to stable tree,
though.

Thanks; thats helpful and also solves a long-standing
puzzlement for me.
(Into my quilt stack it goes)

Next question relates to the port to xtables and friends,
and what I
need to do to convert iptables targets.

I notice a lot of things have moved to net/netfilter/xt_*.c

Is this an absolute requirement, or is there some
compatability system?

I'm porting the layer7 match first, currently when adding a
layer7 rule
I get (dmesg):
[71987.513531] ip_tables: layer7 match: invalid size 0 !=
8452

I know where the error is coming from I just wonder if I
have to do a
complete conversion now, or a quick fix and fuller
conversion later...

thanks

Sam

Amin Azez wrote:
> I notice a lot of things have moved to
net/netfilter/xt_*.c
> 
> Is this an absolute requirement, or is there some
compatability system?

No requirement, things can just stay where they are.

> I'm porting the layer7 match first, currently when
adding a layer7 rule
> I get (dmesg):
> [71987.513531] ip_tables: layer7 match: invalid size 0
!= 8452

8452? Thats ridiculous much .. anyway, you need to
initialize the
.matchsize field to sizeof(struct ...) (the one shared with
userspace).

Patrick McHardy wrote:
> Amin Azez wrote:
>> I notice a lot of things have moved to
net/netfilter/xt_*.c
>>
>> Is this an absolute requirement, or is there some
compatability system?
> 
> No requirement, things can just stay where they are.
> 
>> I'm porting the layer7 match first, currently when
adding a layer7 rule
>> I get (dmesg):
>> [71987.513531] ip_tables: layer7 match: invalid
size 0 != 8452
> 
> 8452? Thats ridiculous much 

too right! Layer7 permits up to 8192 for the regex - like
anyone is
going to want to run a regex that long on every packet (well
the first
few of each connection)

Whats anyones feelings on flexible sizes of iptables match
structs?
Or even on making the kernel size struct bigger?

Right now layer7 does a linear search through a cache of
pre-compiled
regexes with the original regex as the key - PER PACKET - it
would make
much more sense to store a pointer to the cached regex in
the matchinfo
- which is what I'm doing now; but the lack of extra
kernelside fields
is restrictive, so I'm canibalizing (in an ugly way) the
last few bytes
of the refex field.

>.. anyway, you need to initialize the
> .matchsize field to sizeof(struct ...) (the one shared
with userspace).

aye, that works fine, thanks.
To my surprise I'm getting checkinfo called lots of times,
I thought it
would only be called once when a new rule was being accepted
kernel-side. Do you know why it would be called many times
(with the
same value matchinfo pointer)

Sam

Patrick McHardy wrote:
> Amin Azez wrote:
>> I notice a lot of things have moved to
net/netfilter/xt_*.c
>>
>> Is this an absolute requirement, or is there some
compatability system?
> 
> No requirement, things can just stay where they are.
> 
>> I'm porting the layer7 match first, currently when
adding a layer7 rule
>> I get (dmesg):
>> [71987.513531] ip_tables: layer7 match: invalid
size 0 != 8452
> 
> 8452? Thats ridiculous much 

too right! Layer7 permits up to 8192 for the regex - like
anyone is
going to want to run a regex that long on every packet (well
the first
few of each connection)

Whats anyones feelings on flexible sizes of iptables match
structs?
Or even on making the kernel size struct bigger?

Right now layer7 does a linear search through a cache of
pre-compiled
regexes with the original regex as the key - PER PACKET - it
would make
much more sense to store a pointer to the cached regex in
the matchinfo
- which is what I'm doing now; but the lack of extra
kernelside fields
is restrictive, so I'm canibalizing (in an ugly way) the
last few bytes
of the refex field.

>.. anyway, you need to initialize the
> .matchsize field to sizeof(struct ...) (the one shared
with userspace).

aye, that works fine, thanks.
To my surprise I'm getting checkinfo called lots of times,
I thought it
would only be called once when a new rule was being accepted
kernel-side. Do you know why it would be called many times
(with the
same value matchinfo pointer)

Sam

Amin Azez wrote:
> Patrick McHardy wrote:
> 
>>Amin Azez wrote:
>>
>>>I notice a lot of things have moved to
net/netfilter/xt_*.c
>>>
>>>Is this an absolute requirement, or is there
some compatability system?
>>
>>No requirement, things can just stay where they are.
>>
>>
>>>I'm porting the layer7 match first, currently
when adding a layer7 rule
>>>I get (dmesg):
>>>[71987.513531] ip_tables: layer7 match: invalid
size 0 != 8452
>>
>>8452? Thats ridiculous much 
> 
> 
> too right! Layer7 permits up to 8192 for the regex -
like anyone is
> going to want to run a regex that long on every packet
(well the first
> few of each connection)
> 
> Whats anyones feelings on flexible sizes of iptables
match structs?

Its not possible with the current architecture, and
overcoming this
would probably not be worth it measured in both time and the
resulting
uglyness compared to just introducing a netlink interface.

> Right now layer7 does a linear search through a cache
of pre-compiled
> regexes with the original regex as the key - PER PACKET
- it would make
> much more sense to store a pointer to the cached regex
in the matchinfo
> - which is what I'm doing now; but the lack of extra
kernelside fields
> is restrictive, so I'm canibalizing (in an ugly way)
the last few bytes
> of the refex field.

Thats a bit of a different problem. Per-instance private
data will soon
be possible in a nicer way.

>>.. anyway, you need to initialize the
>>.matchsize field to sizeof(struct ...) (the one
shared with userspace).
> 
> 
> aye, that works fine, thanks.
> To my surprise I'm getting checkinfo called lots of
times, I thought it
> would only be called once when a new rule was being
accepted
> kernel-side. Do you know why it would be called many
times (with the
> same value matchinfo pointer)

It gets called whenever you add a new rule. Thats a result
of the
atomic-exchange of the entire ruleset, each
"generation" of your
ruleset (while adding rules incrementally) is an entirely
new
ruleset to the kernel.

Patrick McHardy wrote:
> Amin Azez wrote:
>> To my surprise I'm getting checkinfo called lots
of times, I thought it
>> would only be called once when a new rule was being
accepted
>> kernel-side. Do you know why it would be called
many times (with the
>> same value matchinfo pointer)
> 
> It gets called whenever you add a new rule. Thats a
result of the
> atomic-exchange of the entire ruleset, each
"generation" of your
> ruleset (while adding rules incrementally) is an
entirely new
> ruleset to the kernel.


iptables -Z dsrateentries -t mangle

causes the checkentry to all be called again.
(It just so happens I was doing that every 10 seconds)

I guess iptables -z is not as lightweight as I thought, it
perhaps
re-applies all the rules - or maybe checkentry is wrongly
being called
from -z?

Or maybe checkentry can do more these days and I need to
check why it is
called

Sam

Patrick McHardy wrote:
> Amin Azez wrote:
>> To my surprise I'm getting checkinfo called lots
of times, I thought it
>> would only be called once when a new rule was being
accepted
>> kernel-side. Do you know why it would be called
many times (with the
>> same value matchinfo pointer)
> 
> It gets called whenever you add a new rule. Thats a
result of the
> atomic-exchange of the entire ruleset, each
"generation" of your
> ruleset (while adding rules incrementally) is an
entirely new
> ruleset to the kernel.


iptables -Z dsrateentries -t mangle

causes the checkentry to all be called again.
(It just so happens I was doing that every 10 seconds)

I guess iptables -z is not as lightweight as I thought, it
perhaps
re-applies all the rules - or maybe checkentry is wrongly
being called
from -z?

Or maybe checkentry can do more these days and I need to
check why it is
called

Sam

Amin Azez wrote:
> iptables -Z dsrateentries -t mangle
> 
> causes the checkentry to all be called again.
> (It just so happens I was doing that every 10 seconds)
> 
> I guess iptables -z is not as lightweight as I thought,
it perhaps
> re-applies all the rules - or maybe checkentry is
wrongly being called
> from -z?

I'm actually not sure how this is implemented, I guess it
just
dumps the ruleset to userspace, changes counters and pumps
it back to the kernel again (which means it needs to be
revalidated).

Amin Azez wrote:
> Yasuyuki KOZAKAI wrote:
>   
>> As you says, netfilter_ipv4.h in 2.6.17.x has
unnecessary
>> '#include <config.h>' outside of
__KERNEL__. But it seemd to be deleted
>> in recent kernel as follows.
>>
>>     
>>> commit 62c4f0a2d5a188f73a94f2cb8ea0dba3e7cf0a7f
>>> Author: David Woodhouse <dwmw2infradead.org>
>>> Date:   Wed Apr 26 12:56:16 2006 +0100
>>>
>>>     Don't include linux/config.h from anywhere
else in include/
>>>
>>>     Signed-off-by: David Woodhouse
<dwmw2infradead.org> 
>>>       
>> I don't think/know that this goes to stable tree,
though.
>>     
>
> Thanks; thats helpful and also solves a long-standing
puzzlement for me.
> (Into my quilt stack it goes)
>
> Next question relates to the port to xtables and
friends, and what I
> need to do to convert iptables targets.
>
> I notice a lot of things have moved to
net/netfilter/xt_*.c
>
> Is this an absolute requirement, or is there some
compatability system?
>   
Sorry for this foolish question; I have the answer just by
diffing what
is left in net/ipv4/netfilter/ with earlier kernels
please forgive my laziness.

Sam