Masquerade based on skb->mark ?

I'm now trying to masquerade packets that have been marked
a certain way.  I'm using these commands:

# I'm not sure this is doing the right thing, but it is not
giving errors.
iptables -A POSTROUTING -t nat -j MASQUERADE -m mark --mark
10001

# This appears to work as planned.
iptables -t mangle -A PREROUTING -i eth1  -j MARK --set-mark
10001
iptables -t mangle -A PREROUTING -i eth2  -j MARK --set-mark
10001

I added a u32 'mark' field to the conn-track tuple, and I
now get
different conn-tracking objects for the same source/dest but
with
different 'mark'.  However, the ct->status bit does not
have the
IPS_SRC_NAT flag set.

I think I need to figure out what code creates the initial
conn-track
and make sure it is setting the status bits correctly based
on the
skb->mark, but I am not sure where this code exists.

Any ideas on where to start looking?  I've been trying to
follow
the code path in the netfilter/nat logic, but it's proving
slow going!

Thanks,
Ben

PS.  If anyone does this sort of work for hire, please
contact me off-list.

-- 
Ben Greear <greearbcandelatech.com>
Candela Technologies Inc  http://www.candelatech.com

On Apr 24 2007 20:06, Ben Greear wrote:
>
> I'm now trying to masquerade packets that have been
marked
> a certain way.  I'm using these commands:
>
> # I'm not sure this is doing the right thing, but it is
not giving errors.
> iptables -A POSTROUTING -t nat -j MASQUERADE -m mark
--mark 10001

It does what the programmer said:
  Masquerade only packets with a mark of 10001.

> # This appears to work as planned.
> iptables -t mangle -A PREROUTING -i eth1  -j MARK
--set-mark 10001
> iptables -t mangle -A PREROUTING -i eth2  -j MARK
--set-mark 10001

And this says:
  mark all packets that come from eth1 and eth2.

So in essence, you have "masquerade everything that
came from eth1 and eth2".
A slight bug, but ok. (In most cases, you only want to
masquerade on
some interfaces, not all, so the use of -o is usually
wanted.)

> I added a u32 'mark' field to the conn-track tuple,

Just why?

> and I now get
> different conn-tracking objects for the same
source/dest but with
> different 'mark'.  However, the ct->status bit does
not have the
> IPS_SRC_NAT flag set.
>
> I think I need to figure out what code creates the
initial conn-track
> and make sure it is setting the status bits correctly
based on the
> skb->mark, but I am not sure where this code
exists.
>
> Any ideas on where to start looking?  I've been trying
to follow
> the code path in the netfilter/nat logic, but it's
proving
> slow going!
>
> Thanks,
> Ben
>
> PS.  If anyone does this sort of work for hire, please
contact me off-list.


Jan
-- 

Jan Engelhardt wrote:
> On Apr 24 2007 20:06, Ben Greear wrote:
>> I'm now trying to masquerade packets that have been
marked
>> a certain way.  I'm using these commands:
>>
>> # I'm not sure this is doing the right thing, but
it is not giving errors.
>> iptables -A POSTROUTING -t nat -j MASQUERADE -m
mark --mark 10001
> 
> It does what the programmer said:
>   Masquerade only packets with a mark of 10001.
> 
>> # This appears to work as planned.
>> iptables -t mangle -A PREROUTING -i eth1  -j MARK
--set-mark 10001
>> iptables -t mangle -A PREROUTING -i eth2  -j MARK
--set-mark 10001
> 
> And this says:
>   mark all packets that come from eth1 and eth2.
> 
> So in essence, you have "masquerade everything
that came from eth1 and eth2".
> A slight bug, but ok. (In most cases, you only want to
masquerade on
> some interfaces, not all, so the use of -o is usually
wanted.)
> 
>> I added a u32 'mark' field to the conn-track
tuple,
> 
> Just why?

Because otherwise it seems to me that there is only a single
conn-tracking
tuple for src -- dest, and it also seems to me that the
conn-track entity
has the should-we-NAT flags (in the 'status' bitfield).

My scenario involves
virtual routers (ie, routing tables with rules so that pkts
hit certain routing
tables) and sending packets through (virtual) looped-back
ethernet ports, so
the same source-dest tuple will be seen on multiple
interfaces.  I need
a different tuple for the flow that should be NATed (so only
that flow is NATed),
so that is why I added the MARK rules and the mark field to
the conn-track tuple.

As you noticed, my attempt to MASQ based on skb->mark was
not really the right
thing to do, so I changed it back to mask with -o
[outgoing-device].
This still does not seem to trigger NAT to happen, and I had
no luck figuring out why
yesterday...

I found someone who is interested in considering doing this
for hire,
so I will be sending details and such to them.  If that
works out, hopefully
there will be a patch from him for review sometime soon.

There's also the possibility I have totally mis-understood
what is currently
supported in the kernel and maybe I just need to adjust my
rules or something
like that....

Thanks,
Ben

-- 
Ben Greear <greearbcandelatech.com>
Candela Technologies Inc  http://www.candelatech.com

On Apr 26 2007 12:20, Ben Greear wrote:

>> > iptables -A POSTROUTING -t nat -j MASQUERADE
-m mark --mark 10001
>> > iptables -t mangle -A PREROUTING -i eth1  -j
MARK --set-mark 10001
>> > iptables -t mangle -A PREROUTING -i eth2  -j
MARK --set-mark 10001
>> 
> Because otherwise it seems to me that there is only a
single
> conn-tracking tuple for src -- dest, and it also seems
to me that
> the conn-track entity has the should-we-NAT flags (in
the 'status'
> bitfield).

A ct tuple, to my knowledge, constitutes (srcip, srcport,
dstip, dstport).
Whether a connection is actually NATed or nat, is for you to
decide
(MASQUERADE, SNAT, SAME, you name it.)

> My scenario involves virtual routers (ie, routing
tables with rules
> so that pkts hit certain routing tables)

	ip rule add src 192.168.123.0/24 table 7
or	ip rule add fwmark 999 table 666

for example would do (I assume you do that)

> and sending packets through (virtual) looped-back
ethernet ports,
> so the same source-dest tuple will be seen on multiple
interfaces. 
> I need a different tuple for the flow that should be
NATed (so only
> that flow is NATed), so that is why I added the MARK
rules and the
> mark field to the conn-track tuple.

Why is a different tuple needed?


Regards,
Jan
-- 

Jan Engelhardt wrote:
> On Apr 26 2007 12:20, Ben Greear wrote:
> 
>>>> iptables -A POSTROUTING -t nat -j
MASQUERADE -m mark --mark 10001
>>>> iptables -t mangle -A PREROUTING -i eth1 
-j MARK --set-mark 10001
>>>> iptables -t mangle -A PREROUTING -i eth2 
-j MARK --set-mark 10001
>> Because otherwise it seems to me that there is only
a single
>> conn-tracking tuple for src -- dest, and it also
seems to me that
>> the conn-track entity has the should-we-NAT flags
(in the 'status'
>> bitfield).
> 
> A ct tuple, to my knowledge, constitutes (srcip,
srcport, dstip, dstport).
> Whether a connection is actually NATed or nat, is for
you to decide
> (MASQUERADE, SNAT, SAME, you name it.)

 From looking at this method in
net/ipv4/netfilter/nf_nat_core.c,
I assume it stores NAT decision as well:

/* Do packet manipulations according to nf_nat_setup_info.
*/
unsigned int nf_nat_packet(struct nf_conn *ct,
                            enum ip_conntrack_info ctinfo,
                            unsigned int hooknum,
                            struct sk_buff **pskb)
{
...
         /* Non-atomic: these bits don't change. */
         if (ct->status & statusbit) {
                 struct nf_conntrack_tuple target;

                 /* We are aiming to look like inverse of
other direction. */
                 nf_ct_invert_tuplepr(&target,
&ct->tuplehash[!dir].tuple);


>> My scenario involves virtual routers (ie, routing
tables with rules
>> so that pkts hit certain routing tables)
> 
> 	ip rule add src 192.168.123.0/24 table 7
> or	ip rule add fwmark 999 table 666

Yes, I'm using commands similar to the first line.  I have
not tried
using fwmark.

> for example would do (I assume you do that)
> 
>> and sending packets through (virtual) looped-back
ethernet ports,
>> so the same source-dest tuple will be seen on
multiple interfaces. 
>> I need a different tuple for the flow that should
be NATed (so only
>> that flow is NATed), so that is why I added the
MARK rules and the
>> mark field to the conn-track tuple.
> 
> Why is a different tuple needed?

Isn't the decision to NAT or not stored in the ct->status
bitfield?

If so, then if I want to NAT some packets and not others,
they must belong to different tuples.

If virtual router 1 is routing pkts from 1.1.1.1 to
2.2.2.2,
and virtual router 2 is routing pkts from 1.1.1.1 to
2.2.2.2, and I
only want to NAT pkts leaving virtual router 1, then I think
I
have to somehow force different ct tuples based on which
virtual
router the pkts are flowing through.  I was trying to do
this by
MARKing packets entering a device in a particular virtual
router
and using the mark as part of the tuple....

Thanks,
Ben

> 
> 
> Regards,
> Jan


-- 
Ben Greear <greearbcandelatech.com>
Candela Technologies Inc  http://www.candelatech.com