Advanced routing problem

Hi all,

After having configured my first iptables firewall, it looks
as though
I've already found something it can't handle. Well, I hope
not...

All was fine until a secondary external interface was added.
I was
hoping that I could configure the firewall to allow remote
users to use
either external interface to access a Windows server on a
private net
behind the firewall using VNC.

At first I couldn't access the firewall via the secondary
external
interface at all, because the firewall was routing its
responses out
its default route -- the primary external interface -- and
the ISP was
dropping those packets (hot potato routing). However, after
installing
the advanced routing package (iproute), I configured a
second default
route and solved that problem.

I then set out adjust the firewall, thinking that all I'd
need to do
would be to duplicate some rules to accommodate the new
interface (see
rules below). However, that didn't work because, despite my
efforts,
the firewall loses track of where the packets come from and
sends them
out the primary external interface anyway.

Is there a way around this behavior (advanced DNAT or
something), or is
the only solution to use a public IP address for the
server?

Any hints towards a solution would be appreciated.

Cheers,

Jaap Winius
The Netherlands

======================

# 192.168.13.2 is the Windows server.

iptables -t nat -A PREROUTING -i eth0 -p tcp --dport 5900 
-j DNAT --to-destination 192.168.13.2

iptables -t filter -A FORWARD -i eth0 -p tcp --dport 5900 
-m state --state NEW -j ACCEPT

iptables -t filter -A FORWARD -d 192.168.13.2 -i eth0 -o
eth1 -p tcp 
-m tcp --dport 5900 -m state --state NEW,ESTABLISHED -j
ACCEPT

iptables -t filter -A FORWARD -s 192.168.13.2 -i eth1 -o
eth0 -p tcp 
-m tcp --sport 5900 -m state --state ESTABLISHED -j ACCEPT

iptables -t nat -A PREROUTING -i eth0 -p tcp --dport 1723 
-j DNAT --to-destination 192.168.13.2

iptables -t filter -A FORWARD -i eth0 -p tcp --dport 1723 
-m state --state NEW -j ACCEPT

iptables -t filter -A FORWARD -d 192.168.13.2 -i eth0 -o
eth1 -p tcp 
-m tcp --dport 1723 -m state --state NEW,ESTABLISHED -j
ACCEPT

iptables -t filter -A FORWARD -s 192.168.13.2 -i eth1 -o
eth0 -p tcp 
-m tcp --sport 1723 -m state --state ESTABLISHED -j ACCEPT

iptables -t nat -A PREROUTING -i eth0 -p 47 
-j DNAT --to-destination 192.168.13.2

iptables -t filter -A FORWARD -d 192.168.13.2 -i eth0 -o
eth1 -p 47 
-j ACCEPT

iptables -t filter -A FORWARD -s 192.168.13.2 -i eth1 -o
eth0 -p 47 
-j ACCEPT

iptables -t nat -A POSTROUTING -s 192.168.13.2 -o eth0 -p 47

-j MASQUERADE

======================

Jaap Winius wrote:
> Hi all,
>   

Ha Jaap,

> After having configured my first iptables firewall, it
looks as though
> I've already found something it can't handle. Well, I
hope not...
>
> All was fine until a secondary external interface was
added. I was
> hoping that I could configure the firewall to allow
remote users to use
> either external interface to access a Windows server on
a private net
> behind the firewall using VNC.
>
> At first I couldn't access the firewall via the
secondary external
> interface at all, because the firewall was routing its
responses out
> its default route -- the primary external interface --
and the ISP was
> dropping those packets (hot potato routing). However,
after installing
> the advanced routing package (iproute), I configured a
second default
> route and solved that problem.
>   

That's strange. that means you have some very funny routing.
You'ld 
better explain how your routing is set up. Also post the
output of 'ip ro'.

M4

Quoting Martijn Lievaart <mrtij.nl>:

(nice website, interesting hobby)

> That's strange. that means you have some very funny
routing. You'd
> better explain how your routing is set up. Also post
the output of
> 'ip ro'.

Here's the output from 'ip ro':

196.190.250.17 dev ppp0  proto kernel  scope link  src
63.58.236.234
88.215.195.176/29 dev eth0  proto kernel scope link  src
88.215.195.178
63.58.50.0/25 via 63.58.236.234 dev ppp0  scope link
192.168.1.0/24 dev eth0  proto kernel  scope link  src
192.168.1.1
192.168.14.0/24 dev eth2  proto kernel  scope link  src
192.168.14.1
192.168.13.0/24 dev eth1  proto kernel  scope link  src
192.168.13.1
10.0.0.0/8 dev eth0  proto kernel  scope link  src
10.0.0.100
default via 88.215.195.177 dev eth0

The route to 63.58.50.0/25 is to make some of that ISP's
servers
available, including an SMTP relay that would not be
accessible any
other way.

The second routing table, secnet, has only one line:

   default via 63.58.236.234 dev ppp0

The output of 'ip rule' look like this:

   0:      from all lookup local
   1000:   from 63.58.236.234 lookup secnet
   32766:  from all lookup main
   32767:  from all lookup default

Interfaces:

eth0 -- Primary external interface.
     address 88.215.195.178
     netmask 255.255.255.248
     gateway 88.215.195.177

eth0:0 -- Priv. net behind the prim. external interface's
ADSL modem.
     address 192.168.1.1
     netmask 255.255.255.0

eth0:1 -- Priv. net behind the sec. external interface's
ADSL modem.
     address 10.0.0.100
     netmask 255.0.0.0

eth1 -- Internal segment (UTP)
     address 192.168.13.1
     netmask 255.255.255.0

eth2 -- Internal segment (wireless)
     address 192.168.14.1
     netmask 255.255.255.0

ppp0 -- Secondary external interface.
     address 63.58.236.234
     netmask 255.255.255.255
     P-t-P:  196.190.250.17


A diagram of the network:

                  The
                Internet
               /        
              /          
             /            
            /              
            |               |
       Sec. |         Prim. |
           ADSL            ADSL    modem
          modem-----------modem   /
                            |    /
                            |   /
                       eth0 |  / ttyS0
            ------------Firewall-------------
           |        eth1        eth2         |
           |                                 |
           |                              Wireless
           |                               Access
           |                               Point
        Ethernet
         switch--------Workstation
           |
           |
        Windows
         Server

The secondary ADSL modem has only one Ethernet interface,
while the
firewall (a little Soekris box) has only one interface
available for
Internet connectivity. However, the primary ADSL modem has a
built-in
Ethernet switch, and since the secondary Internet connection
uses PPtP
anyway, I connected it to the primary ADSL modem's switch so
that I
could get to it from the firewall.

The secondary ADSL modem is part of a cheap-o consumer
subscription
that the client did not want to part with. It was there
before I became
responsible for the system, so I thought I might be able to
put it to
good use this way -- it gives the remote users an
alternative access
route in case the primary external interface goes down.

The primary ADSL modem is part of a business subscription.
The ISP
doesn't allow its clients to fiddle with the modem's
configuration, so
I wouldn't have recommended it, but this subscription had
been
requested by the client before I arrived on the scene.
Anyway, I fixed
the situation by asking for a public IP address for the
firewall and
was given 87.215.195.176/29, which allows me to bypass their
modem.

Did I leave anything out? If so, just ask and I'll post it
ASAP.
Thanks very much for your help!

Cheers,

Jaap

Hi all,

To recap, my problem was that I have a Windows server on a
private net
behind a firewall and a NAT that I want to make accessible
to the world
outside through two different Internet connections
simultaneously.
Usually, this is not possible because you have to make a
choice with
your default gateway. I discovered how to make a second
default gateway
by using advanced routing (iproute), which made it possible
for me to
ssh to the firewall via the second Internet connection.
However, if
requests via this connection were then forwarded on to the
server, its
responses were still being routed out the main default
gateway, after
which they would be dropped by the ISP.

The first step towards a solution was to define a second
network,
192.168.15.0/24, on the UTP segment that the server was
attached to.
Luckily, Windows server 2003 allows you to bind additional
IP addresses
to its interfaces. In this case, only the server and the
firewall (via
eth1) have addresses on this network.

The next step in my plan didn't work out. I wanted to use
the firewall
to mark certain packets coming in (into eth1) from this new
network
segment. I thought that this would do the trick:

   iptables -t mangle -A PREROUTING -s 192.168.15.2 -i eth1

      -p tcp -j MARK --set-mark 0x1
   iptables -t mangle -A PREROUTING -s 192.168.15.2 -i eth1

      -p 47 -j MARK --set-mark 0x1

(192.168.15.2 is the server). These marked packets would
then be caught
by iproute and routed according to my secondary routing
table:

   ip rule add fwmark 0x1 table secnet prio 990

I hoped that this would send them out via my secondary
Internet gateway
(ppp0), but it didn't work. In theory, this kind of
configuration will
allow you to route your traffic by protocol (http via one
gateway, smtp
via another, etc.).

Anyway, the second half of my eventual solution had nothing
to do with
the netfilter and everything to do with iproute. It's
simple. Since all
of the packets moving from 192.168/.15.0/24 into the
firewall are
responses to requests that arrived via the secondary
Internet
connection (and should be sent back that way) anyway, I
could use this
one routing rule:

   ip rule add from 192.168.15.0/24 lookup secnet prio 990

This may be less precise than marking the packets (perhaps
also
matching the protocol and state), but in my case this second
approach
will do and is probably more efficient.

Cheers,

Jaap