route change differs from route delete / route add? openbgpd session drops

Hello,

I don't know, if this is a bug, but I can recognize a
strange thing. Im
setting up a redundant pair of routers and run some tests
with carp for
the failover on the lan side. Because of the bug refreshing
the kernel
routing table when changing carp-state I use ifstated with
an route
delete / route add statement. You can read more about this
here:

http://marc.info/?l=openbsd-misc&m=11836843480
7925&w=2

Everything works fine and I want to optimize my config. I
tried to use
"route change" instead of "route delete +
route add". But when I was
running tests on the failover (disabling the link on switch
side) a
strange thing happened: my bgp session to the rest of the
world died.

Jul  7 23:25:48 pinky ifstated[24699]: changing state to
primary213
Jul  7 23:25:48 pinky ifstated[24699]: running route change
-inet
195.140.213.0/24 -interface 195.140.213.1
Jul  7 23:27:56 pinky bgpd[18893]: neighbor 217.79.210.25
(IXEurope Link
2): received notification: HoldTimer expired, unknown
subcode 0
Jul  7 23:27:56 pinky bgpd[18893]: neighbor 194.9.86.8 (iBGP
AS35548):
received notification: HoldTimer expired, unknown subcode 0
Jul  7 23:27:56 pinky ifstated[24699]: changing state to
backup213

When enabling the link from switch side again, the session
came back.

Isn't route change the smartest way to configure the routing
in my case?
Or is there a bug in OpenBGPd or the kernel routing table
update code?

Regards,

Falk

* Falk Brockerhoff <nocsmartterra.de>
[2007-07-07 23:44]:
> Hello,
> 
> I don't know, if this is a bug, but I can recognize a
strange thing. Im
> setting up a redundant pair of routers and run some
tests with carp for
> the failover on the lan side. Because of the bug
refreshing the kernel
> routing table when changing carp-state I use ifstated
with an route
> delete / route add statement. You can read more about
this here:

i believe that isfixed in -current

> http://marc.info/?l=openbsd-misc&m=11836843480
7925&w=2
> 
> Everything works fine and I want to optimize my config.
I tried to use
> "route change" instead of "route delete
+ route add". But when I was
> running tests on the failover (disabling the link on
switch side) a
> strange thing happened: my bgp session to the rest of
the world died.
> 
> Jul  7 23:25:48 pinky ifstated[24699]: changing state
to primary213
> Jul  7 23:25:48 pinky ifstated[24699]: running route
change -inet
> 195.140.213.0/24 -interface 195.140.213.1
> Jul  7 23:27:56 pinky bgpd[18893]: neighbor
217.79.210.25 (IXEurope Link
> 2): received notification: HoldTimer expired, unknown
subcode 0
> Jul  7 23:27:56 pinky bgpd[18893]: neighbor 194.9.86.8
(iBGP AS35548):
> received notification: HoldTimer expired, unknown
subcode 0
> Jul  7 23:27:56 pinky ifstated[24699]: changing state
to backup213
> 
> When enabling the link from switch side again, the
session came back.
> 
> Isn't route change the smartest way to configure the
routing in my case?
> Or is there a bug in OpenBGPd or the kernel routing
table update code?

and that might be a sideissueof that problemtoo.in short,
carp's games 
with routes are... well, spare me the rant. in -current,
it's at least 
shown on the roting socket correctly and bgpd can see what
is going on 
and do the right thing.

-- 
Henning Brauer, hbbsws.de, henningopenbsd.org
BS Web Services, http://bsws.de
Full-Service ISP - Secure Hosting, Mail and DNS Services
Dedicated Servers, Rootservers, Application Hosting -
Hamburg & Amsterdam

Henning Brauer schrieb:

> i believe that isfixed in -current

Oh, this would be really nice. Hm, yesterday I switched off
ospf on both
routers and set static routes to the other loopbacks and
eBGP next-hop
adresses and configured ifstated for a plenty of interfaces
- today it
seems I'm switching back to ospfd and turning off ifstated


> in -current, it's at least 
> shown on the roting socket correctly and bgpd can see
what is going on 
> and do the right thing.

Ok, I'll give it a try. I assume 4.1 GENERIC#320 is
"current enough"?

Regards,

Falk

Falk Brockerhoff schrieb:

> Ok, I'll give it a try. I assume 4.1 GENERIC#320 is
"current enough"?

In this version the bug is NOT fixed. I will try an update
and this
setup again tomorrow.

# ospfctl sh rib
Destination          Nexthop           Path Type    Type    
 Cost
Uptime
195.140.213.0/24     194.9.86.2        Intra-Area   Network 
 20
00:00:25

# ifconfig carp213


carp213: flags=8803<UP,BROADCAST,SIMPLEX,MULTICAST>
mtu 1500
        lladdr 00:00:5e:00:01:01
        carp: INIT carpdev vlan213 vhid 1 advbase 1 advskew
0
        groups: carp
        inet6 fe80::200:5eff:fe00:101%carp213 prefixlen 64
scopeid 0x1f
        inet 195.140.213.1 netmask 0xffffff00 broadcast
195.140.213.255
# route get 195.140.213.30
   route to: 195.140.213.30
destination: 195.140.213.30
  interface: carp213
 if address: 195.140.213.1
      flags: <UP,HOST,DONE,LLINFO,CLONED>
     use  hopcount       mtu    expire
     209         0         0       858

The route should be the one distributed via ospf with
next-hop
194.9.86.2, but it isn't. Ok, perhaps better luck tomorrow


Regards,

Falk

On Sun, Jul 08, 2007 at 11:07:13PM +0200, Falk Brockerhoff
wrote:
> Falk Brockerhoff schrieb:
> 
> > Ok, I'll give it a try. I assume 4.1 GENERIC#320
is "current enough"?
> 
> In this version the bug is NOT fixed. I will try an
update and this
> setup again tomorrow.
> 
> # ospfctl sh rib
> Destination          Nexthop           Path Type   
Type      Cost
> Uptime
> 195.140.213.0/24     194.9.86.2        Intra-Area  
Network   20
> 00:00:25
> 
> # ifconfig carp213
> 
> 
> carp213:
flags=8803<UP,BROADCAST,SIMPLEX,MULTICAST> mtu 1500
>         lladdr 00:00:5e:00:01:01
>         carp: INIT carpdev vlan213 vhid 1 advbase 1
advskew 0
>         groups: carp
>         inet6 fe80::200:5eff:fe00:101%carp213 prefixlen
64 scopeid 0x1f
>         inet 195.140.213.1 netmask 0xffffff00 broadcast
195.140.213.255
> # route get 195.140.213.30
>    route to: 195.140.213.30
> destination: 195.140.213.30
>   interface: carp213
>  if address: 195.140.213.1
>       flags: <UP,HOST,DONE,LLINFO,CLONED>
>      use  hopcount       mtu    expire
>      209         0         0       858
> 
> The route should be the one distributed via ospf with
next-hop
> 194.9.86.2, but it isn't. Ok, perhaps better luck
tomorrow 
> 

There are still some issues with carp and the routes it
modifies on the
fly. Ospfd and bgpd have problems to see carp routes as
connected.
You can force ospfd to redistribute the route by mentioning
the network
directly in the redistribute statement for now until the
real issue is
fixed.

-- 
:wq Claudio

Claudio Jeker schrieb:

> There are still some issues with carp and the routes it
modifies on the
> fly. Ospfd and bgpd have problems to see carp routes as
connected.
> You can force ospfd to redistribute the route by
mentioning the network
> directly in the redistribute statement for now until
the real issue is
> fixed.

The route is redistributed correctly, but the local static
route to the
carp-interface is not deleted when the state changes to init
or backup.

Ok, I think a good way is to use the ifstated workaround a
few days
longer 

Regards,

Falk

On 2007/07/08 23:35, Falk Brockerhoff wrote:
> 
> The route is redistributed correctly, but the local
static route to the
> carp-interface is not deleted when the state changes to
init or backup.

alternatively: put an address on the carp's parent iface.
(ifstated is very cool but I prefer my workaround here,
I would feel differently if I didn't have enough space in
the subnets though 

* Falk Brockerhoff <nocsmartterra.de>
[2007-07-08 23:40]:
> The route is redistributed correctly, but the local
static route to the
> carp-interface is not deleted when the state changes to
init or backup.

it is very debateable wether that is a bug at all.

-- 
Henning Brauer, hbbsws.de, henningopenbsd.org
BS Web Services, http://bsws.de
Full-Service ISP - Secure Hosting, Mail and DNS Services
Dedicated Servers, Rootservers, Application Hosting -
Hamburg & Amsterdam

On Sun, Jul 08, 2007 at 11:35:27PM +0200, Falk Brockerhoff
wrote:
> Claudio Jeker schrieb:
> 
> > There are still some issues with carp and the
routes it modifies on the
> > fly. Ospfd and bgpd have problems to see carp
routes as connected.
> > You can force ospfd to redistribute the route by
mentioning the network
> > directly in the redistribute statement for now
until the real issue is
> > fixed.
> 
> The route is redistributed correctly, but the local
static route to the
> carp-interface is not deleted when the state changes to
init or backup.
> 
> Ok, I think a good way is to use the ifstated
workaround a few days
> longer 
> 

Interface routes should not be removed by the kernel even in
the carp
case. It will cause many more porblems later on (readding
the route will
fail). This will be fixed with route priorities.

-- 
:wq Claudio