It has happened to me a cuple of times when dealing with UDP-based services that, when a server has more than one network interface (either physical or virtual), all the UDP traffic goes out through the interface on the default gateway’s network segment and with that interface’s IP address, even when the original request came through the other interface and was directed to the other IP address.
Graphically, say you have something similar to this:
If the server receives a request on IPa, the response goes out through that same interface and with origin IPa. But if the request arrives on IPb through the interface on the right, the response is also sent through the left interface with IPa. And what happens when the client receives a response from an incorrect IP address? Maybe even from a completely different network segment? And if there’s a fw in between doing NAT?
Of course this breaks the service. This week I’ve had this very problem setting up a L2TP VPN, and it was impossible to establish the tunnel. On some other ocasions I’ve had a similar problem with a DNS server, and the outcome depended on the client’s operating system: some OSes accepted the DNS response even when it came from a different address than that of the server originally queried; others would reject it and even raise a security alert.
I guess that this behaviour can be programmatically controlled. I mean, when you receive a packet you can check the IP address it was sent to, and craft the response so that it gets sent with that same address from the right interface. But it seems that this is seldom done.
Yesterday I got around this issue with the help of iptables and a coworker more knowledgeable than me on routing issues:
- with iptables, you can detect the traffic to “redirect” and mark it
- depending on this mark and using “ip rule/route”, have a special routing table that sends this traffic to the proper GW/through the right interface.
- with iptables again and using the previous mark, do a SNAT on the origin IP address
An example for redirecting all UDP traffic from a certain $PORT using IP address $IPb through gateway $GWb would be:
echo 255 local > /etc/iproute2/rt_tables
echo 254 main >> /etc/iproute2/rt_tables
echo 253 default >> /etc/iproute2/rt_tables
echo 0 unspec >> /etc/iproute2/rt_tables
echo 200 udp >> /etc/iproute2/rt_tables
ip rule add fwmark 1 table udp
ip route add default via $GWb dev eth0 table l2tp
iptables -t mangle -A OUTPUT -p udp -m udp –sport $PORT -j MARK –set-mark 0×1
iptables -t nat -A POSTROUTING -m mark –mark 0×1 -j SNAT –to-source $IPb
