First, let me explain what I'm trying to do - maybe you have already done this and know what to do or know that it won't work and you can skip reading all the foolishness I’ve tried below.
We have ~160 L2L VPNs connected to a Cisco ASA. Each of these has between 2 and 8 /32 hosts on the other end, randomly scattered across IP space - some RFC1918, but a lot are public addresses used inside the tunnels to avoid subnet collisions. To make some applications on this side of those tunnels work (and just to make life easier in general) we are destination NATing all of the endpoints behind one subnet, i.e. hosts on my side connect to a 10.7.0.0/16 address and the ASA NATs it back to the proper address for that tunnel before sending the traffic across.
Anyway, this is apparently not a typical use case for the ASAs because the 500 odd NAT statements are eating up all of the memory and when you get beyond 80% memory utilization the ASA becomes unstable. So I’m trying to create a workaround by sticking a little linux box with bridged adapters inline between the hosts on my side and the ASA that will do all this NATing before the traffic hits the ASA. Another complicating factor here is that the traffic between switch and ASA is VLAN trunked.
What I’ve done:
I got a POC working with non-VLAN traffic and that was a pain in the ass itself. When I switched to sending VLAN tagged packets through it quit working. I spent a couple of hours tinkering with ebtables because I read on some forum somewhere that iptables wouldn’t do anything with VLAN tagged packets because they didn’t hit the kernel the same way. I tried to use ebtables to do the NAT but then realized it only NAT’s mac addresses, not IP. So then I tried using ebtables to mark the traffic so iptables could identify it as needing attention, but iptables ignored those packets just the same.
Finally stumbled across /proc/sys/net/bridge/bridge-nf-filter-vlan-tagged, changed it from 0 to 1 and voila, iptables sees the packets! After banging around with iptables -t nat statements for a while until I finally found a combination of settings that identified the incoming packets, NAT’d them, and sent them on their merry way. I needed to know that this thing would withstand a reboot, so I googled around until I found some very dubious advice on getting the above /proc/…/bridge setting to persist after reboot (which it did not). After the reboot, I added the 1 back to the bridge setting, re-applied my iptables settings and now the damn thing doesn’t work anymore. What’s really weird, is the iptables nat table counters are incrementing - it’s seeing the packets, but for whatever reason they don’t come out the other interface anymore they just seem to disappear into the ether (or perhaps they don’t enter the ether and that’s the problem). I have a network tap between eth0 and the next hop and I never see the NAT’d packets. Regular traffic that iptables should ignore passes just fine and pops out the other end with no problems.
I've tried adding LOG statements to iptables, but that just confuses me even further. If I add a LOG statment before the DNAT, I see it in the log, but it hasn't been NAT'd yet (obviously). If I add it after the DNAT then it never shows up in the log at all, presumably because once it hits the DNAT statement iptables is done with it.
Here are as many of my settings and configs I can think to include. I would really appreciate any help you can provide as I’ve been pulling my hair out on this for 2 days now.
Ubuntu 14.04
packets flow in on eth1 and out eth0 (or at least they're suppose to)
/etc/network/interfaces auto br0 bridge_ports eth0 eth1 bridge_stp off address 192.168.4.3 (this presents it’s own problem because now all the traffic flowing over these interfaces is VLAN’d I can’t reach this host via network anymore - but one thing at a time) netmask 255.255.255.0 gateway 192.168.4.1 /proc/sys/net/bridge/bridge-nf-filter-vlan-tagged set to 1 /proc/sys/net/ipv4/ip_foward set to 1 CHAIN PREROUTING (policy ACCEPT 47 packets, 4000 bytes) pkts bytes target prot opt in out source destination 9299 715K DNAT all -- * * 0.0.0.0/0 10.7.2.3 PHYSDEV match --physdev-in eth1 to:10.247.6.63 INPUT, OUTPUT, and POSTROUTING chains are all empty default ACCEPT (I'll get to the postrouting SNAT after I make the DNAT work) edit: corrected some typos
[link] [comments]