I have a situation where this host (proxy01, 192.168.20.62) becomes inaccessible (from desktop01, 192.168.1.10) after I bring up a vpn client (establish tun0). I've copy/pasted a before and after scenario below. My networking knowledge is generally good, but I'm weak with routing --and I believe that's where the issue is here.
What's really important to mention is that configuration works fine with a host that IS NOT in a different network, or separate VLAN (or rather, lives in PVID 1). For example, I can communicate fine between 192.168.1.20 and 192.168.1.10 with or without the VPN up/down.
enp0s3 is a vagrant management interface. I'm 99% certain this is entirely relevant.
There are iptables in place here, but they make no difference up/down on/off. I've included that far below. All they do is drop inbound on tun0.
I'm pretty certain this is not a VLAN issue, but probably a routing issue. I think the answer is in the 'AFTER' routing table below, I just don't know how to trace it / test.
In addition to the pings I've used a netcat test 'nc -l 1234' before and after. Reflects same results. Packets can't get back.
I've modified some octets below to protect the innocent. No insects were harmed while constructing this test.
BEFORE: enp0s3: flags=4163<UP,BROADCAST,RUNNING,MULTICAST> mtu 1500 inet 10.0.2.15 netmask 255.255.255.0 broadcast 10.0.2.255 inet6 fe80::a00:27ff:fe82:deb prefixlen 64 scopeid 0x20<link> ether 08:00:27:82:0d:eb txqueuelen 1000 (Ethernet) RX packets 48913 bytes 35352375 (33.7 MiB) RX errors 0 dropped 0 overruns 0 frame 0 TX packets 24347 bytes 2712562 (2.5 MiB) TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0 enp0s8: flags=4163<UP,BROADCAST,RUNNING,MULTICAST> mtu 1500 inet6 fe80::a00:27ff:feb7:10fd prefixlen 64 scopeid 0x20<link> ether 08:00:27:b7:10:fd txqueuelen 1000 (Ethernet) RX packets 41593 bytes 29848105 (28.4 MiB) RX errors 0 dropped 0 overruns 0 frame 0 TX packets 22019 bytes 9734282 (9.2 MiB) TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0 enp0s8.20: flags=4163<UP,BROADCAST,RUNNING,MULTICAST> mtu 1500 inet 192.168.20.62 netmask 255.255.255.0 broadcast 192.168.20.255 inet6 fe80::a00:27ff:feb7:10fd prefixlen 64 scopeid 0x20<link> ether 08:00:27:b7:10:fd txqueuelen 1000 (Ethernet) RX packets 27403 bytes 28425883 (27.1 MiB) RX errors 0 dropped 0 overruns 0 frame 0 TX packets 19672 bytes 6716095 (6.4 MiB) TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0 lo: flags=73<UP,LOOPBACK,RUNNING> mtu 65536 inet 127.0.0.1 netmask 255.0.0.0 inet6 ::1 prefixlen 128 scopeid 0x10<host> loop txqueuelen 1 (Local Loopback) RX packets 9 bytes 792 (792.0 B) RX errors 0 dropped 0 overruns 0 frame 0 TX packets 9 bytes 792 (792.0 B) TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0 AFTER: enp0s3: flags=4163<UP,BROADCAST,RUNNING,MULTICAST> mtu 1500 inet 10.0.2.15 netmask 255.255.255.0 broadcast 10.0.2.255 inet6 fe80::a00:27ff:fe82:deb prefixlen 64 scopeid 0x20<link> ether 08:00:27:82:0d:eb txqueuelen 1000 (Ethernet) RX packets 49696 bytes 35410779 (33.7 MiB) RX errors 0 dropped 0 overruns 0 frame 0 TX packets 24794 bytes 2755232 (2.6 MiB) TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0 enp0s8: flags=4163<UP,BROADCAST,RUNNING,MULTICAST> mtu 1500 inet6 fe80::a00:27ff:feb7:10fd prefixlen 64 scopeid 0x20<link> ether 08:00:27:b7:10:fd txqueuelen 1000 (Ethernet) RX packets 50636 bytes 32447956 (30.9 MiB) RX errors 0 dropped 0 overruns 0 frame 0 TX packets 28876 bytes 13075393 (12.4 MiB) TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0 enp0s8.20: flags=4163<UP,BROADCAST,RUNNING,MULTICAST> mtu 1500 inet 192.168.20.62 netmask 255.255.255.0 broadcast 192.168.20.255 inet6 fe80::a00:27ff:feb7:10fd prefixlen 64 scopeid 0x20<link> ether 08:00:27:b7:10:fd txqueuelen 1000 (Ethernet) RX packets 34869 bytes 30811417 (29.3 MiB) RX errors 0 dropped 0 overruns 0 frame 0 TX packets 25732 bytes 9003231 (8.5 MiB) TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0 lo: flags=73<UP,LOOPBACK,RUNNING> mtu 65536 inet 127.0.0.1 netmask 255.0.0.0 inet6 ::1 prefixlen 128 scopeid 0x10<host> loop txqueuelen 1 (Local Loopback) RX packets 9 bytes 792 (792.0 B) RX errors 0 dropped 0 overruns 0 frame 0 TX packets 9 bytes 792 (792.0 B) TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0 tun0: flags=4305<UP,POINTOPOINT,RUNNING,NOARP,MULTICAST> mtu 1500 inet 666.666.191.186 netmask 255.255.255.128 destination 666.666.191.186 inet6 fe80::8180:698:63f4:5537 prefixlen 64 scopeid 0x20<link> unspec 00-00-00-00-00-00-00-00-00-00-00-00-00-00-00-00 txqueuelen 100 (UNSPEC) RX packets 12 bytes 1163 (1.1 KiB) RX errors 0 dropped 0 overruns 0 frame 0 TX packets 109 bytes 26850 (26.2 KiB) TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0 BEFORE: proxy01:/root>netstat -nr Kernel IP routing table Destination Gateway Genmask Flags MSS Window irtt Iface 0.0.0.0 192.168.20.1 0.0.0.0 UG 0 0 0 enp0s8.20 10.0.2.0 0.0.0.0 255.255.255.0 U 0 0 0 enp0s3 192.168.20.0 0.0.0.0 255.255.255.0 U 0 0 0 enp0s8.20 AFTER: proxy01:/root>netstat -nr Kernel IP routing table Destination Gateway Genmask Flags MSS Window irtt Iface 0.0.0.0 666.666.191.129 128.0.0.0 UG 0 0 0 tun0 0.0.0.0 192.168.20.1 0.0.0.0 UG 0 0 0 enp0s8.20 10.0.2.0 0.0.0.0 255.255.255.0 U 0 0 0 enp0s3 128.0.0.0 666.666.191.129 128.0.0.0 UG 0 0 0 tun0 666.666.191.128 0.0.0.0 255.255.255.128 U 0 0 0 tun0 192.168.20.0 0.0.0.0 255.255.255.0 U 0 0 0 enp0s8.20 23.23.113.36 192.168.20.1 255.255.255.255 UGH 0 0 0 enp0s8.20 --- BEFORE: proxy01:/root>tcpdump -ni enp0s8 -e icmp tcpdump: verbose output suppressed, use -v or -vv for full protocol decode listening on enp0s8, link-type EN10MB (Ethernet), capture size 262144 bytes 10:05:00.332772 f0:9f:c2:df:16:01 > 08:00:27:b7:10:fd, ethertype 802.1Q (0x8100), length 102: vlan 20, p 0, ethertype IPv4, 192.168.1.10 > 192.168.20.62: ICMP echo request, id 42734, seq 0, length 64 10:05:00.332878 08:00:27:b7:10:fd > f0:9f:c2:df:16:01, ethertype 802.1Q (0x8100), length 102: vlan 20, p 0, ethertype IPv4, 192.168.20.62 > 192.168.1.10: ICMP echo reply, id 42734, seq 0, length 64 10:05:01.342130 f0:9f:c2:df:16:01 > 08:00:27:b7:10:fd, ethertype 802.1Q (0x8100), length 102: vlan 20, p 0, ethertype IPv4, 192.168.1.10 > 192.168.20.62: ICMP echo request, id 42734, seq 1, length 64 10:05:01.342208 08:00:27:b7:10:fd > f0:9f:c2:df:16:01, ethertype 802.1Q (0x8100), length 102: vlan 20, p 0, ethertype IPv4, 192.168.20.62 > 192.168.1.10: ICMP echo reply, id 42734, seq 1, length 64 10:05:02.369852 f0:9f:c2:df:16:01 > 08:00:27:b7:10:fd, ethertype 802.1Q (0x8100), length 102: vlan 20, p 0, ethertype IPv4, 192.168.1.10 > 192.168.20.62: ICMP echo request, id 42734, seq 2, length 64 10:05:02.370100 08:00:27:b7:10:fd > f0:9f:c2:df:16:01, ethertype 802.1Q (0x8100), length 102: vlan 20, p 0, ethertype IPv4, 192.168.20.62 > 192.168.1.10: ICMP echo reply, id 42734, seq 2, length 64 10:05:03.357284 f0:9f:c2:df:16:01 > 08:00:27:b7:10:fd, ethertype 802.1Q (0x8100), length 102: vlan 20, p 0, ethertype IPv4, 192.168.1.10 > 192.168.20.62: ICMP echo request, id 42734, seq 3, length 64 10:05:03.357376 08:00:27:b7:10:fd > f0:9f:c2:df:16:01, ethertype 802.1Q (0x8100), length 102: vlan 20, p 0, ethertype IPv4, 192.168.20.62 > 192.168.1.10: ICMP echo reply, id 42734, seq 3, length 64 AFTER: proxy01:/root>tcpdump -ni enp0s8 -e icmp tcpdump: verbose output suppressed, use -v or -vv for full protocol decode listening on enp0s8, link-type EN10MB (Ethernet), capture size 262144 bytes 10:05:49.826037 f0:9f:c2:df:16:01 > 08:00:27:b7:10:fd, ethertype 802.1Q (0x8100), length 102: vlan 20, p 0, ethertype IPv4, 192.168.1.10 > 192.168.20.62: ICMP echo request, id 47854, seq 0, length 64 10:05:50.857622 f0:9f:c2:df:16:01 > 08:00:27:b7:10:fd, ethertype 802.1Q (0x8100), length 102: vlan 20, p 0, ethertype IPv4, 192.168.1.10 > 192.168.20.62: ICMP echo request, id 47854, seq 1, length 64 10:05:51.887233 f0:9f:c2:df:16:01 > 08:00:27:b7:10:fd, ethertype 802.1Q (0x8100), length 102: vlan 20, p 0, ethertype IPv4, 192.168.1.10 > 192.168.20.62: ICMP echo request, id 47854, seq 2, length 64 10:05:52.915618 f0:9f:c2:df:16:01 > 08:00:27:b7:10:fd, ethertype 802.1Q (0x8100), length 102: vlan 20, p 0, ethertype IPv4, 192.168.1.10 > 192.168.20.62: ICMP echo request, id 47854, seq 3, length 64 10:05:53.939994 f0:9f:c2:df:16:01 > 08:00:27:b7:10:fd, ethertype 802.1Q (0x8100), length 102: vlan 20, p 0, ethertype IPv4, 192.168.1.10 > 192.168.20.62: ICMP echo request, id 47854, seq 4, length 64 proxy01:/root>iptables -L Chain INPUT (policy ACCEPT) target prot opt source destination ACCEPT all -- anywhere anywhere state RELATED,ESTABLISHED LOG all -- anywhere anywhere limit: avg 5/min burst 5 LOG level notice prefix "iptables denied: " DROP all -- anywhere anywhere Chain FORWARD (policy ACCEPT) target prot opt source destination Chain OUTPUT (policy ACCEPT) target prot opt source destination