<div dir="ltr"><div>Hi Brian,</div><div><br></div><div>Thanks for the feedback. <br></div><div>I just wanted to enable ufw on strongswan installed instance. <br></div><div>I did not configure any special NAT/POSTROUTE just set default forward policy ACCEPT</div><div>and activated ufw. Suddenly everything started working.</div><div>I disabled UFW and it was still working. I rebooted this ubuntu instance and it is still working. I don't know what magic with ufw fixed the problem</div><div>but It is working fine now</div><div><br></div><div>Ismail</div><div><br></div><div><br></div><div><br></div></div><br><div class="gmail_quote"><div dir="ltr" class="gmail_attr">On Tue, Apr 28, 2020 at 1:07 AM Brian Topping <<a href="mailto:brian.topping@gmail.com">brian.topping@gmail.com</a>> wrote:<br></div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><div style="overflow-wrap: break-word;">I did something similar with interfaces and tunnels a year or two ago and am about to remove the tunnels and go to transport. I think you should consider it and while maybe there are solutions, I’ll try to tell you what I know to start. <div><br></div><div>Basically, the only reason I did tunnels was because of the interfaces (similar to your situation), but I wanted the interfaces so I could run OSPF over it. OSPF requires interfaces that can be multicasted on. Transports work via the kernel so packets semi-magically get routed to the correct destination, but only in unicast (ie direct addressing) mode. It seems like it’s conceptually easier to get started on tunnels because they match a paradigm we are used to in networking with interfaces, but times have changed along with expected scale. </div><div><br></div><div>The actual problem that I am referring to is around reconstruction of tunnels after outages. On long term s2s links, outages are to be expected and all that obviously matters is the protected traffic can flow again as soon as the underlying cause is rectified. Tunnel setup under Strongswan requires use of an “updown script”, and the script is sent a unique identifier that doesn’t seem to live as long as the interfaces do. So when Strongswan senses a failure, it tells the updown script what it knows to delete, it’s outdated information and the script does not actually delete the interface. </div><div><br></div><div>This has effects to bringing up the new network because the interface that wasn’t deleted still has the same IP address as the one that needs to be created when the underlying network failure is rectified. Two interfaces with the same addressing, one of which is connected to nothing… it’s a very tricky failure condition to unravel without just stopping the entire Strongswan process, removing all the interfaces and restarting. I’ve had to do that so many times I’ve gotten very good with the vici interface, so I’m not complaining, just pointing it out.</div><div><br></div><div>So while someone might have a solution to this problem of interface identifiers and making the updown scripts work, I have come to the opinion that gratuitous configuration like interfaces themselves are themselves something to be avoided. Given that this stuff isn’t built in to Strongswan, I’m inclined to believe its authors may feel similarly.</div><div><br></div><div>The last question is then “well, how to do dynamic routing?” For that, I am removing all the OSPF and converting to BGP IGP. Since BGP is unicast instead of broadcast based, there’s no need for an interface, only socket reachability, which transport mode excels at. It’s not as nice from a “plug in a new router and dynamically learn” kind of perspective, but alas, if one gets to a network large enough to be adding routers that quickly, they probably have sufficient staff to be more deliberate about their router setups anyway. </div><div><br></div><div>Hope that provides some value...<br><div><br><blockquote type="cite"><div>On Apr 27, 2020, at 2:52 PM, Ismail Yenigul <<a href="mailto:ismailyenigul@gmail.com" target="_blank">ismailyenigul@gmail.com</a>> wrote:</div><br><div><div dir="ltr"><div>Hi,</div><div><br></div><div>I have an ubuntu ec2 instance on AWS and I have other instances in the same vpc and same subnet. This instance has single network interface (10.0.1.100)</div><div>source check disabled and forwarding active<br></div><div><div style="font-size:16px;line-height:16px"><div id="gmail-m_2691933372726085945gmail-crayon-5ea6f0a3eedcf682709820-1">net.ipv4.ip_forward=1</div><div><br></div></div></div><div>I configured site-to-site vpn on this ubuntu server to a remote Juniper box.</div><div>But at Juniper side, their rightsubnet is just a public IP, not internal subnet<br></div><div>I see that tunnel connects</div><div><br></div><div>leftsubnet=<a href="http://10.0.0.0/24" target="_blank">10.0.0.0/24</a><br>rightsubnet=193.x.y.z/32</div><div><br></div><div>I have another Centos instance which has a single network interface (10.0.1.200 ) and public interface enabled in the same VPC</div><div>I added a manual route on this centos instance<br></div><div><br></div><div># route add 93.x.y.z/32 gw 10.0.1.100 </div><div><br></div><div>When I try to traceroute from 10.0.1.200 to 193.x.y.z</div><div>traffic goes to the internet. <br></div><div><br></div><div>Should I create two interfaces on Ubuntu which strongswan installed and site-to-site vpn configured to avoid direct routing from VPC internet gateway?</div><div>This is the routing created by ipsec tunnel</div><div><br></div><div># ip route show table all<br><b>193.201.x.y via 10.0.0.1 dev eth0 table 220 proto static src 10.0.0.100 </b></div><div>default via 10.0.0.1 dev eth0 proto dhcp src 10.0.0.100 metric 100 <br></div><div><br></div><div># ipsec statusall</div><div><br></div><div>Security Associations (1 up, 0 connecting):<br> tosrx[1]: ESTABLISHED 35 minutes ago, 10.0.0.100[18.X.X.X]... Y.Y.Y..Y[Y.Y.Y.Y]<br> tosrx[1]: IKEv2 SPIs: 552fc130bbd_i* 3c9b77da964_r, pre-shared key reauthentication in 7 hours<br> tosrx[1]: IKE proposal: AES_CBC_256/HMAC_SHA1_96/PRF_HMAC_SHA1/MODP_1024<br> tosrx{1}: INSTALLED, TUNNEL, reqid 1, ESP SPIs: cca011de_i deb94bf2_o<br> tosrx{1}: AES_CBC_256/HMAC_SHA1_96, 360 bytes_i (6 pkts, 1238s ago), 2859 bytes_o (50 pkts, 509s ago), rekeying in 6 minutes<br> tosrx{1}: <a href="http://10.0.0.0/24" target="_blank">10.0.0.0/24</a> === 193.X.Y.Z/32</div></div>
</div></blockquote></div><br></div></div></blockquote></div>