<html>
<head>
<meta http-equiv="content-type" content="text/html; charset=UTF-8">
</head>
<body text="#000000" bgcolor="#FFFFFF">
<p>Hello,</p>
<p>Sorry for disturbing.<br>
</p>
<p>I try to start using IPSec VTI technology with single shared VTI
device on Centos 7 with several pfSense routers as roadwarriors.</p>
<p>I want to implement this approach because prefer using dynamic
routing with help of OSPF or BGP protocols.<br>
Aside this the way of setting tunnels over IPSec+GRE works but it
looks more like a conglomeration of different methods for reaching
a desired instead of using something simple.</p>
<p>Despite on the fact of simple configuration and good
documentation describing how to run shared VTI device on Linux, I
have not been able to implement my plan for several weeks.<br>
That is why I have decided to ask for your help.</p>
<p>Here is the config of VPN gateway on Centos 7:</p>
<ol>
<li>iproute2<br>
<blockquote type="cite"># ip tunnel show<br>
ip_vti0: ip/ip remote any local any ttl inherit nopmtudisc key
0<br>
ipsec0: ip/ip remote any local <public gateway IP
address> ttl inherit key 2<br>
<br>
# ip add show ipsec0<br>
5: ipsec0@NONE: <NOARP,UP,LOWER_UP> mtu 1480 qdisc
noqueue state UNKNOWN group default qlen 1000<br>
link/ipip <public gateway IP address> brd 0.0.0.0<br>
inet 10.10.50.1/24 scope global ipsec0<br>
valid_lft forever preferred_lft forever<br>
<br>
# ip xfrm state<br>
src <public gateway IP address> dst <public alice IP
address><br>
proto esp spi 0xc02a1647 reqid 1 mode tunnel<br>
replay-window 0 flag af-unspec<br>
mark 0x2/0xffffffff<br>
auth-trunc hmac(sha256)
0xf4bebd29572077ffd2de2fd94ef5789db9a64bc0d0486840944d8c151ddb1a00
128<br>
enc cbc(aes) 0x3e53b08a64734a080f88ea29c1c4d8d5<br>
anti-replay context: seq 0x0, oseq 0x0, bitmap
0x00000000<br>
src <public alice IP address> dst <public gateway IP
address><br>
proto esp spi 0xc143c654 reqid 1 mode tunnel<br>
replay-window 32 flag af-unspec<br>
auth-trunc hmac(sha256)
0x2685a694d1bba26d113396f34611f31ec19ee7ac8b132a535d6772132616bdd1
128<br>
enc cbc(aes) 0xda453318ab6c3c8a8a15bed3addff236<br>
anti-replay context: seq 0x0, oseq 0x0, bitmap
0x00000000<br>
src <public gateway IP address> dst <public bob IP
address><br>
proto esp spi 0xca6589d4 reqid 1 mode tunnel<br>
replay-window 0 flag af-unspec<br>
mark 0x2/0xffffffff<br>
auth-trunc hmac(sha256)
0x57fcc86f599da0bce04558007094a87e43ad5541539b540146297d266b838c09
128<br>
enc cbc(aes) 0xb48066ff05c8de3a8cab2ff7fa64b3fa<br>
anti-replay context: seq 0x0, oseq 0x0, bitmap
0x00000000<br>
src <public bob IP address> dst <public gateway IP
address><br>
proto esp spi 0xc8cad11b reqid 1 mode tunnel<br>
replay-window 32 flag af-unspec<br>
auth-trunc hmac(sha256)
0x362e088f0d60b4204bde527674952cc80f4855b033d22625c3f2124b3d022d37
128<br>
enc cbc(aes) 0x4f00332b9392bd69cc3c5bfc0de2b1f2<br>
anti-replay context: seq 0x0, oseq 0x0, bitmap
0x00000000<br>
</blockquote>
<br>
</li>
<li>strongswan.conf<br>
<blockquote type="cite">charon {<br>
load_modular = yes<br>
# Install routes into a separate routing table for
established IPSec tunnels.<br>
install_routes = no<br>
<br>
# Install virtual IP addresses.<br>
install_virtual_ip = no<br>
<br>
plugins {<br>
include strongswan.d/charon/*.conf<br>
}<br>
}</blockquote>
<br>
<br>
</li>
<li>ipsec.conf<br>
<blockquote type="cite"># ipsec.conf - strongSwan IPsec
configuration file<br>
config setup<br>
uniqueids=never<br>
charondebug="cfg 1, dmn 1, ike 1, net 0"<br>
<br>
conn %default<br>
leftauth=pubkey<br>
rightauth=pubkey<br>
ike=aes128-sha2_256-modp2048!<br>
ikelifetime=28800s<br>
aggressive=no<br>
esp=aes128-sha2_256-modp2048!<br>
lifetime=3600s<br>
type=tunnel<br>
dpddelay=20s<br>
dpdtimeout=30s<br>
dpdaction=restart<br>
keyexchange=ikev2<br>
rekey=yes<br>
reauth=no<br>
closeaction=restart<br>
leftsubnet=0.0.0.0/0<br>
rightsubnet=0.0.0.0/0<br>
installpolicy=yes<br>
compress=no<br>
mobike=no<br>
<br>
conn alice<br>
auto=route<br>
<a class="moz-txt-link-abbreviated" href="mailto:leftid=@vpn.routers.example.com">leftid=@vpn.routers.example.com</a><br>
leftcert=vpn.routers.example.com.crt<br>
right=%any<br>
<a class="moz-txt-link-abbreviated" href="mailto:rightid=@alice.routers.example.com">rightid=@alice.routers.example.com</a><br>
mark=2<br>
<br>
conn bob<br>
auto=route<br>
<a class="moz-txt-link-abbreviated" href="mailto:leftid=@vpn.routers.example.com">leftid=@vpn.routers.example.com</a><br>
leftcert=vpn.routers.example.com.crt<br>
right=%any<br>
<a class="moz-txt-link-abbreviated" href="mailto:rightid=@bob.routers.example.com">rightid=@bob.routers.example.com</a><br>
mark=2</blockquote>
<br>
</li>
<li>swanctl output:<br>
<blockquote type="cite"># swanctl --list-sas<br>
alice: #2, ESTABLISHED, IKEv2, 8024abf57579427c_i
d9f6a862d18493a9_r*<br>
local 'vpn.routers.example.com' @ xxx.xxx.xx.xx[500]<br>
remote 'alice.routers.example.com' @ yyy.yyy.yyy.yy[500]<br>
AES_CBC-128/HMAC_SHA2_256_128/PRF_HMAC_SHA2_256/MODP_2048<br>
established 4s ago, rekeying in 27954s<br>
gateway-ekb: #2, reqid 1, INSTALLED, TUNNEL,
ESP:AES_CBC-128/HMAC_SHA2_256_128<br>
installed 4s ago, rekeying in 2541s, expires in 3596s<br>
in c143c654 (0x00000002), 0 bytes, 0 packets<br>
out c02a1647 (0x00000002), 0 bytes, 0 packets<br>
local 0.0.0.0/0<br>
remote 0.0.0.0/0<br>
bob: #1, ESTABLISHED, IKEv2, 239d807fda28ae2f_i
d66b0d9da8df6668_r*<br>
local 'vpn.routers.example.com' @ xxx.xxx.xx.xx[500]<br>
remote 'bob.routers.example.com' @ zz.zzz.zz.zz[500]<br>
AES_CBC-128/HMAC_SHA2_256_128/PRF_HMAC_SHA2_256/MODP_2048<br>
established 8s ago, rekeying in 27769s<br>
gateway-krk: #1, reqid 1, INSTALLED, TUNNEL,
ESP:AES_CBC-128/HMAC_SHA2_256_128<br>
installed 8s ago, rekeying in 2657s, expires in 3592s<br>
in c8cad11b (0x00000002), 0 bytes, 0 packets<br>
out ca6589d4 (0x00000002), 0 bytes, 0 packets<br>
local 0.0.0.0/0<br>
remote 0.0.0.0/0</blockquote>
<br>
</li>
<li><br>
</li>
</ol>
<p>Here is the config of Alice that is used on pfSense (config of
Bob is equal except IP addresses and certificates):</p>
<ol>
<li>ipsec.conf:<br>
<blockquote type="cite">conn gateway<br>
reqid = 2000<br>
fragmentation = yes<br>
keyexchange = ikev2<br>
reauth = yes<br>
forceencaps = no<br>
mobike = no<br>
<br>
rekey = no<br>
installpolicy = no<br>
<br>
dpdaction = restart<br>
dpddelay = 10s<br>
dpdtimeout = 60s<br>
auto = start<br>
left = 188.234.247.71<br>
right = vpn.routers.example.com<br>
leftid = fqdn:alice.routers.example.com<br>
ikelifetime = 28800s<br>
lifetime = 3600s<br>
ike = aes128-sha256-modp2048!<br>
esp =
aes128-sha256-modp2048,aes128gcm128-sha256-modp2048!<br>
leftauth = pubkey<br>
rightauth = pubkey<br>
leftcert=/var/etc/ipsec/ipsec.d/certs/cert-2.crt<br>
leftsendcert=always<br>
rightca="<some content>"<br>
rightid = fqdn:vpn.routers.example.com<br>
rightsubnet = 10.10.50.1,0.0.0.0/0<br>
leftsubnet = 10.10.50.2/24,0.0.0.0/0</blockquote>
<br>
</li>
<li>strongswan.conf<br>
<blockquote type="cite">starter {<br>
load_warning = no<br>
config_file = /var/etc/ipsec/ipsec.conf<br>
}<br>
<br>
charon {<br>
# number of worker threads in charon<br>
threads = 16<br>
ikesa_table_size = 32<br>
ikesa_table_segments = 4<br>
init_limit_half_open = 1000<br>
install_routes = no<br>
load_modular = yes<br>
ignore_acquire_ts = yes<br>
<br>
<br>
cisco_unity = no<br>
<br>
<br>
<br>
syslog {<br>
identifier = charon<br>
# log everything under daemon since it ends up
in the same place regardless with our syslog.conf<br>
daemon {<br>
ike_name = yes<br>
dmn = 1<br>
mgr = 1<br>
ike = 2<br>
chd = 2<br>
job = 1<br>
cfg = 2<br>
knl = 1<br>
net = 1<br>
asn = 1<br>
enc = 1<br>
imc = 1<br>
imv = 1<br>
pts = 1<br>
tls = 1<br>
esp = 1<br>
lib = 1<br>
<br>
}<br>
# disable logging under auth so logs aren't
duplicated<br>
auth {<br>
default = -1<br>
}<br>
}<br>
<br>
plugins {<br>
# Load defaults<br>
include
/var/etc/ipsec/strongswan.d/charon/*.conf<br>
<br>
stroke {<br>
secrets_file =
/var/etc/ipsec/ipsec.secrets<br>
}<br>
<br>
unity {<br>
load = no<br>
}<br>
<br>
}<br>
}</blockquote>
</li>
</ol>
<p><br>
</p>
<p>The issue is a weird behavior, which reasons I cannot guess. <br>
I configured Centos 7 and created shared VTI device according to
the documentation of Strongswan. I set up IPSec VTI on pfSense
routers and they can establish connections.<br>
However, when Alice connects to VPN gateway I see the following
XFRM policy:</p>
<p>
<blockquote type="cite"># ip xfrm policy<br>
src 0.0.0.0/0 dst 0.0.0.0/0<br>
dir out priority 399999 ptype main<br>
mark 0x2/0xffffffff<br>
tmpl src <public gateway IP address> dst
<public alice IP address><br>
proto esp spi 0xc02a1647 reqid 1 mode tunnel<br>
src 0.0.0.0/0 dst 0.0.0.0/0<br>
dir fwd priority 399999 ptype main<br>
mark 0x2/0xffffffff<br>
tmpl src <public alice IP address> dst <public
gateway IP address><br>
proto esp reqid 1 mode tunnel<br>
src 0.0.0.0/0 dst 0.0.0.0/0<br>
dir in priority 399999 ptype main<br>
mark 0x2/0xffffffff<br>
tmpl src <public alice IP address> dst <public
gateway IP address><br>
proto esp reqid 1 mode tunnel<br>
src 0.0.0.0/0 dst 0.0.0.0/0<br>
socket in priority 0 ptype main<br>
src 0.0.0.0/0 dst 0.0.0.0/0<br>
socket out priority 0 ptype main<br>
src 0.0.0.0/0 dst 0.0.0.0/0<br>
socket in priority 0 ptype main<br>
src 0.0.0.0/0 dst 0.0.0.0/0<br>
socket out priority 0 ptype main<br>
src ::/0 dst ::/0<br>
socket in priority 0 ptype main<br>
src ::/0 dst ::/0<br>
socket out priority 0 ptype main<br>
src ::/0 dst ::/0<br>
socket in priority 0 ptype main<br>
src ::/0 dst ::/0<br>
socket out priority 0 ptype main</blockquote>
and I can ping endpoints of the tunnel and even resources behind
them if static routes are specified.<br>
</p>
<p>But if Bob connects to the gateway after Alice the XFRM policy is
being changed:</p>
<p>
<blockquote type="cite"># ip xfrm policy<br>
src 0.0.0.0/0 dst 0.0.0.0/0 <br>
dir out priority 399999 ptype main <br>
mark 0x2/0xffffffff<br>
tmpl src <public gateway IP address> dst
<public bob IP address><br>
proto esp spi 0xc17ca64f reqid 1 mode tunnel<br>
src 0.0.0.0/0 dst 0.0.0.0/0 <br>
dir fwd priority 399999 ptype main <br>
mark 0x2/0xffffffff<br>
tmpl src <public bob IP address> dst <public
gateway IP address><br>
proto esp reqid 1 mode tunnel<br>
src 0.0.0.0/0 dst 0.0.0.0/0 <br>
dir in priority 399999 ptype main <br>
mark 0x2/0xffffffff<br>
tmpl src <public bob IP address> dst <public
gateway IP address><br>
proto esp reqid 1 mode tunnel<br>
src 0.0.0.0/0 dst 0.0.0.0/0 <br>
socket in priority 0 ptype main <br>
src 0.0.0.0/0 dst 0.0.0.0/0 <br>
socket out priority 0 ptype main <br>
src 0.0.0.0/0 dst 0.0.0.0/0 <br>
socket in priority 0 ptype main <br>
src 0.0.0.0/0 dst 0.0.0.0/0 <br>
socket out priority 0 ptype main <br>
src ::/0 dst ::/0 <br>
socket in priority 0 ptype main <br>
src ::/0 dst ::/0 <br>
socket out priority 0 ptype main <br>
src ::/0 dst ::/0 <br>
socket in priority 0 ptype main <br>
src ::/0 dst ::/0 <br>
socket out priority 0 ptype main</blockquote>
That means that traffic between the gateway and Alice stops
passing but traffic between the gateway and Bob starts going. <br>
If connections are established to the gateway in opposite order
the behavior is the same.<br>
</p>
<p>The goal is to use one shared VTI device for both simultaneous
connections.</p>
<p>Perhaps my <span class="tlid-translation translation" lang="en"><span
title="" class="">eyes are soiled and I don't see a mistake.
If it is so, please point me to it.<br>
If you already have experience of how to implement the
aforementioned above, I beg you to share that with me.</span></span></p>
<p><span class="tlid-translation translation" lang="en"><span
title="" class="">Sorry for bothering again and thank you in
advance.<br>
</span></span></p>
<div class="moz-signature">-- <br>
Regards, <br>
Aleksey Zolotuhin
<br>
<br>
</div>
</body>
</html>