[strongSwan] VTI device and strongswan ikev2 issue

Miroslav Hostinsky bman at tls.sk
Thu Nov 23 10:05:26 CET 2017


Hello,

it seems, that someone has the same issue but nobody was able to help 
him. He workarounded it with GRE instead of VTI, but I don't want to use 
GRE.

https://www.mail-archive.com/users@lists.strongswan.org/msg12642.html

Maybe it is bug in Strongswan, not my misconfiguration.

BR,
Miroslav

On 2017-11-22 23:27, Miroslav Hostinsky wrote:
> Hello,
> 
> I configured the route source IP as you recommended (I hope the right 
> way):
> 
> # ip -4 r s table all
> 8.8.8.8 via 10.198.54.1 dev eth0
> 10.198.54.0/24 dev eth0 proto kernel scope link src 10.198.54.208 
> metric 100
> 95.115.254.165 via 10.198.54.1 dev eth0
> 195.210.28.0/24 dev vti1 scope link src 95.115.254.161
> broadcast 10.198.54.0 dev eth0 table local proto kernel scope link src
> 10.198.54.208
> local 10.198.54.208 dev eth0 table local proto kernel scope host src
> 10.198.54.208
> broadcast 10.198.54.255 dev eth0 table local proto kernel scope link
> src 10.198.54.208
> local 95.115.254.161 dev vti1 table local proto kernel scope host src
> 95.115.254.161
> broadcast 127.0.0.0 dev lo table local proto kernel scope link src 
> 127.0.0.1
> local 127.0.0.0/8 dev lo table local proto kernel scope host src 
> 127.0.0.1
> local 127.0.0.1 dev lo table local proto kernel scope host src 
> 127.0.0.1
> broadcast 127.255.255.255 dev lo table local proto kernel scope link
> src 127.0.0.1
> 
> but still it doesn't work. I think this is not a routing issue (but
> maybe I am wrong). Now I tried the reverse flow.
> 
> I started to ping 95.115.254.161 (my vpn endpoint IP) from
> 195.210.28.63 (I have configured static route to this net on my vpn
> endpoint). I can see that icmp echo request arrive to vpn server at
> 95.115.254.165 (I have access to this server), then is forwarded via
> ipsec tunnel to my endpoint. I see this on eth0 on my vpn endpoint:
> 
> # tcpdump -i eth0 -n udp
> tcpdump: verbose output suppressed, use -v or -vv for full protocol 
> decode
> listening on eth0, link-type EN10MB (Ethernet), capture size 262144 
> bytes
> 23:06:43.155626 IP 95.115.254.165.ipsec-nat-t >
> 10.198.54.208.ipsec-nat-t: UDP-encap: ESP(spi=0xcde002c4,seq=0x2b8),
> length 136
> 23:06:44.155487 IP 95.115.254.165.ipsec-nat-t >
> 10.198.54.208.ipsec-nat-t: UDP-encap: ESP(spi=0xcde002c4,seq=0x2b9),
> length 136
> 23:06:45.155547 IP 95.115.254.165.ipsec-nat-t >
> 10.198.54.208.ipsec-nat-t: UDP-encap: ESP(spi=0xcde002c4,seq=0x2ba),
> length 136
> 23:06:46.155465 IP 95.115.254.165.ipsec-nat-t >
> 10.198.54.208.ipsec-nat-t: UDP-encap: ESP(spi=0xcde002c4,seq=0x2bb),
> length 136
> 23:06:47.155299 IP 95.115.254.165.ipsec-nat-t >
> 10.198.54.208.ipsec-nat-t: UDP-encap: ESP(spi=0xcde002c4,seq=0x2bc),
> length 136
> 23:06:48.155729 IP 95.115.254.165.ipsec-nat-t >
> 10.198.54.208.ipsec-nat-t: UDP-encap: ESP(spi=0xcde002c4,seq=0x2bd),
> length 136
> 
> For me, it seems that these are encrypted icmp echo requests coming
> from 195.210.28.63 (where ping command is running).
> 
> But those packets are not forwarded to vti1. Instead they are counted 
> as errors:
> 
> # ip -s tunnel show
> vti1: ip/ip  remote 95.115.254.165  local 10.198.54.208  ttl inherit  
> key 38
> RX: Packets    Bytes        Errors CsumErrs OutOfSeq Mcasts
>     0          0            889    0        0        0
> TX: Packets    Bytes        Errors DeadLoop NoRoute  NoBufs
>     0          0            0      0        0        0
> 
> As you can see, there is 0 packets transmitted (I am only receiving
> ping) and lot of received errors (those are ICMP requests I suppose).
> 
> What do you think about this? My amateur opinion is, that there is
> issue with passing decrypted packets to the vti1 device and not the
> routing issue, because packets are not even passed to vti1 device.
> 
> I achieve the same result with TCP instead of ICMP (trying to telnet
> to 95.115.254.161, not pinging 95.115.254.161).
> 
> Thank you for the reply
> 
> BR,
> Miroslav
> 
> On 2017-11-22 21:55, Noel Kuntze wrote:
>> Hi,
>> 
>> Responses are in line.
>> 
>> On 22.11.2017 20:22, Miroslav Hostinsky wrote:
>>> Hello,
>>> 
>>> I am not sure if I understand you, but source IP of the outgoing 
>>> packets on vti1 are right for me (it was always 95.115.254.161).
>>> 
>>> # ip a s vti1
>>> 5: vti1 at NONE: <POINTOPOINT,NOARP,UP,LOWER_UP> mtu 1480 qdisc noqueue 
>>> state UNKNOWN qlen 1
>>>     link/ipip 10.198.54.208 peer 95.115.254.165
>>>     inet 95.115.254.161/32 scope global vti1
>>>        valid_lft forever preferred_lft forever
>>> 
>> 
>> No. That is just an IP that is bound to the interface. It has nothing
>> to do with the source IP of any packets. The kernel does not choose
>> the IP based on an IP that is bound to an interface.
>> It does it based on the routing table and the routing decision.
>> 
>>> 
>>> current ipv4 routing table is following (I simplified my 
>>> configuration, removed second physical interface, removed default 
>>> route):
>>> 
>>> # ip -4 r s table all
>>> 8.8.8.8 via 10.198.54.1 dev eth0
>>> 10.198.54.0/24 dev eth0 proto kernel scope link src 10.198.54.208 
>>> metric 100
>>> 95.115.254.165 via 10.198.54.1 dev eth0
>>> 195.210.28.0/24 dev vti1 scope link
>>> broadcast 10.198.54.0 dev eth0 table local proto kernel scope link 
>>> src 10.198.54.208
>>> local 10.198.54.208 dev eth0 table local proto kernel scope host src 
>>> 10.198.54.208
>>> broadcast 10.198.54.255 dev eth0 table local proto kernel scope link 
>>> src 10.198.54.208
>>> local 95.115.254.161 dev vti1 table local proto kernel scope host src 
>>> 95.115.254.161
>>> broadcast 127.0.0.0 dev lo table local proto kernel scope link src 
>>> 127.0.0.1
>>> local 127.0.0.0/8 dev lo table local proto kernel scope host src 
>>> 127.0.0.1
>>> local 127.0.0.1 dev lo table local proto kernel scope host src 
>>> 127.0.0.1
>>> broadcast 127.255.255.255 dev lo table local proto kernel scope link 
>>> src 127.0.0.1
>>> 
>> 
>> I wonder what the kernel now does, because it has no way to infer the
>> recommended source IP for traffic to 0.0.0.0/0.
>> 
>>> There is the static route 195.210.28.0/24 via vti1. If I do “ping 
>>> 195.210.28.63” I can see using tcpdump on vti1 following output:
>>> 
>>> # tcpdump -i vti1 -n
>>> tcpdump: verbose output suppressed, use -v or -vv for full protocol 
>>> decode
>>> listening on vti1, link-type RAW (Raw IP), capture size 262144 bytes
>>> 20:08:07.776307 IP 95.115.254.161 > 195.210.28.63: ICMP echo request, 
>>> id 1098, seq 480, length 64
>>> 20:08:08.776218 IP 95.115.254.161 > 195.210.28.63: ICMP echo request, 
>>> id 1098, seq 481, length 64
>>> 20:08:09.776228 IP 95.115.254.161 > 195.210.28.63: ICMP echo request, 
>>> id 1098, seq 482, length 64
>>> 20:08:10.776196 IP 95.115.254.161 > 195.210.28.63: ICMP echo request, 
>>> id 1098, seq 483, length 64
>>> 
>>> 
>> 
>> I guess it falls back to the primary IP of the interface that the
>> traffic is sent out of.
>> 
>> 
>>> I can see there, that source IP is 95.115.254.161, so right IP 
>>> address. This flow is running over IKEv2 tunnel.
>> 
>> The IKE protocol in use is irrelevant.
>> 
>>> 
>>> If I do tcpdump on underlying eth0 device:
>>> 
>>> # tcpdump -i eth0 -n udp
>>> tcpdump: verbose output suppressed, use -v or -vv for full protocol 
>>> decode
>>> listening on eth0, link-type EN10MB (Ethernet), capture size 262144 
>>> bytes
>>> 20:09:31.778294 IP 10.198.54.208.ipsec-nat-t > 
>>> 95.115.254.165.ipsec-nat-t: UDP-encap: ESP(spi=0xc39fd86f,seq=0x234), 
>>> length 136
>>> 20:09:31.792601 IP 95.115.254.165.ipsec-nat-t > 
>>> 10.198.54.208.ipsec-nat-t: UDP-encap: ESP(spi=0xc69deaf7,seq=0x24c), 
>>> length 136
>>> 20:09:32.131698 IP 95.115.254.165.ipsec-nat-t > 
>>> 10.198.54.208.ipsec-nat-t: UDP-encap: ESP(spi=0xc69deaf7,seq=0x24d), 
>>> length 104
>>> 20:09:32.778291 IP 10.198.54.208.ipsec-nat-t > 
>>> 95.115.254.165.ipsec-nat-t: UDP-encap: ESP(spi=0xc39fd86f,seq=0x235), 
>>> length 136
>>> 20:09:32.792787 IP 95.115.254.165.ipsec-nat-t > 
>>> 10.198.54.208.ipsec-nat-t: UDP-encap: ESP(spi=0xc69deaf7,seq=0x24e), 
>>> length 136
>>> 20:09:33.778217 IP 10.198.54.208.ipsec-nat-t > 
>>> 95.115.254.165.ipsec-nat-t: UDP-encap: ESP(spi=0xc39fd86f,seq=0x236), 
>>> length 136
>>> 20:09:33.792943 IP 95.115.254.165.ipsec-nat-t > 
>>> 10.198.54.208.ipsec-nat-t: UDP-encap: ESP(spi=0xc69deaf7,seq=0x24f), 
>>> length 136
>>> 
>> 
>> What eth0 does is irrelevant, too.
>> 
>>> I guess that 10.198.54.208->95.115.254.165 is ICMP echo request 
>>> (which I see on vti1 when doing tcpdump) and 
>>> 95.115.254.165->10.198.54.208 is ICMP echo reply (this I only guess, 
>>> but there is no other traffic than this ICMP flow).
>> 
>> It can also be ICMP errors. What traffic passes the other peer?
>> 
>>> 
>>> 
>>> For reference I am pasting statusall outout:
>>> 
>>> # strongswan statusall
>>> Status of IKE charon daemon (strongSwan 5.5.3, Linux 
>>> 3.10.0-693.5.2.el7.x86_64, x86_64):
>>>   uptime: 18 minutes, since Nov 22 19:59:23 2017
>>>   malloc: sbrk 2859008, mmap 0, used 562720, free 2296288
>>>   worker threads: 11 of 16 idle, 5/0/0/0 working, job queue: 0/0/0/0, 
>>> scheduled: 2
>>>   loaded plugins: charon aes des rc2 sha2 sha1 md4 md5 random nonce 
>>> x509 revocation constraints acert pubkey pkcs1 pkcs8 pkcs12 pgp 
>>> dnskey sshkey pem openssl gcrypt fips-prf gmp curve25519 xcbc cmac 
>>> hmac ctr ccm gcm curl attr kernel-netlink resolve socket-default farp 
>>> stroke vici updown eap-identity eap-md5 eap-gtc eap-mschapv2 eap-tls 
>>> eap-ttls eap-peap xauth-generic xauth-eap xauth-pam xauth-noauth dhcp 
>>> unity
>>> Listening IP addresses:
>>>   10.198.54.208
>>>   95.115.254.161
>>> Connections:
>>>      alconet:  %any...ipsec.alconet.sk <http://ipsec.alconet.sk> 
>>>  IKEv2, dpddelay=30s
>>>      alconet:   local:  [bmanovic] uses EAP authentication with EAP 
>>> identity '%any'
>>>      alconet:   remote: [ipsec.alconet.sk <http://ipsec.alconet.sk>] 
>>> uses public key authentication
>>>      alconet:   child:  dynamic === 0.0.0.0/0 ::/0 TUNNEL, 
>>> dpdaction=restart
>>> Security Associations (1 up, 0 connecting):
>>>      alconet[1]: ESTABLISHED 18 minutes ago, 
>>> 10.198.54.208[bmanovic]...95.115.254.165[ipsec.alconet.sk 
>>> <http://ipsec.alconet.sk>]
>>>      alconet[1]: IKEv2 SPIs: aac4fe8159ef3f03_i* d183908ac6976b91_r, 
>>> rekeying disabled
>>>      alconet[1]: IKE proposal: 
>>> AES_CBC_128/HMAC_SHA2_256_128/PRF_HMAC_SHA2_256/CURVE_25519
>>>      alconet{1}:  INSTALLED, TUNNEL, reqid 1, ESP in UDP SPIs: 
>>> c69deaf7_i c39fd86f_o, IPCOMP CPIs: 9858_i 7f79_o
>>>      alconet{1}:  AES_CBC_128/HMAC_SHA2_256_128, 0 bytes_i, 91560 
>>> bytes_o (1090 pkts, 1s ago), rekeying disabled
>>>      alconet{1}:   95.115.254.161/32 === 0.0.0.0/0
>>> 
>>> As I wrote above, I simplified my configuration, but the result is 
>>> the same. It doesn't work.
>>> 
>>> Do you still think, that this is that source IP issue?
>> 
>> Not anymore, but I'm sure some software gets the source IP wrong,
>> because the hints are (still) wrong.
>> Ping might just work, because it does special things.
>> 
>> Kind regards
>> 
>> Noel
>> 
>>> 
>>> Thank you for your help,
>>> 
>>> BR,
>>> Miroslav
>>> 
>>> 
>>>> On 22 Nov 2017, at 19:22, Noel Kuntze 
>>>> <noel.kuntze+strongswan-users-ml at thermi.consulting 
>>>> <mailto:noel.kuntze+strongswan-users-ml at thermi.consulting>> wrote:
>>>> 
>>>> Hi,
>>>> 
>>>> The route over the VTI device does not indicate a preferred source 
>>>> IP, so the kernel takes the one from the default route, which is in 
>>>> term deferred from the route to 10.198.54.0/24,
>>>> which is 10.198.54.208. I'm pretty sure running tcpdump on the VTI 
>>>> device confirms that. So the source IP is wrong, as I wrote before. 
>>>> You need to set the virtual IP as source IP in your updown script.
>>>> 
>>>> Kind regards
>>>> 
>>>> Noel
>>>> 
>>>> On 22.11.2017 18:49, Miroslav Hostinsky wrote:
>>>>> Hello Noel,
>>>>> 
>>>>> I do not know what you exactly mean, but the source IP send over 
>>>>> VTI interface is the same as configured on VTI interface 
>>>>> (95.115.254.161 in this case). As I wrote, I can see that ICMP echo 
>>>>> request & reply are delivered to this IPSEC endpoint machine (but I 
>>>>> only suppose, because packets are encrypted when sniffing using 
>>>>> tcpdump, but there is no other ICMP traffic). It seems that 
>>>>> encrypted echo-reply is delivered to the machine, but  
>>>>> "kernel/ipsec stack" is not able to properly "route" to the VTI 
>>>>> device.
>>>>> 
>>>>> Actually my IPSEC/*swan knowledge is not very good so sorry if my 
>>>>> answer are dumb.
>>>>> 
>>>>> I am attaching logs/configs as you requested.
>>>>> 
>>>>> Thank you,
>>>>> 
>>>>> BR,
>>>>> Miroslav
>>>>> 
>>>>> On 2017-11-22 17:41, Noel Kuntze wrote:
>>>>>> Hello Miroslav,
>>>>>> 
>>>>>> I suspect that the policy lookup for the received packets fail. 
>>>>>> Check
>>>>>> what the source of the packets is that you send over the vti 
>>>>>> device.
>>>>>> Anyway, please provide the full list of information from the
>>>>>> HelpRequests[1] page.
>>>>>> 
>>>>>> [1] 
>>>>>> https://wiki.strongswan.org/projects/strongswan/wiki/HelpRequests
>>>>>> 
>>>>>> Kind regards
>>>>>> 
>>>>>> Noel
>>>>>> 
>>>>>> On 22.11.2017 16:47, Miroslav Hostinsky wrote:
>>>>>>> Hello,
>>>>>>> 
>>>>>>> I have an issue configuring StrongSwan with VTI interface as 
>>>>>>> roadwarrior. This is my configuration:
>>>>>>> 
>>>>>>> ipsec.conf:
>>>>>>> 
>>>>>>> config setup
>>>>>>> 
>>>>>>> conn %default
>>>>>>>   keyexchange=ikev2
>>>>>>>   ikelifetime=60m
>>>>>>>   keylife=20m
>>>>>>>   rekeymargin=3m
>>>>>>>   rekey=no
>>>>>>>   dpdaction=restart
>>>>>>>   dpddelay=30s
>>>>>>>   compress=yes
>>>>>>>   auto=start
>>>>>>> 
>>>>>>> conn acnnet
>>>>>>>   leftupdown=/usr/local/sbin/ipsec-notify.sh
>>>>>>>   left=%defaultroute
>>>>>>>   leftauth=eap
>>>>>>>   leftsourceip=%config4,%config6
>>>>>>>   rightauth=pubkey
>>>>>>>   rightsubnet=0.0.0.0/0,::/0
>>>>>>>   eap_identity=%identity
>>>>>>>   leftid=bman
>>>>>>>   right=mailer.domena.sk <http://mailer.domena.sk>
>>>>>>>   rightid=@mailer.mailer.sk <mailto:rightid=@mailer.mailer.sk>
>>>>>>>   mark=28
>>>>>>> 
>>>>>>> VTI interface is configured using lefupdown script (real commands 
>>>>>>> executed):
>>>>>>> 
>>>>>>> ip tunnel add vti1 local 85.105.254.225 remote 185.210.28.63 mode 
>>>>>>> vti key 28 ikey 28
>>>>>>> ip link set vti1 up
>>>>>>> ip addr add 192.168.228.10 dev vti1
>>>>>>> ip route add 74.99.179.0/24 dev vti1
>>>>>>> sysctl -w net.ipv4.conf.vti1.disable_policy=1
>>>>>>> 
>>>>>>> It seems that outgoing connection via vti1 interface is working 
>>>>>>> (outgoing ICMP echo request to subnet 74.99.179.0/24 ). But I am 
>>>>>>> unable to receive ICMP echo reply. Using tcpdump I can clearly 
>>>>>>> see, that IPSEC encrypted ICMP echo reply is returning via 
>>>>>>> physical interface, but not via vti1.
>>>>>>> 
>>>>>>> 
>>>>>>> I found, that, TX bytes is correctly counted via vti1, but RX 
>>>>>>> shows errors (it seems that each ICMP echo reply packet is 
>>>>>>> counted as +1 error):
>>>>>>> 
>>>>>>> # ip -s tunnel show
>>>>>>> vti1: ip/ip  remote 185.210.28.63  local 85.105.254.225 ttl 
>>>>>>> inherit  key 28
>>>>>>> RX: Packets    Bytes        Errors CsumErrs OutOfSeq Mcasts
>>>>>>>     0          0            805    0        0 0
>>>>>>> TX: Packets    Bytes        Errors DeadLoop NoRoute NoBufs
>>>>>>>     401        68170        0      0        0 0
>>>>>>> ip_vti0: ip/ip  remote any  local any  ttl inherit nopmtudisc key 
>>>>>>> 0
>>>>>>> RX: Packets    Bytes        Errors CsumErrs OutOfSeq Mcasts
>>>>>>>     0          0            0      0        0 0
>>>>>>> TX: Packets    Bytes        Errors DeadLoop NoRoute NoBufs
>>>>>>>     0          0            0      0        0 0
>>>>>>> 
>>>>>>> It seems that, RX Errors on vti1 are currently missing ICMP echo 
>>>>>>> reply packets. But is counted as RX errors, not RX received 
>>>>>>> packets.
>>>>>>> 
>>>>>>> Do you have any idea what's wrong?
>>>>>>> 
>>>>>>> I am using Centos 7.4 with strongswan-5.5.3-1.el7.x86_64 from 
>>>>>>> EPEL. A tried with same result on Archlinux (kernel 4.9 and 
>>>>>>> strongswan 5.6.0).
>>>>>>> 
>>>>>>> Route installation is disabled in charon.conf.
>>>>>>> 
>>>>>>> Normal connection using Virtual IP is working great.
>>>>>>> 
>>>>>>> 
>>>>>>> Thank you very much for any help.
>>>>>>> 
>>>>>>> 
>>>>>>> BR,
>>>>>>> 
>>>>>>> Miroslav
>>>>> 
>>>> 
>>> 

-- 
Miroslav Hostinsky


More information about the Users mailing list