[strongSwan] VTI device and strongswan ikev2 issue
Miroslav Hostinsky
bman at tls.sk
Thu Nov 23 10:05:26 CET 2017
Hello,
it seems, that someone has the same issue but nobody was able to help
him. He workarounded it with GRE instead of VTI, but I don't want to use
GRE.
https://www.mail-archive.com/users@lists.strongswan.org/msg12642.html
Maybe it is bug in Strongswan, not my misconfiguration.
BR,
Miroslav
On 2017-11-22 23:27, Miroslav Hostinsky wrote:
> Hello,
>
> I configured the route source IP as you recommended (I hope the right
> way):
>
> # ip -4 r s table all
> 8.8.8.8 via 10.198.54.1 dev eth0
> 10.198.54.0/24 dev eth0 proto kernel scope link src 10.198.54.208
> metric 100
> 95.115.254.165 via 10.198.54.1 dev eth0
> 195.210.28.0/24 dev vti1 scope link src 95.115.254.161
> broadcast 10.198.54.0 dev eth0 table local proto kernel scope link src
> 10.198.54.208
> local 10.198.54.208 dev eth0 table local proto kernel scope host src
> 10.198.54.208
> broadcast 10.198.54.255 dev eth0 table local proto kernel scope link
> src 10.198.54.208
> local 95.115.254.161 dev vti1 table local proto kernel scope host src
> 95.115.254.161
> broadcast 127.0.0.0 dev lo table local proto kernel scope link src
> 127.0.0.1
> local 127.0.0.0/8 dev lo table local proto kernel scope host src
> 127.0.0.1
> local 127.0.0.1 dev lo table local proto kernel scope host src
> 127.0.0.1
> broadcast 127.255.255.255 dev lo table local proto kernel scope link
> src 127.0.0.1
>
> but still it doesn't work. I think this is not a routing issue (but
> maybe I am wrong). Now I tried the reverse flow.
>
> I started to ping 95.115.254.161 (my vpn endpoint IP) from
> 195.210.28.63 (I have configured static route to this net on my vpn
> endpoint). I can see that icmp echo request arrive to vpn server at
> 95.115.254.165 (I have access to this server), then is forwarded via
> ipsec tunnel to my endpoint. I see this on eth0 on my vpn endpoint:
>
> # tcpdump -i eth0 -n udp
> tcpdump: verbose output suppressed, use -v or -vv for full protocol
> decode
> listening on eth0, link-type EN10MB (Ethernet), capture size 262144
> bytes
> 23:06:43.155626 IP 95.115.254.165.ipsec-nat-t >
> 10.198.54.208.ipsec-nat-t: UDP-encap: ESP(spi=0xcde002c4,seq=0x2b8),
> length 136
> 23:06:44.155487 IP 95.115.254.165.ipsec-nat-t >
> 10.198.54.208.ipsec-nat-t: UDP-encap: ESP(spi=0xcde002c4,seq=0x2b9),
> length 136
> 23:06:45.155547 IP 95.115.254.165.ipsec-nat-t >
> 10.198.54.208.ipsec-nat-t: UDP-encap: ESP(spi=0xcde002c4,seq=0x2ba),
> length 136
> 23:06:46.155465 IP 95.115.254.165.ipsec-nat-t >
> 10.198.54.208.ipsec-nat-t: UDP-encap: ESP(spi=0xcde002c4,seq=0x2bb),
> length 136
> 23:06:47.155299 IP 95.115.254.165.ipsec-nat-t >
> 10.198.54.208.ipsec-nat-t: UDP-encap: ESP(spi=0xcde002c4,seq=0x2bc),
> length 136
> 23:06:48.155729 IP 95.115.254.165.ipsec-nat-t >
> 10.198.54.208.ipsec-nat-t: UDP-encap: ESP(spi=0xcde002c4,seq=0x2bd),
> length 136
>
> For me, it seems that these are encrypted icmp echo requests coming
> from 195.210.28.63 (where ping command is running).
>
> But those packets are not forwarded to vti1. Instead they are counted
> as errors:
>
> # ip -s tunnel show
> vti1: ip/ip remote 95.115.254.165 local 10.198.54.208 ttl inherit
> key 38
> RX: Packets Bytes Errors CsumErrs OutOfSeq Mcasts
> 0 0 889 0 0 0
> TX: Packets Bytes Errors DeadLoop NoRoute NoBufs
> 0 0 0 0 0 0
>
> As you can see, there is 0 packets transmitted (I am only receiving
> ping) and lot of received errors (those are ICMP requests I suppose).
>
> What do you think about this? My amateur opinion is, that there is
> issue with passing decrypted packets to the vti1 device and not the
> routing issue, because packets are not even passed to vti1 device.
>
> I achieve the same result with TCP instead of ICMP (trying to telnet
> to 95.115.254.161, not pinging 95.115.254.161).
>
> Thank you for the reply
>
> BR,
> Miroslav
>
> On 2017-11-22 21:55, Noel Kuntze wrote:
>> Hi,
>>
>> Responses are in line.
>>
>> On 22.11.2017 20:22, Miroslav Hostinsky wrote:
>>> Hello,
>>>
>>> I am not sure if I understand you, but source IP of the outgoing
>>> packets on vti1 are right for me (it was always 95.115.254.161).
>>>
>>> # ip a s vti1
>>> 5: vti1 at NONE: <POINTOPOINT,NOARP,UP,LOWER_UP> mtu 1480 qdisc noqueue
>>> state UNKNOWN qlen 1
>>> link/ipip 10.198.54.208 peer 95.115.254.165
>>> inet 95.115.254.161/32 scope global vti1
>>> valid_lft forever preferred_lft forever
>>>
>>
>> No. That is just an IP that is bound to the interface. It has nothing
>> to do with the source IP of any packets. The kernel does not choose
>> the IP based on an IP that is bound to an interface.
>> It does it based on the routing table and the routing decision.
>>
>>>
>>> current ipv4 routing table is following (I simplified my
>>> configuration, removed second physical interface, removed default
>>> route):
>>>
>>> # ip -4 r s table all
>>> 8.8.8.8 via 10.198.54.1 dev eth0
>>> 10.198.54.0/24 dev eth0 proto kernel scope link src 10.198.54.208
>>> metric 100
>>> 95.115.254.165 via 10.198.54.1 dev eth0
>>> 195.210.28.0/24 dev vti1 scope link
>>> broadcast 10.198.54.0 dev eth0 table local proto kernel scope link
>>> src 10.198.54.208
>>> local 10.198.54.208 dev eth0 table local proto kernel scope host src
>>> 10.198.54.208
>>> broadcast 10.198.54.255 dev eth0 table local proto kernel scope link
>>> src 10.198.54.208
>>> local 95.115.254.161 dev vti1 table local proto kernel scope host src
>>> 95.115.254.161
>>> broadcast 127.0.0.0 dev lo table local proto kernel scope link src
>>> 127.0.0.1
>>> local 127.0.0.0/8 dev lo table local proto kernel scope host src
>>> 127.0.0.1
>>> local 127.0.0.1 dev lo table local proto kernel scope host src
>>> 127.0.0.1
>>> broadcast 127.255.255.255 dev lo table local proto kernel scope link
>>> src 127.0.0.1
>>>
>>
>> I wonder what the kernel now does, because it has no way to infer the
>> recommended source IP for traffic to 0.0.0.0/0.
>>
>>> There is the static route 195.210.28.0/24 via vti1. If I do “ping
>>> 195.210.28.63” I can see using tcpdump on vti1 following output:
>>>
>>> # tcpdump -i vti1 -n
>>> tcpdump: verbose output suppressed, use -v or -vv for full protocol
>>> decode
>>> listening on vti1, link-type RAW (Raw IP), capture size 262144 bytes
>>> 20:08:07.776307 IP 95.115.254.161 > 195.210.28.63: ICMP echo request,
>>> id 1098, seq 480, length 64
>>> 20:08:08.776218 IP 95.115.254.161 > 195.210.28.63: ICMP echo request,
>>> id 1098, seq 481, length 64
>>> 20:08:09.776228 IP 95.115.254.161 > 195.210.28.63: ICMP echo request,
>>> id 1098, seq 482, length 64
>>> 20:08:10.776196 IP 95.115.254.161 > 195.210.28.63: ICMP echo request,
>>> id 1098, seq 483, length 64
>>>
>>>
>>
>> I guess it falls back to the primary IP of the interface that the
>> traffic is sent out of.
>>
>>
>>> I can see there, that source IP is 95.115.254.161, so right IP
>>> address. This flow is running over IKEv2 tunnel.
>>
>> The IKE protocol in use is irrelevant.
>>
>>>
>>> If I do tcpdump on underlying eth0 device:
>>>
>>> # tcpdump -i eth0 -n udp
>>> tcpdump: verbose output suppressed, use -v or -vv for full protocol
>>> decode
>>> listening on eth0, link-type EN10MB (Ethernet), capture size 262144
>>> bytes
>>> 20:09:31.778294 IP 10.198.54.208.ipsec-nat-t >
>>> 95.115.254.165.ipsec-nat-t: UDP-encap: ESP(spi=0xc39fd86f,seq=0x234),
>>> length 136
>>> 20:09:31.792601 IP 95.115.254.165.ipsec-nat-t >
>>> 10.198.54.208.ipsec-nat-t: UDP-encap: ESP(spi=0xc69deaf7,seq=0x24c),
>>> length 136
>>> 20:09:32.131698 IP 95.115.254.165.ipsec-nat-t >
>>> 10.198.54.208.ipsec-nat-t: UDP-encap: ESP(spi=0xc69deaf7,seq=0x24d),
>>> length 104
>>> 20:09:32.778291 IP 10.198.54.208.ipsec-nat-t >
>>> 95.115.254.165.ipsec-nat-t: UDP-encap: ESP(spi=0xc39fd86f,seq=0x235),
>>> length 136
>>> 20:09:32.792787 IP 95.115.254.165.ipsec-nat-t >
>>> 10.198.54.208.ipsec-nat-t: UDP-encap: ESP(spi=0xc69deaf7,seq=0x24e),
>>> length 136
>>> 20:09:33.778217 IP 10.198.54.208.ipsec-nat-t >
>>> 95.115.254.165.ipsec-nat-t: UDP-encap: ESP(spi=0xc39fd86f,seq=0x236),
>>> length 136
>>> 20:09:33.792943 IP 95.115.254.165.ipsec-nat-t >
>>> 10.198.54.208.ipsec-nat-t: UDP-encap: ESP(spi=0xc69deaf7,seq=0x24f),
>>> length 136
>>>
>>
>> What eth0 does is irrelevant, too.
>>
>>> I guess that 10.198.54.208->95.115.254.165 is ICMP echo request
>>> (which I see on vti1 when doing tcpdump) and
>>> 95.115.254.165->10.198.54.208 is ICMP echo reply (this I only guess,
>>> but there is no other traffic than this ICMP flow).
>>
>> It can also be ICMP errors. What traffic passes the other peer?
>>
>>>
>>>
>>> For reference I am pasting statusall outout:
>>>
>>> # strongswan statusall
>>> Status of IKE charon daemon (strongSwan 5.5.3, Linux
>>> 3.10.0-693.5.2.el7.x86_64, x86_64):
>>> uptime: 18 minutes, since Nov 22 19:59:23 2017
>>> malloc: sbrk 2859008, mmap 0, used 562720, free 2296288
>>> worker threads: 11 of 16 idle, 5/0/0/0 working, job queue: 0/0/0/0,
>>> scheduled: 2
>>> loaded plugins: charon aes des rc2 sha2 sha1 md4 md5 random nonce
>>> x509 revocation constraints acert pubkey pkcs1 pkcs8 pkcs12 pgp
>>> dnskey sshkey pem openssl gcrypt fips-prf gmp curve25519 xcbc cmac
>>> hmac ctr ccm gcm curl attr kernel-netlink resolve socket-default farp
>>> stroke vici updown eap-identity eap-md5 eap-gtc eap-mschapv2 eap-tls
>>> eap-ttls eap-peap xauth-generic xauth-eap xauth-pam xauth-noauth dhcp
>>> unity
>>> Listening IP addresses:
>>> 10.198.54.208
>>> 95.115.254.161
>>> Connections:
>>> alconet: %any...ipsec.alconet.sk <http://ipsec.alconet.sk>
>>> IKEv2, dpddelay=30s
>>> alconet: local: [bmanovic] uses EAP authentication with EAP
>>> identity '%any'
>>> alconet: remote: [ipsec.alconet.sk <http://ipsec.alconet.sk>]
>>> uses public key authentication
>>> alconet: child: dynamic === 0.0.0.0/0 ::/0 TUNNEL,
>>> dpdaction=restart
>>> Security Associations (1 up, 0 connecting):
>>> alconet[1]: ESTABLISHED 18 minutes ago,
>>> 10.198.54.208[bmanovic]...95.115.254.165[ipsec.alconet.sk
>>> <http://ipsec.alconet.sk>]
>>> alconet[1]: IKEv2 SPIs: aac4fe8159ef3f03_i* d183908ac6976b91_r,
>>> rekeying disabled
>>> alconet[1]: IKE proposal:
>>> AES_CBC_128/HMAC_SHA2_256_128/PRF_HMAC_SHA2_256/CURVE_25519
>>> alconet{1}: INSTALLED, TUNNEL, reqid 1, ESP in UDP SPIs:
>>> c69deaf7_i c39fd86f_o, IPCOMP CPIs: 9858_i 7f79_o
>>> alconet{1}: AES_CBC_128/HMAC_SHA2_256_128, 0 bytes_i, 91560
>>> bytes_o (1090 pkts, 1s ago), rekeying disabled
>>> alconet{1}: 95.115.254.161/32 === 0.0.0.0/0
>>>
>>> As I wrote above, I simplified my configuration, but the result is
>>> the same. It doesn't work.
>>>
>>> Do you still think, that this is that source IP issue?
>>
>> Not anymore, but I'm sure some software gets the source IP wrong,
>> because the hints are (still) wrong.
>> Ping might just work, because it does special things.
>>
>> Kind regards
>>
>> Noel
>>
>>>
>>> Thank you for your help,
>>>
>>> BR,
>>> Miroslav
>>>
>>>
>>>> On 22 Nov 2017, at 19:22, Noel Kuntze
>>>> <noel.kuntze+strongswan-users-ml at thermi.consulting
>>>> <mailto:noel.kuntze+strongswan-users-ml at thermi.consulting>> wrote:
>>>>
>>>> Hi,
>>>>
>>>> The route over the VTI device does not indicate a preferred source
>>>> IP, so the kernel takes the one from the default route, which is in
>>>> term deferred from the route to 10.198.54.0/24,
>>>> which is 10.198.54.208. I'm pretty sure running tcpdump on the VTI
>>>> device confirms that. So the source IP is wrong, as I wrote before.
>>>> You need to set the virtual IP as source IP in your updown script.
>>>>
>>>> Kind regards
>>>>
>>>> Noel
>>>>
>>>> On 22.11.2017 18:49, Miroslav Hostinsky wrote:
>>>>> Hello Noel,
>>>>>
>>>>> I do not know what you exactly mean, but the source IP send over
>>>>> VTI interface is the same as configured on VTI interface
>>>>> (95.115.254.161 in this case). As I wrote, I can see that ICMP echo
>>>>> request & reply are delivered to this IPSEC endpoint machine (but I
>>>>> only suppose, because packets are encrypted when sniffing using
>>>>> tcpdump, but there is no other ICMP traffic). It seems that
>>>>> encrypted echo-reply is delivered to the machine, but
>>>>> "kernel/ipsec stack" is not able to properly "route" to the VTI
>>>>> device.
>>>>>
>>>>> Actually my IPSEC/*swan knowledge is not very good so sorry if my
>>>>> answer are dumb.
>>>>>
>>>>> I am attaching logs/configs as you requested.
>>>>>
>>>>> Thank you,
>>>>>
>>>>> BR,
>>>>> Miroslav
>>>>>
>>>>> On 2017-11-22 17:41, Noel Kuntze wrote:
>>>>>> Hello Miroslav,
>>>>>>
>>>>>> I suspect that the policy lookup for the received packets fail.
>>>>>> Check
>>>>>> what the source of the packets is that you send over the vti
>>>>>> device.
>>>>>> Anyway, please provide the full list of information from the
>>>>>> HelpRequests[1] page.
>>>>>>
>>>>>> [1]
>>>>>> https://wiki.strongswan.org/projects/strongswan/wiki/HelpRequests
>>>>>>
>>>>>> Kind regards
>>>>>>
>>>>>> Noel
>>>>>>
>>>>>> On 22.11.2017 16:47, Miroslav Hostinsky wrote:
>>>>>>> Hello,
>>>>>>>
>>>>>>> I have an issue configuring StrongSwan with VTI interface as
>>>>>>> roadwarrior. This is my configuration:
>>>>>>>
>>>>>>> ipsec.conf:
>>>>>>>
>>>>>>> config setup
>>>>>>>
>>>>>>> conn %default
>>>>>>> keyexchange=ikev2
>>>>>>> ikelifetime=60m
>>>>>>> keylife=20m
>>>>>>> rekeymargin=3m
>>>>>>> rekey=no
>>>>>>> dpdaction=restart
>>>>>>> dpddelay=30s
>>>>>>> compress=yes
>>>>>>> auto=start
>>>>>>>
>>>>>>> conn acnnet
>>>>>>> leftupdown=/usr/local/sbin/ipsec-notify.sh
>>>>>>> left=%defaultroute
>>>>>>> leftauth=eap
>>>>>>> leftsourceip=%config4,%config6
>>>>>>> rightauth=pubkey
>>>>>>> rightsubnet=0.0.0.0/0,::/0
>>>>>>> eap_identity=%identity
>>>>>>> leftid=bman
>>>>>>> right=mailer.domena.sk <http://mailer.domena.sk>
>>>>>>> rightid=@mailer.mailer.sk <mailto:rightid=@mailer.mailer.sk>
>>>>>>> mark=28
>>>>>>>
>>>>>>> VTI interface is configured using lefupdown script (real commands
>>>>>>> executed):
>>>>>>>
>>>>>>> ip tunnel add vti1 local 85.105.254.225 remote 185.210.28.63 mode
>>>>>>> vti key 28 ikey 28
>>>>>>> ip link set vti1 up
>>>>>>> ip addr add 192.168.228.10 dev vti1
>>>>>>> ip route add 74.99.179.0/24 dev vti1
>>>>>>> sysctl -w net.ipv4.conf.vti1.disable_policy=1
>>>>>>>
>>>>>>> It seems that outgoing connection via vti1 interface is working
>>>>>>> (outgoing ICMP echo request to subnet 74.99.179.0/24 ). But I am
>>>>>>> unable to receive ICMP echo reply. Using tcpdump I can clearly
>>>>>>> see, that IPSEC encrypted ICMP echo reply is returning via
>>>>>>> physical interface, but not via vti1.
>>>>>>>
>>>>>>>
>>>>>>> I found, that, TX bytes is correctly counted via vti1, but RX
>>>>>>> shows errors (it seems that each ICMP echo reply packet is
>>>>>>> counted as +1 error):
>>>>>>>
>>>>>>> # ip -s tunnel show
>>>>>>> vti1: ip/ip remote 185.210.28.63 local 85.105.254.225 ttl
>>>>>>> inherit key 28
>>>>>>> RX: Packets Bytes Errors CsumErrs OutOfSeq Mcasts
>>>>>>> 0 0 805 0 0 0
>>>>>>> TX: Packets Bytes Errors DeadLoop NoRoute NoBufs
>>>>>>> 401 68170 0 0 0 0
>>>>>>> ip_vti0: ip/ip remote any local any ttl inherit nopmtudisc key
>>>>>>> 0
>>>>>>> RX: Packets Bytes Errors CsumErrs OutOfSeq Mcasts
>>>>>>> 0 0 0 0 0 0
>>>>>>> TX: Packets Bytes Errors DeadLoop NoRoute NoBufs
>>>>>>> 0 0 0 0 0 0
>>>>>>>
>>>>>>> It seems that, RX Errors on vti1 are currently missing ICMP echo
>>>>>>> reply packets. But is counted as RX errors, not RX received
>>>>>>> packets.
>>>>>>>
>>>>>>> Do you have any idea what's wrong?
>>>>>>>
>>>>>>> I am using Centos 7.4 with strongswan-5.5.3-1.el7.x86_64 from
>>>>>>> EPEL. A tried with same result on Archlinux (kernel 4.9 and
>>>>>>> strongswan 5.6.0).
>>>>>>>
>>>>>>> Route installation is disabled in charon.conf.
>>>>>>>
>>>>>>> Normal connection using Virtual IP is working great.
>>>>>>>
>>>>>>>
>>>>>>> Thank you very much for any help.
>>>>>>>
>>>>>>>
>>>>>>> BR,
>>>>>>>
>>>>>>> Miroslav
>>>>>
>>>>
>>>
--
Miroslav Hostinsky
More information about the Users
mailing list