[strongSwan] VTI device and strongswan ikev2 issue
Miroslav Hostinsky
bman at tls.sk
Wed Nov 22 23:27:39 CET 2017
Hello,
I configured the route source IP as you recommended (I hope the right
way):
# ip -4 r s table all
8.8.8.8 via 10.198.54.1 dev eth0
10.198.54.0/24 dev eth0 proto kernel scope link src 10.198.54.208 metric
100
95.115.254.165 via 10.198.54.1 dev eth0
195.210.28.0/24 dev vti1 scope link src 95.115.254.161
broadcast 10.198.54.0 dev eth0 table local proto kernel scope link src
10.198.54.208
local 10.198.54.208 dev eth0 table local proto kernel scope host src
10.198.54.208
broadcast 10.198.54.255 dev eth0 table local proto kernel scope link src
10.198.54.208
local 95.115.254.161 dev vti1 table local proto kernel scope host src
95.115.254.161
broadcast 127.0.0.0 dev lo table local proto kernel scope link src
127.0.0.1
local 127.0.0.0/8 dev lo table local proto kernel scope host src
127.0.0.1
local 127.0.0.1 dev lo table local proto kernel scope host src 127.0.0.1
broadcast 127.255.255.255 dev lo table local proto kernel scope link src
127.0.0.1
but still it doesn't work. I think this is not a routing issue (but
maybe I am wrong). Now I tried the reverse flow.
I started to ping 95.115.254.161 (my vpn endpoint IP) from 195.210.28.63
(I have configured static route to this net on my vpn endpoint). I can
see that icmp echo request arrive to vpn server at 95.115.254.165 (I
have access to this server), then is forwarded via ipsec tunnel to my
endpoint. I see this on eth0 on my vpn endpoint:
# tcpdump -i eth0 -n udp
tcpdump: verbose output suppressed, use -v or -vv for full protocol
decode
listening on eth0, link-type EN10MB (Ethernet), capture size 262144
bytes
23:06:43.155626 IP 95.115.254.165.ipsec-nat-t >
10.198.54.208.ipsec-nat-t: UDP-encap: ESP(spi=0xcde002c4,seq=0x2b8),
length 136
23:06:44.155487 IP 95.115.254.165.ipsec-nat-t >
10.198.54.208.ipsec-nat-t: UDP-encap: ESP(spi=0xcde002c4,seq=0x2b9),
length 136
23:06:45.155547 IP 95.115.254.165.ipsec-nat-t >
10.198.54.208.ipsec-nat-t: UDP-encap: ESP(spi=0xcde002c4,seq=0x2ba),
length 136
23:06:46.155465 IP 95.115.254.165.ipsec-nat-t >
10.198.54.208.ipsec-nat-t: UDP-encap: ESP(spi=0xcde002c4,seq=0x2bb),
length 136
23:06:47.155299 IP 95.115.254.165.ipsec-nat-t >
10.198.54.208.ipsec-nat-t: UDP-encap: ESP(spi=0xcde002c4,seq=0x2bc),
length 136
23:06:48.155729 IP 95.115.254.165.ipsec-nat-t >
10.198.54.208.ipsec-nat-t: UDP-encap: ESP(spi=0xcde002c4,seq=0x2bd),
length 136
For me, it seems that these are encrypted icmp echo requests coming from
195.210.28.63 (where ping command is running).
But those packets are not forwarded to vti1. Instead they are counted as
errors:
# ip -s tunnel show
vti1: ip/ip remote 95.115.254.165 local 10.198.54.208 ttl inherit
key 38
RX: Packets Bytes Errors CsumErrs OutOfSeq Mcasts
0 0 889 0 0 0
TX: Packets Bytes Errors DeadLoop NoRoute NoBufs
0 0 0 0 0 0
As you can see, there is 0 packets transmitted (I am only receiving
ping) and lot of received errors (those are ICMP requests I suppose).
What do you think about this? My amateur opinion is, that there is issue
with passing decrypted packets to the vti1 device and not the routing
issue, because packets are not even passed to vti1 device.
I achieve the same result with TCP instead of ICMP (trying to telnet to
95.115.254.161, not pinging 95.115.254.161).
Thank you for the reply
BR,
Miroslav
On 2017-11-22 21:55, Noel Kuntze wrote:
> Hi,
>
> Responses are in line.
>
> On 22.11.2017 20:22, Miroslav Hostinsky wrote:
>> Hello,
>>
>> I am not sure if I understand you, but source IP of the outgoing
>> packets on vti1 are right for me (it was always 95.115.254.161).
>>
>> # ip a s vti1
>> 5: vti1 at NONE: <POINTOPOINT,NOARP,UP,LOWER_UP> mtu 1480 qdisc noqueue
>> state UNKNOWN qlen 1
>> link/ipip 10.198.54.208 peer 95.115.254.165
>> inet 95.115.254.161/32 scope global vti1
>> valid_lft forever preferred_lft forever
>>
>
> No. That is just an IP that is bound to the interface. It has nothing
> to do with the source IP of any packets. The kernel does not choose
> the IP based on an IP that is bound to an interface.
> It does it based on the routing table and the routing decision.
>
>>
>> current ipv4 routing table is following (I simplified my
>> configuration, removed second physical interface, removed default
>> route):
>>
>> # ip -4 r s table all
>> 8.8.8.8 via 10.198.54.1 dev eth0
>> 10.198.54.0/24 dev eth0 proto kernel scope link src 10.198.54.208
>> metric 100
>> 95.115.254.165 via 10.198.54.1 dev eth0
>> 195.210.28.0/24 dev vti1 scope link
>> broadcast 10.198.54.0 dev eth0 table local proto kernel scope link src
>> 10.198.54.208
>> local 10.198.54.208 dev eth0 table local proto kernel scope host src
>> 10.198.54.208
>> broadcast 10.198.54.255 dev eth0 table local proto kernel scope link
>> src 10.198.54.208
>> local 95.115.254.161 dev vti1 table local proto kernel scope host src
>> 95.115.254.161
>> broadcast 127.0.0.0 dev lo table local proto kernel scope link src
>> 127.0.0.1
>> local 127.0.0.0/8 dev lo table local proto kernel scope host src
>> 127.0.0.1
>> local 127.0.0.1 dev lo table local proto kernel scope host src
>> 127.0.0.1
>> broadcast 127.255.255.255 dev lo table local proto kernel scope link
>> src 127.0.0.1
>>
>
> I wonder what the kernel now does, because it has no way to infer the
> recommended source IP for traffic to 0.0.0.0/0.
>
>> There is the static route 195.210.28.0/24 via vti1. If I do “ping
>> 195.210.28.63” I can see using tcpdump on vti1 following output:
>>
>> # tcpdump -i vti1 -n
>> tcpdump: verbose output suppressed, use -v or -vv for full protocol
>> decode
>> listening on vti1, link-type RAW (Raw IP), capture size 262144 bytes
>> 20:08:07.776307 IP 95.115.254.161 > 195.210.28.63: ICMP echo request,
>> id 1098, seq 480, length 64
>> 20:08:08.776218 IP 95.115.254.161 > 195.210.28.63: ICMP echo request,
>> id 1098, seq 481, length 64
>> 20:08:09.776228 IP 95.115.254.161 > 195.210.28.63: ICMP echo request,
>> id 1098, seq 482, length 64
>> 20:08:10.776196 IP 95.115.254.161 > 195.210.28.63: ICMP echo request,
>> id 1098, seq 483, length 64
>>
>>
>
> I guess it falls back to the primary IP of the interface that the
> traffic is sent out of.
>
>
>> I can see there, that source IP is 95.115.254.161, so right IP
>> address. This flow is running over IKEv2 tunnel.
>
> The IKE protocol in use is irrelevant.
>
>>
>> If I do tcpdump on underlying eth0 device:
>>
>> # tcpdump -i eth0 -n udp
>> tcpdump: verbose output suppressed, use -v or -vv for full protocol
>> decode
>> listening on eth0, link-type EN10MB (Ethernet), capture size 262144
>> bytes
>> 20:09:31.778294 IP 10.198.54.208.ipsec-nat-t >
>> 95.115.254.165.ipsec-nat-t: UDP-encap: ESP(spi=0xc39fd86f,seq=0x234),
>> length 136
>> 20:09:31.792601 IP 95.115.254.165.ipsec-nat-t >
>> 10.198.54.208.ipsec-nat-t: UDP-encap: ESP(spi=0xc69deaf7,seq=0x24c),
>> length 136
>> 20:09:32.131698 IP 95.115.254.165.ipsec-nat-t >
>> 10.198.54.208.ipsec-nat-t: UDP-encap: ESP(spi=0xc69deaf7,seq=0x24d),
>> length 104
>> 20:09:32.778291 IP 10.198.54.208.ipsec-nat-t >
>> 95.115.254.165.ipsec-nat-t: UDP-encap: ESP(spi=0xc39fd86f,seq=0x235),
>> length 136
>> 20:09:32.792787 IP 95.115.254.165.ipsec-nat-t >
>> 10.198.54.208.ipsec-nat-t: UDP-encap: ESP(spi=0xc69deaf7,seq=0x24e),
>> length 136
>> 20:09:33.778217 IP 10.198.54.208.ipsec-nat-t >
>> 95.115.254.165.ipsec-nat-t: UDP-encap: ESP(spi=0xc39fd86f,seq=0x236),
>> length 136
>> 20:09:33.792943 IP 95.115.254.165.ipsec-nat-t >
>> 10.198.54.208.ipsec-nat-t: UDP-encap: ESP(spi=0xc69deaf7,seq=0x24f),
>> length 136
>>
>
> What eth0 does is irrelevant, too.
>
>> I guess that 10.198.54.208->95.115.254.165 is ICMP echo request (which
>> I see on vti1 when doing tcpdump) and 95.115.254.165->10.198.54.208 is
>> ICMP echo reply (this I only guess, but there is no other traffic than
>> this ICMP flow).
>
> It can also be ICMP errors. What traffic passes the other peer?
>
>>
>>
>> For reference I am pasting statusall outout:
>>
>> # strongswan statusall
>> Status of IKE charon daemon (strongSwan 5.5.3, Linux
>> 3.10.0-693.5.2.el7.x86_64, x86_64):
>> uptime: 18 minutes, since Nov 22 19:59:23 2017
>> malloc: sbrk 2859008, mmap 0, used 562720, free 2296288
>> worker threads: 11 of 16 idle, 5/0/0/0 working, job queue: 0/0/0/0,
>> scheduled: 2
>> loaded plugins: charon aes des rc2 sha2 sha1 md4 md5 random nonce
>> x509 revocation constraints acert pubkey pkcs1 pkcs8 pkcs12 pgp dnskey
>> sshkey pem openssl gcrypt fips-prf gmp curve25519 xcbc cmac hmac ctr
>> ccm gcm curl attr kernel-netlink resolve socket-default farp stroke
>> vici updown eap-identity eap-md5 eap-gtc eap-mschapv2 eap-tls eap-ttls
>> eap-peap xauth-generic xauth-eap xauth-pam xauth-noauth dhcp unity
>> Listening IP addresses:
>> 10.198.54.208
>> 95.115.254.161
>> Connections:
>> alconet: %any...ipsec.alconet.sk <http://ipsec.alconet.sk>
>> IKEv2, dpddelay=30s
>> alconet: local: [bmanovic] uses EAP authentication with EAP
>> identity '%any'
>> alconet: remote: [ipsec.alconet.sk <http://ipsec.alconet.sk>]
>> uses public key authentication
>> alconet: child: dynamic === 0.0.0.0/0 ::/0 TUNNEL,
>> dpdaction=restart
>> Security Associations (1 up, 0 connecting):
>> alconet[1]: ESTABLISHED 18 minutes ago,
>> 10.198.54.208[bmanovic]...95.115.254.165[ipsec.alconet.sk
>> <http://ipsec.alconet.sk>]
>> alconet[1]: IKEv2 SPIs: aac4fe8159ef3f03_i* d183908ac6976b91_r,
>> rekeying disabled
>> alconet[1]: IKE proposal:
>> AES_CBC_128/HMAC_SHA2_256_128/PRF_HMAC_SHA2_256/CURVE_25519
>> alconet{1}: INSTALLED, TUNNEL, reqid 1, ESP in UDP SPIs:
>> c69deaf7_i c39fd86f_o, IPCOMP CPIs: 9858_i 7f79_o
>> alconet{1}: AES_CBC_128/HMAC_SHA2_256_128, 0 bytes_i, 91560
>> bytes_o (1090 pkts, 1s ago), rekeying disabled
>> alconet{1}: 95.115.254.161/32 === 0.0.0.0/0
>>
>> As I wrote above, I simplified my configuration, but the result is the
>> same. It doesn't work.
>>
>> Do you still think, that this is that source IP issue?
>
> Not anymore, but I'm sure some software gets the source IP wrong,
> because the hints are (still) wrong.
> Ping might just work, because it does special things.
>
> Kind regards
>
> Noel
>
>>
>> Thank you for your help,
>>
>> BR,
>> Miroslav
>>
>>
>>> On 22 Nov 2017, at 19:22, Noel Kuntze
>>> <noel.kuntze+strongswan-users-ml at thermi.consulting
>>> <mailto:noel.kuntze+strongswan-users-ml at thermi.consulting>> wrote:
>>>
>>> Hi,
>>>
>>> The route over the VTI device does not indicate a preferred source
>>> IP, so the kernel takes the one from the default route, which is in
>>> term deferred from the route to 10.198.54.0/24,
>>> which is 10.198.54.208. I'm pretty sure running tcpdump on the VTI
>>> device confirms that. So the source IP is wrong, as I wrote before.
>>> You need to set the virtual IP as source IP in your updown script.
>>>
>>> Kind regards
>>>
>>> Noel
>>>
>>> On 22.11.2017 18:49, Miroslav Hostinsky wrote:
>>>> Hello Noel,
>>>>
>>>> I do not know what you exactly mean, but the source IP send over VTI
>>>> interface is the same as configured on VTI interface (95.115.254.161
>>>> in this case). As I wrote, I can see that ICMP echo request & reply
>>>> are delivered to this IPSEC endpoint machine (but I only suppose,
>>>> because packets are encrypted when sniffing using tcpdump, but there
>>>> is no other ICMP traffic). It seems that encrypted echo-reply is
>>>> delivered to the machine, but "kernel/ipsec stack" is not able to
>>>> properly "route" to the VTI device.
>>>>
>>>> Actually my IPSEC/*swan knowledge is not very good so sorry if my
>>>> answer are dumb.
>>>>
>>>> I am attaching logs/configs as you requested.
>>>>
>>>> Thank you,
>>>>
>>>> BR,
>>>> Miroslav
>>>>
>>>> On 2017-11-22 17:41, Noel Kuntze wrote:
>>>>> Hello Miroslav,
>>>>>
>>>>> I suspect that the policy lookup for the received packets fail.
>>>>> Check
>>>>> what the source of the packets is that you send over the vti
>>>>> device.
>>>>> Anyway, please provide the full list of information from the
>>>>> HelpRequests[1] page.
>>>>>
>>>>> [1]
>>>>> https://wiki.strongswan.org/projects/strongswan/wiki/HelpRequests
>>>>>
>>>>> Kind regards
>>>>>
>>>>> Noel
>>>>>
>>>>> On 22.11.2017 16:47, Miroslav Hostinsky wrote:
>>>>>> Hello,
>>>>>>
>>>>>> I have an issue configuring StrongSwan with VTI interface as
>>>>>> roadwarrior. This is my configuration:
>>>>>>
>>>>>> ipsec.conf:
>>>>>>
>>>>>> config setup
>>>>>>
>>>>>> conn %default
>>>>>> keyexchange=ikev2
>>>>>> ikelifetime=60m
>>>>>> keylife=20m
>>>>>> rekeymargin=3m
>>>>>> rekey=no
>>>>>> dpdaction=restart
>>>>>> dpddelay=30s
>>>>>> compress=yes
>>>>>> auto=start
>>>>>>
>>>>>> conn acnnet
>>>>>> leftupdown=/usr/local/sbin/ipsec-notify.sh
>>>>>> left=%defaultroute
>>>>>> leftauth=eap
>>>>>> leftsourceip=%config4,%config6
>>>>>> rightauth=pubkey
>>>>>> rightsubnet=0.0.0.0/0,::/0
>>>>>> eap_identity=%identity
>>>>>> leftid=bman
>>>>>> right=mailer.domena.sk <http://mailer.domena.sk>
>>>>>> rightid=@mailer.mailer.sk <mailto:rightid=@mailer.mailer.sk>
>>>>>> mark=28
>>>>>>
>>>>>> VTI interface is configured using lefupdown script (real commands
>>>>>> executed):
>>>>>>
>>>>>> ip tunnel add vti1 local 85.105.254.225 remote 185.210.28.63 mode
>>>>>> vti key 28 ikey 28
>>>>>> ip link set vti1 up
>>>>>> ip addr add 192.168.228.10 dev vti1
>>>>>> ip route add 74.99.179.0/24 dev vti1
>>>>>> sysctl -w net.ipv4.conf.vti1.disable_policy=1
>>>>>>
>>>>>> It seems that outgoing connection via vti1 interface is working
>>>>>> (outgoing ICMP echo request to subnet 74.99.179.0/24 ). But I am
>>>>>> unable to receive ICMP echo reply. Using tcpdump I can clearly
>>>>>> see, that IPSEC encrypted ICMP echo reply is returning via
>>>>>> physical interface, but not via vti1.
>>>>>>
>>>>>>
>>>>>> I found, that, TX bytes is correctly counted via vti1, but RX
>>>>>> shows errors (it seems that each ICMP echo reply packet is counted
>>>>>> as +1 error):
>>>>>>
>>>>>> # ip -s tunnel show
>>>>>> vti1: ip/ip remote 185.210.28.63 local 85.105.254.225 ttl
>>>>>> inherit key 28
>>>>>> RX: Packets Bytes Errors CsumErrs OutOfSeq Mcasts
>>>>>> 0 0 805 0 0 0
>>>>>> TX: Packets Bytes Errors DeadLoop NoRoute NoBufs
>>>>>> 401 68170 0 0 0 0
>>>>>> ip_vti0: ip/ip remote any local any ttl inherit nopmtudisc key
>>>>>> 0
>>>>>> RX: Packets Bytes Errors CsumErrs OutOfSeq Mcasts
>>>>>> 0 0 0 0 0 0
>>>>>> TX: Packets Bytes Errors DeadLoop NoRoute NoBufs
>>>>>> 0 0 0 0 0 0
>>>>>>
>>>>>> It seems that, RX Errors on vti1 are currently missing ICMP echo
>>>>>> reply packets. But is counted as RX errors, not RX received
>>>>>> packets.
>>>>>>
>>>>>> Do you have any idea what's wrong?
>>>>>>
>>>>>> I am using Centos 7.4 with strongswan-5.5.3-1.el7.x86_64 from
>>>>>> EPEL. A tried with same result on Archlinux (kernel 4.9 and
>>>>>> strongswan 5.6.0).
>>>>>>
>>>>>> Route installation is disabled in charon.conf.
>>>>>>
>>>>>> Normal connection using Virtual IP is working great.
>>>>>>
>>>>>>
>>>>>> Thank you very much for any help.
>>>>>>
>>>>>>
>>>>>> BR,
>>>>>>
>>>>>> Miroslav
>>>>
>>>
>>
--
Miroslav Hostinsky
More information about the Users
mailing list