[strongSwan] Linux routing issue
Noel Kuntze
noel.kuntze at thermi.consulting
Wed Jan 26 20:29:00 CET 2022
Hello Carlos,
That is great to hear.
> I have not created a vti interface, but I'm using mangle to mark packets destined to the other tunnel endpoint.
I see. Please take a look at the article about route based VPNs[1] which includes VTIs and XFRM interfaces. It's been some months already since I last dealt with configuring a VTI so I can't help you with that particular issue and the reason for it from the top of my head with the information available.
You probably want to check what the mark value on the incoming packets is (best checked with the iptables TRACE target, described on the man page for iptables-extensions), if the inbound policies have the same mark (best checked with ip -d x p), and if there are any problems with decapsulating the packets by the kernel (check /proc/net/xfrm_stat).
The behaviour of the kernel in regards to the decapsulation changes if a VTI with a matching mark exists or not. It's not that much a self contained solution as in "once a VTI for the mark exists the packets just appear in it" - no, the existence (AFAIR) also changes if the packets can be decapsulated at all (if the input policy has a mark but the packets to be decapsulated don't? I don't remember the specifics.
1) set up the VTI with mark_out as described on the wiki
2) completely switch to an XFRM interface and thus don't need to deal with marks and the interaction with the routing engine configuration and the firewall rules
With an XFRM interface you don't have to deal with any marks or any other possible interaction with the routing engine or any other components using marks. XFRM interfaces are basically self contained in regards to if packets can be decapsulated or not. If you have an if_id set on the inbound policy then that just means that the packets will appear in the/a XFRM interfaces with the corresponding if_id and not if they don't exist at all. So it's much easier to distinguish the failure cases with XFRM interfaces.
Kind regards
Noel
[1] https://wiki.strongswan.org/projects/strongswan/wiki/RouteBasedVPN
Am 26.01.22 um 15:21 schrieb Carlos G Mendioroz:
> OK, it seems I've progressed somehow, the mark at the "conn %default" was not being reflected at the policies, now with mark at the actual conn, each connection policy does show mark.
>
> Disabled the route generation and that solved the route priority issue that I was having. Now tunnels come up and I can see traffic being encapsulated, coming back from other end (from a ping test) but not being handled locally.
>
> I have not created a vti interface, but I'm using mangle to mark packets destined to the other tunnel endpoint.
>
> What would be a troubleshooting option to see what happens to the incoming packets ?
> Should I create a vti ? Is there a doc explaining what are the options in using a xfrm interface ?
>
> TIA,
> -Carlos
>
> Noel Kuntze @ 24/1/2022 21:31 -0300 dixit:
>> Hello Carlos,
>>
>> Yes, those marks use the same namespace so to say so they overlap in their usage and meaning.
>> It's the same field in the skb (socketbuffer) used for that purpose everywhere.
>>
>> Kind regards
>> Noel
>>
>> Am 25.01.22 um 00:06 sch=rieb Carlos G Mendioroz:
>>> I transcribed the policy as in ip xfrm policy before, I don't see any reference to a mark there.
>>> Talking about marks, I'm also using them for policy routing and QoS, using shorewall as a manager. Just in case there is any clash.
>>>
>>> -Carlos
>>>
>>> Noel Kuntze @ 24/1/2022 19:23 -0300 dixit:
>>>> Hello Carlos,
>>>>
>>>> I see, yes. Can you check if the XFRM policies have the mark value set you indended in that config?
>>>>
>>>> Kind regards
>>>> Noel
>>>>
>>>> Am 24.01.22 um 23:12 schrieb Carlos G Mendioroz:
>>>>> Noel,
>>>>> thanks again for your time and dedication. FTR, I understand ipsec to some extent, and know the difference between policy and route based.
>>>>> The whole thing is about implementing route based ipsec VPNs to AWS, BTW.
>>>>>
>>>>> I did set mark in the connection configuration:
>>>>>
>>>>> conn %default
>>>>> ikelifetime=60m
>>>>> keylife=20m
>>>>> rekeymargin=3m
>>>>> keyingtries=1
>>>>> keyexchange=ikev2
>>>>> authby=secret
>>>>> mark=32
>>>>>
>>>>> conn vpc-cust-gw1
>>>>> left=<my IP>
>>>>> leftid=vpc-cust-gw1
>>>>> leftsubnet=0.0.0.0/0
>>>>> rightsubnet=0.0.0.0/0
>>>>> leftfirewall=yes
>>>>> keyexchange=ikev2
>>>>> ike=aes256-sha256-modp2048
>>>>> ikelifetime=28800
>>>>> esp=aes256-sha256-modp2048
>>>>> leftauth=psk
>>>>> rightauth=psk
>>>>> dpdtimeout=60
>>>>> lifebytes=0
>>>>> lifepackets=0
>>>>> lifetime=1h
>>>>> right=<AWS IP>
>>>>> auto=start
>>>>>
>>>>> I'm sort of using https://cloud.google.com/community/tutorials/using-cloud-vpn-with-strongswan as an example, but have not decided how to implement the BGP yet (Bird, FRR, ???, main or VRF).
>>>>>
>>>>> -Carlos
>>>>>
>>>>> Noel Kuntze @ 24/1/2022 18:11 -0300 dixit:
>>>>>> Hello Carlos,
>>>>>>
>>>>>> XFRM policies, XFRM states, and XFRM interfaces derive their name from the common subsystem XFRM, not because they would be exclusively used with each other.
>>>>>> By default strongSwan builds a policy based VPN, thus enabling the default use case of IPsec which is protecting traffic between IP addresses without intermediate IP addresses like some types of tunnels. IPsec is meant to be transparent.
>>>>>>
>>>>>> If you want your policies to only apply to an XFRM interface (XFRM interfaces are a relatively recent addition to the Linux kernel), you need to set the if_id_in and if_id_out parameters in the connection configuration.
>>>>>>
>>>>>> If you do not set the if_id_in and if_id_out parameters, the policies are applied regardless of interface (yes you can set the interface as part of the config but that is only taken into account by the kernel for outgoing traffic matching).
>>>>>>
>>>>>> Also please take into consideration that having an XFRM policy with mode "require" (default, can't configure the alternative value "use") implies that traffic matching that policy that is not being processed by a linked state is DROPPED.
>>>>>>
>>>>>> Can you verify for me that you did set if_id_in and if_id_out (assuming your versions of Linux and strongSwan support it), or that you did set mark_in and mark_out (alternatively mark, which is supposed to set mark_in and mark_out)?
>>>>>>
>>>>>> Kind regards
>>>>>> Noel
>>>>>>
>>>>>> Am 24.01.22 um 21:16 schrieb Carlos G Mendioroz:
>>>>>>> Noel Kuntze @ 24/1/2022 17:11 -0300 dixit:
>>>>>>>> Hello Carlos,
>>>>>>>>
>>>>>>>> Well yes but no:
>>>>>>>
>>>>>>> Hmm, I don't follow.
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>> src 0.0.0.0/0 dst 0.0.0.0/0
>>>>>>>> dir out priority 399999
>>>>>>>> tmpl src <my IP> dst <AWS IP>
>>>>>>>> proto esp spi 0xcfef925b reqid 1 mode tunnel
>>>>>>>> src 0.0.0.0/0 dst 0.0.0.0/0
>>>>>>>> dir fwd priority 399999
>>>>>>>> tmpl src <AWS IP> dst <my IP>
>>>>>>>> proto esp reqid 1 mode tunnel
>>>>>>>> src 0.0.0.0/0 dst 0.0.0.0/0
>>>>>>>> dir in priority 399999
>>>>>>>> tmpl src <AWS IP> dst <my IP>
>>>>>>>> proto esp reqid 1 mode tunnel
>>>>>>>>
>>>>>>>> Those are policies that match all traffic.
>>>>>>>>
>>>>>>>> Maybe `ip -d x p` shows the marks if any are set.
>>>>>>>>
>>>>>>>> Kind regards
>>>>>>>> Noel
>>>>>>>
>>>>>>> This is in preparation for a routed VPN, so I wanted a full traffic policy and expecting to bring up a dynamic routing on top. Again, I'm building my understanding... and surprised of the behaviour.
>>>>>>>
>>>>>>> -Carlos
>>>>>>>
>>>>>>>>
>>>>>>>> Am 24.01.22 um 21:09 schrieb Carlos G Mendioroz:
>>>>>>>>> Noel Kuntze @ 24/1/2022 16:55 -0300 dixit:
>>>>>>>>>> Hello Carlos,
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> > The mark did take, but the rest (i.e. non secured traffic) is being affected, I may have been unclear about the
>>>>>>>>>>
>>>>>>>>>> Please check the routing rules and tables too. E.g. ask the kernel what the route would be for an IP address using `ip r get X` and check if it matches what you expect it to be.
>>>>>>>>>
>>>>>>>>> The "ip route get " shows what I would expect, but not what is being done.
>>>>>>>>> Case in point, I do have a tunnel that terminates traffic to a given IP. To be able to serve traffic to that IP, any returning traffic is source routed via a rule (say prio 600) that forces the tunnel as default route. But that would disconnect my local net from testing to that address, so prio 0 has a lookup on local table, which has a route for the local net to the local interface.
>>>>>>>>>
>>>>>>>>> When I started the ipsec SA, all traffic was routed by main table, and sent to default gateway, not paying attention to other rules it would seem.
>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> > The state shows it:
>>>>>>>>>>
>>>>>>>>>> Can you check `ip xfrm policy`? That shows you the policies, which are the crucial parts. States without policies don't do anything. Policies without states drop everything.
>>>>>>>>>
>>>>>>>>> AFAIK, policy looks good too:
>>>>>>>>>
>>>>>>>>> src 0.0.0.0/0 dst 0.0.0.0/0
>>>>>>>>> dir out priority 399999
>>>>>>>>> tmpl src <my IP> dst <AWS IP>
>>>>>>>>> proto esp spi 0xcfef925b reqid 1 mode tunnel
>>>>>>>>> src 0.0.0.0/0 dst 0.0.0.0/0
>>>>>>>>> dir fwd priority 399999
>>>>>>>>> tmpl src <AWS IP> dst <my IP>
>>>>>>>>> proto esp reqid 1 mode tunnel
>>>>>>>>> src 0.0.0.0/0 dst 0.0.0.0/0
>>>>>>>>> dir in priority 399999
>>>>>>>>> tmpl src <AWS IP> dst <my IP>
>>>>>>>>> proto esp reqid 1 mode tunnel
>>>>>>>>> src 0.0.0.0/0 dst 0.0.0.0/0
>>>>>>>>> socket in priority 0
>>>>>>>>> src 0.0.0.0/0 dst 0.0.0.0/0
>>>>>>>>> socket out priority 0
>>>>>>>>> src 0.0.0.0/0 dst 0.0.0.0/0
>>>>>>>>> socket in priority 0
>>>>>>>>> src 0.0.0.0/0 dst 0.0.0.0/0
>>>>>>>>> socket out priority 0
>>>>>>>>> src ::/0 dst ::/0
>>>>>>>>> socket in priority 0
>>>>>>>>> src ::/0 dst ::/0
>>>>>>>>> socket out priority 0
>>>>>>>>> src ::/0 dst ::/0
>>>>>>>>> socket in priority 0
>>>>>>>>> src ::/0 dst ::/0
>>>>>>>>> socket out priority 0
>>>>>>>>>
>>>>>>>>> Note that I have not instantiated an XFRM if yet. I may be missing something obvious, but the change of regular traffic behaviour surprised me.
>>>>>>>>>
>>>>>>>>> -Carlos
>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> Kind regards
>>>>>>>>>> Noel
>>>>>>>>>>
>>>>>>>>>> Am 24.01.22 um 20:49 schrieb Carlos G Mendioroz:
>>>>>>>>>>> Noel,
>>>>>>>>>>> thanks for answering. Please see inline:
>>>>>>>>>>>
>>>>>>>>>>> Noel Kuntze @ 24/1/2022 16:24 -0300 dixit:
>>>>>>>>>>>> Hello Carlos,
>>>>>>>>>>>>
>>>>>>>>>>>> Either the mark didn't take, you're using an old version (some had a different behaviour in regards to marks and how routes are set when marks are set on the connection configuration).
>>>>>>>>>>>
>>>>>>>>>>> I'm using 5.8.2 as distributed by Ubuntu 20.04 LTS.
>>>>>>>>>>> The mark did take, but the rest (i.e. non secured traffic) is being affected, I may have been unclear about the issue.
>>>>>>>>>>>
>>>>>>>>>>> The state shows it:
>>>>>>>>>>>
>>>>>>>>>>> src <my IP> dst <AWS IP>
>>>>>>>>>>> proto esp spi 0xcf54acd4 reqid 1 mode tunnel
>>>>>>>>>>> replay-window 0 flag af-unspec
>>>>>>>>>>> mark 0x20/0xffffffff
>>>>>>>>>>> auth-trunc hmac(sha256) 0xd5... 128
>>>>>>>>>>> enc cbc(aes) 0x1a...
>>>>>>>>>>> encap type espinudp sport 4500 dport 4500 addr 0.0.0.0
>>>>>>>>>>> anti-replay context: seq 0x0, oseq 0x0, bitmap 0x00000000
>>>>>>>>>>> src <AWS IP> <my IP>
>>>>>>>>>>> proto esp spi 0xc1a5cd59 reqid 1 mode tunnel
>>>>>>>>>>> replay-window 32 flag af-unspec
>>>>>>>>>>> auth-trunc hmac(sha256) 0xbe... 128
>>>>>>>>>>> enc cbc(aes) 0xd9...
>>>>>>>>>>> encap type espinudp sport 4500 dport 4500 addr 0.0.0.0
>>>>>>>>>>> anti-replay context: seq 0x22, oseq 0x0, bitmap 0xffffffff
>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>> If you do not require the setting of source IP addresses for the remote subnets, just disable installing of routes, and use XFRM interfaces so you can use routes to direct traffic instead of dealing with the XFRM policies.
>>>>>>>>>>>
>>>>>>>>>>> I'm trying to understand, not to have a working config. For now, at least :)
>>>>>>>>>>>
>>>>>>>>>>> -Carlos
>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>> Kind regards
>>>>>>>>>>>> Noel
>>>>>>>>>>>>
>>>>>>>>>>>> Am 24.01.22 um 12:44 schrieb Carlos G Mendioroz:
>>>>>>>>>>>>> Hi,
>>>>>>>>>>>>> trying to set up a VPN on a lab system with many interfaces
>>>>>>>>>>>>> (Ubuntu 20.04, 2 uplinks, IPv6 tunnel, vlans, openvpn and IPIP tunnel).
>>>>>>>>>>>>>
>>>>>>>>>>>>> It's been a while since I used strongswan, but it was easy to set up using ipsec command and ipsec.conf policies. ipsec route table (220) played fine with my own rules I use mainly to source route to Internet uplinks.
>>>>>>>>>>>>>
>>>>>>>>>>>>> Now I want to setup a routed VPN (AWS transit gateway on the other end) and as soon as link comes up, all my traffic gets routed by main table.
>>>>>>>>>>>>> (I changed policy to any any and at first did not specifiy mark, and it even disconnected from the local net, not nice on a headless server)
>>>>>>>>>>>>> Now with mark it still makes all the traffic ignore rule priorities.
>>>>>>>>>>>>>
>>>>>>>>>>>>> Any pointer to what to check ?
>>>>>>>>>>>>> TIA,
>>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>
>>>>>>>>
>>>>>>>
>>>>>>
>>>>>
>>>>
>>>
>>
>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: OpenPGP_signature
Type: application/pgp-signature
Size: 840 bytes
Desc: OpenPGP digital signature
URL: <http://lists.strongswan.org/pipermail/users/attachments/20220126/f78720ef/attachment-0001.sig>
More information about the Users
mailing list