<html>
<head>
<meta http-equiv="Content-Type" content="text/html; charset=utf-8">
<style type="text/css" style="display:none;"> P {margin-top:0;margin-bottom:0;} </style>
</head>
<body dir="ltr">
<div>
<div dir="ltr">
<div style="font-family:Calibri,Helvetica,sans-serif; font-size:12pt; color:rgb(0,0,0)">
Hi Everyone,</div>
<div style="font-family:Calibri,Helvetica,sans-serif; font-size:12pt; color:rgb(0,0,0)">
<br>
</div>
<div style="font-family:Calibri,Helvetica,sans-serif; font-size:12pt; color:rgb(0,0,0)">
I am scratching my head for weeks now on how to get an IPSec Routed VPN site-to-site to work between a pfSense firewall and OpenWRT.</div>
<div style="font-family:Calibri,Helvetica,sans-serif; font-size:12pt; color:rgb(0,0,0)">
<br>
</div>
<div style="font-family:Calibri,Helvetica,sans-serif; font-size:12pt; color:rgb(0,0,0)">
The setup is currently in a LAB environment to avoid issues with my production networks. The config is as follows:</div>
<div style="font-family:Calibri,Helvetica,sans-serif; font-size:12pt; color:rgb(0,0,0)">
<br>
</div>
<div style="font-family:Calibri,Helvetica,sans-serif; font-size:12pt; color:rgb(0,0,0)">
<b>### pfSense ###<br>
</b></div>
<div style="font-family:Calibri,Helvetica,sans-serif; font-size:12pt; color:rgb(0,0,0)">
<br>
</div>
<div style="font-family:Calibri,Helvetica,sans-serif; font-size:12pt; color:rgb(0,0,0)">
WAN : 192.168.45.10</div>
<div style="font-family:Calibri,Helvetica,sans-serif; font-size:12pt; color:rgb(0,0,0)">
IPsec subnet : 10.10.10.1/30</div>
<div style="font-family:Calibri,Helvetica,sans-serif; font-size:12pt; color:rgb(0,0,0)">
<br>
</div>
<div style="font-family:Calibri,Helvetica,sans-serif; font-size:12pt; color:rgb(0,0,0)">
<b><b>###</b><b>###</b><b>###</b><b>###</b><b>###</b></b></div>
<div style="font-family:Calibri,Helvetica,sans-serif; font-size:12pt; color:rgb(0,0,0)">
<br>
</div>
<div style="font-family:Calibri,Helvetica,sans-serif; font-size:12pt; color:rgb(0,0,0)">
<b>### Arch Linux ###</b></div>
<div style="font-family:Calibri,Helvetica,sans-serif; font-size:12pt; color:rgb(0,0,0)">
<b><b><br>
</b></b></div>
<div style="font-family:Calibri,Helvetica,sans-serif; font-size:12pt; color:rgb(0,0,0)">
<span style="font-size:12pt">WAN : 192.168.45.30</span>
<div style="font-size:12pt">IPsec subnet : 10.10.10.2/30</div>
<b><b><span style="font-size:12pt"></span></b></b></div>
<div style="font-family:Calibri,Helvetica,sans-serif; font-size:12pt; color:rgb(0,0,0)">
<b><b><br>
</b></b></div>
<div style="font-family:Calibri,Helvetica,sans-serif; font-size:12pt; color:rgb(0,0,0)">
<b><b>###<b>###</b><b>###</b><b>###</b><b>###</b><b>##</b></b></b></div>
<div style="font-family:Calibri,Helvetica,sans-serif; font-size:12pt; color:rgb(0,0,0)">
<br>
</div>
<div style="font-family:Calibri,Helvetica,sans-serif; font-size:12pt; color:rgb(0,0,0)">
Pings from the pfSense are reaching the Linux system and are received with no errors :</div>
<div style="font-family:Calibri,Helvetica,sans-serif; font-size:12pt; color:rgb(0,0,0)">
<br>
</div>
<div style="font-family:Calibri,Helvetica,sans-serif; font-size:12pt; color:rgb(0,0,0)">
<span style="font-family:monospace"><span style="color:#000000; background-color:#ffffff">ip_vti1: ip/ip remote 192.168.45.10 local 192.168.45.30 ttl inherit nopmtudisc key 42
</span><br>
RX: Packets Bytes Errors CsumErrs OutOfSeq Mcasts <br>
47185 3963540 0 0 0 0 <br>
TX: Packets Bytes Errors DeadLoop NoRoute NoBufs <br>
0 0 98923 0 98923 0</span><br>
</div>
<div style="font-family:Calibri,Helvetica,sans-serif; font-size:12pt; color:rgb(0,0,0)">
<br>
</div>
<div style="font-family:Calibri,Helvetica,sans-serif; font-size:12pt; color:rgb(0,0,0)">
<br>
</div>
<div style="font-family:Calibri,Helvetica,sans-serif; font-size:12pt; color:rgb(0,0,0)">
Pings from the<span style="font-family:calibri,helvetica,sans-serif; font-size:12pt; color:rgb(0,0,0); background-color:rgba(0,0,0,0)"> Linux system are being seem as errors NoRoute by the tunnel.</span></div>
<div style="font-family:Calibri,Helvetica,sans-serif; font-size:12pt; color:rgb(0,0,0)">
<br>
</div>
<div style="font-family:Calibri,Helvetica,sans-serif; font-size:12pt; color:rgb(0,0,0)">
I have deleted all the rules added into iptables that marks outgoing packets in the hopes of it getting routed into the VTI interface and get marked and therefore tunneled to the pfSense.</div>
<div style="font-family:Calibri,Helvetica,sans-serif; font-size:12pt; color:rgb(0,0,0)">
<br>
</div>
<div style="font-family:Calibri,Helvetica,sans-serif; font-size:12pt; color:rgb(0,0,0)">
Mangle Chain</div>
<div style="font-family:Calibri,Helvetica,sans-serif; font-size:12pt; color:rgb(0,0,0)">
<span style="font-family:monospace"><span style="color:#000000; background-color:#ffffff">-P PREROUTING ACCEPT -c 1002 86651
</span><br>
-P INPUT ACCEPT -c 920 82059 <br>
-P FORWARD ACCEPT -c 0 0 <br>
-P OUTPUT ACCEPT -c 847 101821 <br>
-P POSTROUTING ACCEPT -c 847 101821 <br>
-A PREROUTING -s 192.168.45.10/32 -d 192.168.45.30/32 -p esp -m esp --espspi 3448029224 -c 2871 401940 -j MARK --set-xmark 0x2a/0xffffffff
<br>
-A PREROUTING -d 10.10.10.0/30 -c 143719 14738388 -j NFLOG --nflog-group 5 <br>
-A INPUT -c 337318 30692985 -j NFLOG --nflog-group 6 <br>
-A OUTPUT -c 368660 48238051 -j NFLOG --nflog-group 7 <br>
-A POSTROUTING -c 368923 48266946 -j NFLOG --nflog-group 8</span><br>
</div>
<div style="font-family:Calibri,Helvetica,sans-serif; font-size:12pt; color:rgb(0,0,0)">
<br>
</div>
<div style="font-family:Calibri,Helvetica,sans-serif; font-size:12pt; color:rgb(0,0,0)">
From my understanding the routes are correct as seen below :</div>
<div style="font-family:Calibri,Helvetica,sans-serif; font-size:12pt; color:rgb(0,0,0)">
<br>
</div>
<div style="font-family:Calibri,Helvetica,sans-serif; font-size:12pt; color:rgb(0,0,0)">
<span style="font-family:monospace"><span style="color:#000000; background-color:#ffffff">[root@arch-linux ~]# ip route get 10.10.10.1
</span><br>
10.10.10.1 dev ip_vti1 src 10.10.10.2 uid 0 <br>
cache</span></div>
<div style="font-family:Calibri,Helvetica,sans-serif; font-size:12pt; color:rgb(0,0,0)">
<br>
</div>
<div style="font-family:Calibri,Helvetica,sans-serif; font-size:12pt; color:rgb(0,0,0)">
<span style="font-family:monospace"><span style="color:#000000; background-color:#ffffff">[root@arch-linux ~]# ip route
</span><br>
default via 192.168.45.1 dev ens18 <br>
10.10.10.0/30 dev ip_vti1 scope link <br>
192.168.45.0/24 dev ens18 proto kernel scope link src 192.168.45.30</span></div>
<div style="font-family:Calibri,Helvetica,sans-serif; font-size:12pt; color:rgb(0,0,0)">
<br>
</div>
<div style="font-family:Calibri,Helvetica,sans-serif; font-size:12pt; color:rgb(0,0,0)">
Table 220 has been added but is empty.</div>
<div style="font-family:Calibri,Helvetica,sans-serif; font-size:12pt; color:rgb(0,0,0)">
<br>
</div>
<div style="font-family:Calibri,Helvetica,sans-serif; font-size:12pt; color:rgb(0,0,0)">
<span style="font-family:monospace"><span style="color:#000000; background-color:#ffffff">[root@arch-linux ~]# ip rule
</span><br>
0: from all lookup local <br>
220: from all lookup 220 <br>
32766: from all lookup main <br>
32767: from all lookup default</span><br>
</div>
<div style="font-family:Calibri,Helvetica,sans-serif; font-size:12pt; color:rgb(0,0,0)">
<br>
</div>
<div style="font-family:Calibri,Helvetica,sans-serif; font-size:12pt; color:rgb(0,0,0)">
My Linux IPSec /etc/swanctl/swanctl.conf <span id="x_👇">👇</span><br>
</div>
<div style="font-family:Calibri,Helvetica,sans-serif; font-size:12pt; color:rgb(0,0,0)">
<br>
</div>
<div style="font-family:Calibri,Helvetica,sans-serif; font-size:12pt; color:rgb(0,0,0)">
<span style="font-family:monospace"><span style="color:#000000; background-color:#ffffff">[root@arch-linux ~]# cat /etc/swanctl/swanctl.conf
</span><br>
connections { <br>
ipseclab { <br>
fragmentation = yes <br>
unique = replace <br>
version = 2 <br>
proposals = aes256-sha256-modp2048 <br>
dpd_delay = 10s <br>
dpd_timeout = 60s <br>
rekey_time = 25920s <br>
reauth_time = 0s <br>
over_time = 2880s <br>
rand_time = 2880s <br>
encap = no <br>
mobike = no <br>
local_addrs = 192.168.45.30 <br>
remote_addrs = 192.168.45.10 <br>
local { <br>
id = fqdn:ipsec-lab-openwrt <br>
auth = psk <br>
} <br>
remote { <br>
id = fqdn:ipsec-lab-pfsense <br>
auth = psk <br>
} <br>
children { <br>
con1 { <br>
close_action = start <br>
dpd_action = restart <br>
policies = no <br>
life_time = 3600s <br>
rekey_time = 3240s <br>
rand_time = 360s <br>
start_action = start <br>
remote_ts = 0.0.0.0/0 <br>
local_ts = 10.10.10.2/30 <br>
esp_proposals = aes256gcm128-modp2048 <br>
mark_in = 42 <br>
mark_out = 42 <br>
updown = /root/02.ipsec-log.sh <br>
} <br>
} <br>
} <br>
} <br>
secrets { <br>
ike-0 { <br>
secret = e559c752478188f3fca07ab3e5fcd1ff02f5c55574b576920c67c443 <br>
id-0 = %any <br>
id-1 = fqdn:ipsec-lab-pfsense <br>
} <br>
</span></div>
<div style="font-family:Calibri,Helvetica,sans-serif; font-size:12pt; color:rgb(0,0,0)">
<span style="font-family:monospace">}</span></div>
<div style="font-family:Calibri,Helvetica,sans-serif; font-size:12pt; color:rgb(0,0,0)">
<span style="font-family:monospace"><br>
</span></div>
<div style="font-family:Calibri,Helvetica,sans-serif; font-size:12pt; color:rgb(0,0,0)">
<span style="font-family:monospace"><span><span style="color:#000000; background-color:#ffffff">[root@arch-linux ~]# cat /etc/strongswan.conf
</span><br>
starter { <br>
load_warning = no <br>
} <br>
<br>
charon { <br>
install_routes = no <br>
install_virtual_ip = no # not in pfSense side. Added from internet tutorials.
<br>
load_modular = yes <br>
</span></span></div>
<div style="font-family:Calibri,Helvetica,sans-serif; font-size:12pt; color:rgb(0,0,0)">
<span style="font-family:monospace"><span><br>
</span></span></div>
<div style="font-family:Calibri,Helvetica,sans-serif; font-size:12pt; color:rgb(0,0,0)">
<span style="font-family:monospace"><span> # File Logging <br>
filelog { <br>
charon { <br>
# path to the log file, specify this as section name in versions prior to 5.7.0
<br>
path = /root/charon.log <br>
# add a timestamp prefix <br>
time_format = %b %e %T <br>
# prepend connection name, simplifies grepping <br>
ike_name = yes <br>
# overwrite existing files <br>
append = no <br>
# increase default loglevel for all daemon subsystems <br>
#default = -1 <br>
<br>
app = 4 <br>
asn = 4 <br>
cfg = 4 <br>
chd = 4 <br>
dmn = 4 <br>
enc = 4 <br>
esp = 4 <br>
ike = 4 # set to 2 to troubleshoot <br>
imc = 4 <br>
imv = 4 <br>
job = 4 <br>
knl = 4 # set to 2 to troubleshoot <br>
lib = 4 <br>
mgr = 4 <br>
net = 4 <br>
pts = 4 <br>
tls = 4 <br>
tnc = 4 <br>
</span></span></div>
<div style="font-family:Calibri,Helvetica,sans-serif; font-size:12pt; color:rgb(0,0,0)">
<span style="font-family:monospace"><span><br>
</span></span></div>
<div style="font-family:Calibri,Helvetica,sans-serif; font-size:12pt; color:rgb(0,0,0)">
<span style="font-family:monospace"><span> # flush each line to disk
<br>
flush_line = yes <br>
} <br>
} <br>
syslog { <br>
identifier = charon <br>
# log everything under daemon since it ends up in the same place regardless with our syslog.conf
<br>
daemon { <br>
ike_name = yes <br>
app = -1 <br>
asn = -1 <br>
cfg = -1 <br>
chd = -1 <br>
dmn = -1 <br>
enc = 4 <br>
esp = 4 <br>
ike = -1 # set to 2 to troubleshoot <br>
imc = -1 <br>
imv = -1 <br>
job = -1 <br>
knl = -1 # set to 2 to troubleshoot <br>
lib = 4 <br>
mgr = -1 <br>
net = -1 <br>
pts = -1 <br>
tls = -1 <br>
tnc = -1 <br>
} <br>
# disable logging under auth so logs aren't duplicated <br>
auth { <br>
default = -1 # set to 2 for troubleshooting; -1 to supress
<br>
ike = -1 <br>
} <br>
} <br>
plugins { <br>
include strongswan.d/charon/*.conf <br>
} <br>
}</span></span></div>
<div style="font-family:Calibri,Helvetica,sans-serif; font-size:12pt; color:rgb(0,0,0)">
<span style="font-family:monospace"><span><br>
</span></span></div>
<div style="font-family:Calibri,Helvetica,sans-serif; font-size:12pt; color:rgb(0,0,0)">
<span style="font-family:monospace"><span>And finally, the status of my IPSec daemon :</span><br>
</span></div>
<div style="font-family:Calibri,Helvetica,sans-serif; font-size:12pt; color:rgb(0,0,0)">
<span style="font-family:monospace"><br>
</span></div>
<div style="font-family:Calibri,Helvetica,sans-serif; font-size:12pt; color:rgb(0,0,0)">
<span style="font-family:monospace"><span><span style="color:#000000; background-color:#ffffff">[root@arch-linux ~]# ipsec statusall
</span><br>
Status of IKE charon daemon (strongSwan 5.9.3, Linux 5.13.12-arch1-1, x86_64): <br>
uptime: 66 minutes, since Aug 30 15:52:11 2021 <br>
malloc: sbrk 2965504, mmap 0, used 1061360, free 1904144 <br>
worker threads: 11 of 16 idle, 5/0/0/0 working, job queue: 0/0/0/0, scheduled: 3
<br>
loaded plugins: charon ldap pkcs11 aes des rc2 sha2 sha3 sha1 md5 mgf1 random nonce x509 revocation constraints pubkey pkcs1 pkcs7 pkcs8 pkcs12 pgp dnskey
<br>
sshkey pem openssl fips-prf gmp curve25519 agent chapoly xcbc cmac hmac ntru drbg newhope bliss curl sqlite attr kernel-netlink resolve socket-default bypass<br>
-lan connmark forecast farp stroke vici updown eap-identity eap-sim eap-aka eap-aka-3gpp2 eap-simaka-pseudonym eap-simaka-reauth eap-md5 eap-gtc eap-mschapv2<br>
eap-dynamic eap-radius eap-tls eap-ttls eap-peap xauth-generic xauth-eap xauth-pam xauth-noauth dhcp radattr unity counters
<br>
Listening IP addresses: <br>
192.168.45.30 <br>
10.10.10.2 <br>
Connections: <br>
ipseclab: 192.168.45.30...192.168.45.10 IKEv2, dpddelay=10s <br>
ipseclab: local: [ipsec-lab-openwrt] uses pre-shared key authentication <br>
ipseclab: remote: [ipsec-lab-pfsense] uses pre-shared key authentication <br>
con1: child: 10.10.10.0/30 === 0.0.0.0/0 TUNNEL, dpdaction=restart <br>
Shunted Connections: <br>
Bypass LAN 10.10.10.0/30: 10.10.10.0/30 === 10.10.10.0/30 PASS <br>
Bypass LAN 192.168.45.0/24: 192.168.45.0/24 === 192.168.45.0/24 PASS <br>
Bypass LAN ::1/128: ::1/128 === ::1/128 PASS <br>
Bypass LAN fe80::/64: fe80::/64 === fe80::/64 PASS <br>
Security Associations (1 up, 0 connecting): <br>
ipseclab[2]: ESTABLISHED 66 minutes ago, 192.168.45.30[ipsec-lab-openwrt]...192.168.45.10[ipsec-lab-pfsense]
<br>
ipseclab[2]: IKEv2 SPIs: e38541185347872a_i* 68249287d7529bd9_r, rekeying in 5 hours
<br>
ipseclab[2]: IKE proposal: AES_CBC_256/HMAC_SHA2_256_128/PRF_HMAC_SHA2_256/MODP_2048
<br>
con1{2}: INSTALLED, TUNNEL, reqid 1, ESP SPIs: c3cf4b91_i c742e267_o <br>
con1{2}: AES_GCM_16_256/MODP_2048, 86268 bytes_i, 0 bytes_o, rekeying in 30 minutes
<br>
con1{2}: 10.10.10.0/30 === 0.0.0.0/0</span><br>
</span></div>
<div style="font-family:Calibri,Helvetica,sans-serif; font-size:12pt; color:rgb(0,0,0)">
<span style="font-family:monospace"><br>
</span></div>
<div style="font-family:Calibri,Helvetica,sans-serif; font-size:12pt; color:rgb(0,0,0)">
<span style="font-family:monospace">I have run a tcpdump capture and I can see the ping packet encapsulated but its destination that apparently was generated by IPSec seems incorrect.</span></div>
<div style="font-family:Calibri,Helvetica,sans-serif; font-size:12pt; color:rgb(0,0,0)">
<span style="font-family:monospace"><br>
</span></div>
<div style="font-family:Calibri,Helvetica,sans-serif; font-size:12pt; color:rgb(0,0,0)">
<span style="font-family:monospace">5 0.000002 10.10.10.2 >> 10.10.10.2 ICMP 144 Destination unreachable (Host unreachable)
<div>Frame 15: 144 bytes on wire (1152 bits), 144 bytes captured (1152 bits)</div>
<div>Linux Netfilter NFLOG</div>
<div> Family: IPv4 (2)</div>
<div> Version: 0</div>
<div> Resource id: 7</div>
<div> TLV Type: NFULA_PACKET_HDR (1), Length: 8</div>
<div> Length: 8</div>
<div> .000 0000 0000 0001 = Type: NFULA_PACKET_HDR (1)</div>
<div> HW protocol: IPv4 (0x0800)</div>
<div> Netfilter hook: Local out (3)</div>
<div> TLV Type: NFULA_PREFIX (10), Length: 5</div>
<div> Length: 5</div>
<div> .000 0000 0000 1010 = Type: NFULA_PREFIX (10)</div>
<div> Prefix: </div>
<div> TLV Type: NFULA_IFINDEX_OUTDEV (5), Length: 8</div>
<div> Length: 8</div>
<div> .000 0000 0000 0101 = Type: NFULA_IFINDEX_OUTDEV (5)</div>
<div> IFINDEX_OUTDEV: 1</div>
<div> TLV Type: NFULA_PAYLOAD (9), Length: 116</div>
<div> Length: 116</div>
<div> .000 0000 0000 1001 = Type: NFULA_PAYLOAD (9)</div>
<div>Internet Protocol Version 4, Src: 10.10.10.2, Dst: 10.10.10.2</div>
<div>Internet Control Message Protocol</div>
<div> Type: 3 (Destination unreachable)</div>
<div> Code: 1 (Host unreachable)</div>
<div> Checksum: 0xfcfe [correct]</div>
<div> [Checksum Status: Good]</div>
<div> Unused: 00000000</div>
<div> Internet Protocol Version 4, Src: 10.10.10.2, Dst: 10.10.10.1</div>
<div> 0100 .... = Version: 4</div>
<div> .... 0101 = Header Length: 20 bytes (5)</div>
<div> Differentiated Services Field: 0x00 (DSCP: CS0, ECN: Not-ECT)</div>
<div> Total Length: 84</div>
<div> Identification: 0x03e4 (996)</div>
<div> Flags: 0x40, Don't fragment</div>
<div> Fragment Offset: 0</div>
<div> Time to Live: 64</div>
<div> Protocol: ICMP (1)</div>
<div> Header Checksum: 0x0eaf [validation disabled]</div>
<div> [Header checksum status: Unverified]</div>
<div> Source Address: 10.10.10.2</div>
<div> Destination Address: 10.10.10.1</div>
<div> Internet Control Message Protocol</div>
<div> Type: 8 (Echo (ping) request)</div>
<div> Code: 0</div>
<div> Checksum: 0x90b7 [unverified] [in ICMP error packet]</div>
<div> [Checksum Status: Unverified]</div>
<div> Identifier (BE): 45 (0x002d)</div>
<div> Identifier (LE): 11520 (0x2d00)</div>
<div> Sequence Number (BE): 494 (0x01ee)</div>
<div> Sequence Number (LE): 60929 (0xee01)</div>
<div> Timestamp from icmp data: Aug 30, 2021 14:51:39.000000000 BST</div>
<div> [Timestamp from icmp data (relative): 0.801353000 seconds]</div>
<div> Data (48 bytes)</div>
<div> Data: 0a17040000000000101112131415161718191a1b1c1d1e1f202122232425262728292a2b…</div>
<span> [Length: 48]</span><br>
</span></div>
<div style="font-family:Calibri,Helvetica,sans-serif; font-size:12pt; color:rgb(0,0,0)">
<span style="font-family:monospace"><br>
</span></div>
<div style="font-family:Calibri,Helvetica,sans-serif; font-size:12pt; color:rgb(0,0,0)">
<span style="font-family:monospace"><br>
</span></div>
<div style="font-family:Calibri,Helvetica,sans-serif; font-size:12pt; color:rgb(0,0,0)">
<span style="font-family:monospace">Any help would be appreciated extremely appreciated.
<br>
</span></div>
<div style="font-family:Calibri,Helvetica,sans-serif; font-size:12pt; color:rgb(0,0,0)">
<span style="font-family:monospace"><br>
</span></div>
<div style="font-family:Calibri,Helvetica,sans-serif; font-size:12pt; color:rgb(0,0,0)">
<span style="font-family:monospace">Many Thanks !!<br>
</span></div>
</div>
</div>
</body>
</html>