[strongSwan] Problem with pcrypt
Noel Kuntze
noel.kuntze+strongswan-users-ml at thermi.consulting
Fri Sep 15 19:27:54 CEST 2017
Hi,
I guess ksoftirqd is rotating and kworker, too? If that's the case, you're suffering from
an extremely disadvantageous distribution of ESP packets.
You need to set the number of RX and TX queues on the card to the number of cores and
use RSS to distribute the SAs correctly over all queues. Bind one RX and one TX queue to one core each.
Then use AES based ciphers, so you can use AES-NI. You can then get line speed per CHILD_SA.
Pcrypt has some overhead due to synchronisation, so if your setup's performance problem is not caused
by cipher execution time, pcrypt will not improve the situation.
Use aes128gcm8. aes256gcm16 causes unnecessary overhead and costs more performance.
Disabling replay protection does not improve performance.
Kind regards
NOel
On 15.09.2017 19:17, Sven Anders wrote:
> Hello!
>
> I setup StrongSwan and I am currently performing some throughput tests.
>
> I have two VM-Ware ESXi servers and on each of these is a VM running Ubuntu 16.04.
> Each VM has 8 virtual CPUs. The VM-Ware servers are interconnected with an 10G link.
> The performance on the link is about ~500 MBytes/s. This is because there currently
> other productive instances running in parallel. Without these I will get about
> ~950 MBytes/s. But this should be enough for my tests...
>
>
> Now my problem:
>
> If I run the IPSec connection without the pcrypt module loaded, I will get the
> following throughput:
>
>> ./netperf -H 192.168.184.250 -fM -D 5 -c 1 -C 1 -l 30
> Recv Send Send Utilization Service Demand
> Socket Socket Message Elapsed Send Recv Send Recv
> Size Size Size Time Throughput local remote local remote
> bytes bytes bytes secs. MBytes /s % S % S us/KB us/KB
>
> 87380 16384 16384 30.01 152.25 10.52 6.21 5.397 3.186
> 87380 16384 16384 30.00 157.79 11.05 7.82 5.469 3.872
> 87380 16384 16384 30.01 167.69 11.94 6.79 5.560 3.162
> 87380 16384 16384 30.00 165.32 11.82 7.19 5.585 3.398
>
> If I run this with the pcrypt module loaded, I will get the following throughput:
>
>> ./netperf -H 192.168.184.250 -fM -D 5 -c 1 -C 1 -l 30
> Recv Send Send Utilization Service Demand
> Socket Socket Message Elapsed Send Recv Send Recv
> Size Size Size Time Throughput local remote local remote
> bytes bytes bytes secs. MBytes /s % S % S us/KB us/KB
>
> 87380 16384 16384 30.02 93.67 21.12 17.45 17.615 14.557
> 87380 16384 16384 30.02 87.30 21.81 17.14 19.515 15.338
> 87380 16384 16384 30.02 90.50 21.23 4.29 18.324 3.706
> 87380 16384 16384 30.02 92.18 20.97 4.19 17.771 3.548
>
> [Beware: All throughput values are in MBytes/s not Mbits/s !]
>
> Can anybody explain this?
> I expected the throughput to increase not to decrease...
>
> I loaded the pcrypt module with:
>
> modprobe tcrypt alg="pcrypt(rfc4106(gcm(aes)))" type=3
>
> (and restarted strongswan with "systemctrl restart strongswan")
>
> I played with the "replay_window" option (tried 0 and 64), but it did not help.
>
> What could cause this slowdown?
>
> Any kind of help is appreciated...
>
> Regards
> Sven
>
>
> This is my configuration:
> -------------------------
>
> # /etc/ipsec.conf - strongSwan IPsec configuration file
> config setup
> conn %default
> ikelifetime=60m
> keylife=20m
> rekeymargin=3m
> keyingtries=1
> keyexchange=ikev2
> mobike=no
> esp=aes256gcm16!
> conn net-net
> left=10.10.99.1
> leftcert=moonCert.pem
> leftid=@moon.strongswan.org
> leftsubnet=192.168.183.0/24
> leftfirewall=yes
> right=10.10.99.2
> rightid=@sun.strongswan.org
> rightsubnet=192.168.184.0/24
> auto=add
> replay_window=0
>
> (Same on the other side, just the IPs swapped and with the sunCert.pem certificate ...)
>
> My strongswan output:
>
> charon[4392]: 00[DMN] Starting IKE charon daemon (strongSwan 5.3.5, Linux 4.4.0-93-generic, x86_64)
> charon[4392]: 00[CFG] loading ca certificates from '/etc/ipsec.d/cacerts'
> charon[4392]: 00[CFG] loaded ca certificate "C=CH, O=Linux strongSwan, CN=strongSwan Root CA" from '/etc/ipsec.d/cacerts/strongswanCert.pem'
> charon[4392]: 00[CFG] loading aa certificates from '/etc/ipsec.d/aacerts'
> charon[4392]: 00[CFG] loading ocsp signer certificates from '/etc/ipsec.d/ocspcerts'
> charon[4392]: 00[CFG] loading attribute certificates from '/etc/ipsec.d/acerts'
> charon[4392]: 00[CFG] loading crls from '/etc/ipsec.d/crls'
> charon[4392]: 00[CFG] loading secrets from '/etc/ipsec.secrets'
> charon[4392]: 00[CFG] loaded RSA private key from '/etc/ipsec.d/private/moonKey.pem'
> charon[4392]: 00[CFG] loaded IKE secret for @moon.strongswan.org @sun.strongswan.org
> charon[4392]: 00[CFG] opening triplet file /etc/ipsec.d/triplets.dat failed: No such file or directory
> charon[4392]: 00[CFG] loaded 0 RADIUS server configurations
> charon[4392]: 00[LIB] loaded plugins: charon test-vectors aes rc2 sha1 sha2 md4 md5 random nonce x509 revocation constraints pubkey pkcs1 pkcs7 pkcs8
> pkcs12 pgp dnskey sshkey pem openssl fips-prf gmp agent xcbc hmac gcm attr kernel-netlink resolve socket-default connmark farp stroke updown
> eap-identity eap-sim eap-sim-pcsc eap-aka eap-aka-3gpp2 eap-simaka-pseudonym eap-simaka-reauth eap-md5 eap-gtc eap-mschapv2 eap-dynamic eap-radius
> eap-tls eap-ttls eap-peap eap-tnc xauth-generic xauth-eap xauth-pam xauth-noauth tnc-tnccs tnccs-20 tnccs-11 tnccs-dynamic dhcp lookip error-notify
> certexpire led addrblock unity
> charon[4392]: 00[LIB] dropped capabilities, running as uid 0, gid 0
> charon[4392]: 00[JOB] spawning 16 worker threads
> ipsec_starter[4391]: charon (4392) started after 40 ms
> charon[4392]: 06[CFG] received stroke: add connection 'net-net'
> charon[4392]: 06[CFG] loaded certificate "C=CH, O=Linux strongSwan, CN=moon.strongswan.org" from 'moonCert.pem'
> charon[4392]: 06[CFG] added configuration 'net-net'
> charon[4392]: 06[CFG] received stroke: initiate 'net-net'
> charon[4392]: 07[IKE] initiating IKE_SA net-net[1] to 10.10.99.2
> charon[4392]: 07[IKE] initiating IKE_SA net-net[1] to 10.10.99.2
> charon[4392]: 07[ENC] generating IKE_SA_INIT request 0 [ SA KE No N(NATD_S_IP) N(NATD_D_IP) N(HASH_ALG) ]
> charon[4392]: 07[NET] sending packet: from 10.10.99.1[500] to 10.10.99.2[500] (1124 bytes)
> charon[4392]: 09[NET] received packet: from 10.10.99.2[500] to 10.10.99.1[500] (481 bytes)
> charon[4392]: 09[ENC] parsed IKE_SA_INIT response 0 [ SA KE No N(NATD_S_IP) N(NATD_D_IP) CERTREQ N(HASH_ALG) N(MULT_AUTH) ]
> charon[4392]: 09[IKE] received cert request for "C=CH, O=Linux strongSwan, CN=strongSwan Root CA"
> charon[4392]: 09[IKE] sending cert request for "C=CH, O=Linux strongSwan, CN=strongSwan Root CA"
> charon[4392]: 09[IKE] authentication of 'moon.strongswan.org' (myself) with RSA_EMSA_PKCS1_SHA256 successful
> charon[4392]: 09[IKE] sending end entity cert "C=CH, O=Linux strongSwan, CN=moon.strongswan.org"
> charon[4392]: 09[IKE] establishing CHILD_SA net-net
> charon[4392]: 09[IKE] establishing CHILD_SA net-net
> charon[4392]: 09[ENC] generating IKE_AUTH request 1 [ IDi CERT N(INIT_CONTACT) CERTREQ IDr AUTH SA TSi TSr N(MULT_AUTH) N(EAP_ONLY) ]
> charon[4392]: 09[NET] sending packet: from 10.10.99.1[500] to 10.10.99.2[500] (1596 bytes)
> charon[4392]: 10[NET] received packet: from 10.10.99.2[500] to 10.10.99.1[500] (1532 bytes)
> charon[4392]: 10[ENC] parsed IKE_AUTH response 1 [ IDr CERT AUTH SA TSi TSr N(AUTH_LFT) ]
> charon[4392]: 10[IKE] received end entity cert "C=CH, O=Linux strongSwan, CN=sun.strongswan.org"
> charon[4392]: 10[CFG] using certificate "C=CH, O=Linux strongSwan, CN=sun.strongswan.org"
> charon[4392]: 10[CFG] using trusted ca certificate "C=CH, O=Linux strongSwan, CN=strongSwan Root CA"
> charon[4392]: 10[CFG] checking certificate status of "C=CH, O=Linux strongSwan, CN=sun.strongswan.org"
> charon[4392]: 10[CFG] fetching crl from 'http://crl.strongswan.org/strongswan.crl' ...
> charon[4392]: 10[LIB] unable to fetch from http://crl.strongswan.org/strongswan.crl, no capable fetcher found
> charon[4392]: 10[CFG] crl fetching failed
> charon[4392]: 10[CFG] certificate status is not available
> charon[4392]: 10[CFG] reached self-signed root ca with a path length of 0
> charon[4392]: 10[IKE] authentication of 'sun.strongswan.org' with RSA_EMSA_PKCS1_SHA256 successful
> charon[4392]: 10[IKE] IKE_SA net-net[1] established between 10.10.99.1[moon.strongswan.org]...10.10.99.2[sun.strongswan.org]
> charon[4392]: 10[IKE] IKE_SA net-net[1] established between 10.10.99.1[moon.strongswan.org]...10.10.99.2[sun.strongswan.org]
> charon[4392]: 10[IKE] scheduling reauthentication in 3279s
> charon[4392]: 10[IKE] maximum IKE_SA lifetime 3459s
> charon[4392]: 10[IKE] CHILD_SA net-net{1} established with SPIs c70d5aba_i cb7e5654_o and TS 192.168.183.0/24 === 192.168.184.0/24
> charon[4392]: 10[IKE] CHILD_SA net-net{1} established with SPIs c70d5aba_i cb7e5654_o and TS 192.168.183.0/24 === 192.168.184.0/24
> vpn[4421]: + sun.strongswan.org 192.168.184.0/24 == 10.10.99.2 -- 10.10.99.1 == 192.168.183.0/24
> charon[4392]: 10[IKE] received AUTH_LIFETIME of 3391s, scheduling reauthentication in 3211s
>
>> ipsec statusall
> Status of IKE charon daemon (strongSwan 5.3.5, Linux 4.4.0-93-generic, x86_64):
> uptime: 28 minutes, since Sep 15 18:34:16 2017
> malloc: sbrk 2727936, mmap 0, used 616576, free 2111360
> worker threads: 11 of 16 idle, 5/0/0/0 working, job queue: 0/0/0/0, scheduled: 3
> loaded plugins: charon test-vectors aes rc2 sha1 sha2 md4 md5 random nonce x509 revocation constraints pubkey pkcs1 pkcs7 pkcs8 pkcs12 pgp dnskey
> sshkey pem openssl fips-prf gmp agent xcbc hmac gcm attr kernel-netlink resolve socket-default connmark farp stroke updown eap-identity eap-sim
> eap-sim-pcsc eap-aka eap-aka-3gpp2 eap-simaka-pseudonym eap-simaka-reauth eap-md5 eap-gtc eap-mschapv2 eap-dynamic eap-radius eap-tls eap-ttls
> eap-peap eap-tnc xauth-generic xauth-eap xauth-pam xauth-noauth tnc-tnccs tnccs-20 tnccs-11 tnccs-dynamic dhcp lookip error-notify certexpire led
> addrblock unity
> Listening IP addresses:
> 10.10.133.101
> 10.10.99.1
> 192.168.183.250
> Connections:
> net-net: 10.10.99.1...10.10.99.2 IKEv2
> net-net: local: [moon.strongswan.org] uses public key authentication
> net-net: cert: "C=CH, O=Linux strongSwan, CN=moon.strongswan.org"
> net-net: remote: [sun.strongswan.org] uses public key authentication
> net-net: child: 192.168.183.0/24 === 192.168.184.0/24 TUNNEL
> Security Associations (1 up, 0 connecting):
> net-net[1]: ESTABLISHED 28 minutes ago, 10.10.99.1[moon.strongswan.org]...10.10.99.2[sun.strongswan.org]
> net-net[1]: IKEv2 SPIs: 98a6d3eeeaf1f9c0_i* a25f94aff1cdec12_r, public key reauthentication in 25 minutes
> net-net[1]: IKE proposal: AES_CBC_128/HMAC_SHA1_96/PRF_HMAC_SHA1/MODP_2048
> net-net{2}: INSTALLED, TUNNEL, reqid 1, ESP SPIs: c0b03d25_i c07fc07c_o
> net-net{2}: AES_GCM_16_256/ESN, 94353752 bytes_i, 5965332882 bytes_o, rekeying in 11 seconds
> net-net{2}: 192.168.183.0/24 === 192.168.184.0/24
> net-net{3}: INSTALLED, TUNNEL, reqid 1, ESP SPIs: c770da0f_i c8fe02d2_o
> net-net{3}: AES_GCM_16_256/ESN, 0 bytes_i, 0 bytes_o, rekeying in 11 minutes
> net-net{3}: 192.168.183.0/24 === 192.168.184.0/24
>
>> ip xfrm state
> src 10.10.99.1 dst 10.10.99.2
> proto esp spi 0xc07fc07c reqid 1 mode tunnel
> replay-window 0 flag af-unspec
> aead rfc4106(gcm(aes)) 0xcbcc5acbe14362c202a98286a72ea9b25f2f054435413c7097e476ed3e7c6d0b5d6c6f1d 128
> anti-replay context: seq 0x0, oseq 0x3ef313, bitmap 0x00000000
> src 10.10.99.2 dst 10.10.99.1
> proto esp spi 0xc0b03d25 reqid 1 mode tunnel
> replay-window 0 flag af-unspec
> aead rfc4106(gcm(aes)) 0x58ae85c2c38b8102aabc3d8f14d62988e8935ebbc84a83eff13a3bd831a5fa1b36768b3b 128
> anti-replay context: seq 0x0, oseq 0x0, bitmap 0x00000000
>
>> cat /proc/crypto | grep -C10 pcrypt
> name : seqiv(rfc4106(gcm(aes)))
> driver : seqiv(pcrypt(rfc4106-gcm-aesni))
> module : seqiv
> priority : 500
> refcnt : 6
> selftest : passed
> internal : no
> type : aead
> async : yes
> blocksize : 1
> ivsize : 8
> maxauthsize : 16
> geniv : <none>
>
> name : rfc4106(gcm(aes))
> driver : pcrypt(rfc4106-gcm-aesni)
> module : pcrypt
> priority : 500
> refcnt : 6
> selftest : passed
> internal : no
> type : aead
> async : yes
> blocksize : 1
> ivsize : 8
> maxauthsize : 16
> geniv : <none>
>
>> cpuinfo (8 CPUs for each VM instance):
> processor : 7
> vendor_id : GenuineIntel
> cpu family : 6
> model : 79
> model name : Intel(R) Xeon(R) CPU E5-2630 v4 @ 2.20GHz
> stepping : 1
> microcode : 0xb00001f
> cpu MHz : 2197.455
> cache size : 25600 KB
> physical id : 14
> siblings : 1
> core id : 0
> cpu cores : 1
> apicid : 14
> initial apicid : 14
> fpu : yes
> fpu_exception : yes
> cpuid level : 20
> wp : yes
> flags : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts mmx fxsr sse sse2 ss syscall nx pdpe1gb rdtscp lm
> constant_tsc arch_perfmon pebs bts nopl xtopology tsc_reliable nonstop_tsc aperfmperf eagerfpu pni pclmulqdq ssse3 fma cx16 pcid sse4_1 sse4_2 x2apic
> movbe popcnt tsc_deadline_timer aes xsave avx f16c rdrand hypervisor lahf_lm abm 3dnowprefetch epb fsgsbase tsc_adjust bmi1 hle avx2 smep bmi2 invpcid
> rtm rdseed adx smap xsaveopt dtherm ida arat pln pts
> bugs :
> bogomips : 4394.91
> clflush size : 64
> cache_alignment : 64
> address sizes : 42 bits physical, 48 bits virtual
> power management:
>
>> ip xfrm policy
> src 192.168.184.0/24 dst 192.168.183.0/24
> dir fwd priority 2883
> tmpl src 10.10.99.2 dst 10.10.99.1
> proto esp reqid 1 mode tunnel
> src 192.168.184.0/24 dst 192.168.183.0/24
> dir in priority 2883
> tmpl src 10.10.99.2 dst 10.10.99.1
> proto esp reqid 1 mode tunnel
> src 192.168.183.0/24 dst 192.168.184.0/24
> dir out priority 2883
> tmpl src 10.10.99.1 dst 10.10.99.2
> proto esp reqid 1 mode tunnel
> src 0.0.0.0/0 dst 0.0.0.0/0
> socket in priority 0
> src 0.0.0.0/0 dst 0.0.0.0/0
> socket out priority 0
> src 0.0.0.0/0 dst 0.0.0.0/0
> socket in priority 0
> src 0.0.0.0/0 dst 0.0.0.0/0
> socket out priority 0
> src ::/0 dst ::/0
> socket in priority 0
> src ::/0 dst ::/0
> socket out priority 0
> src ::/0 dst ::/0
> socket in priority 0
> src ::/0 dst ::/0
> socket out priority 0
>
>
>
>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 833 bytes
Desc: OpenPGP digital signature
URL: <http://lists.strongswan.org/pipermail/users/attachments/20170915/445bdd69/attachment-0001.sig>
More information about the Users
mailing list