[strongSwan] Latency and throughput

[sangdrax8]

I am having some problems with throughput in situations with high BPD.  If
anyone has some suggestions it would be much appreciated.

My setup is currently as follows:

2 Ubuntu boxes running Strongswan 5.3.2 and kernel 3.13.0-71-generic.  I am
using IKEv2 (don't think that matters), ESP in tunnel mode, with
aes256-modp3072-esn, and I set the replay_window=0.  I connect both devices
through a third box that I use to adjust the latency between the devices.

If I do not increase latency, I can iperf3 between both servers at 900's
Mbps, both with and with out encryption.  The boxes don't show any
significant load, so I wouldn't expect to have hardware issues after the
latency between them increases.  As I increase latency, my throughput
starts to go down if ipsec is used.  With latency of just 30ms, I see a 1
minute test push near 600 Mbps.  If I simply turn off ipsec, and run the
same test, I hit 900's with in the first 2-3 seconds and stay there.

This gets much worse as I jump up to 100ms.  With some TCP tuning, my
unencrypted transfer reaches the 900's with in about 10 seconds.  My
encrypted transfer just hovers around 100Mbps.  I only achieved this after
reading a previous post about the replay_window.  With replay window on I
wasn't able to push even 10Mbps.

This feels like the replay_window, or the TCP tuning with tcp_rmem/wmem
that I needed to do to account for the extra packets that are in flight
over the link with such high latency and bandwidth.  ESP isn't TCP, so I
have increased wmem_default, wmem_max, rmem_default, and rmem_max to all be
the max value I used when doing my TCP tuning, but I still get the 100Mbps
while encrypting.

Any suggestions/pointers as to other tuning steps that might be needed to
ensure packets are sent at the higher rate I know this setup can achieve?
Is there still some replay_window type delays taking place here that I have
missed?

Thank you.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.strongswan.org/pipermail/users/attachments/20151211/a5a50ae9/attachment.html>