[strongSwan] Maximizing throughput / kernel bottlenecks

[Hose]

What you say...Noel Kuntze (noel at familie-kuntze.de):

> Hello Hose,
> 
> > To keep crypto overhead down the IPsec tunnels are constructed in
> > transport mode with aes128/sha1 for IKE and aes128/md5 for IPsec;
> aes128-sha1 and aes128-md5 are not really the optimum.
> Try to use aesgcm with any key length (lower key lengths are obviously faster).
> There's mysterious packet loss going on in certain configurations.
> Make sure you don't experience it (issue #1220).

So I ended up taking the time and switching all the endpoints in the
network to strongswan w/IKEv2 so I could toss in full aesgcm support (I
know it's supported on IKEv1, but there were a few on racoon and it just
made sense to do the entire network at once). The changes were easy and
I had a full OSPF network back up with barely any issues.

Unfortunately I appear to have the same throughput, both between the
gig-e connected machines and down to the home connection (I'm less
concerned about the latter, but tossing it in as an additional
data point). There is no appreaciable load on any of the systems during
throughput testing.

> > On machines
> > connected via gig-e I'm getting between 150 - 200 mb/s on average over
> > the tunnels (900+ mb/s unencrypted). One of those machines also has a
> > tunnel over to my home via a consumer internet connection (10mb up, 50mb
> > down) but I'm getting relatively slow speeds: ~20mb/s through the
> > tunnel. It pushes the max speed of 50mb/s to the same host when
> > unencrypted.. 
> Don't rely on a consumer connection for testing and benchmarking.
> ISPs do QoS. Test on a direct gigabit link between the boxes. Only then you will
> get reliably benchmark results. Also make sure your kernel uses optimized crypto drivers.
> Google will help you.
> 
> > The only thing I haven't done is migrate over to IKEv2 
> > which is on the roadmap but haven't implemented yet due to some legacy 
> > requirements, however I can't imagine that would actually effect 
> > throughput as that seems to be a kernel bottleneck.
> That wouldn't influence the rate anyway, because the kernel doesn't
> know about IKE.
> 
> There are performance improvements coming in any of the next kernel versions,
> which have been announced on the Netdev 1.1 conference in Seville in February.
> They will introduce a lot of changes, which will improve performance.
> The changes will improve performance nearly twofold.
> A video of the specific talk is on Youtube (https://www.youtube.com/watch?v=JSbU5YE8Hc0).
> The performance numbers are starting to be shown at 26:00.
> 
> -- 
> 
> Mit freundlichen Grüßen/Kind Regards,
> Noel Kuntze
> 

I've read that aes-gcm has been built to scale to 10ge and 40ge, so I'm
kind of stumped at this point; looks like this might be a kernel
bottleneck. Does anyone else have experience with higher throughput on
their IPsec tunnels, whether or not utilizing aes-gcm?

For completeness, this is with IKE aes192gcm8-aesxcbc-modp2048 and ESP
aes128gcm8-aesxcbc-modp1536.

On the positive side, I am now rid of the horrifying racoon config (I'm
looking at you, ipsec-tools.conf); using conn %default has made the
strongswan ipsec.conf file ridiculously easy to parse.

hose