[strongSwan] Strange behaviour

Tue Oct 26 08:20:18 CEST 2010

Martin, Guys,

Sorry to be a pain... but this behaviour is occurring every 6 hours or so
now. This time there are no CLOSE_WAITS in the http server, suggesting that
the initial cause is strongswan, ie when it falls over it affect other
things.

I've added a cron job to restart ipsec every hour - I'll see if this solves
things for now. I've flushed our iptables and xt_recent lists.

Are there any kernel patches / strongswan known issues? We're running a
stock ubuntu-server kernel 2.6.32-24-server x86_64 (Ubuntu Lucid, 10.04)
with strongSwan 4.3.2.

On restart, there are a whole swarm of plugin errors - probably due to the
ubuntu packaging, but for completeness:-
Oct 26 06:00:12 api charon: 01[DMN] Starting IKEv2 charon daemon (strongSwan
4.3.2)
Oct 26 06:00:12 api charon: 01[NET] unable to create raw socket: Address
family not supported by protocol
Oct 26 06:00:12 api charon: 01[NET] could not open IPv6 receive socket, IPv6
disabled
Oct 26 06:00:12 api charon: 01[LIB] loading plugin 'sha1' failed:
/usr/lib/ipsec/plugins/libstrongswan-sha1.so: cannot open shared object
file: No such file or directory
Oct 26 06:00:12 api charon: 01[LIB] loading plugin 'fips-prf' failed:
/usr/lib/ipsec/plugins/libstrongswan-fips-prf.so: cannot open shared object
file: No such file or directory
Oct 26 06:00:12 api charon: 01[KNL] listening on interfaces:
Oct 26 06:00:12 api charon: 01[KNL]   eth0
Oct 26 06:00:12 api charon: 01[KNL]   eth1
Oct 26 06:00:12 api charon: 01[KNL]   bond0
Oct 26 06:00:12 api charon: 01[KNL]     10.0.0.54
Oct 26 06:00:12 api charon: 01[KNL]     xxx.20.214.10
Oct 26 06:00:12 api charon: 01[CFG] loading ca certificates from
'/etc/ipsec.d/cacerts'
Oct 26 06:00:12 api charon: 01[LIB]   loaded certificate file
'/etc/ipsec.d/cacerts/ipsec-certificate-authority.certificate.pem'
Oct 26 06:00:12 api charon: 01[CFG] loading aa certificates from
'/etc/ipsec.d/aacerts'
Oct 26 06:00:12 api charon: 01[CFG] loading ocsp signer certificates from
'/etc/ipsec.d/ocspcerts'
Oct 26 06:00:12 api charon: 01[CFG] loading attribute certificates from
'/etc/ipsec.d/acerts'
Oct 26 06:00:12 api charon: 01[CFG] loading crls from '/etc/ipsec.d/crls'
Oct 26 06:00:12 api charon: 01[CFG] loading secrets from
'/etc/ipsec.secrets'
Oct 26 06:00:12 api charon: 01[CFG]   loaded private key file
'/etc/ssh/ssh_host_rsa_key'
Oct 26 06:00:12 api charon: 01[LIB] loading plugin 'sql' failed:
/usr/lib/ipsec/plugins/libstrongswan-sql.so: cannot open shared object file:
No such file or directory
Oct 26 06:00:12 api charon: 01[LIB] loading plugin 'attr' failed:
/usr/lib/ipsec/plugins/libstrongswan-attr.so: cannot open shared object
file: No such file or directory
Oct 26 06:00:12 api charon: 01[CFG] no RADUIS secret defined
Oct 26 06:00:12 api charon: 01[CFG] RADIUS plugin initialization failed
Oct 26 06:00:12 api charon: 01[LIB] loading plugin 'eapradius' failed:
plugin_create() returned NULL
Oct 26 06:00:12 api charon: 01[CFG] mediation database URI not defined,
skipped
Oct 26 06:00:12 api charon: 01[LIB] loading plugin 'medsrv' failed:
plugin_create() returned NULL
Oct 26 06:00:12 api charon: 01[CFG] mediation client database URI not
defined, skipped
Oct 26 06:00:12 api charon: 01[LIB] loading plugin 'medcli' failed:
plugin_create() returned NULL
Oct 26 06:00:12 api charon: 01[LIB] loading plugin 'nm' failed:
/usr/lib/ipsec/plugins/libstrongswan-nm.so: cannot open shared object file:
No such file or directory
Oct 26 06:00:12 api charon: 01[LIB] loading plugin 'resolv-conf' failed:
/usr/lib/ipsec/plugins/libstrongswan-resolv-conf.so: cannot open shared
object file: No such file or directory
Oct 26 06:00:12 api charon: 01[DMN] loaded plugins: curl ldap random x509
pubkey openssl xcbc hmac agent gmp kernel-netlink stroke updown eapidentity
eapmd5 eapgtc eapaka eapmschapv2

BTW IPv6 is disabled in this server. I don't understand why strongswan
should try listening on eth0 or eth1 - it should only be listening on bond0.

Raph

Raphael Cohn
Managing Director
raphael.cohn at StormMQ.com
StormMQ Limited

UK Office:
Gateshead int'l Business Centre, Mulgrave Terrace, Gateshead, NE8 1AN,
United Kingdom
Telephone: +44 845 3712 567

Registered office:
78 Broomfield Road, Chelmsford, Essex, CM1 1SS, United Kingdom
StormMQ Limited is Registered in England and Wales under Company Number
07175657
StormMQ.com

On 25 October 2010 19:25, Raphael Cohn <raphael.cohn at stormmq.com> wrote:

> Martin,
>
> Thank you.
>
> My config excerpts:-
> config setup
>     crlcheckinterval=180
>     strictcrlpolicy=no
>     plutostart=no
>     charonstart=yes
>
> ca ipsec-certificate-authority
>     cacert=ipsec-certificate-authority.certificate.pem
>     auto=add
>
> include /etc/ipsec.d/ipsec.mesh.conf
>
> (ipsec.mesh.conf)
> conn %default
>     ikelifetime=60m
>     keylife=20m
>     rekeymargin=3m
>     keyexchange=ikev2
>     # My IP address for default route (ie eth0)
>     # We can not use %defaultroute (eg if using DHCP, or multiple routes
> because of an aliased interface)
>     left=10.0.0.54
>     leftcert=certificate.pem
>     leftsendcert=always
>     rightsendcert=always
>     auto=start
>
>
> conn local-amqp
>     right=10.0.0.52
>     rightid="C=GB, ST=County Durham, L=Gateshead, O=StormMQ Limited, OU=
> amqp.stormmq.com, CN=amqp.stormmq.com"
>     rightca="C=GB, ST=County Durham, L=Gateshead, O=StormMQ Limited, OU=
> stormmq.com, CN=stormmq.com IPSec Certificate Authority"
>
> ... others similar
>
> We use bonded interfaces, bond0, with aliased ip addresses - in this case
> bond0 is the internal ip address 10.0.0.54.
>
> We've identified a chain of events, in which many connections to our HTTP
> server end up in CLOSE_WAIT - this seems to happen because mid-connection an
> IP address is being blacklisted by iptables (using xt_recent module). At
> this point, connections in CLOSE_WAIT reach c. 500 or so and ipsec seems to
> then fall over... Blaclkisting occurs because a connection from a valid IP
> and http client is identified as 'Invalid' by iptables - as a solitary ACK
> packet initiating a connection. The source IP address is not one covered by
> ipsec routes.
>
> It's not clear which causes what - iptables, HTTP server or ipsec. My
> suspicision is that CLOSE_WAITs occur because the http releasing a resource
> fails because of blacklisting - but why should ipsec's netlink code then
> fail subsequently?
>
> Raph
>
> Raphael Cohn
> Managing Director
> raphael.cohn at StormMQ.com
> StormMQ Limited
>
> UK Office:
> Gateshead int'l Business Centre, Mulgrave Terrace, Gateshead, NE8 1AN,
> United Kingdom
> Telephone: +44 845 3712 567
>
> Registered office:
> 78 Broomfield Road, Chelmsford, Essex, CM1 1SS, United Kingdom
> StormMQ Limited is Registered in England and Wales under Company Number
> 07175657
> StormMQ.com
>
>
>
> On 25 October 2010 08:39, Martin Willi <martin at strongswan.org> wrote:
>
>> Hi Raph,
>>
>> > creating rekey job for ESP CHILD_SA with SPI cbf0f0af and reqid {3720}
>> > creating rekey job for ESP CHILD_SA with SPI c3a35904 and reqid {3721}
>> > creating delete job for ESP CHILD_SA with SPI c2eb09ce and reqid {3720}
>> > creating delete job for ESP CHILD_SA with SPI cbf0f0af and reqid {3720}
>> > creating delete job for ESP CHILD_SA with SPI c3b5dc9e and reqid {3721}
>> > creating delete job for ESP CHILD_SA with SPI c3a35904 and reqid {3721}
>> > creating rekey job for ESP CHILD_SA with SPI c665f5aa and reqid {3722}
>> > creating rekey job for ESP CHILD_SA with SPI c1ccd29a and reqid {3718}
>> > ...
>>
>> The kernel triggers many rekey/delete events for the installed CHILD_SAs
>> concurrently. What is your rekey configuration (lifetime/margin/fuzz)?
>>
>> Regards
>> Martin
>>
>>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.strongswan.org/pipermail/users/attachments/20101026/e267f54c/attachment.html>