[strongSwan] Strongswan freeze

Thu Jul 4 21:01:13 CEST 2013

No value is set for the number of threads. So I'm assuming the default is used, 16?
We have around 50 active connections.  Next time it crashes I can try to grab more details both with a log dump and output of ipsec statusall. We see them a few times a day, so its just a matter of time.

I am looking at another dump of a log we had from a crash, this one is much larger and seems to be stuck on thread 2 (see below)
What is a good number of threads to have? 
 It seems strange that the app freezes (and doesn't unfreeze) when the thread pool runs out.  We had the application stop accepting traffic for over 4 hours before we noticed.

Jun 22 05:03:46 02[KNL] creating rekey job for ESP CHILD_SA with SPI 0671df25 and reqid {24498}Jun 22 05:03:59 02[KNL] creating rekey job for ESP CHILD_SA with SPI c5f54100 and reqid {24498}Jun 22 05:04:51 02[KNL] creating rekey job for ESP CHILD_SA with SPI 0348c58c and reqid {24507}Jun 22 05:06:03 02[KNL] creating rekey job for ESP CHILD_SA with SPI 0a659d3c and reqid {24508}Jun 22 05:06:04 02[KNL] creating rekey job for ESP CHILD_SA with SPI ce877391 and reqid {24507}Jun 22 05:07:06 02[KNL] creating rekey job for ESP CHILD_SA with SPI c1e0e0c4 and reqid {24508}Jun 22 05:07:29 02[KNL] creating rekey job for ESP CHILD_SA with SPI c2d42cc0 and reqid {24516}Jun 22 05:08:03 02[KNL] creating delete job for ESP CHILD_SA with SPI c5f54100 and reqid {24498}Jun 22 05:08:03 02[KNL] creating delete job for ESP CHILD_SA with SPI 0671df25 and reqid {24498}Jun 22 05:08:06 02[KNL] creating rekey job for ESP CHILD_SA with SPI c94c4290 and reqid {24517}Jun 22 05:08:23 02[KNL] creating rekey job for ESP CHILD_SA with SPI c192a4cf and reqid {24515}Jun 22 05:08:42 02[KNL] creating rekey job for ESP CHILD_SA with SPI 03f54713 and reqid {24515}Jun 22 05:09:15 02[KNL] creating rekey job for ESP CHILD_SA with SPI 0bd422a0 and reqid {24516}Jun 22 05:09:28 02[KNL] creating rekey job for ESP CHILD_SA with SPI 0343a5de and reqid {24517}Jun 22 05:09:52 02[KNL] creating rekey job for ESP CHILD_SA with SPI 02301e9b and reqid {24526}Jun 22 05:09:55 02[KNL] creating delete job for ESP CHILD_SA with SPI ce877391 and reqid {24507}Jun 22 05:09:55 02[KNL] creating delete job for ESP CHILD_SA with SPI 0348c58c and reqid {24507}Jun 22 05:09:56 02[KNL] creating acquire job for policy 23.63.227.152/32[tcp/https] === 10.253.0.88/32[tcp/49633] with reqid {24507}Jun 22 05:10:11 02[KNL] creating rekey job for ESP CHILD_SA with SPI c76976cc and reqid {24526}Jun 22 05:10:17 02[KNL] creating rekey job for ESP CHILD_SA with SPI c232c236 and reqid {24474}Jun 22 05:10:22 02[KNL] creating rekey job for ESP CHILD_SA with SPI 03916f37 and reqid {24474}Jun 22 05:10:27 02[KNL] creating rekey job for ESP CHILD_SA with SPI c3ff1926 and reqid {24528}Jun 22 05:10:30 02[KNL] creating delete job for ESP CHILD_SA with SPI c1e0e0c4 and reqid {24508}Jun 22 05:10:30 02[KNL] creating delete job for ESP CHILD_SA with SPI 0a659d3c and reqid {24508}Jun 22 05:11:50 02[KNL] creating rekey job for ESP CHILD_SA with SPI c312f057 and reqid {24529}Jun 22 05:11:54 02[KNL] creating rekey job for ESP CHILD_SA with SPI 08f151c2 and reqid {24532}Jun 22 05:11:56 02[KNL] creating rekey job for ESP CHILD_SA with SPI cea1302d and reqid {24532}Jun 22 05:12:39 02[KNL] creating rekey job for ESP CHILD_SA with SPI 08391a9c and reqid {24528}Jun 22 05:12:49 02[KNL] creating delete job for ESP CHILD_SA with SPI c192a4cf and reqid {24515}Jun 22 05:12:49 02[KNL] creating delete job for ESP CHILD_SA with SPI 03f54713 and reqid {24515}Jun 22 05:12:52 02[KNL] creating delete job for ESP CHILD_SA with SPI c2d42cc0 and reqid {24516}Jun 22 05:12:52 02[KNL] creating delete job for ESP CHILD_SA with SPI 0bd422a0 and reqid {24516}Jun 22 05:12:56 02[KNL] creating rekey job for ESP CHILD_SA with SPI 03366e42 and reqid {24534}Jun 22 05:12:57 02[KNL] creating delete job for ESP CHILD_SA with SPI c94c4290 and reqid {24517}Jun 22 05:12:57 02[KNL] creating delete job for ESP CHILD_SA with SPI 0343a5de and reqid {24517}Jun 22 05:13:05 02[KNL] creating rekey job for ESP CHILD_SA with SPI 0c14915a and reqid {24529}Jun 22 05:13:11 02[KNL] creating rekey job for ESP CHILD_SA with SPI ce798aeb and reqid {24537}Jun 22 05:13:46 02[KNL] creating rekey job for ESP CHILD_SA with SPI 0592d687 and reqid {24539}Jun 22 05:13:50 02[KNL] creating rekey job for ESP CHILD_SA with SPI 0382742f and reqid {24533}Jun 22 05:14:01 02[KNL] creating rekey job for ESP CHILD_SA with SPI c3ee3505 and reqid {24533}Jun 22 05:14:32 02[KNL] creating rekey job for ESP CHILD_SA with SPI 0650351e and reqid {24538}Jun 22 05:14:35 02[KNL] creating rekey job for ESP CHILD_SA with SPI c1f396e6 and reqid {24534}Jun 22 05:14:37 02[KNL] creating rekey job for ESP CHILD_SA with SPI 06ba9eaa and reqid {24537}Jun 22 05:15:19 02[KNL] creating delete job for ESP CHILD_SA with SPI c232c236 and reqid {24474}Jun 22 05:15:19 02[KNL] creating delete job for ESP CHILD_SA with SPI 03916f37 and reqid {24474}Jun 22 05:15:23 02[KNL] creating rekey job for ESP CHILD_SA with SPI c7010733 and reqid {24539}Jun 22 05:15:50 02[KNL] creating delete job for ESP CHILD_SA with SPI c76976cc and reqid {24526}Jun 22 05:15:50 02[KNL] creating delete job for ESP CHILD_SA with SPI 02301e9b and reqid {24526}Jun 22 05:15:58 02[KNL] creating delete job for ESP CHILD_SA with SPI c3ff1926 and reqid {24528}Jun 22 05:15:58 02[KNL] creating delete job for ESP CHILD_SA with SPI 08391a9c and reqid {24528}Jun 22 05:16:06 02[KNL] creating rekey job for ESP CHILD_SA with SPI c032d496 and reqid {24538}Jun 22 05:16:41 02[KNL] creating delete job for ESP CHILD_SA with SPI c312f057 and reqid {24529}Jun 22 05:16:41 02[KNL] creating delete job for ESP CHILD_SA with SPI 0c14915a and reqid {24529}Jun 22 05:17:06 02[KNL] creating delete job for ESP CHILD_SA with SPI cea1302d and reqid {24532}Jun 22 05:17:06 02[KNL] creating delete job for ESP CHILD_SA with SPI 08f151c2 and reqid {24532}Jun 22 05:17:25 02[KNL] creating delete job for ESP CHILD_SA with SPI c3ee3505 and reqid {24533}Jun 22 05:17:25 02[KNL] creating delete job for ESP CHILD_SA with SPI 0382742f and reqid {24533}Jun 22 05:17:39 02[KNL] creating delete job for ESP CHILD_SA with SPI c1f396e6 and reqid {24534}Jun 22 05:17:39 02[KNL] creating delete job for ESP CHILD_SA with SPI 03366e42 and reqid {24534}Jun 22 05:17:56 02[KNL] creating delete job for ESP CHILD_SA with SPI ce798aeb and reqid {24537}Jun 22 05:17:56 02[KNL] creating delete job for ESP CHILD_SA with SPI 06ba9eaa and reqid {24537}Jun 22 05:18:25 02[KNL] creating acquire job for policy 76.13.21.16/32[tcp/https] === 10.253.0.88/32[tcp/49272] with reqid {24507}Jun 22 05:19:07 02[KNL] creating delete job for ESP CHILD_SA with SPI c032d496 and reqid {24538}Jun 22 05:19:07 02[KNL] creating delete job for ESP CHILD_SA with SPI 0650351e and reqid {24538}Jun 22 05:19:07 02[KNL] creating delete job for ESP CHILD_SA with SPI c7010733 and reqid {24539}Jun 22 05:19:07 02[KNL] creating delete job for ESP CHILD_SA with SPI 0592d687 and reqid {24539}Jun 22 05:22:10 02[KNL] creating acquire job for policy 76.13.21.16/32[tcp/https] === 10.253.0.88/32[tcp/49272] with reqid {24507}Jun 22 05:25:55 02[KNL] creating acquire job for policy 76.13.21.16/32[tcp/https] === 10.253.0.88/32[tcp/49272] with reqid {24507}Jun 22 05:28:13 02[KNL] creating acquire job for policy 17.158.8.60/32[tcp/imaps] === 10.255.0.81/32[tcp/53612] with reqid {24532}Jun 22 05:29:23 02[KNL] creating acquire job for policy 134.170.0.216/32[tcp/https] === 10.255.0.22/32[tcp/55364] with reqid {24533}

Joshua J. Gross

> Date: Thu, 4 Jul 2013 20:46:30 +0200
> From: andreas.steffen at strongswan.org
> To: grossjo2 at hotmail.com
> CC: users at lists.strongswan.org
> Subject: Re: [strongSwan] Strongswan freeze
> 
> Hi Joshua,
> 
> how many connections and how many charon threads are you running.
> I just see from the log that always thread 16 is answering, so
> it might be that you have run out of threads.
> 
>   ipsec statusall
> 
> gives you the current jobs/threads situation.
> 
> Regards
> 
> Andreas
> 
> On 07/04/2013 07:55 PM, joshua gross wrote:
> > Hi, we have a n issue where we see a number of there messages in our
> > logs (messages below) and then strongswan stops putting through traffic
> > and accepting new connections.  Essentially it hangs.
> > Can anyone provide some insight into this?
> > 
> > Jul  3 19:25:05 16[KNL] creating rekey job for ESP CHILD_SA with SPI
> > c04e9aee and reqid {1115}
> > Jul  3 19:25:07 16[KNL] creating rekey job for ESP CHILD_SA with SPI
> > 06295e90 and reqid {1155}
> > Jul  3 19:25:10 16[KNL] creating rekey job for ESP CHILD_SA with SPI
> > 0b7ffc36 and reqid {1127}
> > Jul  3 19:25:14 16[KNL] creating rekey job for ESP CHILD_SA with SPI
> > 099af4b4 and reqid {1152}
> > Jul  3 19:25:19 16[KNL] creating delete job for ESP CHILD_SA with SPI
> > c22c511c and reqid {734}
> > Jul  3 19:25:19 16[KNL] creating delete job for ESP CHILD_SA with SPI
> > 0b9de2c7 and reqid {734}
> > Jul  3 19:25:22 16[KNL] creating rekey job for ESP CHILD_SA with SPI
> > c7f78aa6 and reqid {1165}
> > Jul  3 19:25:22 16[KNL] creating rekey job for ESP CHILD_SA with SPI
> > ced489dc and reqid {1160}
> > Jul  3 19:25:27 16[KNL] creating rekey job for ESP CHILD_SA with SPI
> > cef09c72 and reqid {1133}
> > Jul  3 19:25:31 16[KNL] creating rekey job for ESP CHILD_SA with SPI
> > cc84aaa8 and reqid {1156}
> > 
> > 
> > 
> > ------------------------------------------------------------------------
> > 
> > 
> > 
> > Joshua J. Gross
> 
> ======================================================================
> Andreas Steffen                         andreas.steffen at strongswan.org
> strongSwan - the Linux VPN Solution!                www.strongswan.org
> Institute for Internet Technologies and Applications
> University of Applied Sciences Rapperswil
> CH-8640 Rapperswil (Switzerland)
> ===========================================================[ITA-HSR]==
> 

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.strongswan.org/pipermail/users/attachments/20130704/8e0207d6/attachment.html>