<html><body><div style="color:#000; background-color:#fff; font-family:lucida console, sans-serif;font-size:12pt"><div><var id="yui-ie-cursor"></var></div><span><div><font face="Times New Roman">
</font></div><div class="MsoNormal" style="margin: 0in 0in 10pt;"><span style='font-family: "Arial","sans-serif";'>Hi Martin,<o:p></o:p></span></div><div><font face="Times New Roman">
</font></div><div><font face="Times New Roman">
</font></div><div class="MsoNormal" style="margin: 0in 0in 10pt;"><span style='font-family: "Arial","sans-serif";'>I did profiling
the Charon implementation (using perf profiler tool) to find the main
bottleneck with 25k IPsec connections (without data traffic). The perf tool
generates an output file called perf.data. That file can then be analyzed using
the perf report. I found from call stack in the perf profiler ,: gmpn_addmul_1
function in<span style="mso-spacerun: yes;"> </span>libgmp.so.3.4.1 consumes
most of the CPU cycles on both the Linux systems ( IKE Initiator as well as <span style="mso-spacerun: yes;"> </span>IKE Responder) . It was clearly the hottest
procedure in the Chardon keying daemon (IKEv2). What I understand, the
strongswan uses the gmp library for the implementation of DH and we are using
the DH group as modp1024 at both ends.<o:p></o:p></span></div><div><font face="Times New Roman">
</font></div><div class="MsoNormal" style="margin: 0in 0in 10pt;"><span style='font-family: "Arial","sans-serif";'>Here goes the
results of #perf report<o:p></o:p></span></div><div><font face="Times New Roman">
</font></div><div class="MsoNormal" style="margin: 0in 0in 10pt;"><span style='font-family: "Arial","sans-serif";'>3.72%<span style="mso-spacerun: yes;"> </span>charon<span style="mso-spacerun: yes;"> </span>libgmp.so.3.4.1<span style="mso-spacerun: yes;">
</span>[.] __gmpn_addmul_1<o:p></o:p></span></div><div><font face="Times New Roman">
</font></div><div class="MsoNormal" style="margin: 0in 0in 10pt;"><span style='font-family: "Arial","sans-serif";'><span style="mso-spacerun: yes;"> </span>0.86%<span style="mso-spacerun: yes;"> </span>charon<span style="mso-spacerun: yes;"> </span>libcharon.so.0.0.0<span style="mso-spacerun: yes;">
</span>[.] checkout_by_message<o:p></o:p></span></div><div><font face="Times New Roman">
</font></div><div class="MsoNormal" style="margin: 0in 0in 10pt;"><span style='font-family: "Arial","sans-serif";'><span style="mso-spacerun: yes;"> </span>0.38%<span style="mso-spacerun: yes;"> </span>charon<span style="mso-spacerun: yes;"> </span>libc-2.11.1.so<span style="mso-spacerun: yes;">
</span>[.] memcmp<o:p></o:p></span></div><div><font face="Times New Roman">
</font></div><div class="MsoNormal" style="margin: 0in 0in 10pt;"><span style='font-family: "Arial","sans-serif";'><span style="mso-spacerun: yes;"> </span>0.19%<span style="mso-spacerun: yes;"> </span>charon<span style="mso-spacerun: yes;"> </span>libgmp.so.3.4.1<span style="mso-spacerun: yes;">
</span><span style="mso-spacerun: yes;"> </span>[.] __gmpn_add_n<o:p></o:p></span></div><div><font face="Times New Roman">
</font></div><div class="MsoNormal" style="margin: 0in 0in 10pt;"><span style='font-family: "Arial","sans-serif";'><span style="mso-spacerun: yes;"> </span>0.15%<span style="mso-spacerun: yes;"> </span>charon<span style="mso-spacerun: yes;"> </span>libgmp.so.3.4.1<span style="mso-spacerun: yes;">
</span>[.] __gmpn_mul_1<o:p></o:p></span></div><div><font face="Times New Roman">
</font></div><div class="MsoNormal" style="margin: 0in 0in 10pt;"><span style='font-family: "Arial","sans-serif";'><span style="mso-spacerun: yes;"> </span>0.10%<span style="mso-spacerun: yes;"> </span>charon<span style="mso-spacerun: yes;"> </span>libgmp.so.3.4.1<span style="mso-spacerun: yes;">
</span><span style="mso-spacerun: yes;"> </span>[.] 0x00000000022ed8<o:p></o:p></span></div><div><font face="Times New Roman">
</font></div><div class="MsoNormal" style="margin: 0in 0in 10pt;"><span style='font-family: "Arial","sans-serif";'><span style="mso-spacerun: yes;"> </span>0.09%<span style="mso-spacerun: yes;"> </span>charon<span style="mso-spacerun: yes;"> </span>libgmp.so.3.4.1<span style="mso-spacerun: yes;">
</span>[.] __gmpn_sqr_basecase<o:p></o:p></span></div><div><font face="Times New Roman">
</font></div><div class="MsoNormal" style="margin: 0in 0in 10pt;"><span style='font-family: "Arial","sans-serif";'><span style="mso-spacerun: yes;"> </span>0.09%<span style="mso-spacerun: yes;"> </span>charon<span style="mso-spacerun: yes;"> </span>libgmp.so.3.4.1<span style="mso-spacerun: yes;"> </span><span style="mso-spacerun: yes;"> </span>[.]
__gmpn_sqr_diagonal<o:p></o:p></span></div><div><font face="Times New Roman">
</font></div><div class="MsoNormal" style="margin: 0in 0in 10pt;"><span style='font-family: "Arial","sans-serif";'><span style="mso-spacerun: yes;"> </span>0.07%<span style="mso-spacerun: yes;"> </span>charon<span style="mso-spacerun: yes;"> </span>libgmp.so.3.4.1<span style="mso-spacerun: yes;">
</span>[.] __gmpn_lshift<o:p></o:p></span></div><div><font face="Times New Roman">
</font></div><div class="MsoNormal" style="margin: 0in 0in 10pt;"><span style='font-family: "Arial","sans-serif";'><span style="mso-spacerun: yes;"> </span>0.04%<span style="mso-spacerun: yes;"> </span>charon<span style="mso-spacerun: yes;"> </span>libgmp.so.3.4.1<span style="mso-spacerun: yes;"> </span><span style="mso-spacerun: yes;"> </span>[.]
__gmpn_sub_n<o:p></o:p></span></div><div><font face="Times New Roman">
</font></div><div class="MsoNormal" style="margin: 0in 0in 10pt;"><span style='font-family: "Arial","sans-serif";'><span style="mso-spacerun: yes;"> </span>0.04%<span style="mso-spacerun: yes;"> </span>charon<span style="mso-spacerun: yes;"> </span>libc-2.11.1.so<span style="mso-spacerun: yes;">
</span>[.] 0x00000000096284<o:p></o:p></span></div><div><font face="Times New Roman">
</font></div><div class="MsoNormal" style="margin: 0in 0in 10pt;"><span style='font-family: "Arial","sans-serif";'><span style="mso-spacerun: yes;"> </span>0.04%<span style="mso-spacerun: yes;"> </span>charon<span style="mso-spacerun: yes;"> </span>libpthread-2.11.1.so<span style="mso-spacerun: yes;"> </span><span style="mso-spacerun: yes;"> </span>[.]
pthread_rwlock_rdlock<o:p></o:p></span></div><div><font face="Times New Roman">
</font></div><div class="MsoNormal" style="margin: 0in 0in 10pt;"><span style='font-family: "Arial","sans-serif";'><span style="mso-spacerun: yes;"> </span>0.04%<span style="mso-spacerun: yes;"> </span>charon<span style="mso-spacerun: yes;"> </span>libpthread-2.11.1.so<span style="mso-spacerun: yes;">
</span>[.] __pthread_rwlock_unlock<o:p></o:p></span></div><div><font face="Times New Roman">
</font></div><div class="MsoNormal" style="margin: 0in 0in 10pt;"><span style='font-family: "Arial","sans-serif";'><span style="mso-spacerun: yes;"> </span>0.02%<span style="mso-spacerun: yes;"> </span>charon<span style="mso-spacerun: yes;"> </span>libc-2.11.1.so<span style="mso-spacerun: yes;">
</span>[.] __libc_malloc<o:p></o:p></span></div><div><font face="Times New Roman">
</font></div><div class="MsoNormal" style="margin: 0in 0in 10pt;"><span style='font-family: "Arial","sans-serif";'><span style="mso-spacerun: yes;"> </span>0.02%<span style="mso-spacerun: yes;"> </span>charon<span style="mso-spacerun: yes;"> </span>libgmp.so.3.4.1<span style="mso-spacerun: yes;">
</span>[.] __gmpn_mul_basecase<o:p></o:p></span></div><div><font face="Times New Roman">
</font></div><div class="MsoNormal" style="margin: 0in 0in 10pt;"><span style='font-family: "Arial","sans-serif";'><span style="mso-spacerun: yes;"> </span>0.02%<span style="mso-spacerun: yes;"> </span>charon<span style="mso-spacerun: yes;"> </span>libgmp.so.3.4.1<span style="mso-spacerun: yes;">
</span>[.] __gmpz_powm<o:p></o:p></span></div><div><font face="Times New Roman">
</font></div><div class="MsoNormal" style="margin: 0in 0in 10pt;"><span style='font-family: "Arial","sans-serif";'><span style="mso-spacerun: yes;"> </span>0.02%<span style="mso-spacerun: yes;"> </span>charon<span style="mso-spacerun: yes;"> </span>libstrongswan-sha1.so<span style="mso-spacerun: yes;">
</span><span style="mso-spacerun: yes;"> </span>[.] SHA1Transform<o:p></o:p></span></div><div><font face="Times New Roman">
</font></div><div class="MsoNormal" style="margin: 0in 0in 10pt;"><span style='font-family: "Arial","sans-serif";'><span style="mso-spacerun: yes;"> </span>0.02%<span style="mso-spacerun: yes;"> </span>charon<span style="mso-spacerun: yes;"> </span>libc-2.11.1.so<span style="mso-spacerun: yes;">
</span>[.] cfree<o:p></o:p></span></div><div><font face="Times New Roman">
</font></div><div class="MsoNormal" style="margin: 0in 0in 10pt;"><span style='font-family: "Arial","sans-serif";'><span style="mso-spacerun: yes;"> </span>0.02%<span style="mso-spacerun: yes;"> </span>charon<span style="mso-spacerun: yes;"> </span>[kernel.kallsyms]<span style="mso-spacerun: yes;">
</span><span style="mso-spacerun: yes;"> </span>[k] sha_transform<o:p></o:p></span></div><div><font face="Times New Roman">
</font></div><div class="MsoNormal" style="margin: 0in 0in 10pt;"><span style='font-family: "Arial","sans-serif";'><span style="mso-spacerun: yes;"> </span>0.02%<span style="mso-spacerun: yes;"> </span>charon<span style="mso-spacerun: yes;"> </span>libc-2.11.1.so<span style="mso-spacerun: yes;">
</span>[.] vfprintf<o:p></o:p></span></div><div><font face="Times New Roman">
</font></div><div class="MsoNormal" style="margin: 0in 0in 10pt;"><span style='font-family: "Arial","sans-serif";'><span style="mso-spacerun: yes;"> </span>0.02%<span style="mso-spacerun: yes;"> </span>charon<span style="mso-spacerun: yes;"> </span>libgmp.so.3.4.1<span style="mso-spacerun: yes;">
</span><span style="mso-spacerun: yes;"> </span>[.] __gmpn_kara_mul_n<o:p></o:p></span></div><div><font face="Times New Roman">
</font></div><div class="MsoNormal" style="margin: 0in 0in 10pt;"><span style='font-family: "Arial","sans-serif";'><span style="mso-spacerun: yes;"> </span>0.01%<span style="mso-spacerun: yes;"> </span>charon<span style="mso-spacerun: yes;"> </span>[kernel.kallsyms]<span style="mso-spacerun: yes;">
</span>[k] finish_task_switch<o:p></o:p></span></div><div><font face="Times New Roman">
</font></div><div class="MsoNormal" style="margin: 0in 0in 10pt;"><span style='font-family: "Arial","sans-serif";'><span style="mso-spacerun: yes;"> </span>0.01%<span style="mso-spacerun: yes;"> </span>charon<span style="mso-spacerun: yes;"> </span>libgmp.so.3.4.1<span style="mso-spacerun: yes;">
</span><span style="mso-spacerun: yes;"> </span>[.]
__gmpn_sqr_n<o:p></o:p></span></div><div><font face="Times New Roman">
</font></div><div class="MsoNormal" style="margin: 0in 0in 10pt;"><span style='font-family: "Arial","sans-serif";'><span style="mso-spacerun: yes;"> </span>0.01%<span style="mso-spacerun: yes;"> </span>charon<span style="mso-spacerun: yes;"> </span>[kernel.kallsyms]<span style="mso-spacerun: yes;">
</span>[k] _raw_spin_unlock_irqrestore<o:p></o:p></span></div><div><font face="Times New Roman">
</font></div><div class="MsoNormal" style="margin: 0in 0in 10pt;"><span style='font-family: "Arial","sans-serif";'><span style="mso-spacerun: yes;"> </span>0.01%<span style="mso-spacerun: yes;"> </span>charon<span style="mso-spacerun: yes;"> </span>libcharon.so.0.0.0<span style="mso-spacerun: yes;"> </span><span style="mso-spacerun: yes;"> </span>[.]
vlog<o:p></o:p></span></div><div><font face="Times New Roman">
</font></div><div class="MsoNormal" style="margin: 0in 0in 10pt;"><span style='font-family: "Arial","sans-serif";'><span style="mso-spacerun: yes;"> </span>0.01%<span style="mso-spacerun: yes;"> </span>charon<span style="mso-spacerun: yes;"> </span>[kernel.kallsyms]<span style="mso-spacerun: yes;">
</span>[k] smp_call_function_many<o:p></o:p></span></div><div><font face="Times New Roman">
</font></div><div class="MsoNormal" style="margin: 0in 0in 10pt;"><span style='font-family: "Arial","sans-serif";'><span style="mso-spacerun: yes;"> </span>0.01%<span style="mso-spacerun: yes;"> </span>charon<span style="mso-spacerun: yes;"> </span>libstrongswan-sha1.so<span style="mso-spacerun: yes;"> </span><span style="mso-spacerun: yes;"> </span>[.]
SHA1Update<o:p></o:p></span></div><div><font face="Times New Roman">
</font></div><div class="MsoNormal" style="margin: 0in 0in 10pt;"><span style='font-family: "Arial","sans-serif";'><span style="mso-spacerun: yes;"> </span>0.00%<span style="mso-spacerun: yes;"> </span>charon<span style="mso-spacerun: yes;"> </span>libc-2.11.1.so<span style="mso-spacerun: yes;">
</span>[.] memcpy<o:p></o:p></span></div><div><font face="Times New Roman">
</font></div><div class="MsoNormal" style="margin: 0in 0in 10pt;"><span style='font-family: "Arial","sans-serif";'><span style="mso-spacerun: yes;"> </span>0.00%<span style="mso-spacerun: yes;"> </span>charon<span style="mso-spacerun: yes;"> </span>libc-2.11.1.so<span style="mso-spacerun: yes;"> </span><span style="mso-spacerun: yes;"> </span>[.]<o:p></o:p></span></div><div><font face="Times New Roman">
</font></div><div class="MsoNormal" style="margin: 0in 0in 10pt;"><span style='font-family: "Arial","sans-serif";'>Do I need to
use the Libgcrypt instead of GMP library?<span style="mso-spacerun: yes;">
</span>If yes, please suggest how to do that. Or will you suggest drilling down
into gmpn_addmul_1 function (GMP software component) to figure out the real
cause?</span></div><div class="MsoNormal" style="margin: 0in 0in 10pt;"><span style='font-family: "Arial","sans-serif";'>Thanks in advance for your help and suggestions.</span></div><div><font face="Times New Roman">
</font></div><div class="MsoNormal" style="margin: 0in 0in 10pt;"><span style='font-family: "Arial","sans-serif";'><o:p> </o:p></span></div><div><font face="Times New Roman">
</font></div><div class="MsoNormal" style="margin: 0in 0in 10pt;"><span style='font-family: "Arial","sans-serif";'>Regards,<o:p></o:p></span></div><div><font face="Times New Roman">
</font></div><div class="MsoNormal" style="margin: 0in 0in 10pt;"><span style='font-family: "Arial","sans-serif";'>Chinmaya<o:p></o:p></span></div><div><font face="Times New Roman">
</font></div></span><div></div><div><br></div> <div style="font-family: lucida console, sans-serif; font-size: 12pt;"> <div style="font-family: times new roman, new york, times, serif; font-size: 12pt;"> <div dir="ltr"> <div class="hr" style="margin: 5px 0px; padding: 0px; border: 1px solid rgb(204, 204, 204); height: 0px; line-height: 0; font-size: 0px;" contenteditable="false" readonly="true"></div> <font face="Arial" size="2"> <b><span style="font-weight: bold;">From:</span></b> Martin Willi <martin@strongswan.org><br> <b><span style="font-weight: bold;">To:</span></b> Chinmaya Dwibedy <ckdwibedy@yahoo.com> <br><b><span style="font-weight: bold;">Cc:</span></b> "users@lists.strongswan.org" <users@lists.strongswan.org> <br> <b><span style="font-weight: bold;">Sent:</span></b> Wednesday, September 25, 2013 1:10 PM<br> <b><span style="font-weight: bold;">Subject:</span></b> Re: [strongSwan] Performance issue with 20k IPsec tunnels
(using 5.0.4 strongswan and load-tester plugin)<br> </font> </div> <div class="y_msg_container"><br><br clear="none">> I find, there are lots of retransmissions (as it prints the status of<br clear="none">> the initiation with *character mostly) in console. I know, these are<br clear="none">> certainly considered to be bad. But I have set the retransmit_timeout<br clear="none">> and retransmit_tries to 300 seconds and 300 times respectively, which<br clear="none">> is a huge.<br clear="none"><br clear="none">The retransmissions usually indicate that one of the peers is<br clear="none">overloaded. Increasing retransmission timeouts can't solve your<br clear="none">performance limitations; this might help to work around the issues you<br clear="none">see in your lab, but certainly does not resemble what you have on a real<br clear="none">setup. Further, the charon.half_open_timeout strongswan.conf setting<br clear="none">defaulting to 30s
will delete the IKE_SA on the responder if it does not<br clear="none">come up within that timeout.<br clear="none"><br clear="none">As said before, I think you should focus on finding the bottleneck of<br clear="none">your setup rather than adjusting your client configuration. Use a<br clear="none">profiling tool.<div class="yqt2777541139" id="yqtfd90171"><br clear="none"><br clear="none">Regards<br clear="none">Martin<br clear="none"><br clear="none"></div><br><br></div> </div> </div> </div></body></html>