[strongSwan] Charon Core files observed with aes-ni driver.

[Thomas Egerer]

Hi Bhargav,

On 10/16/2014 08:11 PM, bhargav p wrote:
> Hi,
> 
> Can someone please reply to the query?
> 
> On Mon, Oct 13, 2014 at 12:13 PM, bhargav p <bhargav.1226 at gmail.com <mailto:bhargav.1226 at gmail.com>> wrote:
> 
> 
>     Hi,
> 
>     I am observering charon generated core files when I configured aes as an encryption alogorithm.
> 
>     aes-ni is enabled on my device.
I seriously doubt that this is a aes-ni-related problem, try running
charon with the environment-variable OPENSSL_ia32cap set, such that
bit #57 is cleared [1] (should be the value ~0x200000000000000).
> 
>     strongswan version I am using is 4.3.6 and openssl version openssl-1.0.0.
> 
>     I ran gdb on the core file, it is pointing to the below point
> 
> 
>     #4  0x00007fabd3eee1b7 in EVP_CIPHER_CTX_set_key_length (c=0x7fabcc99e8b0, keylen=16) at evp_enc.c:520
>     520evp_enc.c: No such file or directory.
>     in evp_enc.c
>     (gdb) p *c
>     $1 = {cipher = 0x0, engine = 0x0, encrypt = 1, buf_len = 0, oiv = '\000' <repeats 15 times>, iv = '\000' <repeats 15 times>, buf = '\000' <repeats 31 times>, num = 0,
>     app_data = 0x0, 
>       key_len = 0, flags = 256, cipher_data = 0x0, final_used = 0, block_mask = 0, final = '\000' <repeats 31 times>}
A little more context (as in backtrace: '(gdb) help bt') would
certainly help! But I'm guessing it leads somewhere to [2], which
as you can see, calls EVP_CIPHER_CTX_set_key_length, regardless
of EVP_CipherInit_ex's return code. Martin fixed this in [3] which
is first available with strongswan 5.0.2, so maybe you check this
version out.
However: my guess is, that openssl has problems dealing with the
cipher, use gdb to print *this->cipher from openssl_crypt.c to shed
some more light on this.
Btw: which openssl-1.0.0 version are you using: [d-n] are available)?

>     cipher is becoming NULL.
Judging from the code, this is impossible, if EVP_CipherInit_ex has
been called :/


Cheers, let's have a drink now!
Thomas

[1] https://www.openssl.org/docs/crypto/OPENSSL_ia32cap.html
[2]
https://git.strongswan.org/?p=strongswan.git;a=blob;f=src/libstrongswan/plugins/openssl/openssl_crypter.c;h=a8923ab56b3f4a3801b905c0a55c7e32d2f509cf;hb=7daf5226b74e14a6e0f1a888b0be26f3d246f9f8#l137
[1] https://git.strongswan.org/?p=strongswan.git;a=commit;h=e35abbe5