[strongSwan] Test for Unrecognized payload types and critical bit is set fails

[kumuda]

Hi,

Have configured IKEv2 as responder and using strongSwan 5.2.0.

Test is to verify that IKEv2 sends a CREATE_CHILD_SA response with a 
Notify payload of type
UNSUPPORTED_CRITICAL_PAYLOAD for the CREATE_CHILD_SA request with invalid
payload type value (1) and critical bit is set.

As per RFC:

If the critical flag is set
    and the payload type is unrecognized, the message MUST be rejected
    and the response to the IKE request containing that payload MUST
    include a Notify payload UNSUPPORTED_CRITICAL_PAYLOAD, indicating an
    unsupported critical payload was included

Charon log shows that decrypting the notify payload fails..

Oct 20 01:47:16 16[ENC] <tahi_ikev2_test|1>   parsing rule 0 U_INT_8
Oct 20 01:47:16 16[ENC] <tahi_ikev2_test|1>    => 41
Oct 20 01:47:16 16[ENC] <tahi_ikev2_test|1>   parsing rule 1 RESERVED_BIT
Oct 20 01:47:16 16[ENC] <tahi_ikev2_test|1>    => 1
Oct 20 01:47:16 16[ENC] <tahi_ikev2_test|1>   parsing rule 2 RESERVED_BIT
Oct 20 01:47:16 16[ENC] <tahi_ikev2_test|1>    => 0
Oct 20 01:47:16 16[ENC] <tahi_ikev2_test|1>   parsing rule 3 RESERVED_BIT
Oct 20 01:47:16 16[ENC] <tahi_ikev2_test|1>    => 0
Oct 20 01:47:16 16[ENC] <tahi_ikev2_test|1>   parsing rule 4 RESERVED_BIT
Oct 20 01:47:16 16[ENC] <tahi_ikev2_test|1>    => 0
Oct 20 01:47:16 16[ENC] <tahi_ikev2_test|1>   parsing rule 5 RESERVED_BIT
Oct 20 01:47:16 16[ENC] <tahi_ikev2_test|1>    => 0
Oct 20 01:47:16 16[ENC] <tahi_ikev2_test|1>   parsing rule 6 RESERVED_BIT
Oct 20 01:47:16 16[ENC] <tahi_ikev2_test|1>    => 0
Oct 20 01:47:16 16[ENC] <tahi_ikev2_test|1>   parsing rule 7 RESERVED_BIT
Oct 20 01:47:16 16[ENC] <tahi_ikev2_test|1>    => 0
Oct 20 01:47:16 16[ENC] <tahi_ikev2_test|1>   parsing rule 8 RESERVED_BIT
Oct 20 01:47:16 16[ENC] <tahi_ikev2_test|1>    => 0
Oct 20 01:47:16 16[ENC] <tahi_ikev2_test|1>   parsing rule 9 PAYLOAD_LENGTH
Oct 20 01:47:16 16[ENC] <tahi_ikev2_test|1>    => 4
Oct 20 01:47:16 16[ENC] <tahi_ikev2_test|1>   parsing rule 10 U_INT_32
Oct 20 01:47:16 16[ENC] <tahi_ikev2_test|1>    => 553648136
Oct 20 01:47:16 16[ENC] <tahi_ikev2_test|1>   parsing rule 11 U_INT_32
Oct 20 01:47:16 16[ENC] <tahi_ikev2_test|1>    => 16391
Oct 20 01:47:16 16[ENC] <tahi_ikev2_test|1>   parsing rule 12 (1258)
Oct 20 01:47:16 16[ENC] <tahi_ikev2_test|1> could not decrypt payloads
Oct 20 01:47:16 16[IKE] <tahi_ikev2_test|1> message parsing failed
Oct 20 01:47:16 16[ENC] <tahi_ikev2_test|1> added payload of type NOTIFY 
to message
Oct 20 01:47:16 16[ENC] <tahi_ikev2_test|1> added payload of type NOTIFY 
to message
Oct 20 01:47:16 16[ENC] <tahi_ikev2_test|1> generating CREATE_CHILD_SA 
response 2 [ N(INVAL_SYN) ]
Oct 20 01:47:16 16[ENC] <tahi_ikev2_test|1> insert payload NOTIFY into 
encrypted payload

Comparing charon log of 4.6.4, I see difference in the nonce payload 
length and parsing of next payload 41.
Oct 20 03:27:20 16[ENC] <tahi_ikev2_test|1>   parsing rule 0 U_INT_8
Oct 20 03:27:20 16[ENC] <tahi_ikev2_test|1>    => 41
Oct 20 03:27:20 16[ENC] <tahi_ikev2_test|1>   parsing rule 1 FLAG
Oct 20 03:27:20 16[ENC] <tahi_ikev2_test|1>    => 1
Oct 20 03:27:20 16[ENC] <tahi_ikev2_test|1>   parsing rule 2 RESERVED_BIT
Oct 20 03:27:20 16[ENC] <tahi_ikev2_test|1>    => 0
Oct 20 03:27:20 16[ENC] <tahi_ikev2_test|1>   parsing rule 3 RESERVED_BIT
Oct 20 03:27:20 16[ENC] <tahi_ikev2_test|1>    => 0
Oct 20 03:27:20 16[ENC] <tahi_ikev2_test|1>   parsing rule 4 RESERVED_BIT
Oct 20 03:27:20 16[ENC] <tahi_ikev2_test|1>    => 0
Oct 20 03:27:20 16[ENC] <tahi_ikev2_test|1>   parsing rule 5 RESERVED_BIT
Oct 20 03:27:20 16[ENC] <tahi_ikev2_test|1>    => 0
Oct 20 03:27:20 16[ENC] <tahi_ikev2_test|1>   parsing rule 6 RESERVED_BIT
Oct 20 03:27:20 16[ENC] <tahi_ikev2_test|1>    => 0
Oct 20 03:27:20 16[ENC] <tahi_ikev2_test|1>   parsing rule 7 RESERVED_BIT
Oct 20 03:27:20 16[ENC] <tahi_ikev2_test|1>    => 0
Oct 20 03:27:20 16[ENC] <tahi_ikev2_test|1>   parsing rule 8 RESERVED_BIT
Oct 20 03:27:20 16[ENC] <tahi_ikev2_test|1>    => 0
Oct 20 03:27:20 16[ENC] <tahi_ikev2_test|1>   parsing rule 9 PAYLOAD_LENGTH
Oct 20 03:27:20 16[ENC] <tahi_ikev2_test|1>    => 4
Oct 20 03:27:20 16[ENC] <tahi_ikev2_test|1>   parsing rule 10 UNKNOWN_DATA
Oct 20 03:27:20 16[ENC] <tahi_ikev2_test|1>    => => 0 bytes @ (nil)
Oct 20 03:27:20 16[ENC] <tahi_ikev2_test|1> parsing (1) payload finished

Does the nonce payload influence parsing this invalid payload?
What could be causing this message parsing fail?

Regards,
Kumuda G