[strongSwan] Kernel panic with VTI tunnel
Mike Noordermeer
mike at normi.net
Mon Mar 16 10:30:31 CET 2015
Hi,
In the mail conversation, Steffen mentions that the patch should never
be necessary, since outer_mode should not become NULL. So I doubt the
patch would be the proper fix? It may fix this issue, but if the
maintainer says the patch should not be necessary it seems like the
wrong fix to me.
Regards,
Mike
On 16 March 2015 at 10:19, André Valentin <avalentin at marcant.net> wrote:
> Hi!
>
> Please try the patch which is attached to the initial email. That shoud fix it. There is another bug with IPv6 which at first I ahrought at, but that's only with NAT. So please ignore that. So diffinig vanilla isn't needed.
>
> Kind regards,
>
> André
>
> Am 16.03.2015 um 09:50 schrieb Mike Noordermeer:
>> Thanks... that looks very much like the same bug indeed. I will diff
>> the various files of the Debian kernel and 3.18 vanilla to see if I
>> can spot the change that introduced it.
>>
>> Regards,
>>
>> Mike
>>
>> On 16 March 2015 at 09:42, André Valentin <avalentin at marcant.net> wrote:
>>> Hi,
>>>
>>> take a look at this thread:
>>> http://marc.info/?t=142495092700001&r=1&w=2
>>> The initial mail is attached. I couldn't verfy the error with vanilla, but your error looks like mine.
>>> Have fun;-)
>>>
>>> André
>>>
>>>
>>> Am 16.03.2015 um 09:18 schrieb Mike Noordermeer:
>>>> Hi,
>>>>
>>>> Do you happen to have any more specific info on this bugfix? I would
>>>> rather not deviate from the Debian default kernels, so it would be
>>>> nice if I could point the maintainers to a specific fix that should be
>>>> backported.
>>>>
>>>> Thanks,
>>>>
>>>> Mike
>>>>
>>>>
>>>> On 15 March 2015 at 17:02, Andre Valentin <avalentin at marcant.net> wrote:
>>>>> Hi!
>>>>>
>>>>> Try kernel 3.18. There's a bugfix for an issue like this.
>>>>>
>>>>> Kind regards,
>>>>>
>>>>> André
>>>>>
>>>>>
>>>>> Am 15.03.2015 um 15:15 schrieb Mike Noordermeer:
>>>>>> Hi,
>>>>>>
>>>>>> I am currently experiencing the same kernel panic on multiple hosts,
>>>>>> with a quite recent Linux kernel, and was wondering if anyone here has
>>>>>> an idea of what the issue could be, or how I could further debug it.
>>>>>> Any help is appreciated.
>>>>>>
>>>>>> I am using Linux 3.16 (3.16.7-ckt4-3~bpo70+1 from Debian
>>>>>> wheezy-backports) and Strongswan 5.2.1 (5.2.1-5~bpo70+1 form Debian
>>>>>> wheezy-backports). I have a fairly 'simple' tunnel with a mark and a
>>>>>> left/right subnet of 0/0, and disabled install_routes in Strongswan.
>>>>>> Then I have a VTI device configured with the same mark. This all works
>>>>>> well, but causes a kernel panic every few hours, always on the same
>>>>>> spot. As far as I can see, no fixes for such an issue have been
>>>>>> committed to the kernel since version 3.16.
>>>>>>
>>>>>> From the backtrace it seems that xfrm_input() in the kernel is hitting
>>>>>> a NULL dereference, when dereferencing 'outer_mode' in the xfrm_state
>>>>>> struct, this line to be precise:
>>>>>> https://github.com/torvalds/linux/blob/2e71029e2c32ecd59a2e8f351517bfbbad42ac11/include/net/xfrm.h#L1807
>>>>>>
>>>>>> Any idea on why this could be NULL? Some config details and the full
>>>>>> backtrace are below.
>>>>>>
>>>>>> Thanks,
>>>>>>
>>>>>> Mike
>>>>>>
>>>>>> ----------------------------------------
>>>>>> Simplified ipsec.conf:
>>>>>> ----------------------------------------
>>>>>>
>>>>>> config setup
>>>>>>
>>>>>> conn %default
>>>>>> keyexchange = ikev2
>>>>>> dpdaction = restart
>>>>>> esp = aes128gcm128-modp4096!
>>>>>> ike = aes128gcm128-prfsha256-modp4096!
>>>>>> mobike = no
>>>>>> auto = route
>>>>>>
>>>>>> conn myconnection
>>>>>> left = x.x.x.x
>>>>>> leftcert = leftcert.crt
>>>>>> leftsubnet = 0.0.0.0/0
>>>>>> right = y.y.y.y
>>>>>> rightcert = rightcert.crt
>>>>>> rightsubnet = 0.0.0.0/0
>>>>>> mark = 15
>>>>>>
>>>>>> ----------------------------------------
>>>>>> ip xfrm policy
>>>>>> ----------------------------------------
>>>>>>
>>>>>> src 0.0.0.0/0 dst 0.0.0.0/0
>>>>>> dir fwd priority 3075 ptype main
>>>>>> mark 15/0xffffffff
>>>>>> tmpl src y.y.y.y dst x.x.x.x
>>>>>> proto esp reqid 1 mode tunnel
>>>>>> src 0.0.0.0/0 dst 0.0.0.0/0
>>>>>> dir in priority 3075 ptype main
>>>>>> mark 15/0xffffffff
>>>>>> tmpl src y.y.y.y dst x.x.x.x
>>>>>> proto esp reqid 1 mode tunnel
>>>>>> src 0.0.0.0/0 dst 0.0.0.0/0
>>>>>> dir out priority 3075 ptype main
>>>>>> mark 15/0xffffffff
>>>>>> tmpl src x.x.x.x dst y.y.y.y
>>>>>> proto esp reqid 1 mode tunnel
>>>>>> src 0.0.0.0/0 dst 0.0.0.0/0
>>>>>> socket in priority 0 ptype main
>>>>>> src 0.0.0.0/0 dst 0.0.0.0/0
>>>>>> socket out priority 0 ptype main
>>>>>> src 0.0.0.0/0 dst 0.0.0.0/0
>>>>>> socket in priority 0 ptype main
>>>>>> src 0.0.0.0/0 dst 0.0.0.0/0
>>>>>> socket out priority 0 ptype main
>>>>>> src ::/0 dst ::/0
>>>>>> socket in priority 0 ptype main
>>>>>> src ::/0 dst ::/0
>>>>>> socket out priority 0 ptype main
>>>>>> src ::/0 dst ::/0
>>>>>> socket in priority 0 ptype main
>>>>>> src ::/0 dst ::/0
>>>>>> socket out priority 0 ptype main
>>>>>>
>>>>>> ----------------------------------------
>>>>>> ip xfrm state
>>>>>> ----------------------------------------
>>>>>>
>>>>>> src x.x.x.x dst y.y.y.y
>>>>>> proto esp spi 0xcb5c6f72 reqid 1 mode tunnel
>>>>>> replay-window 32 flag af-unspec
>>>>>> mark 15/0xffffffff
>>>>>> aead rfc4106(gcm(aes)) 0x3d1c9ae2f921fc088b2e54a1d1efcd3e4441e502 128
>>>>>> src y.y.y.y dst x.x.x.x
>>>>>> proto esp spi 0xcd742975 reqid 1 mode tunnel
>>>>>> replay-window 32 flag af-unspec
>>>>>> mark 15/0xffffffff
>>>>>> aead rfc4106(gcm(aes)) 0x439dd5bf790a1f7ba1979d798757bab94f62776c 128
>>>>>> src x.x.x.x dst y.y.y.y
>>>>>> proto esp spi 0xc79db590 reqid 1 mode tunnel
>>>>>> replay-window 32 flag af-unspec
>>>>>> mark 15/0xffffffff
>>>>>> aead rfc4106(gcm(aes)) 0x7bf0811323a4df1118680d30d4117ed403b60bd8 128
>>>>>> src y.y.y.y dst x.x.x.x
>>>>>> proto esp spi 0xc8e198f5 reqid 1 mode tunnel
>>>>>> replay-window 32 flag af-unspec
>>>>>> mark 15/0xffffffff
>>>>>> aead rfc4106(gcm(aes)) 0x1f1f32fc74a0d8ba38b9aab67fbbfff1024cf265 128
>>>>>>
>>>>>> ----------------------------------------
>>>>>> Kernel oops backtrace
>>>>>> ----------------------------------------
>>>>>>
>>>>>> [31202.487290] BUG: unable to handle kernel NULL pointer dereference
>>>>>> at 0000000000000034
>>>>>> [31202.499656] IP: [<ffffffff814e4a12>] xfrm_input+0x3d2/0x590
>>>>>> [31202.502444] PGD 0
>>>>>> [31202.503479] Oops: 0000 [#1] SMP
>>>>>> [31202.505121] Modules linked in: seqiv xfrm6_mode_tunnel
>>>>>> xfrm4_mode_tunnel xfrm_user xfrm4_tunnel tunnel4 ipcomp xfrm_ipcomp
>>>>>> esp4 ah4 af_key xfrm_algo act_police cls_basic cls_flow cls_fw cls_u32
>>>>>> sch_tbf sch_prio sch_hfsc sch_htb sch_ingress sch_sfq xt_statistic
>>>>>> xt_CT xt_realm xt_LOG iptable_raw xt_connlimit xt_addrtype xt_comment
>>>>>> xt_nat xt_recent ipt_ULOG ipt_REJECT ipt_MASQUERADE ipt_ECN
>>>>>> ipt_CLUSTERIP ipt_ah nf_nat_tftp nf_nat_snmp_basic nf_conntrack_snmp
>>>>>> nf_nat_sip nf_nat_pptp nf_nat_proto_gre nf_nat_irc nf_nat_h323
>>>>>> nf_nat_ftp xt_set ip_set nf_nat_amanda nf_conntrack_tftp
>>>>>> nf_conntrack_sip nf_conntrack_sane nf_conntrack_proto_udplite
>>>>>> nf_conntrack_proto_sctp nf_conntrack_pptp nf_conntrack_proto_gre
>>>>>> nf_conntrack_netlink nf_conntrack_netbios_ns nf_conntrack_broadcast
>>>>>> nf_conntrack_irc ts_kmp nf_conntrack_amanda nf_conntrack_h323
>>>>>> nf_conntrack_ftp xt_time xt_TCPMSS xt_TPROXY xt_tcpmss xt_sctp
>>>>>> xt_policy xt_pkttype xt_physdev xt_owner xt_NFLOG nfnetlink_log
>>>>>> xt_NFQUEUE xt_multiport xt_mark xt_mac xt_limit xt_length xt_iprange
>>>>>> xt_helper xt_hashlimit xt_DSCP xt_dscp xt_dccp xt_connmark xt_CLASSIFY
>>>>>> ip6t_REJECT xt_AUDIT xt_tcpudp iptable_nat nf_nat_ipv4 xt_state nf_nat
>>>>>> nf_conntrack_ipv6 nf_conntrack_ipv4 nf_defrag_ipv6 nf_defrag_ipv4
>>>>>> xt_conntrack nf_conntrack iptable_mangle ip6table_raw ip6table_mangle
>>>>>> nfnetlink iptable_filter ip6table_filter ip6_tables ip_tables x_tables
>>>>>> ip_vti ip_tunnel loop coretemp vmwgfx ttm crct10dif_pclmul
>>>>>> drm_kms_helper crc32_pclmul ghash_clmulni_intel drm aesni_intel
>>>>>> aes_x86_64 lrw gf128mul glue_helper vmw_balloon ablk_helper cryptd
>>>>>> psmouse i2c_piix4 i2c_core serio_raw pcspkr evdev vmw_vmci shpchp
>>>>>> battery parport_pc parport processor thermal_sys ac button ext4 crc16
>>>>>> mbcache jbd2 dm_mod sr_mod cdrom sg sd_mod crc_t10dif crct10dif_common
>>>>>> ata_generic crc32c_intel floppy ata_piix e1000 libata mptspi
>>>>>> scsi_transport_spi mptscsih mptbase scsi_mod
>>>>>> [31202.591173] CPU: 0 PID: 3829 Comm: charon Not tainted
>>>>>> 3.16.0-0.bpo.4-amd64 #1 Debian 3.16.7-ckt4-3~bpo70+1
>>>>>> [31202.595671] Hardware name: VMware, Inc. VMware Virtual
>>>>>> Platform/440BX Desktop Reference Platform, BIOS 6.00 04/14/2014
>>>>>> [31202.600531] task: ffff88002b3112f0 ti: ffff88002bef4000 task.ti:
>>>>>> ffff88002bef4000
>>>>>> [31202.603967] RIP: 0010:[<ffffffff814e4a12>] [<ffffffff814e4a12>]
>>>>>> xfrm_input+0x3d2/0x590
>>>>>> [31202.607734] RSP: 0000:ffff880031003b98 EFLAGS: 00010286
>>>>>> [31202.610241] RAX: 0000000000000000 RBX: ffff880030a33d00 RCX: 0000000000000000
>>>>>> [31202.613640] RDX: 0000000000000001 RSI: 0000000000000200 RDI: ffffffff814e1633
>>>>>> [31202.617023] RBP: 0000000000000002 R08: ffff880030916c00 R09: 0000000000000002
>>>>>> [31202.620272] R10: 0000000000000032 R11: 00000000033993db R12: 0000000000000032
>>>>>> [31202.623532] R13: 0000000000000032 R14: ffff880030916c00 R15: 0000000000000000
>>>>>> [31202.626860] FS: 00007f669aafa700(0000) GS:ffff880031000000(0000)
>>>>>> knlGS:0000000000000000
>>>>>> [31202.630585] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
>>>>>> [31202.633234] CR2: 0000000000000034 CR3: 000000002146e000 CR4: 00000000000407f0
>>>>>> [31202.636588] Stack:
>>>>>> [31202.637589] ffffffff81486050 00000000a1339d6c ffffffff818b7bc0
>>>>>> 0000000030a33d00
>>>>>> [31202.641338] ffff88002925769e 5059f5ca00000002 0000000000000032
>>>>>> 01000000260ae8c0
>>>>>> [31202.645024] ffff88002a902000 ffff880030a33d00 ffffffffa02df040
>>>>>> ffffffff818b7bc0
>>>>>> [31202.648700] Call Trace:
>>>>>> [31202.649879] <IRQ>
>>>>>> [31202.650797] [<ffffffff81486050>] ? ip_rcv_finish+0x370/0x370
>>>>>> [31202.653769] [<ffffffff814d87b7>] ? xfrm4_esp_rcv+0x37/0x70
>>>>>> [31202.656423] [<ffffffff814860ee>] ? ip_local_deliver_finish+0x9e/0x200
>>>>>> [31202.659449] [<ffffffff8144b15b>] ? __netif_receive_skb_core+0x57b/0x700
>>>>>> [31202.662551] [<ffffffff8101e0c5>] ? read_tsc+0x5/0x20
>>>>>> [31202.664889] [<ffffffff8144ba6f>] ? netif_receive_skb_internal+0x1f/0x90
>>>>>> [31202.668100] [<ffffffff8144c3d8>] ? napi_gro_receive+0x128/0x1b0
>>>>>> [31202.670892] [<ffffffffa00af36b>] ? e1000_clean_rx_irq+0x2db/0x560 [e1000]
>>>>>> [31202.674112] [<ffffffffa00b0313>] ? e1000_clean+0x273/0x980 [e1000]
>>>>>> [31202.677012] [<ffffffffa00b0406>] ? e1000_clean+0x366/0x980 [e1000]
>>>>>> [31202.679902] [<ffffffff8104dab1>] ? ack_apic_level+0x81/0x170
>>>>>> [31202.682591] [<ffffffff8144cb21>] ? net_rx_action+0x121/0x230
>>>>>> [31202.685246] [<ffffffff81072c0e>] ? __do_softirq+0xde/0x2e0
>>>>>> [31202.687941] [<ffffffff8104dab1>] ? ack_apic_level+0x81/0x170
>>>>>> [31202.690708] [<ffffffff81073066>] ? irq_exit+0x86/0xb0
>>>>>> [31202.693130] [<ffffffff8154c856>] ? do_IRQ+0x66/0x110
>>>>>> [31202.695531] [<ffffffff8154a6ed>] ? common_interrupt+0x6d/0x6d
>>>>>> [31202.698241] <EOI>
>>>>>> [31202.699165] Code: ff ff 85 c0 0f 85 c1 fd ff ff e9 05 fd ff ff 66
>>>>>> 2e 0f 1f 84 00 00 00 00 00 48 83 7b 40 00 0f 84 5b fd ff ff 49 8b 86
>>>>>> e0 02 00 00 <f6> 40 34 01 0f 84 85 fd ff ff e9 45 fd ff ff 0f 1f 80 00
>>>>>> 00 00
>>>>>> [31202.712413] RIP [<ffffffff814e4a12>] xfrm_input+0x3d2/0x590
>>>>>> [31202.715102] RSP <ffff880031003b98>
>>>>>> [31202.716751] CR2: 0000000000000034
>>>>>> [31202.719064] ---[ end trace cebe794b0c57af5e ]---
>>>>>> [31202.721593] Kernel panic - not syncing: Fatal exception in interrupt
>>>>>> [31202.724814] Kernel Offset: 0x0 from 0xffffffff81000000 (relocation
>>>>>> range: 0xffffffff80000000-0xffffffff9fffffff)
>>>>>> _______________________________________________
>>>>>> Users mailing list
>>>>>> Users at lists.strongswan.org
>>>>>> https://lists.strongswan.org/mailman/listinfo/users
>>>>> --
>>>>> Systemadministration / Projektkoordination
>>>>>
>>>>> --
>>>>> ++++++WIR ZIEHEN UM++++++
>>>>> Ab dem 09.03.2015 erreichen Sie uns unter folgender Anschrift.
>>>>>
>>>>> MarcanT GmbH
>>>>> Herforder Straße 163 a
>>>>> 33609 Bielefeld
>>>>>
>>>>> Bitte beachten Sie, dass ab dem 09.03.2015 alle Rechnungen und
>>>>> Korrespondenz nur noch auf die oben genannte Anschrift ausgestellt
>>>>> werden. Aktualisieren Sie bitte Ihre Stammdaten entsprechend. Wir
>>>>> wünschen uns, den Umzug für Sie und uns ohne Beeinträchtigungen des
>>>>> Tagesgeschäftes abwickeln zu können. Sollte es dennoch zu Schwierigkeiten
>>>>> kommen, hoffen wir auf Ihr Verständnis.
>>>>> Auf unser Rechenzentrum hat der Umzug keinen Einfluss; die Funktionen
>>>>> wurden bereits Anfang Februar auf unsere Redundanzrechenzentren verteilt.
>>>>> ++++++++++++++++++++++++++++++++++++
>>>>>
>>>>> MarcanT GmbH, Ravensberger Str. 10 G, D - 33602 Bielefeld
>>>>> Fon: +49 (521) 95945-0 | Fax: +49 (521) 95945-18
>>>>> URL: http://www.marcant.net | http://www.global-m2m.com
>>>>>
>>>>> Internet * Netzwerk * Mobile Daten
>>>>> Citrix Silver Solution Advisor
>>>>>
>>>>> Geschäftsführer: Thorsten Hojas
>>>>> Handelsregister: AG Bielefeld, HRB 35827 USt-ID Nr.: DE 190203238
>>>>> ___________________________________________________________
>>>>> Ausserhalb unserer Geschäftszeiten (Montag bis Freitag von 8:30 Uhr bis
>>>>> 17:30 Uhr, ausgenommen gesetzliche Feiertage in NRW) stehen wir Ihnen
>>>>> gemäß Ihrer jeweiligen Service-Level-Agreements unter der Ihnen
>>>>> mitgeteilten Telefonnummer für Störungen und Notfälle zur Verfügung.
>>>>> Sie können natürlich auch gerne jederzeit unter support at marcant.net ein
>>>>> Ticket eröffnen, welches am nächsten Arbeitstag bearbeitet wird.
>>>>>
>>>>>
>>>>>
>>>>> Mit freundlichen Grüßen
>>>>> André Valentin
>>>>> Systemadministrator
>>>>> --
>>>>>
>>>>> ++++++WIR SIND UMGEZOGEN++++++
>>>>> Seit dem 09.03.2015 erreichen Sie uns unter folgender Anschrift.
>>>>>
>>>>> MarcanT GmbH
>>>>> Herforder Straße 163a
>>>>> 33609 Bielefeld
>>>>>
>>>>> Bitte beachten Sie, dass ab dem 09.03.2015 alle Rechnungen und
>>>>> Korrespondenz nur noch auf die oben genannte Anschrift ausgestellt
>>>>> werden. Aktualisieren Sie bitte Ihre Stammdaten entsprechend.
>>>>> ++++++++++++++++++++++++++++++++++++
>>>>>
>>>>> MarcanT GmbH, Herforder Straße 163a, D - 33609 Bielefeld
>>>>> Fon: +49 (521) 95945-0 | Fax: +49 (521) 95945-18
>>>>> URL: http://www.marcant.net | http://www.global-m2m.com
>>>>>
>>>>> Internet * Netzwerk * Mobile Daten
>>>>> Citrix Silver Solution Advisor
>>>>>
>>>>> Geschäftsführer: Thorsten Hojas
>>>>> Handelsregister: AG Bielefeld, HRB 35827 USt-ID Nr.: DE 190203238
>>>>> ___________________________________________________________
>>>>> Ausserhalb unserer Geschäftszeiten (Montag bis Freitag von 8:30 Uhr bis
>>>>> 17:30 Uhr, ausgenommen gesetzliche Feiertage in NRW) stehen wir Ihnen
>>>>> gemäß Ihrer jeweiligen Service-Level-Agreements unter der Ihnen
>>>>> mitgeteilten Telefonnummer für Störungen und Notfälle zur Verfügung.
>>>>> Sie können natürlich auch gerne jederzeit unter support at marcant.net ein
>>>>> Ticket eröffnen, welches am nächsten Arbeitstag bearbeitet wird.
>>>>> _______________________________________________
>>>>> Users mailing list
>>>>> Users at lists.strongswan.org
>>>>> https://lists.strongswan.org/mailman/listinfo/users
>>> Mit freundlichen Grüßen
>>> André Valentin
>>>
>>> Systemadministration / Projektkoordination
>>> --
>>> MarcanT GmbH, Ravensberger Str. 10 G, D - 33602 Bielefeld
>>> Fon: +49 (521) 95945-0 | Fax: +49 (521) 95945-18
>>> URL: http://www.marcant.net | http://www.global-m2m.com
>>>
>>> Internet * Netzwerk * Mobile Daten
>>> Citrix Silver Solution Advisor
>>>
>>> Geschäftsführer: Thorsten Hojas
>>> Handelsregister: AG Bielefeld, HRB 35827 USt-ID Nr.: DE 190203238
>>> _____________________________________________________________________
>>> Ausserhalb unserer Geschäftszeiten (Montag bis Freitag von 8:30 Uhr
>>> bis 17:30 Uhr, ausgenommen gesetzliche Feiertage in NRW) stehen wir
>>> Ihnen gemäß Ihrer jeweiligen Service-Level-Agreements unter der Ihnen
>>> mitgeteilten Telefonnummer für Störungen und Notfälle zur Verfügung.
>>> Sie können natürlich auch gerne jederzeit unter support at marcant.net
>>> ein Ticket eröffnen, welches am nächsten Arbeitstag bearbeitet wird.
>>>
>
> Mit freundlichen Grüßen
> André Valentin
>
> Systemadministration / Projektkoordination
> --
> MarcanT GmbH, Ravensberger Str. 10 G, D - 33602 Bielefeld
> Fon: +49 (521) 95945-0 | Fax: +49 (521) 95945-18
> URL: http://www.marcant.net | http://www.global-m2m.com
>
> Internet * Netzwerk * Mobile Daten
> Citrix Silver Solution Advisor
>
> Geschäftsführer: Thorsten Hojas
> Handelsregister: AG Bielefeld, HRB 35827 USt-ID Nr.: DE 190203238
> _____________________________________________________________________
> Ausserhalb unserer Geschäftszeiten (Montag bis Freitag von 8:30 Uhr
> bis 17:30 Uhr, ausgenommen gesetzliche Feiertage in NRW) stehen wir
> Ihnen gemäß Ihrer jeweiligen Service-Level-Agreements unter der Ihnen
> mitgeteilten Telefonnummer für Störungen und Notfälle zur Verfügung.
> Sie können natürlich auch gerne jederzeit unter support at marcant.net
> ein Ticket eröffnen, welches am nächsten Arbeitstag bearbeitet wird.
>
More information about the Users
mailing list