Kernel-panic with 3.13.0-64.104 generic kernel (BUG at net/core/skbuff:1290)

Bug #1497184 reported by Ioann
24
This bug affects 4 people
Affects Status Importance Assigned to Milestone
linux (Ubuntu)
Invalid
High
Unassigned
Trusty
Fix Released
High
Unassigned
Vivid
Fix Released
High
Unassigned

Bug Description

Hi.
I get randomly kernel panic with 3.13-64 kernel after some hours of work.
This happens on 2 different computers with x86 and x86_kernel.
First(x86_64): Toshiba Tecra A10-181 Intel Core i3-370M 8Gb DDR3 ram
Second(x86): Desktop with AMD Athlon 64 x2 4800+ 4Gb DDR1 ram

In attached photo you can see panic on laptop.
Every time i other kernel panic message.

With kernel 3.13-63-generic kernel i have uptime more than week.

Techa host: uname -a
Linux nb-ubuntu-ioann 3.13.0-64-generic #104-Ubuntu SMP Wed Sep 9 12:36:12 UTC 2015 x86_64 x86_64 x86_64 GNU/Linux

Desktop host:
Linux ioann-old-desktop 3.13.0-64-generic #104-Ubuntu SMP Wed Sep 9 12:36:12 UTC 2015 x86 x86 x86 GNU/Linux

Description: Ubuntu 14.04.3 LTS
Release: 14.04

ProblemType: Bug
DistroRelease: Ubuntu 14.04
Package: linux-image-3.13.0-64-generic 3.13.0-64.104
ProcVersionSignature: Ubuntu 3.13.0-64.104-generic 3.13.11-ckt26
Uname: Linux 3.13.0-64-generic x86_64
ApportVersion: 2.14.1-0ubuntu3.13
Architecture: amd64
AudioDevicesInUse:
 USER PID ACCESS COMMAND
 /dev/snd/controlC0: unim95 2004 F.... pulseaudio
CRDA: Error: [Errno 2] No such file or directory: 'iw'
CurrentDesktop: XFCE
CurrentDmesg:
 [ 26.552940] IPv6: ADDRCONF(NETDEV_CHANGE): eth0: link becomes ready
 [ 52.301084] audit_printk_skb: 108 callbacks suppressed
 [ 52.301088] type=1400 audit(1442566720.780:48): apparmor="STATUS" operation="profile_replace" profile="unconfined" name="/usr/lib/cups/backend/cups-pdf" pid=2283 comm="apparmor_parser"
 [ 52.301096] type=1400 audit(1442566720.780:49): apparmor="STATUS" operation="profile_replace" profile="unconfined" name="/usr/sbin/cupsd" pid=2283 comm="apparmor_parser"
 [ 52.301679] type=1400 audit(1442566720.780:50): apparmor="STATUS" operation="profile_replace" profile="unconfined" name="/usr/sbin/cupsd" pid=2283 comm="apparmor_parser"
Date: Fri Sep 18 11:02:21 2015
HibernationDevice: RESUME=UUID=7bb2db85-2152-4e84-ba1a-fb4a3c7c82ca
MachineType: TOSHIBA TECRA A11
ProcFB: 0 inteldrmfb
ProcKernelCmdLine: BOOT_IMAGE=/boot/vmlinuz-3.13.0-64-generic root=UUID=314c867a-d6f1-4af2-b92b-510042a6940b ro quiet splash vt.handoff=7
RelatedPackageVersions:
 linux-restricted-modules-3.13.0-64-generic N/A
 linux-backports-modules-3.13.0-64-generic N/A
 linux-firmware 1.127.15
SourcePackage: linux
UpgradeStatus: No upgrade log present (probably fresh install)
dmi.bios.date: 06/27/2011
dmi.bios.vendor: TOSHIBA
dmi.bios.version: Version 3.20
dmi.board.asset.tag: 0000000000
dmi.board.name: Portable PC
dmi.board.vendor: TOSHIBA
dmi.board.version: Version A0
dmi.chassis.asset.tag: 0000000000
dmi.chassis.type: 10
dmi.chassis.vendor: TOSHIBA
dmi.chassis.version: Version 1.0
dmi.modalias: dmi:bvnTOSHIBA:bvrVersion3.20:bd06/27/2011:svnTOSHIBA:pnTECRAA11:pvrPTSE0E-0LP046GR:rvnTOSHIBA:rnPortablePC:rvrVersionA0:cvnTOSHIBA:ct10:cvrVersion1.0:
dmi.product.name: TECRA A11
dmi.product.version: PTSE0E-0LP046GR
dmi.sys.vendor: TOSHIBA

Revision history for this message
Ioann (yanunim95) wrote :
Revision history for this message
Ioann (yanunim95) wrote :

Here is second part of screen photo of panic.

Revision history for this message
Brad Figg (brad-figg) wrote : Status changed to Confirmed

This change was made by a bot.

Changed in linux (Ubuntu):
status: New → Confirmed
Andy Whitcroft (apw)
summary: - Kernel-panic with 3.13-64 generic kernel
+ Kernel-panic with 3.13-64 generic kernel (BUG at net/core/skbuff:1290
summary: - Kernel-panic with 3.13-64 generic kernel (BUG at net/core/skbuff:1290
+ Kernel-panic with 3.13-64 generic kernel (BUG at net/core/skbuff:1290)
summary: - Kernel-panic with 3.13-64 generic kernel (BUG at net/core/skbuff:1290)
+ Kernel-panic with 3.130-64.104 generic kernel (BUG at
+ net/core/skbuff:1290)
Andy Whitcroft (apw)
tags: added: regression-proposed
Revision history for this message
Luis Henriques (henrix) wrote :

I suspect the issue is related with commit

commit 738ac1ebb96d02e0d23bc320302a6ea94c612dec
Author: Herbert Xu <email address hidden>
Date: Mon Jul 13 16:04:13 2015 +0800

    net: Clone skb before setting peeked flag

This commit was included in Trusty 3.13.0-64.104, and introduces a use-after-free bug, fixed with upstream commit

commit a0a2a6602496a45ae838a96db8b8173794b5d398
Author: Herbert Xu <email address hidden>
Date: Tue Aug 4 15:42:47 2015 +0800

    net: Fix skb_set_peeked use-after-free bug

which is not in Trusty yet.

I've built a test kernel that includes this commit here: http://people.canonical.com/~henrix/lp1497184/amd64/
Could you please try to reproduce the issue using this kernel? Thanks

summary: - Kernel-panic with 3.130-64.104 generic kernel (BUG at
+ Kernel-panic with 3.13.0-64.104 generic kernel (BUG at
net/core/skbuff:1290)
Andy Whitcroft (apw)
Changed in linux (Ubuntu Trusty):
status: New → Confirmed
Changed in linux (Ubuntu Vivid):
status: New → Confirmed
Changed in linux (Ubuntu):
importance: Undecided → High
Changed in linux (Ubuntu Trusty):
importance: Undecided → High
Changed in linux (Ubuntu Vivid):
importance: Undecided → High
penalvch (penalvch)
tags: added: bios-outdated-3.50
Revision history for this message
Ioann (yanunim95) wrote :

Hi. Got panic again. New panic messages are in attach.
It was direct while router updated my IP address.

Revision history for this message
Arjen Lentz (arjen-lentz) wrote :

Hi henrix - I'm trying your patched kernel, will update soon with results.
I too get the kernel panics in 3.13.0-64, but not in -63.
In -64 they are very frequent, so it shouldn't take long to see one again it re-occurs anyway with your patch in place.

Revision history for this message
Ioann (yanunim95) wrote :

henrix.

Now I'm using you kerenel. It made 24 hours uptime. So your patch is working.

Revision history for this message
Arjen Lentz (arjen-lentz) wrote :

I concur with loann, although mine has only been up for 2h20. With the unpatched kernel it would've long panicked.

Revision history for this message
Andy Whitcroft (apw) wrote :

@Arjen -- this package has not been released into the LTS it is only in -proposed for testing. This is exactly why we build and publish these kernels into -proposed and why they sit there for some time before being released into -updates. In this case the kernel carries a regression, one which has been detected before release to the general population. It would not be recommended to run any kernel from -proposed on a production system, though testing on less critical systems is always appreciated.

Revision history for this message
Arjen Lentz (arjen-lentz) wrote :

@apw / @henrix - ah, that's good. No worries.
That explains why only my laptop has had the issue, as indeed I do pull in updates from proposed there and that's fine - happy to help catch stuff early. So, all is as it should be! awesome. thanks!

Luis Henriques (henrix)
Changed in linux (Ubuntu Vivid):
status: Confirmed → Fix Committed
Changed in linux (Ubuntu Trusty):
status: Confirmed → Fix Committed
Changed in linux (Ubuntu):
status: Confirmed → Invalid
Revision history for this message
aleandro (aleandrodasilva) wrote :

Hello. I updated with the proposed updates my ununtu 14.04 and installed the 3.13.0-64.104 kernel on the 24 of september. I got a first kernel panic after many hours of work and after reboot even after about an hour. I removed this kernel and installed the 3.13.0-65.-generic which was in the repositories of ubuntu. I hope this new one is not affected by that bug.

Revision history for this message
Launchpad Janitor (janitor) wrote :
Download full text (5.4 KiB)

This bug was fixed in the package linux - 3.13.0-65.105

---------------
linux (3.13.0-65.105) trusty; urgency=low

  [ Brad Figg ]

  * Release Tracking Bug
    - LP: #1498108

  [ Upstream Kernel Changes ]

  * net: Fix skb_set_peeked use-after-free bug
      - LP: #1497184

linux (3.13.0-64.104) trusty; urgency=low

  [ Luis Henriques ]

  * Release Tracking Bug
    - LP: #1493803

  [ Chris J Arges ]

  * [Config] DEFAULT_IOSCHED="deadline" for ppc64el
    - LP: #1469829

  [ Upstream Kernel Changes ]

  * tcp: fix recv with flags MSG_WAITALL | MSG_PEEK
    - LP: #1486146
  * libceph: abstract out ceph_osd_request enqueue logic
    - LP: #1488035
  * libceph: resend lingering requests with a new tid
    - LP: #1488035
  * n_tty: Refactor input_available_p() by call site
    - LP: #1397976
  * tty: Fix pty master poll() after slave closes v2
    - LP: #1397976
  * md: use kzalloc() when bitmap is disabled
    - LP: #1493305
  * ata: pmp: add quirk for Marvell 4140 SATA PMP
    - LP: #1493305
  * libata: add ATA_HORKAGE_BROKEN_FPDMA_AA quirk for HP 250GB SATA disk
    VB0250EAVER
    - LP: #1493305
  * libata: add ATA_HORKAGE_NOTRIM
    - LP: #1493305
  * libata: force disable trim for SuperSSpeed S238
    - LP: #1493305
  * libata: increase the timeout when setting transfer mode
    - LP: #1493305
  * libata: Do not blacklist M510DC
    - LP: #1493305
  * mac80211: clear subdir_stations when removing debugfs
    - LP: #1493305
  * ALSA: hda - Add new GPU codec ID 0x10de007d to snd-hda
    - LP: #1493305
  * drm: Stop resetting connector state to unknown
    - LP: #1493305
  * usb: dwc3: Reset the transfer resource index on SET_INTERFACE
    - LP: #1493305
  * usb: xhci: Bugfix for NULL pointer deference in xhci_endpoint_init()
    function
    - LP: #1493305
  * xhci: Calculate old endpoints correctly on device reset
    - LP: #1493305
  * xhci: report U3 when link is in resume state
    - LP: #1493305
  * xhci: prevent bus_suspend if SS port resuming in phase 1
    - LP: #1493305
  * xhci: do not report PLC when link is in internal resume state
    - LP: #1493305
  * USB: OHCI: Fix race between ED unlink and URB submission
    - LP: #1493305
  * usb-storage: ignore ZTE MF 823 card reader in mode 0x1225
    - LP: #1493305
  * blkcg: fix gendisk reference leak in blkg_conf_prep()
    - LP: #1493305
  * tile: use free_bootmem_late() for initrd
    - LP: #1493305
  * Input: usbtouchscreen - avoid unresponsive TSC-30 touch screen
    - LP: #1493305
  * md/raid1: fix test for 'was read error from last working device'.
    - LP: #1493305
  * mmc: omap_hsmmc: Fix DTO and DCRC handling
    - LP: #1493305
  * isdn/gigaset: reset tty->receive_room when attaching ser_gigaset
    - LP: #1493305
  * mmc: sdhci-pxav3: fix platform_data is not initialized
    - LP: #1493305
  * mmc: block: Add missing mmc_blk_put() in power_ro_lock_show()
    - LP: #1493305
  * mmc: sdhci-esdhc: Make 8BIT bus work
    - LP: #1493305
  * bonding: correctly handle bonding type change on enslave failure
    - LP: #1493305
  * net: Clone skb before setting peeked flag
    - LP: #1493305
  * bridge: mdb: fix double add notification
    - LP: #1493305
  * usb: gadget: mv_udc_c...

Read more...

Changed in linux (Ubuntu Trusty):
status: Fix Committed → Fix Released
Revision history for this message
Launchpad Janitor (janitor) wrote :
Download full text (13.2 KiB)

This bug was fixed in the package linux - 3.19.0-30.33

---------------
linux (3.19.0-30.33) vivid; urgency=low

  [ Brad Figg ]

  * Release Tracking Bug
    - LP: #1498065
  * Revert "[Config]
    MFD_INTEL_LPSS/MFD_INTEL_LPSS_ACPI/MFD_INTEL_LPSS_PCI=m"
    - LP: #1498137
  * [Config] Disable the MFD_INTEL_LPSS* driver

linux (3.19.0-30.32) vivid; urgency=low

  [ Brad Figg ]

  * Release Tracking Bug
    - LP: #1498065

  [ Upstream Kernel Changes ]

  * net: Fix skb_set_peeked use-after-free bug
    - LP: #1497184

linux (3.19.0-29.31) vivid; urgency=low

  [ Luis Henriques ]

  * Release Tracking Bug
    - LP: #1493902

  [ Ander Conselvan de Oliveira ]

  * SAUCE: i915_bpo: Set ddi_pll_sel in DP MST path
    - LP: #1483320

  [ Chris J Arges ]

  * [Config] DEFAULT_IOSCHED="deadline" for ppc64el
    - LP: #1469829

  [ Chris Wilson ]

  * SAUCE: i915_bpo: drm/i915: Flag the execlists context object as dirty
    after every use
    - LP: #1489501

  [ Daniel Vetter ]

  * SAUCE: i915_bpo: drm/i915: Only dither on 6bpc panels
    - LP: #1489501

  [ David Henningsson ]

  * SAUCE: drm/i915: Add audio pin sense / ELD callback
    - LP: #1490895
  * SAUCE: drm/i915: Call audio pin/ELD notify function
    - LP: #1490895
  * SAUCE: ubuntu/i915: Call audio pin/ELD notify function
    - LP: #1490895
  * SAUCE: ALSA: hda - Add "hdac_acomp" global variable
    - LP: #1490895
  * SAUCE: ALSA: hda - allow codecs to access the i915 pin/ELD callback
    - LP: #1490895
  * SAUCE: ALSA: hda - Wake the codec up on pin/ELD notify events
    - LP: #1490895

  [ Jani Nikula ]

  * SAUCE: i915_bpo: Revert "drm/i915: Allow parsing of variable size child
    device entries from VBT"
    - LP: #1489501

  [ Maarten Lankhorst ]

  * SAUCE: i915_bpo: drm/i915: calculate primary visibility changes instead
    of calling from set_config
    - LP: #1489501
  * SAUCE: i915_bpo: drm/i915: Commit planes on each crtc separately.
    - LP: #1489501

  [ Thulasimani,Sivakumar ]

  * SAUCE: i915_bpo: Revert "drm/i915: Add eDP intermediate frequencies for
    CHV"
    - LP: #1489501
  * SAUCE: i915_bpo: drm/i915: remove HBR2 from chv supported list
    - LP: #1489501
  * SAUCE: i915_bpo: drm/i915: Avoid TP3 on CHV
    - LP: #1489501

  [ Timo Aaltonen ]

  * Revert "SAUCE: i915_bpo: drm/i915: Allow parsing of variable size child
    device entries from VBT, addendum v2"
    - LP: #1489501
  * SAUCE: Migrate Broadwell to i915_bpo.
    - LP: #1483320

  [ Upstream Kernel Changes ]

  * tcp: fix recv with flags MSG_WAITALL | MSG_PEEK
    - LP: #1486146
  * powerpc/powernv: Fix the overflow of OPAL message notifiers head array
    - LP: #1487085
  * xhci: call BIOS workaround to enable runtime suspend on Intel Braswell
    - LP: #1489292
  * PM / QoS: Make it possible to expose device latency tolerance to
    userspace
    - LP: #1488395
  * ACPI / PM: Attach ACPI power domain only once
    - LP: #1488395
  * Driver core: wakeup the parent device before trying probe
    - LP: #1488395
  * klist: implement klist_prev()
    - LP: #1488395
  * driver core: implement device_for_each_child_reverse()
    - LP: #1488395
  * mfd: make mfd_remove_devices() iterate in reverse order
    ...

Changed in linux (Ubuntu Vivid):
status: Fix Committed → Fix Released
Revision history for this message
Matt Riedemann (mriedem) wrote :
Download full text (6.5 KiB)

Ever since we started picking up the 3.13.0-65 kernel in the openstack CI system on 9/29 we've been seeing lots of random failures across 3 branches (juno/kilo/master) with ebtables, details are in:

https://bugs.launchpad.net/nova/+bug/1501558

An example failure:

http://logs.openstack.org/62/229962/1/check/gate-tempest-dsvm-full/28bb6b6/logs/screen-n-cpu.txt.gz?level=TRACE#_2015-10-01_17_39_59_1872015-10-01 17:39:59.187 6060 ERROR nova.compute.manager [instance: 79207baa-d735-401c-9f81-4f7ea75aaa4f] Traceback (most recent call last):
2015-10-01 17:39:59.187 6060 ERROR nova.compute.manager [instance: 79207baa-d735-401c-9f81-4f7ea75aaa4f] File "/opt/stack/new/nova/nova/compute/manager.py", line 2155, in _build_resources
2015-10-01 17:39:59.187 6060 ERROR nova.compute.manager [instance: 79207baa-d735-401c-9f81-4f7ea75aaa4f] yield resources
2015-10-01 17:39:59.187 6060 ERROR nova.compute.manager [instance: 79207baa-d735-401c-9f81-4f7ea75aaa4f] File "/opt/stack/new/nova/nova/compute/manager.py", line 2009, in _build_and_run_instance
2015-10-01 17:39:59.187 6060 ERROR nova.compute.manager [instance: 79207baa-d735-401c-9f81-4f7ea75aaa4f] block_device_info=block_device_info)
2015-10-01 17:39:59.187 6060 ERROR nova.compute.manager [instance: 79207baa-d735-401c-9f81-4f7ea75aaa4f] File "/opt/stack/new/nova/nova/virt/libvirt/driver.py", line 2444, in spawn
2015-10-01 17:39:59.187 6060 ERROR nova.compute.manager [instance: 79207baa-d735-401c-9f81-4f7ea75aaa4f] block_device_info=block_device_info)
2015-10-01 17:39:59.187 6060 ERROR nova.compute.manager [instance: 79207baa-d735-401c-9f81-4f7ea75aaa4f] File "/opt/stack/new/nova/nova/virt/libvirt/driver.py", line 4516, in _create_domain_and_network
2015-10-01 17:39:59.187 6060 ERROR nova.compute.manager [instance: 79207baa-d735-401c-9f81-4f7ea75aaa4f] xml, pause=pause, power_on=power_on)
2015-10-01 17:39:59.187 6060 ERROR nova.compute.manager [instance: 79207baa-d735-401c-9f81-4f7ea75aaa4f] File "/opt/stack/new/nova/nova/virt/libvirt/driver.py", line 4446, in _create_domain
2015-10-01 17:39:59.187 6060 ERROR nova.compute.manager [instance: 79207baa-d735-401c-9f81-4f7ea75aaa4f] guest.launch(pause=pause)
2015-10-01 17:39:59.187 6060 ERROR nova.compute.manager [instance: 79207baa-d735-401c-9f81-4f7ea75aaa4f] File "/opt/stack/new/nova/nova/virt/libvirt/guest.py", line 141, in launch
2015-10-01 17:39:59.187 6060 ERROR nova.compute.manager [instance: 79207baa-d735-401c-9f81-4f7ea75aaa4f] self._encoded_xml, errors='ignore')
2015-10-01 17:39:59.187 6060 ERROR nova.compute.manager [instance: 79207baa-d735-401c-9f81-4f7ea75aaa4f] File "/usr/local/lib/python2.7/dist-packages/oslo_utils/excutils.py", line 195, in __exit__
2015-10-01 17:39:59.187 6060 ERROR nova.compute.manager [instance: 79207baa-d735-401c-9f81-4f7ea75aaa4f] six.reraise(self.type_, self.value, self.tb)
2015-10-01 17:39:59.187 6060 ERROR nova.compute.manager [instance: 79207baa-d735-401c-9f81-4f7ea75aaa4f] File "/opt/stack/new/nova/nova/virt/libvirt/guest.py", line 136, in launch
2015-10-01 17:39:59.187 6060 ERROR nova.compute.manager [instance: 79207baa-d735-401c-9f81-4f7ea75aaa4f] return self._domain.cr...

Read more...

Revision history for this message
Stefan Bader (smb) wrote :

Matt, please open a new bug report. As your problem neither does seem to be unexpected complete hangs or crashes, nor isolated to 3.13.0-64.104 this unlikely is the same issue. Ideally run "ubuntu-bug linux" from an affected machine. Note also that 3.13.0-64.104 never left the proposed stage. So only people that have the proposed archive enabled have ever seen that kernel version. Thanks.

To post a comment you must log in.
This report contains Public information  
Everyone can see this information.

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.