obsolete out-of-tree ivsc dkms in favor of in-tree one

Bug #2061747 reported by You-Sheng Yang
42
This bug affects 6 people
Affects Status Importance Assigned to Milestone
ipu6-drivers (Ubuntu)
Fix Released
High
You-Sheng Yang
Noble
Fix Committed
High
You-Sheng Yang
ivsc-driver (Ubuntu)
Invalid
Undecided
Unassigned
Noble
Invalid
Undecided
Unassigned
linux (Ubuntu)
In Progress
High
You-Sheng Yang
Noble
Fix Committed
High
You-Sheng Yang
linux-meta (Ubuntu)
In Progress
High
You-Sheng Yang
Noble
Fix Committed
High
You-Sheng Yang
linux-meta-oem-6.8 (Ubuntu)
Invalid
High
You-Sheng Yang
Noble
In Progress
High
You-Sheng Yang
linux-oem-6.8 (Ubuntu)
Invalid
High
You-Sheng Yang
Noble
In Progress
High
You-Sheng Yang

Bug Description

[SRU Justification]

BugLink: https://bugs.launchpad.net/bugs/2061747

========== linux-meta, linux-meta-unstable, linux-meta-oem-6.8 ==========

[Impact]

Starting from kernel v6.8, Intel demands the use of in-tree VSC driver
instead of the out-of-tree dkms originated from
https://github.com/intel/ivsc-driver.

[Fix]

The main fixes are in kernel & dkms packages. This adds ivsc-modules to
Provides list of linux-image-generic as other built-in linux-modules-foo
packages do.

[Test Case]

The generated meta package linux-image-generic should then provides
"ivsc-modules".

[Where problems could occur]

Meta package changes only. No real effect but in package dependency.

[Other Info]

While the ivsc-dkms obsoletion only happens for kernel >= 6.8, this will
only be nominated for Unstable/Noble/OEM-6.8.

========== ipu6-drivers, linux, linux-oem-6.8 ==========

[Impact]

Starting from kernel v6.8, Intel demands the use of in-tree VSC driver instead of the out-of-tree dkms originated from https://github.com/intel/ivsc-driver.

[Fix]

The in-tree vsc driver as of v6.8 still needs a few fixes to achieve the same support level to launch Intel IPU6 Camera devices. Commit 1,3, and 4 are to add supported devices and platforms. Commit 2 resolves an issue after resumed.

[Test Case]

This is supposed to work together with the updated dkms, which shall also be built along with the kernel itself as linux-modules-ipu6-<variant>. Install the corresponding kernel/modules packages and test camera functions.

[Where problems could occur]

While this is the first time we switch to in-tree VSC driver, and the out-of-tree driver is not aligned at the time of transition and probably never will, the provided functions and verified stability issues may vary.

[Other Info]

The dkms is created to be compatible of multiple kernel versions, yet the in-tree vsc driver transitioning should only happen for kernel >= v6.8. That is, oem-6.8, noble and linux-unstable will be nominated.

========== original bug description ==========

Starting from kernel v6.8 (yet from the code diff it's v6.6), Intel demands the use of in-tree IVSC drivers instead of out-of-tree dkms from https://github.com/intel/ivsc-driver.

Revision history for this message
Timo Aaltonen (tjaalton) wrote :
Changed in linux-oem-6.8 (Ubuntu Noble):
status: New → Fix Released
You-Sheng Yang (vicamo)
description: updated
You-Sheng Yang (vicamo)
description: updated
Revision history for this message
You-Sheng Yang (vicamo) wrote :
Changed in linux (Ubuntu Noble):
status: New → In Progress
importance: Undecided → High
assignee: nobody → You-Sheng Yang (vicamo)
Revision history for this message
You-Sheng Yang (vicamo) wrote :
Changed in linux-oem-6.8 (Ubuntu Noble):
status: Fix Released → In Progress
importance: Undecided → High
assignee: nobody → You-Sheng Yang (vicamo)
Changed in ivsc-driver (Ubuntu Noble):
status: New → Invalid
Revision history for this message
You-Sheng Yang (vicamo) wrote :

No further update to ivsc-driver shall be necessary as we're removing it.

Changed in ipu6-drivers (Ubuntu Noble):
status: New → In Progress
importance: Undecided → High
assignee: nobody → You-Sheng Yang (vicamo)
Revision history for this message
You-Sheng Yang (vicamo) wrote :

PPA: https://launchpad.net/~vicamo/+archive/ubuntu/ppa-2061747

Proposed DKMS update for ipu6-drivers/noble based on new upstream release Release_20240409.

Need sponsor.

Revision history for this message
Ubuntu Foundations Team Bug Bot (crichton) wrote :

The attachment "ipu6-drivers_0~git202404110253.97c94720-0ubuntu1~ppa.1.debdiff" seems to be a debdiff. The ubuntu-sponsors team has been subscribed to the bug report so that they can review and hopefully sponsor the debdiff. If the attachment isn't a patch, please remove the "patch" flag from the attachment, remove the "patch" tag, and if you are member of the ~ubuntu-sponsors, unsubscribe the team.

[This is an automated message performed by a Launchpad user owned by ~brian-murray, for any issue please contact him.]

tags: added: patch
description: updated
You-Sheng Yang (vicamo)
Changed in linux-meta-oem-6.8 (Ubuntu Noble):
assignee: nobody → You-Sheng Yang (vicamo)
importance: Undecided → High
status: New → Triaged
Changed in linux-meta (Ubuntu Noble):
assignee: nobody → You-Sheng Yang (vicamo)
importance: Undecided → High
status: New → Triaged
Revision history for this message
Steve Langasek (vorlon) wrote : Please test proposed package

Hello You-Sheng, or anyone else affected,

Accepted ipu6-drivers into noble-proposed. The package will build now and be available at https://launchpad.net/ubuntu/+source/ipu6-drivers/0~git202404110253.97c94720-0ubuntu1 in a few hours, and then in the -proposed repository.

Please help us by testing this new package. See https://wiki.ubuntu.com/Testing/EnableProposed for documentation on how to enable and use -proposed. Your feedback will aid us getting this update out to other Ubuntu users.

If this package fixes the bug for you, please add a comment to this bug, mentioning the version of the package you tested, what testing has been performed on the package and change the tag from verification-needed-noble to verification-done-noble. If it does not fix the bug for you, please add a comment stating that, and change the tag to verification-failed-noble. In either case, without details of your testing we will not be able to proceed.

Further information regarding the verification process can be found at https://wiki.ubuntu.com/QATeam/PerformingSRUVerification . Thank you in advance for helping!

N.B. The updated package will be released to -updates after the bug(s) fixed by this package have been verified and the package has been in -proposed for a minimum of 7 days.

Changed in ipu6-drivers (Ubuntu Noble):
status: In Progress → Fix Committed
tags: added: verification-needed verification-needed-noble
Revision history for this message
Martin D. Weinberg (martin-weinberg-5) wrote (last edit ):

I did the followings steps:
- Added the noble-proposed repo
- Installed the driver(s) using: `sudo apt install intel-ipu6-dkms/noble-proposed`
- Confirmed that the correct package was installed:
```
$ apt policy intel-ipu6-dkms
intel-ipu6-dkms:
  Installed: 0~git202404110253.97c94720-0ubuntu1
  Candidate: 0~git202404110253.97c94720-0ubuntu1
  Version table:
 *** 0~git202404110253.97c94720-0ubuntu1 100
        100 http://us.archive.ubuntu.com/ubuntu noble-proposed/universe amd64 Packages
        100 /var/lib/dpkg/status
     0~git202311240921.07f0612e-0ubuntu2 500
        500 http://us.archive.ubuntu.com/ubuntu noble/universe amd64 Packages
```
- Rebooted

I see no evidence that a ipu6 device was detected. E.g. nothing in kern.log from intel-ipu6 after the new driver package installed. Perhaps I've missed something important here? LMK if there's another test I can perform.

tags: added: verification-failed-noble
removed: verification-needed-noble
Revision history for this message
Anthony Wong (anthonywong) wrote :

This fix also requires kernel patches in this series: https://lists.ubuntu.com/archives/kernel-team/2024-April/150378.html

Revision history for this message
Martin D. Weinberg (martin-weinberg-5) wrote :

Thanks, I tried the NVidia kernel 6.8.0-1005.5+1 from noble-proposed; still no detection. Could be that this predates that patch series? Not sure.

Revision history for this message
You-Sheng Yang (vicamo) wrote :
Changed in linux-meta (Ubuntu Noble):
status: Triaged → In Progress
Changed in linux-meta-oem-6.8 (Ubuntu Noble):
status: Triaged → In Progress
Revision history for this message
You-Sheng Yang (vicamo) wrote :

Please don't fail the verification for things you're not sure.

tags: added: verification-done verification-done-noble
removed: verification-failed-noble verification-needed
description: updated
You-Sheng Yang (vicamo)
description: updated
Revision history for this message
Martin D. Weinberg (martin-weinberg-5) wrote (last edit ):

Sorry, just trying to help by providing reports and trying to follow the rules. I posted the results of my test because the post above asked for testers. Obviously, I'm not helping. I'll refrain from posting moving forward. I'm happy to test but I'll wait for someone to ask.

Changed in linux (Ubuntu Noble):
status: In Progress → Fix Committed
Changed in linux-meta (Ubuntu Noble):
status: In Progress → Fix Committed
Revision history for this message
Launchpad Janitor (janitor) wrote :

This bug was fixed in the package ipu6-drivers - 0~git202404110253.97c94720-0ubuntu1

---------------
ipu6-drivers (0~git202404110253.97c94720-0ubuntu1) noble; urgency=medium

  [ You-Sheng Yang ]
  * New upstream Release_20240409 release (LP: #2061747)
    - drop upstreamed "backport: replace strlcpy with strscpy" patch
    - debian: refresh patches
    - UBUNTU: SAUCE: firmware: fallback to old firmware path
    - UBUNTU: SAUCE: don't fail probing at hwcfg checks

 -- You-Sheng Yang <email address hidden> Thu, 11 Apr 2024 18:38:46 +0800

Changed in ipu6-drivers (Ubuntu):
status: Fix Committed → Fix Released
Timo Aaltonen (tjaalton)
Changed in linux-oem-6.8 (Ubuntu):
status: In Progress → Invalid
Changed in linux-meta-oem-6.8 (Ubuntu):
status: In Progress → Invalid
Revision history for this message
Martin D. Weinberg (martin-weinberg-5) wrote :

I have upgraded to the noble-proposed ipu6-drivers package. Is there a particular kernel that needs to be installed to get the camera to work? Or is the kernel fix not yet available?

Revision history for this message
Agoston Horvath (horvath-agoston) wrote :

I've installed the proposed update, my dkms looks like this:

$ dkms status
ipu6-drivers/0~git202311240921.07f0612e-0ubuntu2, 6.8.0-31-generic, x86_64: installed

Yet, I still get the dmesg that I've seen so far:

[...]
[ 26.463603] int3472-discrete INT3472:06: cannot find GPIO chip INTC100B:00, deferring
[ 26.463986] intel-ipu6 0000:00:05.0: Device 0xa75d (rev: 0x0)
[ 26.464034] intel-ipu6 0000:00:05.0: physical base address 0x624c000000
[ 26.464036] intel-ipu6 0000:00:05.0: mapped as: 0x0000000099d97458
[ 26.464311] intel-ipu6 0000:00:05.0: IPU in secure mode
[ 26.464315] intel-ipu6 0000:00:05.0: IPU secure touch = 0x0
[ 26.464317] intel-ipu6 0000:00:05.0: IPU camera mask = 0x0
[ 26.483089] intel-ipu6 0000:00:05.0: IPC reset done
[ 26.483099] intel-ipu6 0000:00:05.0: cpd file name: intel/ipu6ep_fw.bin
[ 26.486833] intel-ipu6 0000:00:05.0: FW version: 20230925
[ 29.125733] usb 3-9: reset full-speed USB device number 3 using xhci_hcd
[ 29.258062] int3472-discrete INT3472:06: cannot find GPIO chip INTC100B:00, deferring
[ 29.258609] intel-ipu6 0000:00:05.0: Device 0xa75d (rev: 0x0)
[ 29.258686] intel-ipu6 0000:00:05.0: physical base address 0x624c000000
[ 29.258689] intel-ipu6 0000:00:05.0: mapped as: 0x00000000d8f2888d
[ 29.259205] intel-ipu6 0000:00:05.0: IPU in secure mode
[ 29.259213] intel-ipu6 0000:00:05.0: IPU secure touch = 0x0
[ 29.259216] intel-ipu6 0000:00:05.0: IPU camera mask = 0x0
[ 29.278774] intel-ipu6 0000:00:05.0: IPC reset done
[ 29.278781] intel-ipu6 0000:00:05.0: cpd file name: intel/ipu6ep_fw.bin
[ 29.281654] intel-ipu6 0000:00:05.0: FW version: 20230925
[...]

And I have no /dev/video0.

Revision history for this message
Ubuntu Kernel Bot (ubuntu-kernel-bot) wrote :

This bug is awaiting verification that the linux/6.8.0-32.32 kernel in -proposed solves the problem. Please test the kernel and update this bug with the results. If the problem is solved, change the tag 'verification-needed-noble-linux' to 'verification-done-noble-linux'. If the problem still exists, change the tag 'verification-needed-noble-linux' to 'verification-failed-noble-linux'.

If verification is not done by 5 working days from today, this fix will be dropped from the source code, and this bug will be closed.

See https://wiki.ubuntu.com/Testing/EnableProposed for documentation how to enable and use -proposed. Thank you!

tags: added: kernel-spammed-noble-linux-v2 verification-needed-noble-linux
removed: verification-done
Revision history for this message
Martin D. Weinberg (martin-weinberg-5) wrote (last edit ):

Thank you for all of this work. I am not sure that I should be changing the tags based on previous experience, but here is my test report.

For testing, I updated to the linux/6.8.0-32.32 generic kernel in noble-proposed. It seemed that I needed to reinstall the intel-ipu6-dkms after installing the kernel packages.

Overall, this is a huge improvement. The kernel detects the camera sensor and loads the ipu6 drivers. About half the time, the sensor is detected correctly and registered. Then, applications like vlc, zoom, gUM test in the browser all work reliably. Successful kernel messages look like (in part):

2024-05-06T17:26:42.688374-04:00 raven kernel: intel-ipu6-isys intel-ipu6-isys0: bind ov2740 3-0036 nlanes is 2 port is 1
2024-05-06T17:26:42.688378-04:00 raven kernel: intel-ipu6-isys intel-ipu6-isys0: All sensor registration completed.

And during operation:

2024-05-06T17:39:24.330879-04:00 raven kernel: intel-ipu6-isys intel-ipu6-isys0: stream on ov2740 3-0036

The other half of the time, there is a problem detecting the sensor and loading the module. When that happens, kernel messages are:

2024-05-06T17:11:53.825968-04:00 raven kernel: ov2740 i2c-INT3474:01: error -EIO: failed to find sensor
2024-05-06T17:11:53.825968-04:00 raven kernel: ov2740: probe of i2c-INT3474:01 failed with error -5

When this happens, reboot once or twice gets it back to the working state.

One time I noticed a seemingly correct boot, but when I tried to use the camera, it failed and I saw the following in the log:

2024-05-06T17:20:21.587753-04:00 raven kernel: intel-ipu6-isys intel-ipu6-isys0: stream on ov2740 3-0036
2024-05-06T17:20:21.730058-04:00 raven kernel: intel-ipu6-isys intel-ipu6-isys0: csi2-1 error: Payload checksum (CRC) error
2024-05-06T17:20:21.906327-04:00 raven kernel: intel-ipu6-isys intel-ipu6-isys0: csi2-1 error: Payload checksum (CRC) error
2024-05-06T17:20:21.906343-04:00 raven kernel: intel-ipu6-isys intel-ipu6-isys0: csi2-1 error: Transfer FIFO overflow
2024-05-06T17:20:21.906354-04:00 raven kernel: intel-ipu6-isys intel-ipu6-isys0: csi2-1 error: Frame sync error

And this is an error with the similar user-side symptoms but a different message:

2024-05-06T18:37:01.015739-04:00 raven kernel: intel-ipu6-isys intel-ipu6-isys0: stream on ov2740 3-0036
2024-05-06T18:37:01.135909-04:00 raven kernel: ov2740 i2c-INT3474:01: failed to reset
2024-05-06T18:37:03.183800-04:00 raven kernel: intel-ipu6-isys intel-ipu6-isys0: stream stop time out

In both of these failure cases, the driver was then hung and then camera was stuck until the next successful reboot.

No configuration or kernel packages were added or removed during these tests. Same kernel from noble-proposed:

$ uname -a
Linux raven 6.8.0-32-generic #32-Ubuntu SMP PREEMPT_DYNAMIC Wed May 1 15:30:59 UTC 2024 x86_64 x86_64 x86_64 GNU/Linux

Revision history for this message
Luke Marsden (lukemarsden) wrote :

Some work ongoing on resolving camera flakiness towards the end of this thread: https://github.com/intel/ipu6-drivers/issues/187

Revision history for this message
Martin D. Weinberg (martin-weinberg-5) wrote :

Wow. Thanks for all of that effort on behalf of Carbon X1 owners. My machine is also a Lenovo 21HMCTO1WW. Looks like the same issues. So this is an upstream problem.

Revision history for this message
Ubuntu SRU Bot (ubuntu-sru-bot) wrote : Autopkgtest regression report (linux-meta/6.8.0-32.32)

All autopkgtests for the newly accepted linux-meta (6.8.0-32.32) for noble have finished running.
The following regressions have been reported in tests triggered by the package:

cryptsetup/unknown (arm64)
dropbear/2022.83-4 (arm64)
dropbear/unknown (armhf)
zfs-linux/2.2.2-0ubuntu9 (arm64)

Please visit the excuses page listed below and investigate the failures, proceeding afterwards as per the StableReleaseUpdates policy regarding autopkgtest regressions [1].

https://people.canonical.com/~ubuntu-archive/proposed-migration/noble/update_excuses.html#linux-meta

[1] https://wiki.ubuntu.com/StableReleaseUpdates#Autopkgtest_Regressions

Thank you!

Revision history for this message
Sander Tuit (sandertuit) wrote :

Kernel 6.8.0-32.32 doesn't boot for me on AMD64 (can't find the root disk). 6.8.0-31-generic boots just fine. I'm not using root disk encryption or zfs, so the cryptsetup/zfs regressions probably don't have anything to do with that.

To post a comment you must log in.
This report contains Public information  
Everyone can see this information.

Duplicates of this bug

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.