[realtime app] not possible to redirect drivers/nvme IRQs from realtime cpuset

This bug is missing log files that will aid in diagnosing the problem. While running an Ubuntu kernel (not a mainline or third-party kernel) please enter the following command in a terminal window:

apport-collect 1831566

and then change the status of the bug to 'Confirmed'.

If, due to the nature of the issue you have encountered, you are unable to run this command, please add a comment stating that fact and change the bug status to 'Confirmed'.

This change has been made by an automated script, maintained by the Ubuntu Kernel Team.

It looks like the supplied patch is a no-op. A module parameter is introduced with a .set function, but the parameter does not have any write permission specified so the .set function can never execute (since userspace won't have permission to write to the module parameter file).

module_param_cb(cq_cpulist, &cq_cpulist_ops, NULL, 0);
                                                 ~~^~

This 0 provided to module_param_cb() is the octal UNIX permissions argument to module_param_cb(). The correct value should be 0200 in this case, to specify write-only permission.

The permission is set to 0 on a purpose -- to prevent runtime changes. The parameter can be still used on modprobe/insmod or on kernel command line.

Thanks for the clarification, that makes sense to me now. It doesn't look like the patch handles CPU hotplugging though; a cpumask filled with *only* offline CPUs can be passed as a parameter, or all of the CPUs in the cpumask can be offlined after the setup is finished. This isn't a problem with the default behavior because num_possible_cpus() guarantees that there will always be at least one online CPU for the nvme queue (since the CPU hotplug code doesn't allow all CPUs to be offlined).

@mykizi-ericsson Could I get an update for my last comment? If CPU hotplugging can't be addressed then perhaps it'd be alright to just mark the new module parameter as unsafe...

I agree that it is wise to intersect cq_cpumask with online_cpumask in cq_cpulist_set(), and if cpuset_weight(cq_cpumask & online_cpumask) == 0 -- ignore cq_cpumask and stick with default behavior.

I don't think cpu offlining is problematic here. If CPU is offlined -- the IRQ (and queue) will be served by another online CPU. Yes, it will break IRQ affinity and make cq_cpumask setting pointless, but this is system administrator's problem. The parameter is only useful for fine tuning and some resource planning in advance is required. I don't think that this side effect is enough to motivate "unsafe" tag.

Sorry for the late action. I made a patch (attached) adding the nitpicks we discussed above, and a couple others. Please take a look and let me know what you think. If all is well, I'll submit this all to the mailing list to be merged.

These patch sets are quite invasive for a stable kernel. It looks like the default behavior is untouched if cq_cpumask is zero. However, I would like to see some discussion with upstream as to the efficacy of IRQ affinity with this device.

Mykyta - can you develop and send a patch for current upstream review ?