Comment 32 for bug 661248

Thanks for the additional dmesg daemonx.

[ 25.243600] nvidia 0000:01:00.0: enabling device (0000 -> 0003)
[ 25.243608] nvidia 0000:01:00.0: PCI INT A -> GSI 16 (level, low) -> IRQ 16
[ 25.243619] nvidia 0000:01:00.0: setting latency timer to 64
[ 25.243625] vgaarb: device changed decodes: PCI:0000:01:00.0,olddecodes=io+mem,decodes=none:owns=io+mem
[ 25.243660] NVRM: The NVIDIA GPU 0000:01:00.0 (PCI ID: 10de:0407) installed
[ 25.243661] NVRM: in this system is not supported by the 270.41.06 NVIDIA Linux
[ 25.243662] NVRM: graphics driver release. Please see 'Appendix A -
[ 25.243663] NVRM: Supported NVIDIA GPU Products' in this release's README,
[ 25.243664] NVRM: available on the Linux graphics driver download page at
[ 25.243666] NVRM: www.nvidia.com.
[ 25.243674] nvidia 0000:01:00.0: PCI INT A disabled
[ 25.243682] nvidia: probe of 0000:01:00.0 failed with error -1
[ 25.243714] NVRM: The NVIDIA probe routine failed for 1 device(s).
[ 25.243716] NVRM: None of the NVIDIA graphics adapters were initialized!

I zeroed in on the

 vgaarb: device changed decodes: PCI:0000:01:00.0,olddecodes=io+mem,decodes=none:owns=io+mem

and looked at the git commit history of the VGA Arbitrator. It is present in v2.6.32 and therefore it's presence isn't the cuplrit.

I note that the nvidia drivers helpfully tells us that

nvidia: probe of 0000:01:00.0 failed with error -1

-1 is symbolically known as -ENOPERM (see include/asm-generic/errno-base.h) but in this case is not accurate. Looking at the nvidia kernel interface source-code I see:

    else
    {
        nv_check_pci_config_space(nv, FALSE);

        if (rm_is_supported_device(sp, nv) != RM_OK)
        {
            nv_printf(NV_DBG_ERRORS,
                "NVRM: The NVIDIA GPU %02x:%02x.%x (PCI ID: %04x:%04x) installed\n"
                "NVRM: in this system is not supported by the %s NVIDIA Linux\n"
                "NVRM: graphics driver release. Please see 'Appendix A -\n"
                "NVRM: Supported NVIDIA GPU Products' in this release's README,\n"
                "NVRM: available on the Linux graphics driver download page at\n"
                "NVRM: www.nvidia.com.\n",
                nv->bus, nv->slot, PCI_FUNC(dev->devfn), nv->vendor_id, nv->device_id,
                NV_VERSION_STRING);
            goto err_not_supported;
        }

so the crux is at:

rm_is_supported_device(sp, nv) != RM_OK

which is an Nvidia Resource Manager function:

$ grep 'rm_is_supported_device' /proc/kallsyms

f9e6fc68 t rm_is_supported_device [nvidia]

which is contained in the binary blob 'nv-kernel.o' for which we don't have source code.

grep -rn 'rm_is_supported_device' *
nv.c:4773: if (rm_is_supported_device(sp, nv) != RM_OK)
nv.h:618:BOOL NV_API_CALL rm_is_supported_device (nv_stack_t *, nv_state_t *);
Binary file nv-kernel.o matches

So this is an Nvidia issue with how Resource Manager compares the NV state pointer nvl->nv_state and the stack pointer sp.