Comment 7 for bug 1746316

Hello, the MRE has been discussed many times, e.g. I did the same work a lot of times, including describing the MRE exception paperwork.

e.g. LP: #1594493
LP: #1629870 (here you can see why the upstream testsuite is better than autopkgtests)
LP: #1674819
LP: #1683965
LP: #1729568

If we compare the debdiff I got uploaded for trusty, the debdiff is even worse

diff from 4.3.10-dfsg-1 (in Debian) to 4.3.34-dfsg-1+deb8u1ubuntu1.14.04.1 (13.0 MiB)

Also xenial has been bumped from 5.0.18 to 5.0.40, just in 4 minor updates, but the debdiff between what is in release, and what is in updates is scary too.

Just to be clear on all the above scary debdiffs, the TOTAL number of bugs opened because of them is...

ONE :)
(I named the virtualbox and virtualbox-hwe file init files with the same alias, so installing them without purging the other one resulted in a init system warning).
I fixed that one in a minor update, as soon as I found out the root cause for it.

If we consider the upstream debdiff, ZERO new bugs found.

That said, I understand the rationale for having a reproducing testcase, while security fixes don't need it usually, I can say that the case for testing is:

- install a xenial/artful guest machine.
  - check if 3d works, both enabled and disabled.
  - check if other guest machines still work, e.g. Windows/Other linuxes
  - check if 3d with HWE stack still work, reboot and see if everything works as expected.
  - run xenial with old and new hwe kernels, it should boot in both cases

  ^^ do the same with virtualbox-guest-additions-iso package, after removing the guest* apt stuff

This way you can test the guest stuff

- install virtualbox inside that xenial/artful guest machine, and install another linux inside that VM.
  - everything should run as expected, except for being slow (a VM inside a VM needs a lot of powerful hardware)

This is the testsuite I usually run, before asking for an SRU/MRE accept.

I also ask a lot of colleagues to test my ppa, and their machines, so I can say I'm happy with the upload, even before filing the paperwork.

That said, upstream has a really serious (not public) testsuite, they run it with the packages their provide, and this is the best reason for sticking with upstream new releases, instead of just cherry-picking new patches, because I don't want to diverge from their code, this makes regressions easier to track/reproduce, and fixes to be applied.

I never got a regression, even if in all the cases I was scared about the debdiffs, and I really think even with this upload we will be safe to go.

I case the package goes in proposed, I'll test artful in my laptop (I still run artful here :p )
and xenial with the VM I usually use.

I'll report back all the new testing done, if you care, but honestly the tests we will need, will come as new bug reports in a matter of *minutes* after the upload goes in proposed
(virtualbox is a really strange package, each time I mess up with something, it really takes minutes to get the first bug report in Debian or Ubuntu).
I can't test different hardware/CPU vendors (specially because of such CVEs), but I think monitoring new bug reports is the best way we can test it.

(sorry for the verbosity, it was intended as a bullet list, but even if it seems "zero" work, there is a lot of stuff going behind the scenes there)