Alternate CD cdrom-detect.postinst does not consider USB flash drive/key.

Thank you for your work! However, the changes you are requesting really require more of a discussion, which should be done on an appropriate mailing list or forum. [WWW] http://www.ubuntu.com/support/community/mailinglists might be a good start for determining which mailing list to use.

I initiated contact with the developers of the "https://blueprints.launchpad.net/ubuntu/+spec/usb-installation-images" project (https://blueprints.launchpad.net/ubuntu/+spec/usb-installation-images). Let see where it brings me too.

I don't think this patch is appropriate as it stands. You have to be careful not to mount a partition from the hard disk. Remember that cdrom-detect is used on regular Ubuntu CDs too. Let's say a normal from-CD Ubuntu installation happened to encounter a partition on the hard disk that happened to look a bit like an Ubuntu CD. With your patch, it might well end up mounting it and merrily carry on with installing from it. Unfortunately, if you have any partitions on a hard disk mounted, you can't usefully perform partitioning on that hard disk, because the kernel will have the partition table locked. This would make the installer fall over in a big heap.

I'm not saying this idea is fundamentally wrong, but it needs a bit more work. What if you're installing *to* a USB stick? You'll have the same problem as above. Perhaps this needs to be controllable using a kernel command-line option rather than something that's done by default.

Colin, your comment is valid. However, my patch tries to mount /cdrom as vfat or iso9660. The likelihood that /cdrom on a hard disk exist as vfat is remote. The likelihood that /cdrom exist on another USB stick you are trying to install is also remote. How many users will be burned by this?

If I had the time, I would fix it. However, I am busy with my current open source project http://autopoweroff.sourceforge.net

I agree that it is not robust. Probably we should add a test after mounting /cdrom to check if /cdrom indeed contains a valid Ubuntu image. `list-devices maybe-usb-floppy` should probably report the USB device/partition in the first place and thus `list-device partition` would not be necessary. I will leave to the author of this script to fix the issue. It has been months since I played with this thus my memory is failing.

Couldn't these issues be solved by simply checking for a matching .disk/info, like casper does?

Tromod, your solution is obviously the best. Check for a file specific to the installation image that is expected to be found only on the /cdrom device if it has been mounted from a USB device.

And if someone has a VFAT32 /cdrom with a copy of a valid installation image on their hard drive, then though for that fellow. This should not happen ever.

Is there any place I could put my script to create a usb key with the installation image?

Hans, regarding your script, there is a new package usb-creator in Intrepid which should do the job. I don't know if it works with Alternate CDs yet. I guess cdrom-detect would need to be patched if usb-creator is going to work as well.

The best would maybe be if the installer, after not finding a real CD, would ask if it should scan USB drives as well. Anyway, I think support for install from USB devices is very useful. There are many PC's without CD drives (for instance the recent netbooks) and burning CDs (and using USB CD drives) seems wasteful now that USB sticks are everywhere and for cheap. So if we could try to accommodate it in the official CD, it would avoid all the hacks people have to come up with today.

If it does not support the alternate CD, then my script would remain useful. I will not test Intrepid (no time to play around) and continue to stick with 08.04.b

Regardless, all should be plug & play. The user should not be prompted to scan for USB if a USB key has been used for the installation. Try to mount a VFAT 32bits FS on /cdrom and check for a specific file would be good enough. Nobody should have another /cdrom with the same characteristics.

As a first step, I suggest to extend list-devices() with a "usb-partition" option which returns only partitions on USB drives. See https://code.launchpad.net/~tormodvolden/debian-installer-utils/usb-drive-install

Now debian/cdrom-detect.postinst can use this option either 1) all the time or 2) when a special boot option like "usb-media" has been entered or 3) after asking the user if USB drives should be scanned. The alternatives are listed in increasing coding difficulty :) If 1 or 2 are acceptable, I can quickly come up with some patches.

Hans, one problem is that the installer initrd has no idea whether is was booted from a USB drive or not, so the question whether to just scan for USB automatically can not be based on this. On the other hand, I don't see that it can do much harm to scan all USB devices anyway. cdrom-detect already checks for .disk/info. The current situation is not 100% foolproof either, for instance there can be several CD drives each with a different Ubuntu CD and one could be booted while the other is picked for package scan.

I vote for option 1) all the time. We do not need a 100% foolproof solution because lets face it, how many people will have two sources of installation on a system? I cannot imagine why someone would put two install USBs or two CDs on a system at the same time. However, installation should be done with the least interaction with the user. USB support should be plug&play.

One easy foolproof would be to check how many installation sources are there and if there is more than one, list them on the screen with a message asking to remove/rename one of the sources and retry. No need to program interaction with the user; the install would simply abort with the message. Put an easter egg (nothing fancy, just some ascii art) there if you want for fun and lets see if anybody would ever see it. :)

Thank you Tormod for you involvement with the project. It is appreciated.

Looking closer at cdrom-detect (which probably would need to be replaced by something like "detect-media") is that it does not iterate through devices. It really assumes there is one CD available. It just picks the first that satisfies a rough "cdrom" criteria, and then goes on to verify that single device. Just adding "list-devices usb-partition" would just grab the first USB drive's first partition (it will check for real CD devices first though). If there are several USB drives and partitions and it picks the wrong, it is stuck. Like Colin already mentioned, this is a problem if you want to install to another USB drive.

Similarly in the current situation, if there are two CDs, and only one is a Ubuntu CD, it fails if it happens to detect another CD first.

I therefore adjusted cdrom-detect.postinst to check for .disk/info each time it tries to mount a "CD" drive. See https://code.launchpad.net/~tormodvolden/cdrom-detect/usb-drive-install

We can now apply a slightly modified version of Hans' original patch: add "list-devices usb-partition" and probe devices using vfat in addition to iso9660. See http://bazaar.launchpad.net/~tormodvolden/cdrom-detect/usb-drive-install/revision/428

This bug was fixed in the package debian-installer-utils - 1.59ubuntu3

---------------
debian-installer-utils (1.59ubuntu3) intrepid; urgency=low

  * list-devices: new "usb-partition" type for detecting partitions on
    USB drives (LP: #234185)

 -- Tormod Volden <email address hidden> Mon, 22 Sep 2008 00:31:05 +0200

I really prefer Tormod's option 2: make this controlled by a boot option, please. usb-creator can then set that.

Hans, you may laugh at the notion that more than one thing that looks like an installable CD image may be available, but I'm not just making this up to be awkward; *this really does happen in practice*. It so happens that the easiest way to create a recovery partition (of the sort that OEM vendors like to make available) is to simply copy the CD image to a partition on the hard disk. If the installer trawls through disks looking for CD-like things, then this can easily misfire. In fact, Dell contributed a casper patch to help them avoid exactly this problem (bug 209847).

Tormod, your debian-installer-utils branch looks fine and I've merged and uploaded it; thanks! However, I'm not keen on your cdrom-detect branch. Firstly, your revision 427 that checks for .disk/info further up has the effect that if somebody tries to use a misbuilt CD that's missing the .disk directory (a very common mistake), then instead of getting an "Incorrect CD-ROM detected" message, they'll get a message that tells them that the CD couldn't be mounted. I think this would be both inaccurate and confusing. Secondly, your revision 428 has the problem as described above (at least if somebody is using an external disk), and I think it would be better to make this conditional on a debconf template that could be preseeded by usb-creator.

I would suggest adding a cdrom-detect/try-usb template, defaulting to false. If it is found to be true by cdrom-detect.postinst, then it should try 'list-devices usb-partition' and try mounting each of those and checking .disk/info (since in this specific case it's expected that there will be non-CD-like partitions, so it makes more sense to skip over them silently). .disk/info should not be checked for CD devices, so this needs to be a separate for loop. I'm thinking of something like http://paste.ubuntu.com/50052/ - perhaps you could test this and see if it works?

(I agree, by the way, that the two-CD-drives situation is a problem; I'm just not convinced that this is quite the right way to fix it, and would rather fix one thing at a time where possible.)

Well, my approach was to treat CDs and USB removables in as much as possible the same way. But I guess that will need interface changes and rewrites to be perfect. And with Intrepid Beta around the corner this is not going to happen now.

I made a new revision, where I included the cdrom-detect/try-usb template. I chose to scan USB devices in addition to CDs. But then separating out the scan loop actually makes a cleaner patch, and I ended up doing pretty much what Colin suggested :)

http://bazaar.launchpad.net/~tormodvolden/cdrom-detect/usb-drive-install/revision/431?compare_revid=426

Attempted iso qa testing on the 20080930.4 amd64 alternate iso, and the usb flashdrive was not detected as a valid cd-rom device.

All installation attempts fail, including "Check CD for Defects." Currently unable to test the iso with an actual cd as I do not have any available at the moment. But normal installation using desktop isos are possible using usb-creator (tested yesterday).

Harvey, the code mentioned here hasn't been released yet. But it would be great if you could take the time to try it out. Basically you'll need the debian/cdrom-detect.postinst file from http://bazaar.launchpad.net/~tormodvolden/cdrom-detect/usb-drive-install/files and replace the one in your initrd (var/lib/dpkg/info/cdrom-detect.postinst).

Tormod,

Sorry about the noise. I wrongly assumed that "fix released" meant that it was.

I replaced cdrom-detect.postinst in the initrd using the file identified above, but I still get the "Incorrect CD-ROM detected" error message when attempting any of the options.

I'll repeat the test this evening, to ensure it just wasn't operator error.

Harvey

The "fix released" was in the debian-installer-utils task. The cdrom-detect task is still open :)
I forgot to say that you have to boot with the boot parameter cdrom-detect/try-usb=true

Tormod,

It appears to be an iso problem. I burned the iso to cdrom at home, and it fails to detect the cdrom also. The md5 hash checks out on the iso, but I'm going to redownload and retest anyway.

So my issue may not be relevant to this bug. I'll report back any relevant results.

Harvey

I've merged Tormod's patch (with a small change to debian/changelog), and uploaded it. It'll be usable after the first debian-installer upload after beta.

Evan, could you arrange to have usb-creator pass cdrom-detect/try-usb=true for alternate CDs? (It's harmless for desktop CDs, in case it's hard for usb-creator to tell the difference.) That should be the last piece of this bug.

cdrom-detect accepted, closing this task.

Hi, I'm struggling with booting Ubuntu 8.10 Beta (Alternate) from USBdisk which I created via UNetbootin 282.

I have read throu this tread and have a proposl that might sove your problem with multiple detected cdroms. I vote for scanning all media (cd-drives aswell as usb-devices) and if more than one installationmedia is found, ask the user what media to use! I don't like it when things get to automated and hidden from the user.

In use-cases where more than one installation-media is found, the user have probably good knowledge about the medias found and is capable of choosing the correct one.

Another use-case, which I am part of, is the ability to have two or three iso-files on a large USB-stick, say for example "ubuntu_8.10_alternate.iso", "debian_lenny.iso" and "ubuntu_eee_8.10.iso" and during bootup or in cdrom-detect I would like to chose the proper iso-file for installation depending where i am installing at the moment.
For this to work cdrom-detect needs to search for *.iso in all medias (usb-disks mostly) and loop-mount what it find as a virtual cd.

Thats my two cents
// Anders

The .iso case needs to be handled by iso-scan in the hd-media initrds (that's its whole purpose!), not by cdrom-detect. USB media creation software needs to be aware of that.

However, I don't think this is a priority for usb-creator. It is much more urgent for us to get it working *at all* than to make it more configurable, which can come later.

This bug was fixed in the package usb-creator - 0.1.8

---------------
usb-creator (0.1.8) intrepid; urgency=low

  * Add cdrom-detect/try-usb=true to the kernel command line so that
    usb-creator is usable with alternate CDs (LP: #234185).
  * Properly notify that the the user needs to insert a CD, not a USB disk,
    when no CD is inserted.

 -- Evan Dandrea <email address hidden> Thu, 16 Oct 2008 03:53:04 -0400

As discussed on #ubuntu-installer:
17:29:46 < evand> hrm, cdrom-detect/ask-usb fails to account for the situation where you have a CD in the drive as well
*snip*
17:34:13 < evand> Right now I have an Ubuntu image on USB stick plugged in and a Windows drivers CD inserted into the computer.
17:34:44 < evand> It first mounts the CD, then goes into the try-usb block and mounts the usb disk over top of that.
17:35:38 < evand> If I stick a if [ "$mounted" = "1" ]; then break; fi before try-usb it goes with the Windows CD, which obviously fails.
17:36:29 < cjwatson> oh, hmm
17:36:33 < evand> I suspect in this scenario, it should try to mount, then check to make sure we're dealing with an Ubuntu image, if not go to the next device, if there are no devices left, fail
17:37:36 < cjwatson> I see what you mean, that's badly wrong

Untested patch. This will give USB disks precedence over CDs and search all inserted CDs for an Ubuntu image, failing with cdrom-detect/wrong-cd if none of the inserted CDs contain a valid CD image. Previously, cdrom-detect would go to cdrom-detect/wrong-cd on the first CD it could mount, rather than checking to see if there were other CDs with an Ubuntu image.

Most importantly, it no longer double mounts on /cdrom if both a CD and USB disk are inserted.

Testing with a bogus CD and an Ubuntu USB disk inserted was successful. There was no double mount.

This bug was fixed in the package cdrom-detect - 1.28ubuntu4

---------------
cdrom-detect (1.28ubuntu4) intrepid; urgency=low

  * Try to find an Ubuntu image on every CD device, not just the first
    (LP: #234185).

 -- Evan Dandrea <email address hidden> Sun, 19 Oct 2008 17:29:04 -0400

This bug is still reproducible in Ubuntu 8.10 Release

Tested with usb-creater v0.1.8
with ubuntu-8.10-alternate-amd64.iso, (ISO's MD5sum checks out)

But boot still gets stuck and keeps trying to read from CD drive

Reopening & confirming this bug in the 8.10 Release

also
$ usb-creator --version
gives
0.1.8

While synaptic confirms that the package version is indeed 0.1.10

I've just used unetbootin (latest - 292 I think) to burn the 8.10 release alternate CD to a USB key and I can confirm that the installer cannot proceed past the "Detect CDROM" stage. I'm giving up for now and using 8.10 Desktop, but this means that I can't use full-disk encryption (a requirement for my work) which means that I can't use Ubuntu as planned at my organisation (without a lot of hassle - buying CDROMs for the laptops (assuming I can) - burning CD's, etc).

I've tried simply mounting the USB into /cdrom using terminal two of the installer, which works breifly but it then fails with a "failed to copy file from CDROM" error message. This may have something to do with the use of Unetbootin, but I don't really understand grub, the installer script, or about 99% of this thread, so I can't comment.

My preference for a solution would be to include full-disk encryption on the Ubuiqity Live CD, but since that's been deferred for the last two releases, I think getting it working here is more likely.

Neil, I don't know what unetbootin is, but please try the usb-creator, which is officially supported.

The point of unetbootin (or isotostick.sh) is that you run them on your current o/s (or even windows) to create a key that can install any other. Isn't usb-creator an intrepid tool? If I had intrepid successfully installed somewhere, I could try it - although my understanding from the forums was that it created a "live" session, not an "alternative" session, which is what I need to do this full-disk encryption.

If usb-creator is still relevant here and there's a way for me to use, please let me know and I'll give it a try.

The point of my post was really to re-open the cd-rom detect issue in the alternative installer. It says fix-released, but that doesn't appear to be the case in my experience. I couldn't really follow the patches talked about in this thread, so I thought it was worth posting again.

May I suggest that someone build a USB key with 08.10, then dd an image and make it available for downloading? Canonical should do this. This way users get out of the chicken and the egg trap.

This assuming of course that one can dd from one model of USB key to another if there is enough space. I know that this works for USB keys of same size and model.

The Alternate CD needs to be booted with cdrom-detect/try-usb=true as a boot option. usb-creator automcatically adds this.

Well that was long winded - I downloaded the standard desktop image, cut a CD from it and booted another PC live, then used USB-Creator to make an alternative boot key that actually worked and doesn't fail at detect-cd roms.

Surely it shouldn't be that hard to get the alternative CD to work as a USB key install? Is it not better to fix the issue with CD-ROM detect, than mask it with USB-Creator?

Anyway, thanks for the feedback in this bug. It's got me a fully disk-encrypted laptop and I'm now on track to introducing at least another 4 such machines at my work.

intel,
I'm not sure why you reopened the usb-creator task. The bug is clearly fixed there, given the reports here and elsewhere. However, if using USB creator to create an alternate USB disk is not working for you, please file a new bug.

If you take issue with the functionality being disabled by default on the alternate CD, please re-read Colin's comment and propose a solution that accounts for the problems he mentions:
https://bugs.launchpad.net/ubuntu/+source/usb-creator/+bug/234185/comments/5

Thanks!

intel,
Sorry I missed your earlier comment. Please open a new bug against cdrom-detect. Put `set -x` at the top of /var/lib/dpkg/info/cdrom-detect.postinst before that code runs and attach /var/log/syslog after the machine starts to hang.

Thanks

Hi

I am using a hardy desktop with an intrepid usb-creator deb.
It installed without complaint.
ii usb-creator 0.1.10 Ubuntu USB desktop image creator

I've made a jaunty-alpha6 alternate x86 iso onto a stick with usb-creator which did not boot
at all. The dell optiplex 760 said no operating system found.

I remade the usb stick with unetbootin with both jaunty-alpha6 and intrepid
(both alternate amd64) and they both boot.

Both get stuck at cdrom-detect.
I described some experience here (Jan Groenewald's posts)
https://bugs.launchpad.net/ubuntu/+source/debian-installer/+bug/260672

I then found this thread and patch, http://launchpadlibrarian.net/14665232/cdrom-detect.postinst.usb-detect.patch
and patched and repacked cdrom-detect and put it in pool/ , and put that on the stick, but it still stops
at cdrom-detect.

Well blow me over.

I re-did the usb-creator due to the above thread's insights.
Apparantly the first time I did this some files were corrupted or the disk didn't sync properly.
The jaunty-alternate-amd64-alpha-6 now boots OK and cdrom-detects works great.

Accepted cdrom-detect into hardy-proposed, the package will build now and be available in a few hours. Please test and give feedback here. See https://wiki.ubuntu.com/Testing/EnableProposed for documentation how to enable and use -proposed. Thank you in advance!

Accepted debian-installer-utils into hardy-proposed, the package will build now and be available in a few hours. Please test and give feedback here. See https://wiki.ubuntu.com/Testing/EnableProposed for documentation how to enable and use -proposed. Thank you in advance!

The USB approach is STILL not working properly.
I've tested it with 9.04 Server and with 9.10 Server, Both keep complaining about CD-ROM even when booted from a USB stick.

I've confirmed this fix for hardy. I wrote a copy of the 8.04.4 candidate server ISO to a USB stick using the lucid version of usb-creator, then booted it editing the kernel commandline to remove 'cdrom-detect/try-usb=true' and confirmed that the installer failed when looking for the CD. Then I booted again without editing the kernel commandline, and the installer gets past this point just fine and begins loading installer components from the USB stick.

This bug was fixed in the package cdrom-detect - 1.25ubuntu3

---------------
cdrom-detect (1.25ubuntu3) hardy-proposed; urgency=low

  * Add support for using the contents of the CD image on a USB device
    with the cdrom-detect/try-usb=true preseed option (LP: #234185).

 -- Evan Dandrea <email address hidden> Thu, 27 Aug 2009 11:05:18 +0100

This bug was fixed in the package debian-installer-utils - 1.50ubuntu4

---------------
debian-installer-utils (1.50ubuntu4) hardy-proposed; urgency=low

  * Add 'usb-partition' type in support of the new cdrom-detect option
    to try USB devices (LP: #234185).

 -- Evan Dandrea <email address hidden> Thu, 27 Aug 2009 11:29:21 +0100

I'm still seeing this bug on the 9.10 alternative cd, even with cdrom-detect/try-usb=true set, anyone else?

Yes I'm having this problem too!

Still reproducable on lucid alternate CD even with cdrom-detect/try-usb=true set

I can confirm it's still reproductable on lucid alternate, I've tried adding "cdrom-detect/try-usb=true" on boot but nothing changed.

Then I tried another solution that was successful on 9.04 (didn't remember the exact syntax, but something like this, the idea was to mount the key itself into the /cdrom directory and then point it when it says the cdrom can't be found):
- switch to the second virtual console during the first couple of dialogs by pressing the "ALT-2"
- mkdir /cdrom /dev/cdroms
- cd /dev/cdroms
- ln -s ../sda1 cdrom0 (where sda1 is your USB stick)
- mount -t vfat /dev/cdroms/cdrom0 /cdrom
- switch back to the first virtual console by pressing "ALT-1" and continue installation
But this was also unsuccessful.

In the end I tried to copy the iso image in the usb key and, similar to above, mounting the iso in loop as a cd.. but no changes.

It's useful to install the alternate cd to configure lvm, and if you have a netbook using the usb key is the unique solution..

Ps: the server version does the same thing as the alternate.

I also tried this solution, always on lucid alternate, but didn't work: https://bugs.launchpad.net/ubuntu/+source/cdrom-detect/+bug/195614/comments/8

Same here, still a problem in 10.04. Installing from usb needs work. Ubuntu is too fixated on optical media.

I put Ubuntu Server 10.04 (amd64) on a 1G usb flash drive. This is for installation on a netbook with an Atom N450 and without a CD drive. Just getting Ubuntu Server on the flash drive was too much work. There was no easy way to put Ubuntu on a flash drive from within Arch Linux! The executable from the unetbootin web site won't run unless version 12 of libpng is present (Arch moved on to version 14 some time ago), and setting up usb-creator outside of Ubuntu looked to be a lot of trouble. I finally booted into Windows to do that part. (You should be ashamed that you made me boot up Windows.)

On the netbook, Ubuntu Server booted fine from the usb drive. But the installation process insists on finding a CDROM drive, and that netbook has none. I tried F6 and typed in "install cdrom-detect/try-usb=true" on the kernel boot parameters. Didn't make any difference.

I think it's fixed in 10.10.
10.10 Alternate beta i386 works fine for me from a usb stick.
I haven't tried server version yet.

I would like to confirm Ubuntu 10.10 Alternate RC i386 iso works fine for me.

I've created the usb pendrive with the "Startup Disk Creator". I was able to boot it and I was able to install a working system.

Not completely fixed in all versions of 10.10 (at least server CD is still afflicted as of today's daily buiild of the RC) , but since everything is marked as resolved here, perhaps it would be best to transition to this existing open bug that isn't focused on alternate CD: Bug 582427.

This still fails for me on 10.10 alternate amd64. I used Ubootin to create the USB image, Im adding cdrom-detect/try-usb=true and it doesn't complain about not finding the cdrom, but when it starts to copy if says there was a problem reading the cdrom and that it failed to copy a file.

It seems to me that the bug still exists only if the USB media is created with Unetbootin, with "Startup disk creator" the image works fine, I didn't add any string on boot. Andrew can you try to do the same thing but this time with "Startup disk creator"?

@Andrew Henry: the issue is that some udeb files in the Ubuntu alternate iso have more than 64-character filenames, but unfortunately they are truncated to 64 characters (and hence can't be found by the installer) because 7-zip, which is used by UNetbootin for extracting iso files, doesn't support the RockRidge extensions which are used by the Ubuntu alternate iso for having 64+ characters in filenames. See https://bugs.launchpad.net/unetbootin/+bug/373089 and http://sourceforge.net/tracker/?func=detail&aid=2928072&group_id=14481&atid=114481 for the bug reports detailing the issues. Unfortunately this means that UNetbootin isn't going to be able to support Ubuntu alternate iso files, even with the magical "cdrom-detect/try-usb=true" option, until the bug is fixed in 7-zip.

I was running into this same issue with the 10.10 Alternate using a 2GB Flash Voyager on my Dell Studio XPS. However, it works fine using my 16 GB Patriot XT. Both were created using the Setup Disk Creator v 0.2.22.1 on Ubuntu 10.04.

One difference I noticed is that the Flash Voyager drive didn't show up as a USB Device in the BIOS boot menu...I had to select External Device (or something like that). However, the Patriot shows up as a USB Device...as expected. I erased and formated both of them with the Setup Disk Creator.

Hardy has seen the end of its life and is no longer receiving any updates. Marking the Hardy task for this ticket as "Won't Fix".