Automounting removable storage doesn't always honour the utf8 mount option

Bug #199165 reported by Andrew
32
Affects Status Importance Assigned to Milestone
KDE Base
Fix Released
Medium
kdebase (Ubuntu)
Fix Released
Medium
Unassigned
kdebase-kde4 (Ubuntu)
Invalid
Undecided
Unassigned

Bug Description

I often have to mount flash drives having russian symbols in file names. KDE4 doesn't seem to honour the utf8 mount option always.

This is an example of an incorrectly mounted one (This SD card is mounted through WM5Storage from a wm based pda)
/dev/sdi1 on /media/!SD type vfat (rw,nosuid,nodev,uid=1000)

And here's the same card inserted into the cardreader:

/dev/sdd1 on /media/!SD type vfat (ro,nosuid,nodev,noatime,flush,uid=1000,utf8,shortname=lower
(everything OK)

However, on a different SD card the same charset problem happens while inserted into the cardreader as well as the pda.
The removable flash drives also seem to differ: some mount OK, with proper charset, some without. I still cannot figure out what affects it.

A simple 512mb flash stick:
/dev/sdi1 on /media/ya_flashko type vfat (rw,nosuid,nodev,uid=1000) (same here)

Here are the dmesg outputs:
1. SD card through WM5Storage:

[ 811.117862] usb 5-2: new full speed USB device using uhci_hcd and address 3
[ 811.316927] usb 5-2: configuration #1 chosen from 1 choice
[ 811.327920] scsi9 : SCSI emulation for USB Mass Storage devices
[ 811.328068] usb-storage: device found at 3
[ 811.328072] usb-storage: waiting for device to settle before scanning
[ 816.325755] usb-storage: device scan complete
[ 816.328761] scsi 9:0:0:0: Direct-Access WM5torag Acer n300 Handhe 1.74 PQ: 0 ANSI: 0 CCS
[ 816.333749] sd 9:0:0:0: [sdi] 4022272 512-byte hardware sectors (2059 MB)
[ 816.336847] sd 9:0:0:0: [sdi] Write Protect is off
[ 816.336851] sd 9:0:0:0: [sdi] Mode Sense: 03 00 00 00
[ 816.336854] sd 9:0:0:0: [sdi] Assuming drive cache: write through
[ 816.341736] sd 9:0:0:0: [sdi] 4022272 512-byte hardware sectors (2059 MB)
[ 816.344734] sd 9:0:0:0: [sdi] Write Protect is off
[ 816.344737] sd 9:0:0:0: [sdi] Mode Sense: 03 00 00 00
[ 816.344739] sd 9:0:0:0: [sdi] Assuming drive cache: write through
[ 816.344742] sdi: sdi1 sdi2
[ 816.352810] sd 9:0:0:0: [sdi] Attached SCSI disk
[ 816.352856] sd 9:0:0:0: Attached scsi generic sg9 type 0
[ 827.746161] usb 5-2: USB disconnect, address 3

2). same SD card through card reader:

[ 1714.673467] sd 6:0:0:0: [sdd] 4022272 512-byte hardware sectors (2059 MB)
[ 1714.674465] sd 6:0:0:0: [sdd] Write Protect is off
[ 1714.674469] sd 6:0:0:0: [sdd] Mode Sense: 03 00 00 00
[ 1714.674472] sd 6:0:0:0: [sdd] Assuming drive cache: write through
[ 1714.675461] sd 6:0:0:0: [sdd] 4022272 512-byte hardware sectors (2059 MB)
[ 1714.676338] sd 6:0:0:0: [sdd] Write Protect is off
[ 1714.676342] sd 6:0:0:0: [sdd] Mode Sense: 03 00 00 00
[ 1714.676344] sd 6:0:0:0: [sdd] Assuming drive cache: write through
[ 1714.676348] sdd: sdd1 sdd2

3).

[ 1841.375714] usb 2-5: new high speed USB device using ehci_hcd and address 7
[ 1841.509281] usb 2-5: configuration #1 chosen from 1 choice
[ 1841.509499] scsi11 : SCSI emulation for USB Mass Storage devices
[ 1841.509667] usb-storage: device found at 7
[ 1841.509669] usb-storage: waiting for device to settle before scanning
[ 1846.506731] usb-storage: device scan complete
[ 1846.507488] scsi 11:0:0:0: Direct-Access Ut163 USB2FlashStorage 0.00 PQ: 0 ANSI: 2
[ 1846.509090] sd 11:0:0:0: [sdi] 1007616 512-byte hardware sectors (516 MB)
[ 1846.509714] sd 11:0:0:0: [sdi] Write Protect is off
[ 1846.509717] sd 11:0:0:0: [sdi] Mode Sense: 00 00 00 00
[ 1846.509720] sd 11:0:0:0: [sdi] Assuming drive cache: write through
[ 1846.511838] sd 11:0:0:0: [sdi] 1007616 512-byte hardware sectors (516 MB)
[ 1846.512463] sd 11:0:0:0: [sdi] Write Protect is off
[ 1846.512465] sd 11:0:0:0: [sdi] Mode Sense: 00 00 00 00
[ 1846.512468] sd 11:0:0:0: [sdi] Assuming drive cache: write through
[ 1846.512471] sdi: sdi1
[ 1846.620657] sd 11:0:0:0: [sdi] Attached SCSI removable disk
[ 1846.620706] sd 11:0:0:0: Attached scsi generic sg9 type 0

Now I used ivman (don't use it to mount - just to run scripts) to get the hal debug messages for all three test cases
(Will attach them in a separate file since they're quite long)

Revision history for this message
Andrew (aifiltr0) wrote :

Here go the hal messages got from ivman.

Revision history for this message
Andrew (aifiltr0) wrote :

And here's what kde outputs to stout:

I have tried messing up with hal storage policies, but no luck so far kde seems as usual to silently ingnore them (

Revision history for this message
Alexander Frolushkin (hokum) wrote :

I confirm this. All my usb flash sticks mounts without "utf8" in kde 4.03 and therefore I have "?????.???" in filenames instead of russian characters.
In gnome everything works fine.

Revision history for this message
Vítor E. Silva Souza (vitorsouza) wrote :

I just installed Kubuntu Hardy with kde4 and the same thing happens to me. I'm Brazilian and my external HD has lots of folders with accents in their names. When kde4 mounts them through "Recently plugged devices", the letters with accents become a funny "?" and I can't look inside that folder. If I manually mount it, works fine.

I've tried putting a new line on fstab so kde would use the options I put there, but in this case it's just not able to mount the drive. When double-clicking on it on "Recently plugged devices" Dolphin opens my home folder and the command mount shows that the drive hasn't been mounted.

Anyone knows any workarounds already? This is a really annoying bug... (if a bug at all, maybe I'm missing something...)

Thanks

Revision history for this message
Shinjo Park (peremen) wrote :

Me, too. My USB flash disk has *lots of* Korean filenames, but I have to remount to see all Korean characters properly.

Revision history for this message
Léonard G (leonard-gerard) wrote :

Confirmed with vfat utf8 external usb drives ( basic european utf8 ) and kde 4.1 beta

Kenobi (kazmirzak)
Changed in kdebase-kde4:
status: New → Confirmed
Revision history for this message
Anton Pushkarev (drop-06) wrote :

I have the same trouble with Russian letters. If I mount usb disk through hal:

$ pmount
Printing mounted removable devices:
/dev/sdb5 on /media/data type fuseblk (rw,nosuid,nodev,noatime,relatime,user_id=0,group_id=0,allow_other)
/dev/sdb1 on /media/disk type vfat (rw,nosuid,nodev,noatime,relatime,uid=1000,fmask=0022,dmask=0022,codepage=cp437,iocharset=iso8859-1,utf8)

and if through 'pmount'

$ pmount /dev/sdb5
$ pmount /dev/sdb1
$ pmount
Printing mounted removable devices:
/dev/sdb5 on /media/sdb5 type fuseblk (rw,nosuid,nodev,noexec,noatime,user_id=0,group_id=1000,default_permissions,allow_other)
/dev/sdb1 on /media/sdb1 type vfat (rw,nosuid,nodev,noexec,uid=1000,gid=1000,fmask=0177,dmask=0077,codepage=cp437,iocharset=utf8,shortname=mixed,quiet)

note: sdb1 is vfat, sdb5 is ntfs
what's differs: sdb5: fmask dmask codepage iocharset in the second try have other settings than ones in the first try

But I have some files on sdb5 and as you look there are not big difference between pmount output for this device, but if i mount through hal i can't see non-English letters at all. If i mount this disk through 'pmount' all works fine. I don't know does this bug appear on other file systems, but it's appear on ntfs.

Revision history for this message
c_t (chefturner) wrote :

I can confirm this! While it was possible to specify some mount options depending on the filesystem-type (like iocharset, shortname, etc) for removable media in KDE 3, I have not found any way to acchieve this in KDE 4 (using KDE 4.1 RC1 at the moment).
This is quite annoying! Mounting through pmount-hal instead of clicking on the device in the "Devices recently plugged in"-Widget yields the correct mount options...

Revision history for this message
Anton Pushkarev (drop-06) wrote :

c_t , I use KDE 3.5.9 (I was afraid of installing 4.*) but this trouble is here too.
When I used Kubuntu 7.10 (I don't remember which KDE version is there by default) there was no trouble like this.
Now I have Kubuntu 8.04 with last updates

Revision history for this message
c_t (chefturner) wrote :

zolle, in KDE 3.5.9 (latest 8.04) I can rightclick on the desktop-icon of a removable drive, select "Properties" and then select some options, like automounting, shortname-behavior and iocharset. Don't you have these options?

Revision history for this message
Anton Pushkarev (drop-06) wrote :

c_t , yes i do. Hmmm, I have never used this properties page before. I thought that it's a rubbish information and all what i desire i can get through command line & pmount. ;-) Changing information on this page is not affected my hdd (but it was mounted, so this behaviour is normal) And moreover, in previous version of Kubuntu i could merely click on a usb disk icon and HAL mounts it with no problem. So i did not look into this page because of lack of need. May be because of necessery of this page it was deleted? Am i right?

I am sure that usb disks mounting shall be made with UTF-8 codepage for compatibility.

Revision history for this message
Olivier (olivier-lacroix) wrote :

I have this problem on an up to date intrepid with KDE4. Rather annoying issue....

Revision history for this message
Kenobi (kazmirzak) wrote :

This bug does NOT affect XUbuntu 8.04. Xubuntu is based on hal, too.

Can someone please check the XUbuntu hal settings and compare them to the KUbuntu hal settings? I think this way the problem should be solvable.

Revision history for this message
Andrew (aifiltr0) wrote :

> This bug does NOT affect XUbuntu 8.04. Xubuntu is based on hal, too.

And it doen't affect ubuntu and kubuntu-kde3.
AFAIK xubuntu uses exo-utils to mount storage and ubuntu - gnome volumemanager, or such. This is not the matter of hal, but the way kde4 mounts the storage and the bug should be searched in the kde source.

Revision history for this message
Alexander Frolushkin (hokum) wrote :

Still appears in recently released kde 4.1

Revision history for this message
Harald Sitter (apachelogger) wrote :

Moving to kdebase (new source package name in Intrepid)

Changed in kdebase-kde4:
importance: Undecided → Medium
Revision history for this message
Kurt J. Bosch (kujub-deactivatedaccount) wrote :

Me too on Hardy with KDE 4.1 (USB-Stick with german Umlauts which works normal when using pmount).

Even tried KDE 4.1 on ArchLinux (which is pretty vanilla) - its the same there.

So this seems to be a bug from upstream.

Revision history for this message
Kurt J. Bosch (kujub-deactivatedaccount) wrote :

Added URL for upstream bug. (There is a quick hack included for those willing to patch and build from source, but I didn't try it.)

Please vote there ! Hopefully it helps ;-)

Changed in kdebase:
status: Unknown → Confirmed
Revision history for this message
Garbor (garbor) wrote :

This is for me an important bug.

It stopped me from using KDE, and sent me back to my Gnome Ubuntu desktop, please fix it soon.

Revision history for this message
Kurt J. Bosch (kujub-deactivatedaccount) wrote :

Attached proposed patch from upstream bug report.

Seems to works good here on ArchLinux KDE 4.1.0. :-)

Changed in kdebase-kde4:
status: New → Invalid
Revision history for this message
Olivier (olivier-lacroix) wrote :

There is a non-hackish patch posted upstream. Could that patch be included in kubuntu builds ? That would be great.

Changed in kdebase:
status: Confirmed → Triaged
Revision history for this message
Vítor E. Silva Souza (vitorsouza) wrote :

Would anyone be so kind as to provide more detailed description on how to patch the system so we can still use KDE 4.1 while waiting for the definitive solution to come from the developers?

The patches (mount-utf8.patch and solid.patch at the KDE-bugs page) change a file named halstorageaccess.cpp, but I don't have that file in my file system. Do I have to install a src package? Which one? Do I have to recompile something? What? How? Just make; make install or is there more to it?

Sorry if I'm the only noob that doesn't understand what to do with the proposed patches. :)

Thanks!

Changed in kdebase:
status: Confirmed → Invalid
Revision history for this message
kws (kenneth.westelinck) wrote :

It seems that this it is still not solved in KDE 4.1.2, available for Hardy. As indicated earlier in this thread, using pmount works. For your interest, below are some relevant lines taken from /etc/mtab when using pmount and Dolhpin:

pmount:
-----------
/dev/sdd /media/sdd vfat rw,noexec,nosuid,nodev,quiet,shortname=mixed,uid=1002,gid=513,umask=077,fmask=0177,dmask=0077 0 0

dolphin:
----------
/dev/sdd /media/music vfat rw,nosuid,nodev,uhelper=hal,uid=1002,codepage=437,iocharset=utf8 0 0

Revision history for this message
Kenobi (kazmirzak) wrote :

KDE is dead as long as this problem is not fixed. It is unbelievable for me that the last KDE update didn't include that fix - its like selling cars where the doors won't lock. OK, it is not the main function of the car - but who's going to buy it?

No one. Who is responsible?

Revision history for this message
Harald Sitter (apachelogger) wrote :

Kenobi, this is a bug tracking system, not a blame tracking system. Stop whining and fix it yourself if you want this fixed. And for the car comparison: we are providing this car for free, we provide the construction plans for free, we even provide the support for free, and the majority of does it in their spare time.
Just so you know, I removed this bug from my todo list. Next time you should think about how appropriate your comment is, before posting it. Thank you.

Revision history for this message
Kenobi (kazmirzak) wrote :

in the transition from KDE 3.5 to KDE 4 someone made a little mistake. That's a fact. Maybe it is just a handful of lines that is missing. He did it in his freetime, and I appreciate all the work he put into KDE 4. But working on KDE 4 is not fun, it is a job where you have responsibilities, too. The community depends on him. Millions of hours have been put into KDE 4 by hundreds of people.

I don't say that the problem has to be fixed until next week or next month. But I think it is just fair for all the other developers who work hard to make KDE4 more popular through their programs, that the respective specialist does everything in his powers to track down the bug, and communicates to the others (WORK IN PROGRESS) or (JUST MARRIED, BUT NEXT MONTH I WILL).

If I could fix it, I would. But I'm not a specialist. If the developers don't take this responsibility, because they think that working for KDE is just fun, then KDE has no future. I would regret that.

This comment is meant to be productive criticism. Responsibility needs to be taken.

Revision history for this message
Vítor E. Silva Souza (vitorsouza) wrote :

Just so you know, this is fixed in Intrepid (Kubuntu 8.10) beta. At least it works for me here and I did no patching (since I didn't know what to do - see my previous comment). I just kept updating the system until finally one day I gave it a shot again and it worked.

Revision history for this message
Kenobi (kazmirzak) wrote :

Hey, Vítor Souza!

Thanks for these great news. I'm going to check this as soon as I can get the beta (will have to wait due to slow internet connection).

Can anyone else confirm this? If there are two more confirmations or so than this bug can finally be marked as fixed.

Revision history for this message
Marc Ermshaus (mermshaus) wrote :

It works for me in the same way Vítor described. Take a look at the corresponding discussions in the KDE bug tracking system (http://bugs.kde.org/show_bug.cgi?id=161588, http://bugs.kde.org/show_bug.cgi?id=161673). This "bug" seems to be fixed, at least partially, in KDE 4.1.2.

Revision history for this message
kws (kenneth.westelinck) wrote :

I see. That's good to hear. But does it also work for USB drives with vfat file systems where folders were created on a Windows machine (e.g. Jérome) and later accessed on a kubuntu machine? The problem is that dolphin apparently always mounts the file system with iocharset encoding, whereas pmount respects the encoding of the system (i.e. what is configured in /etc/default/locale).

Revision history for this message
Vítor E. Silva Souza (vitorsouza) wrote :

I put a 128 MB USB pen drive in a notebook with Windows Vista, asked to format it using FAT and then created a folder called "Avião Rápido". It was the first Portuguese expression that I could think of that included a tilde, a blank space and an accent in the name. :)

Removed it, plugged in my notebook with Kubuntu Intrepid (8.10) beta and the Devide Notifier popped up. I cliked in the devide and it opened Dolphin, showing the folder correctly, no weird characters. No problem at all...

Revision history for this message
Vítor E. Silva Souza (vitorsouza) wrote :

Sorry for the typo:

devide -> device

Since it happened twice, I can only conclude that there is a bug in my brain now :)

Revision history for this message
kws (kenneth.westelinck) wrote :

Cool. I've downloaded the latest kubuntu 8.10 beta and I will try this on VMWare (probably tomorrow or so). To be continued.

Revision history for this message
kws (kenneth.westelinck) wrote :

Just an update. I downloaded kubuntu untrepid 8.10 and installed it in VMWare. After upgrading and dist-upgrading everything seems to work. I have tested this with my Trekstor Vibez MP3 player containing a folder "Café del mar" and it does not show any strange characters in Dolphin. This is good!
I was just wondering if this could be solved in 8.04 as well, as this is still my main OS on the desktop and I have no intentions (yet :) ) to upgrade to 8.10.

Changed in kdebase:
status: Invalid → Unknown
Revision history for this message
Jonathan Thomas (echidnaman) wrote :

Probably not. The fixes were made to a component that didn't exist back in KDE3. :(
You could always install dolphin-kde4 from the Kubuntu KDE4 ppa for use in Hardy. That way you don't have to install all of KDE4 and you don't have to upgrade to Intrepid.

Anyway, closing as fixed since a fix has been made for Intrepid.

Changed in kdebase:
status: Triaged → Fix Released
Revision history for this message
kws (kenneth.westelinck) wrote :

I _am_ running KDE 4 on Hardy. And I am not running any KDE 3 applications, unless they don't exist for KDE 4 yet. And I _am_ following the debs provided by ppa. So why doesn't this work then.

Changed in kdebase:
status: Unknown → Fix Released
Changed in kdebase:
importance: Unknown → Medium
To post a comment you must log in.
This report contains Public information  
Everyone can see this information.

Duplicates of this bug

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.