Set $LANGUAGE if the user picks a different locale in gdm, so that language-selector and gdm stop disagreeing

Tried Beta 2 on April 9th. It becomes more interesting.

reproduce steps.

1. enable simplified Chinese. reboot. The localization on menu is good.
2. enable Thai. reboot. The menu is mixed in Thai and in Simplified Chinese. Screen shot and locale are attached.

This bug was fixed in the package language-selector - 0.5.5

---------------
language-selector (0.5.5) lucid; urgency=low

  * QtLanguageSelector.py: fix crash when no language is selected
    in the install window (LP: #553729)
  * LanguageSelector.py: if gdm is running, write the user LANG setting
    to ~/.dmrc and /var/cache/gdm/$USER/dmrc (LP: #553162)
  * updated translations from launchpad
  * remove dangling ImSwitch symlinks on startup (LP: #500594)
  * check for write permission on ~/.profile (LP: #560881)
  * check for read permission on /etc/default/locale and
    /etc/environment (LP: #554617)
 -- Arne Goetje <email address hidden> Thu, 15 Apr 2010 23:41:40 +0800

Issue still exists in 2010-04-18 image.

Reproduce steps.

1. Download daily build from cdimage on 2010-04-18
2. Fresh install Lucid with Japanese as as default language and with no Internet connection during installation.
3. reboot system after installation.
4. The language of menu is in English which should be Japanese. please see Screenshot in the attachment.

5. Connect internet and install the uninstalled language packages. Please see screenshot-1 in th attachment.
6. reboot system after installation. The language of menu is in Japanese.

6. I wan to use English UI. Therefore I move English (United States), then reboot the system. After system the rebooted, the language in menu is still in Japanese. Please see screenshot-2 with locale info.

7. In Language Support, move "English" after "English (United States)", then reboot the system. After system the rebooted, the language in menu is in English. Also, while system reboot, system pop up an window asks user if user wants to change the folder name (Documents, music......)

After those steps, can you please copy&paste the output of

  echo $LANG
  echo $LANGUAGE
  cat ~/.dmrc
  cat /var/cache/gdm/$USER/dmrc

Thanks!

reproduce with Lucid-RC

1. Fresh install Lucid with Japanese as as default language and with no Internet connection during installation.
2. reboot and connect to internet. the language of menu on the top panel is in English which should be in Japanese.
Screenshot: locale and other information provided.
3. System pops up the language installation is incomplete. Download, install the language pack.
Screenshot-1:
4. Reboot the system. the language of menu on the top panel is in JAPANESE.
screenshot-2: locale and other information provided.

5. Install Traditional Chinese Language pack, move Chinese (Taiwan) to the first one in the language options
6. Reboot the system. the language of menu on the top panel is in Traditional Chinese, EXCEPT Firefox browser
 is in Japanese screenshot-3

One new bug here. originally, I saved the screenshot file on the desktop. I moved the file on the screen, it suddenly disappeared. I checked the desktop folder, the screenshot files are still there.
screenshot-4
screenshot-5: locale and other information provided.

6. move English (United States) to the first one in the language options.
Reboot the system. the language of menu on the top panel is still in Traditional Chinese, EXCEPT Firefox browser is in Japanese. Strange, all the files in desktop folder are showed again.
screenshot-6
screenshot-7: locale and other information provided.

7. reboot one more time. no change. move move English after English (United States) in the language options
screenshot-8

8. reboot system. the language of menu on the top panel is in English. however the folder name is in Traditional chinese. Usually system shoudl pop up a screen which ask users if they want to keep the old name, but no this time.

screenshot-9: locale and other information provided.

Thanks for the detailed info Kevin.

El dv 23 de 04 de 2010 a les 08:16 +0000, en/na Kevin Huang va escriure:
> reproduce with Lucid-RC
>
> 1. Fresh install Lucid with Japanese as as default language and with no Internet connection during installation.
> 2. reboot and connect to internet. the language of menu on the top panel is in English which should be in Japanese.
> Screenshot: locale and other information provided.

This is expected behaviour. Due to space constraints, we can only fit a
few language packs on the CD, and Japanese is one of them.

These can be downloaded post-installation (well, or automatically during
the installation, if there is network and Internet connectivity), which
is why we show the notification message below.

> 3. System pops up the language installation is incomplete. Download, install the language pack.
> 4. Reboot the system. the language of menu on the top panel is in JAPANESE.

Yes. Up until here this is expected behaviour. See
http://testcases.qa.ubuntu.com/Install/NonEnglishLanguage#Installation%
20No%20Network

>
> 5. Install Traditional Chinese Language pack, move Chinese (Taiwan) to the first one in the language options
> 6. Reboot the system. the language of menu on the top panel is in Traditional Chinese, EXCEPT Firefox browser
> is in Japanese screenshot-3
>
> One new bug here. originally, I saved the screenshot file on the desktop. I moved the file on the screen, it suddenly disappeared. I checked the desktop folder, the screenshot files are still there.
> screenshot-4
> screenshot-5: locale and other information provided.
>
> 6. move English (United States) to the first one in the language options.
> Reboot the system. the language of menu on the top panel is still in Traditional Chinese, EXCEPT Firefox browser is in Japanese. Strange, all the files in desktop folder are showed again.
> screenshot-6
> screenshot-7: locale and other information provided.
>
> 7. reboot one more time. no change. move move English after English (United States) in the language options
> screenshot-8
>
> 8. reboot system. the language of menu on the top panel is in English.
> however the folder name is in Traditional chinese. Usually system
> shoudl pop up a screen which ask users if they want to keep the old
> name, but no this time.
>

I believe all this should be related to the issue described in bug
553162 and the fact that some programs, most notably Firefox, ignore the
LANGUAGE value and use LANG instead.
--
David Planella
Ubuntu Translations Coordinator
david(dot)planella(at)ubuntu(dot)com
www.ubuntu.com

I'm experiencing this problem on 10.04 Final AMD64. Clean install.

Changing language in GDM does not affect anything except for Firefox and the Home folder names.

I am also experiencing this on 64 bit 10.04 final, upgrading from 9.10.
Lots of perl "locale" errors on the terminal while upgrading... then,
gnome-language-selector unusable (window disappeared half a second after launch), until I changed (I think) export LC_ALL=C, did sudo dpkg-reconfigure locales
and then managed to run gnome-language-selector.
Anyway, now EVERYTHING (gnome menus, calendar, firefox, bash window, ...) is in English except a handful of applications ("update manager", "about gnome") which are half-english half Spanish.
>echo $LANG
ca_ES.utf8

>echo $LANGUAGE
es_ES:ca:es:en_GB:en

>echo $GDM_LANG
ca_ES.utf8

>cat .dmrc

[Desktop]
Language=ca_ES.utf8
Layout=es

and gnome menus in English (Appliations, Places, Systems) keep appearing whatever language I choose in the login screen.
This is a show-stopper for people which does not speak English (my father is happy with ubuntu 9.10 but I don't dare to upgrade should he lose Spanish support): an operating system which does not speak your language is almost as useless as one that hangs.

Please don't interpret my last sentence despectively... I was thinking in "ubuntu for the masses" and what a not-so-cultivated non-English user would think of Ubuntu's reputation if this bug hits him. My experience with Ubuntu 9.10 was great.

This was just discussed in #ubuntu-desktop. The basic problem here is that language-selector now is able to set system-wide locale settings which gdm does not understand. gdm is only able to set a locale (i. e. $LANG), while language-selector can set a locale and additionally a language list (which overrides LC_MESSAGES from $LANG) in $LANGUAGE.

I think a reasonable compromise would be:

 * As long as the locale picked in gdm matches the system locale, keep $LANGUAGE, since it's explicit configuration that the admin did for this machine, and we need to respect it. (cf. bug 407300)

 * Once the user selects a different locale, $LANGUAGES should be unset. This can be one of:

    - $LANGUAGES is set system-wide, and $GDM_LANG != the system wide locale
    - $LANGUAGES is set in ~/.profile, and $GDM_LANG != $LANG set from ~/.profile or ~/.dmrc

This needs to be done in /etc/gdm/Xsession.

This bug has yet another affection of i18n breakage,

 - GDM read ~/.dmrc 's "Language" value and set LANG/LANGUAGEenvironment varibale.
 - ~/.dmrc has "Language=(langname).utf8" form by language-selector in some cases.
    (such as Lucid clean install without "no touch" boot. after install, ~/.dmrc has ".utf8" values)
 - So, LANG/LANGUAGE environment varibales sets "Language=(langname).utf8" by GDM.

But, LANG/LANGUAGE environment varibales expected 'LANG="(langname).UTF-8"' style, not "(langname).utf8".

This LANG/LANGUAGE settings behavior brakes some i18n codepages related applications (such as xarchiver, file-roller with archiver that has "codepage handling").

We must recover(or mitigate) dmrc's "Language" value too.

Fumihito YOSHIDA [2010-05-11 12:51 -0000]:
> But, LANG/LANGUAGE environment varibales expected
> 'LANG="(langname).UTF-8"' style, not "(langname).utf8".

That's not true. ".utf8" is the canonical name now, and .UTF-8 is an
alias.

> This LANG/LANGUAGE settings behavior brakes some i18n codepages related
> applications (such as xarchiver, file-roller with archiver that has
> "codepage handling").

These would be bugs in those applications then, not gdm.

Thanks!

Martin
--
Martin Pitt | http://www.piware.de
Ubuntu Developer (www.ubuntu.com) | Debian Developer (www.debian.org)

hmm, I just installed German support for my girlfriend under her user. Then, logging off, changing to my user acount choosing "English" as a language (german keyboard layout) everything stays german.

clausen@clausen-ubuntu1004:~$ echo $LANG
en_GB.utf8
clausen@clausen-ubuntu1004:~$ echo $LANGUAGE
de_DE:de:en_GB:en
clausen@clausen-ubuntu1004:~$ echo $LANG
en_GB.utf8
clausen@clausen-ubuntu1004:~$ echo $LANGUAGE
de_DE:de:en_GB:en
clausen@clausen-ubuntu1004:~$ cat ~/.dmrc

[Desktop]
Language=en_GB.utf8
Layout=de
clausen@clausen-ubuntu1004:~$ cat /var/cache/gdm/$USER/dmrc

[Desktop]
Language=en_GB.utf8
Layout=de

> That's not true. ".utf8" is the canonical name now, and .UTF-8 is an
> alias.
>
>> This LANG/LANGUAGE settings behavior brakes some i18n codepages related
>> applications (such as xarchiver, file-roller with archiver that has
>> "codepage handling").
>
> These would be bugs in those applications then, not gdm.

We have a similar problem at Opera with Ubuntu users, they are filing reports with us that their locale doesn't work if they use gdm.

All these applications - including Opera - call standard X11 functions to set X's locale (see XSupportsLocale(3)). The problem is that some locales in the *.utf8 form are not supported by X11 as distributed with Ubuntu, but their *.UTF-8 counterparts are. An example of this would be bg_BG.utf8, which is one of the locales that gdm will set.

This can be fixed in two ways: gdm should use *.UTF-8 instead (as kdm does on Kubuntu), or X11 should be fixed to support the various *.utf8 locales that are missing (this can be done by adding the locale to X11's locale.alias file).

I don't have an exhaustive list of locales that don't work - I only know about bg_BG.utf8 and ru_UA.utf8 from bug reports by our users - but in the specific case of bg_BG.utf8 it's worth noting that that entry is in fact present in locale.alias on Ubuntu, but misspelled as be_BG.utf8. There is no entry for ru_UA.utf8.

The bug has been assigned to the canonical desktop team for a while without change, pitti do you know want to work on this when you have a free slot or should be just unassign the bug for now?

Without change means no progress.

Kevin Huang, Canonical

On 2010/8/3, at 19:42, Sebastien Bacher <email address hidden> wrote:

> The bug has been assigned to the canonical desktop team for a while
> without change, pitti do you know want to work on this when you have a
> free slot or should be just unassign the bug for now?
>
> ** Changed in: gdm (Ubuntu)
> Assignee: Canonical Desktop Team (canonical-desktop-team) => Martin Pitt (pitti)
>
> --
> Unset $LANGUAGES if the user picks a different locale in gdm, so that language-selector and gdm stop disagreeing
> https://bugs.launchpad.net/bugs/553162
> You received this bug notification because you are a direct subscriber
> of the bug.

Just for the record, this seems to affect Kubuntu as well. I've had at least two reports on IRC from people with different languages in LANGUAGE and LANG, with a half translated system (one in one language and one in the other).

There are also some people who install Ubuntu in English and then simply change the the language, and are affected by this. The first place where they notice is on Firefox, which does not follow LANGUAGE (bug 539761) and then appears in English.

Due to the number of duplicates, the ease to reproduce it and the usability impact, would it be possible to raise the importance of this one?

Thanks!

As per

https://wiki.ubuntu.com/Bugs/Importance

"It has a moderate impact on a large portion of (all non english) Ubuntu users"

As per

https://wiki.ubuntu.com/Bugs/Importance

"It has a moderate impact on a large portion of (all non english) Ubuntu users"

raising the importance.

As per

https://wiki.ubuntu.com/Bugs/Importance

"It has a moderate impact on a large portion of (all non english) Ubuntu users"

I raise the importance of the bug.

I believe that the problems that are addressed in this bug mainly fall into either or both of two categories:

1. - The LANGUAGE and LANG env. variables are often mixed up. LANGUAGE is meant to control the display language, while LANG is meant to control other locale aspects such as date formats etc.

2. - Changes in language-selector of LANGUAGE get overridden by gdm.

I have uploaded two branches to Launchpad and proposed that they are merged into lp:ubuntu/gdm respektive lp:ubuntu/lucid-updates/language-selector. Related patches that take care of both problem categories have been attached to this bug.

We also have the Firefox related problem, described both here and in LP: #550222. I added a patch to the latter that makes Firefox behave like one would expect.

Actually, with these three patches implemented I can seamlessly switch languages and other locales back and forth without them interfering with each other. Hopefully those of you who test the patches will make the same experience.

Can't see how unsetting $LANGUAGE would not create new problems, so I changed the summary.

Gunnar,

Making up a $LANGUAGE from $LANG is hard to impossible. I think we should just unset it, as per comment 10. Also, the gdm script should not _ever_ modify files aside from ~/.wmrc, in particular not the user's profile. The current one doesn't take into account different syntax variants for setting and exporting the variable, and it can't do anything about setting it from /etc/default/locale anyway.

@Martin
Well, I don't consider it to be a problem, as you say in comment #10, that language-selector distinguishes between language and other locale aspects. That's just as it should be IMO. As far as I understand, the compromise you currently advocate would mean that Ubuntu's localisation model would keep being inconsistent, and an annoyance for e.g. people who prefer German locale settings while messages are displayed in English. Not to mention people who often switch language and/or locale settings...

We need to take into account how the different packages work together. language-selector (via LanguageSelector.py) modifies both ~/.profile and /etc/default/locale already.

I really think that this issue deserves a thorough discussion. To start with, it would be great if you could test the two debdiffs I attached to this bug report. Until somebody has proved me wrong, I believe that it is possible, and that I have figured out a good way to make gdm and language-selector work seamlessly together. :)

Btw, if you want to see that it also can work with Firefox, please see the solution I provide in bug 550222.

I don't see why it should be inconsistent. If you pick a new language in gdm, then presumably you actually want to use this language (why else did you do it?), so in this case we should really empty $LANGUAGE. We just need to care about leaving $LANGUAGE alone if the selected language in gdm already matches the one that the user has in his profile, in other words, when he did not change it.

for example user's LANGUAGE is set zh_CN:zh_TW:zh_HK:en_US:en_GB

and in gdm he/her set to en_GB

can we do it like this:

gdm set LANGUAGE to

en_GB:zh_CN:zh_TW:zh_HK:en_US

or even

en_GB:zh_CN:zh_TW:zh_HK:en_US:en_GB

it is simple and can work.

@ Martin
language-selector sets $LANGUAGE for language, and $LANG for other locales. That's why it's inconsistent that gdm sometimes uses $LANG when dealing with the language to be used for message display.

I have attached a modified gdm-patch. These are the news:

- As you pointed out in comment #24, the previous code didn't take into account different syntax variants for setting and exporting $LANGUAGE. Now the code for editing ~/.profile is safer.

- The $LANGUAGE priority list is preserved, as was suggested in comment #27 by YunQiang Su.

- $LC_MESSAGES is set to take care of applications that don't recognize $LANGUAGE.

The third item seems to solve reported problems with Mozilla apps (Firefox/Thunderbird), which means that the solution I provided at bug 550222 already can be regarded as superseded. Also other programs that apparently don't recognize $LANGUAGE, e.g. FileZilla and Cervisia, now display menues etc. in the expected language.

Personally I think we should give high priority to this issue, and try to attain consensus about a solution. I'd be happy to contribute to achieve that goal ( seems like I have started... :) ). I noticed at ubuntu-devel that David Planella asked for suggestions for translation focus aspects with an eye to 11.04. Wouldn't a consensus solution to this issue be a suitable item on the list that David is preparing at https://wiki.ubuntu.com/Translations/Roadmaps/11.04 ?

Hi Gunnar,

Thanks for your work on this. I'll let Martin comment on the technical
aspects and add only a couple of comments.

El dl 04 de 10 de 2010 a les 08:38 +0000, en/na Gunnar Hjalmarsson va
escriure:
> @ Martin
> language-selector sets $LANGUAGE for language, and $LANG for other
> locales.

Language selector sets $LANGUAGE to use the possibility of setting
fallback languages. I haven't looked at the code to see if it sets $LANG
as well or it leaves it to gdm to do it, but it does allow setting the
LC_* categories individually.

> That's why it's inconsistent that gdm sometimes uses $LANG when
> dealing with the language to be used for message display.
>

GDM does not currently write $LANGUAGE.

> I have attached a modified gdm-patch. These are the news:
>
> - As you pointed out in comment #24, the previous code didn't take into
> account different syntax variants for setting and exporting $LANGUAGE.
> Now the code for editing ~/.profile is safer.
>
> - The $LANGUAGE priority list is preserved, as was suggested in comment
> #27 by YunQiang Su.
>
> - $LC_MESSAGES is set to take care of applications that don't recognize
> $LANGUAGE.
>

For the messages' locale, after evaluating LANGUAGE, the order of
evaluation in setlocale() is LC_ALL, LC_MESSAGES, LANG. So setting LANG
as gdm and language-selector do, should be enough. The problems I've
observed in Firefox, OpenOffice.org and other applications were related
to the fact that they did not support reading LANGUAGE. Were they not
reading LANG, either?

Rather than using a workaround, would it not be better to fix the bug in
the affected applications? I see you've followed up on Firefox already.

> The third item seems to solve reported problems with Mozilla apps
> (Firefox/Thunderbird), which means that the solution I provided at bug
> 550222 already can be regarded as superseded. Also other programs that
> apparently don't recognize $LANGUAGE, e.g. FileZilla and Cervisia, now
> display menues etc. in the expected language.
>
> Personally I think we should give high priority to this issue, and try
> to attain consensus about a solution. I'd be happy to contribute to
> achieve that goal ( seems like I have started... :) ). I noticed at
> ubuntu-devel that David Planella asked for suggestions for translation
> focus aspects with an eye to 11.04. Wouldn't a consensus solution to
> this issue be a suitable item on the list that David is preparing at
> https://wiki.ubuntu.com/Translations/Roadmaps/11.04 ?
>

As I noted on the follow up e-mail and on the blog post, that roadmap is
for translations community plans. It is not thought as a feature list
for Launchpad or a list of bugs to fix.

However, you should feel free to continue your work on this and address
the maintainer's comments. Keep up the good work!

Regards,
David.
--
David Planella
Ubuntu Translations Coordinator
www.ubuntu.com / www.davidplanella.wordpress.com
www.identi.ca/dplanella / www.twitter.com/dplanella

@ David
Thanks for your praising comment.

It seems to me that you, and possibly also Martin, assume that $LANG ought to correspond to the first language in the $LANGUAGE list. That, I believe, is a misconception, at least from a language-selector perspective, and I fear that we have talked at cross-purposes so far.

In an attempt to help changing that, I wrote a wiki-page:
https://wiki.ubuntu.com/GdmLanguageSelectorDissonance

It summarizes my view on where we stand, where we want to go ("Desired behavior"), and my proposed solution in English. It would be great if you, Martin and anybody else with an interest in this issue could study the page and add your possible objections and comments.

Basically, what I hope to convince you about is that since it's only language that the users can set from GDM, and not any other locale aspects, GDM should not do anything with the $LANG variable. Not at any point. Never.
Instead GDM should update $LANGUAGE and set $LC_MESSAGES. That appears to be sufficient to reliably control which language is used in menus and messages. $LANG is used for the non-translation locale aspects, and its value shall not be used as a fall back by programs that don't understand $LANGUAGE.

As regards apps that don't understand $LANGUAGE:
> Rather than using a workaround, would it not be better to fix the
> bug in the affected applications? I see you've followed up on
> Firefox already.

If I have understood it correctly, $LANGUAGE is a gettext specific variable, and certain applications' inability to understand that variable is not a bug per se. Consequently I think that Ubuntu in any case shall provide the preferred translation language also via the $LC_MESSAGES variable, and I don't think that's a workaround. Rather should the Firefox fix I provide at bug 550222 be considered a workaround IMO.

I noted that you don't consider this issue a matter for the translation community. But it's a borderline case, isn't it? ;-)

Hello Gunnar,

Gunnar Hjalmarsson [2010-10-04 8:38 -0000]:
> language-selector sets $LANGUAGE for language, and $LANG for other
> locales. That's why it's inconsistent that gdm sometimes uses $LANG
> when dealing with the language to be used for message display.

gdm doesn't have such an elaborate locale/language configuration as
the language-selector, it just as a simple locale selector. It simply
doesn't support setting a list of language fallbacks for $LANGUAGE.
It's not supposed to either, the main point is that you can start a
desktop session in a different language. You can then fine-tune it
with language-selector.

Thus gdm is not really inconsistent, it just doesn't have feature
parity with language-selector.

Therefore it should not try to change your $LANGUAGE settings either.
It should just clear it when it changes $LANG, so that $LANG actually
becomes effective.

> - As you pointed out in comment #24, the previous code didn't take into
> account different syntax variants for setting and exporting $LANGUAGE.
> Now the code for editing ~/.profile is safer.

No, please let's not. gdm is _not_ a tool for setting $LANGUAGES.
That's what language-selector is for.

> - $LC_MESSAGES is set to take care of applications that don't recognize
> $LANGUAGE.

That should be fixed in language-selector, not gdm. gdm already sets
$LANG, which firefox & friends do recognize.

Thanks!

Martin
--
Martin Pitt | http://www.piware.de
Ubuntu Developer (www.ubuntu.com) | Debian Developer (www.debian.org)

Gunnar Hjalmarsson [2010-10-05 4:13 -0000]:
> It seems to me that you, and possibly also Martin, assume that $LANG
> ought to correspond to the first language in the $LANGUAGE list.

No, it shouldn't. I wasn't saying that, I was just saying that, since
gdm doesn't have a way to configure $LANGUAGES, there is no point in
faking $LANGUAGES, _because_ $LANG and $LANGUAGES are independent.

> Basically, what I hope to convince you about is that since it's only
> language that the users can set from GDM, and not any other locale
> aspects

No, it's exactly the other way around. gdm has a locale selector, but
not a language list selector.

> If I have understood it correctly, $LANGUAGE is a gettext specific
> variable, and certain applications' inability to understand that
> variable is not a bug per se.

That's correct, $LANGUAGE is a GNU extension.

> Consequently I think that Ubuntu in any case shall provide the
> preferred translation language also via the $LC_MESSAGES variable,
> and I don't think that's a workaround.

I agree, this should indeed be fixed in language-selector.

Thanks,

Martin
--
Martin Pitt | http://www.piware.de
Ubuntu Developer (www.ubuntu.com) | Debian Developer (www.debian.org)

Martin,

On 2010-10-07 11:11, Martin Pitt wrote:
> [gdm] just has a simple locale selector ... the main point is that
> you can start a desktop session in a different language.

That's crystal clear to me. And as long as language-selector didn't distinguish between language and other locale aspects, the natural way to serve that purpose was to let GDM set $LANG. There was no reason to do it otherwise.

As from Lucid (revision 79 in language-selector, I think) language-selector works differently, and currently it does not play well with GDM. Making GDM and language-selector work well together in this respect, i.e. preventing unexpected behavior and user confusion, is the focus I have. Please note that I suggest changes to both language-selector and GDM.

Keeping the limited purpose of GDM's locale selector in mind, i.e. allowing users to start a session in a different language, I believe that _now_ the best way to serve that purpose is to update $LANGUAGE and set $LC_MESSAGES, while leaving $LANG to language-selector. In short: Keep the purpose, change the method.

So, why do I persist, and claim that my method is better? As far as I can tell, the patches I wrote make language-selector and GDM work seamlessly together, without any cause left for users to get confused. At the same time I can't see how going the other way, and just unset $LANGUAGE, would be sufficient to prevent undesired user surprises.

In an attempt to make my point I added this code to Xsession:

    if [ $( echo $GDM_LANG | sed 's/\..*//' ) != \
         $( echo $LANGUAGE | sed 's/:.*//' ) ]; then
        unset LANGUAGE
    fi

Then I made sure that everything was set to Swedish to start with. Now, please consider this scenario:

* I decide to switch the language (and nothing else) to en_US, and do so from GDM.

* While now seeing English in menus and messages, I also see those ambigous American date formats etc., which I don't want.

* Without actually understanding what happened, I go to Language Support and open the tab for non-language locale settings. To my surprise, it appears as if Swedish is still selected. Weird! Maybe a temporary glitch...?

* I restart the computer. No change.

* This time, when in the Language Support tab for non-language locale settings, I change (from Swedish) to something else, and then back to Swedish. (I happen to know by now that that makes a difference, but does the average user know? Probably not.)

* Restart.

* Now Swedish is the pre-selected _language_ in GDM. Seems like we are back at square one. :(

This is certainly not what I want, and I'm sure that you do not want it like that either.

So, what else can I say? Suppose most that can be said have been said by now, so I simply beg you to reconsider and give my patches a chance. If I'm proved wrong, by you or somebody else, I promise that I'll shut my mouth. But only then. ;-)

Hello Gunnar,

Gunnar Hjalmarsson [2010-10-07 22:34 -0000]:
> Keeping the limited purpose of GDM's locale selector in mind, i.e.
> allowing users to start a session in a different language, I believe
> that _now_ the best way to serve that purpose is to update $LANGUAGE and
> set $LC_MESSAGES, while leaving $LANG to language-selector.

If gdm's UI becomes a pure language selector, I agree. This requires
some more intrusive code changes, though, and some thought about
handling country specific languages like Portugese from Portugal vs.
Brazil, or simplified/traditional Chinese. This should be
coordinated/discussed in an upstream bug, since this is going to be a
fairly large patch.

> So, why do I persist, and claim that my method is better? As far as I
> can tell, the patches I wrote make language-selector and GDM work
> seamlessly together, without any cause left for users to get confused.

Your gdm patch just changes how the result from the locale selector is
handled, but it doesn't change the locale selector to become a
language selector. So if I'm choosing "German (Switzerland)" on an
en_US-by-default box, I'd get German strings, but time/currency/etc
format would still be US. This would be by design with your idea (set
LANGUAGE=de and keep LANG=en_US), but gdm shouldn't offer me three
different country options for Germany, since they won't make a
difference.

> At the same time I can't see how going the other way, and just unset
> $LANGUAGE, would be sufficient to prevent undesired user surprises.

In my scenario above, unsetting $LANGUAGE when I change LANG away from
the default en_US, would ensure that I actually get a fully German
desktop.

> * I decide to switch the language (and nothing else) to en_US, and do so
> from GDM.

You can't right now. en_US is not a language, it's a locale.

> * While now seeing English in menus and messages, I also see those
> ambigous American date formats etc., which I don't want.

Right, I understand (see above). That's exactly my point above, gdm
shouldn't offer a country selection if it stops being able to set
country specific settings.

> So, what else can I say? Suppose most that can be said have been said by
> now, so I simply beg you to reconsider and give my patches a chance. If
> I'm proved wrong, by you or somebody else, I promise that I'll shut my
> mouth. But only then. ;-)

Oh, please don't misunderstand me. I don't think your idea is wrong,
it just changes some use cases. However, if we want to implement it,
it should be done properly, not just halfway (and thus become even
more inconsistent).

Thanks,

Martin
--
Martin Pitt | http://www.piware.de
Ubuntu Developer (www.ubuntu.com) | Debian Developer (www.debian.org)

Hi again, Martin!

On 2010-10-08 13:08, Martin Pitt wrote:
> Gunnar Hjalmarsson [2010-10-07 22:34 -0000]:
>> Keeping the limited purpose of GDM's locale selector in mind, i.e.
>> allowing users to start a session in a different language, I believe
>> that _now_ the best way to serve that purpose is to update $LANGUAGE and
>> set $LC_MESSAGES, while leaving $LANG to language-selector.
>
> If gdm's UI becomes a pure language selector, I agree. This requires
> some more intrusive code changes, though, and some thought about
> handling country specific languages like Portugese from Portugal vs.
> Brazil, or simplified/traditional Chinese.

I thought that GDM was already prepared for handling region specific languages (see below).

> Your gdm patch just changes how the result from the locale selector is
> handled, but it doesn't change the locale selector to become a
> language selector. So if I'm choosing "German (Switzerland)" on an
> en_US-by-default box, I'd get German strings, but time/currency/etc
> format would still be US. This would be by design with your idea

So far I'm with you (I think...).

> (set LANGUAGE=de and keep LANG=en_US),

Not true as regards LANGUAGE. If you would choose "German (Switzerland)", my patch would make de_CH, i.e. including the country code, become the first language/country combination in the LANGUAGE list. (It would also set LC_MESSAGES to "de_CH.utf8".)

Please note that the LANGUAGE list, that is currently maintained from language-selector only, includes country codes when available. Consequently the code I patched does so too.

> but gdm shouldn't offer me three different country options for Germany,

Why not? Ok, it should offer you six different country options, because that's the available number. ;-)

> since they won't make a difference.

They will (se above). As regards translated strings, that is. Not as regards other locale aspects.

While I agree that GDM's UI should be reconsidered if a change along these lines is implemented, I'm not convinced, at least not yet, that there is actually a need to change it.

>> * While now seeing English in menus and messages, I also see those
>> ambigous American date formats etc., which I don't want.
>
> Right, I understand (see above). That's exactly my point above, gdm
> shouldn't offer a country selection if it stops being able to set
> country specific settings.

Well, with my patch it _would_ be able to set a country/region specific language. Personally I think that the GDM UI explains it: The label is "Language", and the list includes region specific options.

Or do you mean that users are currently led to believe that GDM changes _all_ locale settings? Personally I don't think they are, possibly with the exception of experienced Ubuntu users, who may remember how it used to work...

> Oh, please don't misunderstand me. I don't think your idea is wrong,
> it just changes some use cases. However, if we want to implement it,
> it should be done properly, not just halfway (and thus become even
> more inconsistent).

Absolutely.

Let me say that I like the direction this conversation now has taken, and I'd be happy to keep participating in the process. As a n...

I agree with Gunnar Hjalmarsson, if a user changes the language in GDM, it should only change the interface language(set LANGUAGE, LC_MESSAGE variable), not ALL locale seetings.

As a Taiwan user, the default display language after Ubuntu installed is Traditional Chinese, but I prefer my interface message displays in English and keep the currency, datetime, formate, input method is zh_TW.

To have language-selector and location-selector option in GDM would be better.

I believe this requirement is also needed by foreigns live in Taiwan. (who may are married with Taiwanese or are studying in Taiwan school.)

I can't understand how the developers or testers of the language selector have missed the very basic test scenario of choosing different languages for text and numbert etc. and then starting the most commonly used application: Firefox.

Some additional thoughts.

Irrespective of which changes we agree upon at last, considering the high importance of this issue they ought to be implemented, and people given sufficient time to test them, in good time for the Natty release. To make testing easier, I created ppa:gunnarhj/locale-test and uploaded versions of the gdm and language-selector packages that include the code changes I suggest. It means that we now can test the solution by just installing the binary files (provided that we upgrade to Ubuntu 10.10). Example:

    sudo dpkg -i language-selector_0.6.6localefix1_all.deb

Martin, you mentioned an upstream bug, because of the assumed size of the GDM patch. As regards a significant GDM UI change, I still don't believe it will be necessary. I believe that we are just about to turn GDM's locale selector into a country/region specific language selector. (I took pains to clarifying my view when I formulated the PPA changelog entries.)

Problably _some_ remaining code tweaking will prove to be motivated, and I'll be happy to help with it. But wouldn't it be better if our further discussion and testing take place concurrently?

/ Gunnar

Hmm.. Thought that Launchpad would make a link from ppa:gunnarhj/locale-test. Apparently not.

This is the address to the PPA: https://launchpad.net/~gunnarhj/+archive/locale-test

Hello Gunnar,

Gunnar Hjalmarsson [2010-10-08 13:55 -0000]:
> > but gdm shouldn't offer me three different country options for
> Germany,
>
> Why not? Ok, it should offer you six different country options, because
> that's the available number. ;-)

I mainly mentioned this because for many languages there isn't
actually much difference. E. g. for German, Launchpad/Ubuntu langpacks
just ship "de", since the variations in the different countries are
negligible and just lead to bloat or inconsistent translations. The
major exceptions are Portugese, Chinese, and British English.

So if we turn this into a pure language selector, it could/should
become much smaller. However, I agree that this is only a secondary
issue.

> Or do you mean that users are currently led to believe that GDM changes
> _all_ locale settings? Personally I don't think they are, possibly with
> the exception of experienced Ubuntu users, who may remember how it used
> to work...

It's what gdm has been so far. TBH I don't know how many people would
expect that this also changes the time/date/currency/etc. format for
them, but I don't think this expectation is unreasonable, especially
not if you can pick a country in the list.

So, after this discussion I'm not opposed to turning gdm into a pure
language selector, and ask users to use language-selector (ugh, how
confusing :) ) to setup the full locale. But if we do that, I'd like
it to be done properly, i. e. upstream, and have gdm set $GDM_LANG
without requiring extra hacks like sed 's/^en\./en_US./' (remember
that this stuff runs on every boot and costs precious boot time).

Also, changing ~/.profile in Xsession.in is still an absolute no-go
for me. gdm writes its settings to ~/.dmrc, and that's where it should
live. grepping/testing/writing ~/.profile at each boot is just wrong.
I hope I'll be able to convince you about this point, then we'd have
consensus. :-)

Thanks, and have a nice evening,

Martin

--
Martin Pitt | http://www.piware.de
Ubuntu Developer (www.ubuntu.com) | Debian Developer (www.debian.org)

El dl 11 de 10 de 2010 a les 12:35 +0000, en/na August Karlstrom va
escriure:
> I can't understand how the developers or testers of the language
> selector have missed the very basic test scenario of choosing different
> languages for text and numbert etc. and then starting the most commonly
> used application: Firefox.
>

Hi August,

In bug reports, we concentrate on fixing problems. While your input is
appreciated, it does not provide any insight on how to fix the bug and
might be better suited to a forum.

The testing scenario to reproduce this bug was not as trivial as you
were describing, and the bug was discovered early enough in the process
and reported. If it hasn't been fixed until now it's because the
solution is not trivial, either, but as you can see there has been
already very productive discussion towards a proper fix.

If you'd like to join that discussion, or help testing, please feel free
to jump in.

Thanks!

David; my input provided insight into what the end user percieve as a problem. I think to many it is not clear what this bug is all about. The testing scenario to reproduce the bug I reported (bug 626722) and which was considered to be the same as this one by Gunnar Hjalmarsson really is as trivial as I described.

I marked bug 550222 as a duplicate of this bug report. Even if that may not be all true, I hope and believe that also those issues will be resolved by the solution that is currently being discussed here.

Hi Martin,

I considered your comments, and rewrote the patches. Even if it resulted in a few more lines of code, I believe you will find that it makes sense.

As regards language-selector, it now updates an LC_MESSAGES line in ~/.profile. Given that, it was possible to put the sedding/writing stuff in GDM into a function, which _only_ will be called when the user has changed the language from the greeter.

On 2010-10-13 18:49, Martin Pitt wrote:
> ... for many languages there isn't
> actually much difference. E. g. for German, Launchpad/Ubuntu
> langpacks just ship "de", since the variations in the different
> countries are negligible and just lead to bloat or inconsistent
> translations. The major exceptions are Portugese, Chinese, and
> British English.
>
> So if we turn this into a pure language selector, it could/should
> become much smaller. However, I agree that this is only a secondary
> issue.

Didn't know that about German. Of course it would make more sense if the language lists only displayed options with some significance.

There are details involved that make things a little more difficult. For instance, since there is no 'de.utf8' locale, and if you would select 'de' without a country code, the current code assigns 'de_AT.utf8' to LC_MESSAGES. The reason is that you need to stick to available locales when you set LC_MESSAGES, or else the system produces annoying warning messages (and Austria shows up first when you sort the locales...).

> So, after this discussion I'm not opposed to turning gdm into a pure
> language selector, and ask users to use language-selector (ugh, how
> confusing :) ) to setup the full locale. But if we do that, I'd like
> it to be done properly, i. e. upstream,

I'm not sure of what you mean by upstream in this case. Had a look at Debian's Xsession file, and there seems to be more differences than similarities. Anyway, my latest merge proposals are with lp:language-selector respective lp:gdm. Please let me know what else I can do.

> and have gdm set $GDM_LANG
> without requiring extra hacks like sed 's/^en\./en_US./' (remember
> that this stuff runs on every boot and costs precious boot time).
>
> Also, changing ~/.profile in Xsession.in is still an absolute no-go
> for me. gdm writes its settings to ~/.dmrc, and that's where it
> should live. grepping/testing/writing ~/.profile at each boot is
> just wrong.

Well, most of that is no longer done at each boot, but only when the users change language in GDM. Hopefully that's sufficient to relieve you from the concern, considering that writing to disk is necessary for changes to be persistent. Or is there any other way?

Ok, there are still a couple of "seds" to be able to tell if the user changed anything, but please note that they use basic regular expressions, and don't load any heavy-weight regex engine like e.g. Perl's ditto.

> I hope I'll be able to convince you about this point, then we'd have
> consensus. :-)

I believe I addressed your concerns, but with another approach.

The latest code is available for testing (in Maverick) at https://launchpad.net/~gunnarhj/+archive/locale-test

For the case you feel it's time to 'press the button', pleas...

On 2010-10-13 18:49, Martin Pitt wrote:
> ... for German, Launchpad/Ubuntu langpacks just ship "de",
> since the variations in the different countries are negligible
> and just lead to bloat or inconsistent translations. The
> major exceptions are Portugese, Chinese, and British English.
>
> So if we turn this into a pure language selector, it could/should
> become much smaller.

Have thought some more about that. Maybe the language list should basically be the intersection of available locales and available langpacks? I wrote a quick-and-dirty Perl script based on that idea (attached), and on my box it outputs this:

de.utf8 German
en.utf8 English
en_AU.utf8 English (Australia)
en_CA.utf8 English (Canada)
en_GB.utf8 English (United Kingdom)
en_NZ.utf8 English (New Zealand)
en_US.utf8 English (United States)
sv.utf8 Swedish

LC_MESSAGES also set system-wide, i.e. in /etc/default/locale and /etc/environment

No language update from Xsession for guest sessions; system-wide language applied.

Some 'sed-ing' and 'grep-ing' replaced with built-in shell features (for efficiency purposes)

Bugfix (of this bugfix...) and some more code tweaking

Martin,

On 2010-10-15 23:29, Gunnar Hjalmarsson wrote:
> On 2010-10-13 18:49, Martin Pitt wrote:
>> So, after this discussion I'm not opposed to turning gdm into a pure
>> language selector, and ask users to use language-selector (ugh, how
>> confusing :) ) to setup the full locale. But if we do that, I'd like
>> it to be done properly, i. e. upstream,
>
> I'm not sure of what you mean by upstream in this case.

I realize now that it was GNOME and the GIT repository you were referring to. Anyway, I'll await your comments now before I take any further steps.

Rgds,
Gunnar

Hello Gunnar,

just so you know that I'm not ignoring you: UDS is happening this
week, and before I had to wrap up my OEM term. Next week is Plumber's,
but after that I promise I'll spend some time on this. So, sorry for
the delay here.

Thanks!

Martin
--
Martin Pitt | http://www.piware.de
Ubuntu Developer (www.ubuntu.com) | Debian Developer (www.debian.org)

Hi Martin,

Thanks for saying that, but it's absolutely no problem. I'm a web scripting hobbyist (Perl/CGI) who am busy learning new stuff. The impressing organisation, the tools, the insight that shell scripting involves more than 'cp' and 'mv'... So please see my posts and code tweaking in the light of that, and not as repeated attempts to nudge you. ;-)

Best regards,

Gunnar

New patch due to a discussion at bugzilla.gnome.org
https://bugzilla.gnome.org/show_bug.cgi?id=633295

Because of some reluctance upstream (GNOME) to make the suggested changes to GDM, I figured out a reasonably sensible way to make all the code changes in the Ubuntu specific language-selector package.

https://code.launchpad.net/~gunnarhj/language-selector/fix-553162

If /etc/gdm/Xsession isn't altered, I believe that language-selector is where the code best belong.

[ moving discussion from bug 655427, so more people get notified ]

On 2010-11-10 03:10, trespasser wrote:
> I agree this is a duplicate to the certain point but (please, correct
> me if I am wrong or missed something in discussion on bug 553162)
> your patch makes sure $LANG, $LANGUAGE and $LC_MESSAGES all point to
> the same language.

Ok, then I'll correct you. :-) The patch makes sure that LANGUAGE and LC_MESSAGES state the same language, so messages and menus are translated as expected. LANG is separated to handle other localization aspects.

With 10.04 language-selector introduced a separation between language for message translation and other localization stuff. However, a couple of related issues were left unsolved. I'm trying to address those issues and with that help complete the separation.

> My issue is to separate those variables as I want to see Polish
> currency (zł or PLN) but English/US decimal separator (. instead of
> ,) as well as Polish date format (2010-11-10) but English name of
> days (Wednesday instead of Środa) and Polish days order (weeks in
> Poland traditionally starts on Monday).

It sounds like you want to not only separate the variables you mention, but rather set each LC_* variable (except for LC_ALL) individually. In addition to that, you are affected by the issue that names of days and months are not actually subject to translation. To completely satisfy such a complex wishlist, you'd need to create a custom locale.

However, I believe that you can come close (except for the decimal separator) by doing the following:

* Install the patch I mentioned.

* Use language-selector for setting LANGUAGE to 'en_GB:en' (and with
  that setting LC_MESSAGES to 'en_GB.utf8').

* Use language-selector for setting LANG to 'pl_PL.utf8'.

* Edit your ~/.profile file manually and add:
  export LC_TIME="en_DK.utf8"

> Therefore, in my opinion, language-selector should be reduced to
> choosing a language only

Please note that that would be a step backwards.

> and more advanced changes should be done in
> custom locale editor (such thing should be introduced). What's more,
> standard regarding locale should be better documented and guidance
> for developers on using particular locale variables should be
> outlined with strong examples.

Personally I hope that the GUI will be enough intuitive to limit the need for comprehensive docs.

Script for updating /etc/default/locale and /etc/environment added.

Minor change in LanguageSelector/LocaleInfo.py: ~/.dmrc may be read in connection with scanning for the user LANGUAGE setting, not the user LANG setting.

Update of ~/.profile (setting of LC_MESSAGES) added to install script.

Gunnar,

finally I have some time to get back to this. I looked at your recent gdm patch, and have some questions:

 - in language_update(), why do we need the "en" special case?

 - in the second part of language_update(), why would $GDM_LANG not be a valid locale? this does some rather expensive things like locale -a and grep and iterating over all available locales. So far we trusted that $GDM_LANG was the format of a locale name, and also a valid locale (gdm already tests that); under that assumption, couldn't we simply set $LC_MESSAGES to $GDM_LANG?

 - in the new_language check at the bottom, I think we should additionally check that $GDM_LANG != $LANG; for people who already have $LANG set to $GDM_LANG (as we do in our installer by default, etc.) there is IMHO no reason to change anything in the environment or do any expensive operations.

 - in the final patch I'd like to replace some external program calls (expr, grep) with internal shell constructs for efficiency; that's just a FYI, though

Thanks!

Martin

Hi Martin,

Thanks for your comments and questions, both here and on the merge proposal!

On 2010-11-24 10:40, Martin Pitt wrote:
> I looked at your recent gdm patch,

Please note that the gdm patch, i.e. the Xsession code changes, is not part of my suggestion any longer. The reason why I still didn't remove the patch is that I wasn't sure about which route to take. My apologies if that caused some confusion.

As you know, I tried to 'sell' the idea upstream, but wasn't very successful...
https://bugzilla.gnome.org/show_bug.cgi?id=633295

As a result of that, I replaced the previously suggested Xsession changes with two new language-selector files:
/etc/profile.d/gdm-lang-unset.sh
/etc/X11/xinit/xinitrc.d/language-environment.sh

OTOH, since language_update() is now part of language-environment.sh, your questions and comments still apply.

> - in language_update(), why do we need the "en" special case?

That's an attempt to imitate what the LANGUAGE list would have looked like if the change had been made via the language tab in LanguageSelector. If a user picks 'en' in the GDM greeter, the whole LANGUAGE list should consist of nothing but 'en'.

> - in the second part of language_update(), why would $GDM_LANG not
> be a valid locale? this does some rather expensive things like
> locale -a and grep and iterating over all available locales. So far
> we trusted that $GDM_LANG was the format of a locale name, and also
> a valid locale (gdm already tests that); under that assumption,
> couldn't we simply set $LC_MESSAGES to $GDM_LANG?

I think that the reason why there is a need to ensure that LC_MESSAGES is assigned a valid locale is that ~/.dmrc and /var/cache/gdm/$USER/dmrc are now updated when LANGUAGE is set. Hence $GDM_LANG (or $GDM_LANGUAGE) may well be just 'en.utf8', 'sv.utf8' or 'de.utf8', none of which is a valid locale (at least not on my box).

As regards efficiency ("expensive things"), please note that the language_update() function is called only when the user changes the language from the GDM greeter.

> - in the new_language check at the bottom, I think we should
> additionally check that $GDM_LANG != $LANG; for people who already
> have $LANG set to $GDM_LANG (as we do in our installer by default,
> etc.) there is IMHO no reason to change anything in the environment
> or do any expensive operations.

No, no, please don't go back to mixing up LANG and LANGUAGE/LC_MESSAGES. ;-)

Assume a user with these ~/.profile settings:

export LANG="de_DE.utf8"
export LANGUAGE="en_GB:en"
export LC_MESSAGES="en_GB.utf8"

If s/he picks German (Germany) from the GDM greeter, i.e. changes the language from English to German, $GDM_LANG/$GDM_LANGUAGE would equal $LANG. But now, when we complete the separation of LANG from the translation language, that has nothing to do with it any longer, has it?

> - in the final patch I'd like to replace some external program
> calls (expr, grep) with internal shell constructs for efficiency;
> that's just a FYI, though

Ok. I did a few such replacements already, but there may well be further opportunities that I missed.

Please also see my comments on the merge proposal.

I'd like to raise a coup...

Martin,

During our talk in #ubuntu-desktop last friday, you mentioned the possibility to drop ~/.profile for storing the user language environment and use /var/cache/gdm/$USER/dmrc instead. I think that may be a practicable approach with the advantage that there is a safe method in place for writing to disk. I added the keys "Langlist" and "LCMess" to dmrc and rewrote language-environment.sh (attached).

There are a couple of matters I'd like to talk over before I proceed. You mentioned that the "Language" value in dmrc must represent a valid locale because of other *dms, but I fear a problem there and don't quite follow you. Aren't we working with the Ubuntu model for locale environment irrespective of the desktop GUI?

Kubuntu is the other pending issue, which I wrote about in comment #64 above.

Rgds,
Gunnar

... and here is the GDM diff

Hello Gunnar,

Gunnar Hjalmarsson [2010-11-24 17:20 -0000]:
> Please note that the gdm patch, i.e. the Xsession code changes, is not
> part of my suggestion any longer. [...] As you know, I tried to
> 'sell' the idea upstream, but wasn't very successful...

Right, but I think we should still patch the Xsession script in the
Ubuntu gdm package instead of this profile.d hacking; first, it would
be a bit confusing for someone who looks at gdm's script, and second,
it hasn't been tested with other login managers and might just break
stuff there.

I think the gdm patch will be rather small, so that it won't be a
burden to maintain.

> As a result of that, I replaced the previously suggested Xsession
> changes with two new language-selector files:
> /etc/profile.d/gdm-lang-unset.sh
> /etc/X11/xinit/xinitrc.d/language-environment.sh

That would again duplicate code, though? Also, it would mean that gdm
behaves differently when language-selector is installed or not. I'm
not totally avert to this, but I'd rather avoid it. (See above)

> > - in the second part of language_update(), why would $GDM_LANG not
> > be a valid locale? this does some rather expensive things like
> > locale -a and grep and iterating over all available locales. So far
> > we trusted that $GDM_LANG was the format of a locale name, and also
> > a valid locale (gdm already tests that); under that assumption,
> > couldn't we simply set $LC_MESSAGES to $GDM_LANG?
>
> I think that the reason why there is a need to ensure that LC_MESSAGES
> is assigned a valid locale is that ~/.dmrc and /var/cache/gdm/$USER/dmrc
> are now updated when LANGUAGE is set.

That's the part I don't understand. The format of LANG and LC_MESSAGES
is the same -- a valid locale string. There shouldn't ever be a change
which sets "Language=" in .dmrc to a non-locale string (cf.
compatibility to older gdm and other *dm); is there?

> Hence $GDM_LANG (or $GDM_LANGUAGE) may well be just 'en.utf8',
> 'sv.utf8' or 'de.utf8', none of which is a valid locale (at least
> not on my box).

Right, but I think $GDM_LANG assuming these values is a bug in the
first place.

> As regards efficiency ("expensive things"), please note that the
> language_update() function is called only when the user changes the
> language from the GDM greeter.

Thanks, noted. I agree that we can add some computational overhead
just for this case.

> > - in the new_language check at the bottom, I think we should
> > additionally check that $GDM_LANG != $LANG [...]
> [...]
> If s/he picks German (Germany) from the GDM greeter, i.e. changes the
> language from English to German, $GDM_LANG/$GDM_LANGUAGE would equal
> $LANG. But now, when we complete the separation of LANG from the
> translation language, that has nothing to do with it any longer, has it?

Of course, you are right. So let's forget this.

> I'd like to raise a couple of Kubuntu related questions.

I'm afraid I don't know anything about the KDE bits; it's worth
speaking to the kubuntu-dev folks about it. However, I think as a
first step we should make these changes to /etc/gdm/Xsession first and
see how it goes out in the field. Then we can see how to apply these
to kdm and whet...

Gunnar Hjalmarsson [2010-11-29 13:52 -0000]:
> During our talk in #ubuntu-desktop last friday, you mentioned the
> possibility to drop ~/.profile for storing the user language environment
> and use /var/cache/gdm/$USER/dmrc instead.

Note that this was just a strawman idea, I haven't looked into this
at all. We can also keep this separate from this initial change, in
order to avoid putting too many changes into one step.

> I think that may be a practicable approach with the advantage that
> there is a safe method in place for writing to disk. I added the
> keys "Langlist" and "LCMess" to dmrc and rewrote
> language-environment.sh (attached).

It's an interesting idea indeed. I brought this up in the context of
language-selector, to avoid having to parse/change ~/.profile there.
Back then I didn't actually think about the gdm integration, but if it
can be done in gdm itself, so much the better.

This needs to be tested with other window managers, though, that they
don't stumble over these new fields.

> You mentioned that the "Language" value in dmrc must represent a valid
> locale because of other *dms

.. and also for backwards compatibility with older gdm. Imagine shared
home directories on a network, which folks access from Lucid and Natty
boxes.

> , but I fear a problem there and don't quite follow you. Aren't we
> working with the Ubuntu model for locale environment irrespective of
> the desktop GUI?

It would be nice, but since $GDM_LANG is specific to gdm, this won't
automatically apply to kdm, lxdm, and others.

Gunnar Hjalmarsson [2010-11-29 13:53 -0000]:
> ... and here is the GDM diff

This looks nicely generic. Would upstream consider taking this?

Thanks,

Martin
--
Martin Pitt | http://www.piware.de
Ubuntu Developer (www.ubuntu.com) | Debian Developer (www.debian.org)

Hi Martin,

Thanks for your latest comments. This reply is long, but I sum up my
POV at the end.

On 2010-11-29 16:21, Martin Pitt wrote:
> ... I think we should still patch the Xsession script in the
> Ubuntu gdm package instead of this profile.d hacking; first, it would
> be a bit confusing for someone who looks at gdm's script,

I agree if you by confusing mean that it would be more difficult to
figure out, by following the code, what's going on at login. OTOH, to
non-language-selector users it would be less confusing, since they
wouldn't need to worry about my language_update() function etc. ;-)

> and second, it hasn't been tested with other login managers and might
> just break stuff there.

Don't see the difference in that respect between an Xsession patch and
sub-scripts. In any case we'll need to be attentive to those login
managers, that may be used by language-selector users.

> I think the gdm patch will be rather small, so that it won't be a
> burden to maintain.

Doesn't the maintenance aspect speak in favour of sub-scripts that
belong to the language-selector package? I mean, since it looks like the
Xsession related changes won't likely be accepted upstream, at least not
short-term, we are about to modify the locale environment handling for
language-selector users only. With the use of sub-scripts, i.e. most of
the code in the language-selector package, certain subsequent changes
might be easier to make.

>> ... I replaced the previously suggested Xsession
>> changes with two new language-selector files:
>> /etc/profile.d/gdm-lang-unset.sh
>> /etc/X11/xinit/xinitrc.d/language-environment.sh
>
> That would again duplicate code, though?

Don't understand. To me it's a question about which file the code best
belongs to (unless you think of that tiny Xsession section with
LANG="$GDM_LANG", of course...).

> Also, it would mean that gdm
> behaves differently when language-selector is installed or not.

Yes. Isn't that what we want for the time being, considering that the
idea was rejected upstream?

> I'm not totally avert to this, but I'd rather avoid it. (See above)

The sub-script solution was triggered by Takao Fujiwara's response at
https://bugzilla.gnome.org/show_bug.cgi?id=633295. At first I too
thought it was hackish, but I have come to feel that the advantages are
not insignificant.
- We leave the GDM behavior untouched for users without
language-selector or with older language-selector versions.
- Easier to maintain if as much of the i18n related code as possible
resides in the language-selector package.

>>> - in the second part of language_update(), why would $GDM_LANG not
>>> be a valid locale?
>> ...
>> I think that the reason why there is a need to ensure that LC_MESSAGES
>> is assigned a valid locale is that ~/.dmrc and /var/cache/gdm/$USER/dmrc
>> are now updated by language-selector when LANGUAGE is set.
>
> That's the part I don't understand. The format of LANG and LC_MESSAGES
> is the same -- a valid locale string. There shouldn't ever be a change
> which sets "Language=" in .dmrc to a non-locale string (cf.
> compatibility to older gdm and other *dm); is there?

Hmm.. If I recall it correctly, we agreed a...

Hi all,

a few comments for clarification:

On 11/30/2010 08:12 PM, Gunnar Hjalmarsson wrote:
> Hmm.. If I recall it correctly, we agreed a few weeks ago to
> conceptually convert GDM's locale selector to a pure language picker, in
> order to make it play well with the language tab in language-selector.
> It should be noted that language-selector's language tab is not designed
> to set LC_MESSAGES, but to build LANGUAGE lists. The language tab
> menu includes both ll_CC combinations and pure ll options, and that's
> the kind of values that has been passed to the dmrc Language field (and
> with that $GDM_LANG) in all the variants I have uploaded the past few
> weeks. Even if a .utf8 extension is appended, no testing of whether the
> resulting string represents a valid locale is done, AFAIK.
>
> To get a string that equals the name of a valid locale, and hence can be
> assigned LC_MESSAGES, language-selector uses similar code with locale -a
> etc. as you have noted in the language_update() function. Typically an
> ll_CC combination plus .utf8 makes up the name of a valid locale, while
> you need to have the program pick an arbitrary country code in the case
> of a pure ll option.
>
> The dmrc Language field and GDM_LANG serve the purposes of
> * determining the pre-selected option in GDM's language picker, and
> * holding the new value if another language is picked from the greeter.
>
> For the above reasons, LC_MESSAGES should not be assigned the GDM_LANG
> value just like that.
>
> So, why doesn't language-selector pass the same string to dmrc as it
> does to LC_MESSAGES? My reason for not wanting to do it that way is that
> it would result in incorrect pre-selected options in GDM's language
> picker. For instance, if you would drag "Deutch" to the top of the menu
> in language-selector's language tab, you would see "German (Austria)" as
> the pre-selected option at next login. Certainly not a disastrous bug,
> but significant enough IMO to not introduce intentionally.
>
> In other words, I suggest that we start using dmrc's Language field and
> $GDM_LANG in a way that differs from the original intention, but only
> for Ubuntu users with language-selector. As you pointed out above, we
> would make GDM behave differently when the new language-selector version
> is installed compared to when it's not, and that ought to take care of
> the risks for backwards compatibility issues as well as issues with
> other dms.

The code in language-selector is a bit hackish and unprofessional, since
Iʼm not an experience programmer or python coder and this whole thing
more or less got dumped on me and I was under pressure to deliver
features in time for the Ubuntu release. :(
The reason *why* I implemented this feature was, because of the
inability of Gnome to let the user choose a different locale setting
from his desktop language. Something, that KDE for example has had for a
long time already.

Now for the locale handling code in language-selector:
There are currently no checks done if a generated locale code is "valid"
according to /usr/share/i18n/SUPPORTED. Instead, the code just appends
".utf8" to any existing ll_CC combination on the system, just...

Hello Arne,

Thanks a bunch for your valuable comments.

On 2010-11-30 17:43, Arne Goetje wrote:
> The code in language-selector is a bit hackish and unprofessional, since
> Iʼm not an experience programmer or python coder ...

Maybe that's the reason why I'm fairly comfortable with the code. ;-)

> and this whole thing more or less got dumped on me and I was under
> pressure to deliver features in time for the Ubuntu release. :(

You have made great contributions, AFAICT.

> KDE has it's own means to handle the desktop language separate from the
> system locale, and therefor the QT version on language-selector does not
> include this functionality. The QT version of l-s is embedded in the
> regular KDE language handling interface, so duplicating this
> functionality would be counter-productive.

That was especially clarifying to me.

I can see that LANG and LANGUAGE have been written to
/etc/default/locale and /etc/environment also for KDE users. I'm
suggesting that also LC_MESSAGES is written to those files, and so far I
have assumed that that change would at least not be a problem for KDE.
Are you able to tell whether my assumption is correct?

> Thanks for taking over the language-selector development, btw. ;)

Well, that's not exactly what I'm doing. Don't consider myself qualified
for such a commitment. So far my goal has been to help solve the issues
in this bug report.

I'd be happy to keep contributing as a member of some formal or informal
l-s team, though. Your participation in such a team on a volunteer, less
demanding basis would be much appreciated.

Best regards,

Gunnar

For the record, below please find Martin Pitts comments on a superseded (and deleted) merge proposal, followed by my reply.

--------------------------------
Martin Pitt wrote on 2010-11-24:
--------------------------------
Hello Gunnar,

many thanks for working on this! Some comments:

- in the beginning of writeUserLanguageSetting() you moved the first line of code above the function's docstring, please revert.

- Thanks for moving the _update_gdm_dmrc() from the "system.." to the "user.." configuration writing, nice catch.

- the stuff that data/gdm-lang-unset.sh and data/language-environment.sh are doing, can we please intregrate this into language-selector itself instead of inflicting it on every login? I don't think we need to try too hard to fix the variables on upgrades. There's too much that can go wrong, and it'll lead to unintended changes for some people. I think we should update the system and ~/.dmrc and ~/.profile variables in language-selector only, when you run it and change your configuration.

Thank you!

-----------------------------------------
Gunnar Hjalmarsson replied on 2010-11-24:
-----------------------------------------
I moved the first line of code because there are now two docstrings in writeUserLanguageSetting(), explaining one section each, and the conffiles variable is used in both sections. Do you still want me to revert?

Actually, _update_gdm_dmrc() was moved from writeUserLangSetting(). Diffs may play a trick on you sometimes. :)

data/gdm-lang-unset.sh makes it possible to by-pass the LANG="$GDM_LANG" line in Xsession, and as far as I can tell, it has to be done at each login in order to keep LANG separated from the translation language.

I understand that you are anxious to avoid that expensive code is run unnecessarily at login, and I share that view. I think that my first Xsession suggestion(s) included such unnecessary code, and that was stupid. However, in data/language-environment.sh there is the language_update() function which holds the possibly expensive stuff, and that function is only called if the user changes the language from the greeter; otherwise only the variable checks at the bottom are done. In other words I would say that my suggestion does not include any expensive code that would be run at each login.

For the case the user does change the language from the greeter, OTOH, I can't see how we could avoid doing something at login. Setting LANGUAGE and LC_MESSAGES to new values is reasonably necessary. Then why do I suggest that ~/.profile is updated as well? Right now I come to think of two reasons:

1. Efficiency. If ~/.profile was not updated, ~/.dmrc would not correspond with ~/.profile, and if the user does not go to "Language Support" and redo the language switch (and why would s/he find such a measure to make sense?), it would mean that data/language-environment.sh would need to set LANGUAGE and LC_MESSAGES at each login. In other words, in the long run it's more efficient to have data/language-environment.sh update ~/.profile as well.

2. Avoid confusion. A user, who checks the contents of ~/.profile, would find that the LANGUAGE and LC_MESSAGES lines would not correspond with t...

Martin,

I have updated the patches and merge proposals, and personally I feel that they have been sufficiently penetrated to be committable. Consequently I ask you to please complete your sponsoring of this stage of the project, so the usefulness can be tested more widely.

I don't know if you have had the time yet to read my reply above to your latest comments. In any case I'd like to add that IMO I've had my opportunities to argue for my ideas. As from now I'll be less disinclined to comply with your change requests, if any. Promise. ;-)

Best regards,

Gunnar

Hi Gunnar,

On 12/02/2010 05:36 AM, Gunnar Hjalmarsson wrote:
> On 2010-11-30 17:43, Arne Goetje wrote:
>> The code in language-selector is a bit hackish and unprofessional, since
>> Iʼm not an experience programmer or python coder ...
>
> Maybe that's the reason why I'm fairly comfortable with the code. ;-)

heh. :)

>> and this whole thing more or less got dumped on me and I was under
>> pressure to deliver features in time for the Ubuntu release. :(
>
> You have made great contributions, AFAICT.

Thanks, it's nice to hear that once in a while. :)

>> KDE has it's own means to handle the desktop language separate from the
>> system locale, and therefor the QT version on language-selector does not
>> include this functionality. The QT version of l-s is embedded in the
>> regular KDE language handling interface, so duplicating this
>> functionality would be counter-productive.
>
> That was especially clarifying to me.
>
> I can see that LANG and LANGUAGE have been written to
> /etc/default/locale and /etc/environment also for KDE users. I'm
> suggesting that also LC_MESSAGES is written to those files, and so far I
> have assumed that that change would at least not be a problem for KDE.
> Are you able to tell whether my assumption is correct?

That should be correct.

>> Thanks for taking over the language-selector development, btw. ;)
>
> Well, that's not exactly what I'm doing. Don't consider myself qualified
> for such a commitment. So far my goal has been to help solve the issues
> in this bug report.

he he, it was meant to be ironic. :) Sadly, that was what happened to
me. I started to make some contributions and suddenly the whole project
was dumped on me, although it was never my intention to take over the
whole project.

> I'd be happy to keep contributing as a member of some formal or informal
> l-s team, though. Your participation in such a team on a volunteer, less
> demanding basis would be much appreciated.

If I can find time to do that, I'd be happy to help out. But I don't
have much time, since I have two young and demanding kids at home.

But yes, it would be nice to have a team to take over this project.

Cheers
Arne
--
Arne Götje (高盛華) <email address hidden>
PGP/GnuPG key: 1024D/685D1E8C
Fingerprint: 2056 F6B7 DEA8 B478 311F 1C34 6E9F D06E 685D 1E8C
Key available at wwwkeys.pgp.net. Encrypted e-mail preferred.

Hi again,

On 2010-12-02 03:10, Arne Goetje wrote:
> On 12/02/2010 05:36 AM, Gunnar Hjalmarsson wrote:
>> I can see that LANG and LANGUAGE have been written to
>> /etc/default/locale and /etc/environment also for KDE users. I'm
>> suggesting that also LC_MESSAGES is written to those files, and so
>> far I have assumed that that change would at least not be a problem
>> for KDE. Are you able to tell whether my assumption is correct?
>
> That should be correct.

Thanks for that confirmation.

> I started to make some contributions and suddenly the whole project
> was dumped on me, although it was never my intention to take over
> the whole project.

I wonder why that pattern sounds so familiar, kind of? :( Believe me
when I say that I have been there a few times as well...

>> I'd be happy to keep contributing as a member of some formal or
>> informal l-s team, though. Your participation in such a team on a
>> volunteer, less demanding basis would be much appreciated.
>
> If I can find time to do that, I'd be happy to help out. But I don't
> have much time, since I have two young and demanding kids at home.
>
> But yes, it would be nice to have a team to take over this project.

Yeah, but personally I believe there is still a need for _one_ responsible
team leader.

See you soon! (at least on line)

Gunnar

Would someone be able to comment if communication with upstream is still active please?

@Martin, what do you consider to be the next step?

Hi Dave,

On 2010-12-03 12:12, Dave Walker wrote:
> Would someone be able to comment if communication with upstream is
> still active please?

Communication with upstream (GNOME) is applicable for two reasons.

Previously I suggested some changes to /etc/gdm/Xsession, and tried to 'sell' those changes upstream.
https://bugzilla.gnome.org/show_bug.cgi?id=633295
That discussion with upstream appears to be over, doesn't it? Two other ways to achieve the same thing for Ubuntu have been discussed:
1. Sub-scripts to Xsession, belonging to language-selector
2. Xsession patch in the Ubuntu gdm package
I included option 1 in the latest merge proposals, since I can see a couple of advantages with it. Waiting for Martin's response.

I'm also suggesting changes to the GDM source files
daemon/gdm-session-settings.c and daemon/gdm-session-settings.h, which haven't been discussed with upstream yet. Please see remark in the description field of https://code.launchpad.net/~gunnarhj/ubuntu/natty/gdm/lp-553162/+merge/42415

@Martin
I have made an attempt to "juggle all the files", as you put it the other day in #ubuntu-desktop, and I believe it's now almost as it ought to be.
* Patched the changes to Xsession.in, gdm-session-settings.c and
  gdm-session-settings.h. Sub-scripts belonging to the
  language-selector package dropped - yes, I changed my mind. ;-)
* I suggest that the patches are not forwarded upstream, since they
  address issues as to the interaction with the Ubuntu specific
  language-selector package.
* Installation script in language-selector dropped

Natty problem:
When I uploaded the GDM source to my ppa, it failed to build for Natty.
https://launchpad.net/~gunnarhj/+archive/locale-test/+build/2083108
The upload for Maverick (same stuff) built ok, though.

@everyone
Binaries with the changes are available for testing in Maverick or Natty.
https://launchpad.net/~gunnarhj/+archive/locale-test
As regards Natty, you can test the changes despite of the GDM build failure. Simply download the GDM binary for Maverick and install that .deb file:

sudo dpkg -i gdm_2.30.5-0ubuntu5dmrc3m_amd64.deb

Rgds,
Gunnar

Hello Gunnar,

I see a ton of new MPs and replies here, so I'll reply/catch up with
them one by one. So some of my replies here might already be obsolete
now.

Gunnar Hjalmarsson [2010-11-30 12:12 -0000]:
> > and second, it hasn't been tested with other login managers and might
> > just break stuff there.
>
> Don't see the difference in that respect between an Xsession patch and
> sub-scripts.

If you patch /etc/gdm/Xsession, it'll just affect gdm. If you add
something to /etc/profile.d/ and xinitrc.d/, it will affect all login
managers, and will even run for console and ssh logins.

> Doesn't the maintenance aspect speak in favour of sub-scripts that
> belong to the language-selector package?

I don't think it makes much difference where the script changes are
maintained (in gdm or l-s), but in gdm they'll only apply to gdm,
which I think is the safer approach for now. There's another aspect to
this: The upstream gdm Xsession script already sets $LANG. With the
changes that you have in mind, it needs to stop doing that; with
separate .d directory scripts, you can only additionally set $LC_*,
but the original $LANG value is lost.

> > Also, it would mean that gdm behaves differently when
> > language-selector is installed or not.
>
> Yes. Isn't that what we want for the time being, considering that the
> idea was rejected upstream?

It sounds strange to me. Why would you desktop language suddenly
change if you uninstall (or install) language-selector? I don't think
that this meets the principle of least surprise.

> >> I think that the reason why there is a need to ensure that LC_MESSAGES
> >> is assigned a valid locale is that ~/.dmrc and /var/cache/gdm/$USER/dmrc
> >> are now updated by language-selector when LANGUAGE is set.
> >
> > That's the part I don't understand. The format of LANG and LC_MESSAGES
> > is the same -- a valid locale string. There shouldn't ever be a change
> > which sets "Language=" in .dmrc to a non-locale string (cf.
> > compatibility to older gdm and other *dm); is there?
>
> Hmm.. If I recall it correctly, we agreed a few weeks ago to
> conceptually convert GDM's locale selector to a pure language picker, in
> order to make it play well with the language tab in language-selector.

Right, but without upstream taking some of those more intrusive
changes we need to have a compromise where we don't change the meaning
of .dmrc. Since other DMs also use that file, we can't just redefine
the meaning of the Language= option there anyway.

> It should be noted that language-selector's language tab is not
> designed to set LC_MESSAGES, but to build LANGUAGE lists.

Correct, but that's orthogonal to gdm. language-selector has both a
language list (for $LANGUAGE) in the first tab, and a locale picker in
the second tab (for time, currency format, etc.).

> The language tab menu includes both ll_CC combinations and pure ll
> options, and that's the kind of values that has been passed to the
> dmrc Language field (and with that $GDM_LANG) in all the variants I
> have uploaded the past few weeks.

Right, and that's what I'm continuing to object to :) We can't do that
only in Ubuntu without getting agreement between other users ...

Gunnar Hjalmarsson [2010-12-02 7:24 -0000]:
> Yeah, but personally I believe there is still a need for _one_ responsible
> team leader.

For the record: Indeed l-s currently doesn't have a lead developer,
but we'll get one soon.

Right now it's on collaborative maintenance mode. (There are some
changes from me in bzr as well)

Martin

--
Martin Pitt | http://www.piware.de
Ubuntu Developer (www.ubuntu.com) | Debian Developer (www.debian.org)

Hello Gunnar,

Gunnar Hjalmarsson [2010-12-07 8:42 -0000]:
> * Patched the changes to Xsession.in, gdm-session-settings.c and
> gdm-session-settings.h. Sub-scripts belonging to the
> language-selector package dropped - yes, I changed my mind. ;-)

Oh, great :)

> Natty problem:
> When I uploaded the GDM source to my ppa, it failed to build for Natty.
> https://launchpad.net/~gunnarhj/+archive/locale-test/+build/2083108

I see. This seems to be unrelated to your changes, so I guess I'll fix
that first in our ubuntu-desktop gdm branch, and then apply the merge
on top of this.

Martin
--
Martin Pitt | http://www.piware.de
Ubuntu Developer (www.ubuntu.com) | Debian Developer (www.debian.org)

Unsubscribing sponsors, for the record. I think the discussion here has become way too complex, this will be handled between Gunnar and me now.

Hi Martin,

Thanks for your comments! Before replying to some of them, let me say
that I finally realized that the .d/ sub-scripts solution I suggested
previously was too obfuscated. Whatever advantages I saw, they could not
justify the resulting obfuscation. All code in a project like this has
to be reasonably easy to understand and maintain.

I'm not sure, but judging from some of your comments, I have a feeling
that even you had some difficulties in seeing through what the code
actually did. My fault, in that case; sorry.

Besides my replies below, I have added a comment to the GDM merge
proposal.

On 2010-12-07 10:34, Martin Pitt wrote:
> If you patch /etc/gdm/Xsession, it'll just affect gdm. If you add
> something to /etc/profile.d/ and xinitrc.d/, it will affect all
> login managers, and will even run for console and ssh logins.
> ...
> There's another aspect to this: The upstream gdm Xsession script
> already sets $LANG. With the changes that you have in mind, it needs
> to stop doing that; with separate .d directory scripts, you can only
> additionally set $LC_*, but the original $LANG value is lost.

I believe that I had taken care of both those aspects. The code in
/etc/profile.d/, which would have run before the spot where Xsession
sets $LANG, looked like this:

    if [ -n "$GDM_LANG" ]; then
        export GDM_LANGUAGE=$GDM_LANG
        unset GDM_LANG
    fi

The code in xinitrc.d/ would not have done anything but a variable test,
if [ -n "$GDM_LANGUAGE" ] had returned false.

Even if it's history now, I'd appreciate a hint if I missed anything.

> ... we need to have a compromise where we don't change the meaning of
> .dmrc. Since other DMs also use that file, we can't just redefine the
> meaning of the Language= option there anyway.

I didn't know that other DMs use the .dmrc file; thought it was a GDM
related file like the GDM_LANG variable. Hmm.. Spontaneously I don't
think it's an issue that hasn't already been taken care of, though,
since language-selector updates .dmrc only if GDM is running. Other
login managers can safely keep saving its settings in .dmrc.

I removed a section, where you further emphasize that we need to prevent
undesired and/or surprising effects to other login managers but GDM. On
that we are 100% agreed. My apologies if I have given you some other
impression.

Please note that the "redefinition" I suggest would only apply when GDM
and language-selector play together. Replace GDM, and l-s stops writing
to .dmrc. Remove l-s, and GDM falls back to its original behavior. No
harm done.

>> So, why doesn't language-selector pass the same string to dmrc as
>> it does to LC_MESSAGES? My reason for not wanting to do it that way
>> is that it would result in incorrect pre-selected options in GDM's
>> language picker. For instance, if you would drag "Deutch" to the
>> top of the menu in language-selector's language tab, you would see
>> "German (Austria)" as the pre-selected option at next login.
>> Certainly not a disastrous bug, but significant enough IMO to not
>> introduce intentionally.
>
> I agree that this is confusing, but I think that's the kind of
> compromise we have to make if this is going to be a ...

It gives me SO much headache. I understand what I propose is a separate approach but can someone please specify what drawbacks lie behind creating a "language-creator" instead of "language-selector"? There are only 10 types of users: those finding default language settings for their country convenient and those needing custom locale files. So why there's no creator that would allow to specify system messages language, date in a form of %Y-%M-%D or YYYY-MM-DD, put in a decimal separator, currency name, first day of the week and so on? It would auto-normalize after a time by users' statistics (e.g. it can automatically sent statistics after a few days without changes to the locales and after the user agrees) so every popular scheme would "come up" shortly and could be coming predefined with new system distributions. Why do I have to trying out all the locales one by one to find out where's the one with convenient date, or decimal separator, with learning whole lang file structure and creating one myself as the only alternative? Language-selector, when fixed, will only allow to speed-up the "trying out" part but after trying every default set I will still end up editing files I do not understand. Your approach gives only N/M-1 probability of satisfying the user, where N is the number of prepared sets and M is the number of possible language/country combinations. And those are small chances.

For the record, current ubuntu-desktop gdm bzr head (and the version in natty) build fine again.

Hello Gunnar,

Gunnar Hjalmarsson [2010-12-09 1:49 -0000]:
> > There's another aspect to this: The upstream gdm Xsession script
> > already sets $LANG. With the changes that you have in mind, it needs
> > to stop doing that; with separate .d directory scripts, you can only
> > additionally set $LC_*, but the original $LANG value is lost.
>
> I believe that I had taken care of both those aspects. The code in
> /etc/profile.d/, which would have run before the spot where Xsession
> sets $LANG, looked like this:
>
> if [ -n "$GDM_LANG" ]; then
> export GDM_LANGUAGE=$GDM_LANG
> unset GDM_LANG
> fi

Ah, I haven't paid attention to this bit before (I didn't review the
code in that detail yet, just took a look at the structure so far).
That would certainly have done the trick, although it's quite
non-obvious to someone who just reads the /etc/gdm/Xsession script.

> I didn't know that other DMs use the .dmrc file

At least lxdm and kdm do, I didn't check/know about others.

But exactly because it's a shared config file, I'm not too worried
about adding new fields to it. DMs should just ignore fields which
they can't deal with, and the .ini file format is meant for this. We
just need to pay attention to not change the semantics of existing
fields, which is why I'm so insistant on keeping Language= as a valid
locale.

> > I'd rather add a new LanguageList (or similar) field to it on
> > demand.
>
> Actually I played with that thought when preparing the current branches,
> but concluded that there is no need. Convince me that I'm wrong, and
> I'll be happy to do some copy-n-pasting again. ;-)

Oh, don't get me wrong -- if we can make-do with the existing Language
field, so much the better! I just had the impression that we also
wanted to store a list of languages for the $LANGUAGE variable, in
which case we'd probably need a new field.

> Sounds reasonable to me. Is there a more appropriate place than bug
> reports to hold this kind of complex (at least for a newbie like me)
> design discussions?

It's good enough IMHO. Well, an even better one is being in a room
together with a whiteboard :)

It just became too complex for a regular sponsor to pick up.

Thanks,

Martin
--
Martin Pitt | http://www.piware.de
Ubuntu Developer (www.ubuntu.com) | Debian Developer (www.debian.org)

This bug was fixed in the package language-selector - 0.8

---------------
language-selector (0.8) natty; urgency=low

  * Set LC_MESSAGES for applications that don't recognize LANGUAGE
    (LP: #553162).
  * GDM related fixes (LP: #553162):
    - Update dmrc when LANGUAGE and LC_MESSAGES are set, not when LANG
      is set.
    - Use the new dmrc fields "Langlist" and "LCMess" to store the
      LANGUAGE and LC_MESSAGES values on disk.
 -- Gunnar Hjalmarsson <email address hidden> Tue, 14 Dec 2010 22:20:37 +0100

This bug was fixed in the package gdm - 2.32.0-0ubuntu2

---------------
gdm (2.32.0-0ubuntu2) natty; urgency=low

  [ Martin Pitt ]
  * 06_run_xsession.d.patch: Don't trip over directories and other non-files
    in Xsession.d/. (LP: #654578)

  [ Gunnar Hjalmarsson ]
  * debian/patches/35_langlist_and_lsmess_dmrc_fields.patch:
    - Addition of the fields "Langlist" and "LCMess", which make ~/.dmrc
      and /var/cache/gdm/$USER/dmrc able to store the user language
      environment (LP: #553162).
  * debian/patches/36_language_environment_settings.patch:
    - Changes to Xsession's way to use GDM_LANG. It now sets LANGUAGE
      and LC_MESSAGES, which makes it possible to keep the user language
      for message translation apart from other locale settings
      (LP: #553162).
 -- Martin Pitt <email address hidden> Tue, 14 Dec 2010 22:24:40 +0100

It costs almost One year !

--
YunQiang Su

@trespasser (comment #87)
Personally I would like to see a GUI where users can seemlessly set their locale preferences without knowing what a locale is, so I find your suggestion interesting. However, it's beyond the scope of this bug report. These are two places that may be more suitable to discuss your idea:

http://brainstorm.ubuntu.com/

https://lists.ubuntu.com/mailman/listinfo/ubuntu-devel

A wishlist bug might also be an option.

/ Gunnar

Hi all,

As you may have seen, fixes to this bug report were released in Natty on December 14, and yesterday Martin Pitt approved the last 'follow-up fix'.

The issues were introduced in Lucid (Ubuntu 10.04), so I have prepared gdm and language-selector packages for Lucid and Maverick with similar changes. For the time being they are available at https://launchpad.net/~gunnarhj/+archive/locale-test

Your feedback on the fixes would be much appreciated. For the case you would find buggy behavior that was discussed here, but has not been properly dealt with, please submit new, specific bugs. I for one picked up one such issue, and reported it as bug 693337.

/ Gunnar

This introduces quite a substantial behaviour change, including the additional setting of $LC_MESSAGES (which now causes a lot of confusion when using ssh), has very intrusive patches, and this isn't a data loss or security bug, thus I am afraid it doesn't qualify for an SRU.

Ok Martin, no problem. You weren't very keen on the SRU idea in December either, but you mentioned backports as a possibly passable option. To me, the branches that this bug report resulted in are principally fixes of annoying, buggy behavior. However, I'm sure there are good reasons for the restrictive SRU criteria.

At https://help.ubuntu.com/community/UbuntuBackports I read:
"Please, for bugfixes, try https://wiki.ubuntu.com/StableReleaseUpdates first. We will reject any Backports requests for bugfixes if SRU has not been attempted."

I take it that I now can consider the SRU nomination in the making 'officially' rejected, and start checking out the backport process more closely. :-)

Hi all,

I have filed the bug 719815, where I request that the fixes of this bug be backported to Lucid and Maverick.

https://help.ubuntu.com/community/UbuntuBackports

Once test builds have been made, please feel free to help process the request by testing the packages and posting feedback and possible other comments on bug 719815.

Fixes of this bug for Lucid and Maverick are now available in official backports packages. To make Synaptic check for backports updates you can do:

o System -> Administration -> Update Manager -> Settings...

o Select the "Updates" tab and check the "Unsupported updates" option.

More about Ubuntu backports: https://help.ubuntu.com/community/UbuntuBackports

re-tested GDM in Natty, and found the issues still remain.

languages pre-installed: English, Traditional Chinese, Simplified Chinese
GDM: 2.32.0-0ubuntu14

Reproduce steps

A. issue at GDM screen.
1. logout
2. select English on the bottom panel
3. Ideally, "密碼" should change to "password", but it remains no change. Please see the attached screen shot.

B.issue after login
1. logout
2. select ”漢語 台灣“ （Traditional Chinese） on the bottom panel
3. input password then login the system
4. the language keeps no changes as English

Kevin,

On 2011-04-05 11:02, Kevin Huang wrote:
> A. issue at GDM screen.
> 1. logout
> 2. select English on the bottom panel
> 3. Ideally, "密碼" should change to "password", but it remains no
> change.

That is as it should be. The language chooser on the bottom panel sets
the user language, while it's the system language that controls in which
language the labels on the login screen are displayed. Please use
System -> Administration -> Language Support
to change language system-wide, if that is what you want to do.

> B.issue after login
> 1. logout
> 2. select ”漢語 台灣“ （Traditional Chinese） on the bottom panel
> 3. input password then login the system
> 4. the language keeps no changes as English

If you select "Chinese (Taiwan)" or "Chinese (Hong Kong)" with the
language chooser on the login screen, and then log in, the session
indeed should be displayed in traditional Chinese. Did you really log
out first, or could it possibly be that you just returned to an
already existing session?

Hi Gunnar,

let me reproduce it in details. Hope you don't mind to refer to I captured the screen shots in sequence attached, because I am not sure if I can explain it in details in English.

Environment
1. Fresh Unity Beta 1 installation with all updates on April 7th
2. Language selected in OOBE at installation: Simplified Chinese
3. GDM version: 2.32.0-0ubuntu15
4. Languages preinstalled: English, Simplified Chinese, Arabia (added after installation)

Testing steps

1. starting Locale: Simplified Chinese
2. Screen shot1 (bug): For some reason, Gwibber is in Aribic. Except Gwibber, other application names look fine.
3. log out
4. Screen shot2 (bug): The "Universal Access Preferences" icon on the right bottom is missing.
5. Screen shot3: password and language selection are in Simplified Chinese
6. Screen shot4: select language: English (America). The "password" is still kept as "密碼". From user's point of view, I would prefer it could be changed to "password" after I select language in English. However, I would say this is at lower priority.
7. login
8. Screen shot5: all names of preinstalled applications are in English. There are several applications in "Apps available for download" in Simplified Chinese.
9. logout
10. screen shot6: password and language selection are in Simplified Chinese which should be in English.
11. screen shot7: all languages in language selection list are in Simplified Chinese.
12. screen shot8: select language: 阿拉伯语 (Aribirc).
13: login
14: screen shot9: Most names of preinstalled application are in Arabic although the translation quality is not as high as in Simplified Chinese. There are several applications in "Apps available for download" in Simplified Chinese.
15: logout, select English, login, then install traditional Chinese and Japanese.
16: logout
17: no matter I select Japanese or Traditional Chinese, the GDM is always in Simplified Chinese, and several applications in "Apps available for download" are in Simplified Chinese.

Hi again, Kevin.

Two things you need to be aware about:

1. The display language can be set on two levels: system wide,
which determines what's shown on the login screen, and user specific.
It's only the user language that can be set from the login screen.

2. The principal tool for setting language is Language Support. The
language chooser on the login screen is just a supplement.

I have recently written a help document, which shows up if you click the
"Help" button in Language Support. Your latest comment triggered me to
make some changes to the document, but they are currently queued since
we are in BetaFreeze (https://wiki.ubuntu.com/BetaFreeze). You can still
reach those very latest changes by installing language-selector from my
PPA: https://launchpad.net/~gunnarhj/+archive/i18n-docs

I recommend you to study the document carefully, to get a better
understanding of how languages are handled in Ubuntu.

On 2011-04-08 05:13, Kevin Huang wrote:
> Hope you don't mind to refer to I captured the screen shots in
> sequence attached, because I am not sure if I can explain it in
> details in English.

Of course I don't mind. :) It's a good way to clarify your points and
prevent misunderstandings.

> 1. starting Locale: Simplified Chinese
> 2. Screen shot1 (bug): For some reason, Gwibber is in Aribic. Except
> Gwibber, other application names look fine.

Probably you either made Arabic be included in the language priority
list, or you picked an Arabic locale on the "Regional Formats" tab in
Language Support.

> 3. log out
> 4. Screen shot2 (bug): The "Universal Access Preferences" icon on the
> right bottom is missing.

I know that that bug has been addressed, and I'd be surprised if it's
not fixed before the 11.04 release.

> 5. Screen shot3: password and language selection are in Simplified
> Chinese
> 6. Screen shot4: select language: English (America). The "password"
> is still kept as "密碼". From user's point of view, I would prefer it
> could be changed to "password" after I select language in English.
> However, I would say this is at lower priority.

There is a wish for such a feature in bug 24935. Can't tell whether it
would be reasonable to implement it.

> 7. login
> 8. Screen shot5: all names of preinstalled applications are in
> English. There are several applications in "Apps available for
> download" in Simplified Chinese.

Probably those apps ignore both the LANGUAGE and LC_MESSAGES variables,
and use the $LANG value directly to determine the display language.

I have a vague idea of what might be done to improve Ubuntu's Language
Support in this respect, but it would need to be discussed first.

> 9. logout
> 10. screen shot6: password and language selection are in Simplified
> Chinese which should be in English.

No, it should not. Since you haven't changed the system language, it's
still simplified Chinese.

> 11. screen shot7: all languages in language selection list are in
> Simplified Chinese.

Since you haven't changed the system language, it's still simplified
Chinese.

> 12. screen shot8: select language: 阿拉伯语 (Aribirc).
> 13: login
> 14: screen shot9: Most names of preinstalled application are in
> Arabic although ...

Hi Gunnar, Thanks for your clear explanation about two level setting of display language. Issue closed

You're welcome, Kevin. Let me thank you for your effort with the detailed questions. They helped me improve the new Language Support Help document.

Also, I'm glad that it wasn't yet another bug. ;-)

The patch #36 (36_language_environment_settings.patch) damaged my desktop environment by setting LANGUAGES.

I'm Japanese and my LANG is "ja_JP.UTF-8", but because of some reasons my LC_MESSAGES is set to "C" by .gnomerc and .zshenv to display non-translated messages/texts. This worked very well for me.

After upgrading to Natty, however, it went badly. Because GDM now sets LANGUAGES as "ja_JP:en_US:en" and this setting overrides LC_MESSAGES in many applications.

Mine is not a major requirement, but setting LANGUAGE is too strong and overriding various settings including LC_XXX.

Let me correct my previous comment:

s/LANGUAGES/LANGUAGE/g

@Kentaro
To have menus and messages displayed in English, just drag "English" to the top of the combobox list on the "Language" tab of Language Support.

@Gunnar,

I guess there are very few users (not developers) know two level language set up designs, system wide on language support and user specific on the login screen. It actually has confused me for a while until I read your clear explanation: https://bugs.launchpad.net/ubuntu/+source/gdm/+bug/553162/comments/102

@Gunnar, (#107)

Thank you for your help. I tested it and it seems to be working at this moment.

I was in doubt that selecting "English" of Language menu affects all locale settings such as LC_TIME or LC_NUMERIC.
Let me see what happens with my settings. Now I have "export LANG=ja_JP.UTF-8; export LC_MESSAGES=C" in my
.gnomerc and .zshenv. "echo $LANGUAGE" says "LANGUAGE=en_US:en". Gnome Clock displays current date in Japanese
format.

@Kevin
Yes, I remember that you were confused by system vs. user specific level. Language Support Help, which you can reach by clicking the "Help" button at the bottom left of the Language Support window, will hopefully contribute to reduce similar confusion henceforth. Also, at the UDS in Budapest it was decided to let a design team review the work flow and UI of Language Support.

Do you have a specific suggestion in mind?

@Kentaro
In Lucid and Maverick, choosing a locale for regional formats, that does not match the selected language, often leads to unexpected results, which I suppose that your special code in .gnomerc and .zshenv aims at correcting. In Natty that code ought to be redundant, since the problems have been fixed, so you should be able to remove the special code without any undesired behavior.

I am not sure if this is related, but I am experiencing problems with GDM overriding language selections I apply on a per-programme basis via the env variable.

Normally, when I select Chinese (China) at the login screen, I can still run programmes in English by prepending an env argument, e.g. "env LANG=en_US.UTF-8 firefox". That way I can run an English instance of Firefox in a Chinese language Ubuntu session. This was working fine up to Lucid. Maverick was the first Ubuntu release that caused problems. After installing Maverick, whatever language I specified at the login screen would stick, and I couldn't run programmes in an alternative language by prepending env. The issue disappeared somewhere along the way after an update, though. However, I just finished setting up a fresh install of Natty and the problem reappaeared.

Is this related to this bug or described somewhere else? Has somebody else experienced similar problems?

A screenshot demonstrating the conflict between gdm and env.

I selected "Chinese (China)" at the login screen to run a Chinese language session of Ubuntu. In this Chinese Ubuntu session I am trying to run an English instance of Firefox by prepending "env LANG=en_US.UTF-8". Note how Firefox comes up in Chinese instead of English.

@dreamon
You are probably right when assuming that your observations are related to the fixes of this bug.

If your regional formats setting (second tab in Language Support) is something else but "Chinese (China)", your choosing "Chinese (China)" as the display language makes gdm set the LC_MESSAGES variable to "zh_CN.UTF-8". Since LC_MESSAGES overrides LANG, setting LANG before launching an app is not sufficient. Try this instead:

env LC_MESSAGES=en_US.UTF-8 firefox

Gunnar, thank you for your quick reply. Unfortunately, this didn't make a difference. Firefox still comes up in Chinese. This is on a fresh Natty install, by the way (had to reinstall since my last post for a different reason).

Suppose that means the bug is not fixed then?

Any other suggestions? Any additional test you'd like me to run?

It makes a difference to me.

Since you said "an English instance of Firefox", maybe I should point out that you can't have a Chinese and an English FF instance simultaneously. In other words, Firefox must not be running when you execute the command I suggested.

Yes, of course. Firefox wasn't running when I executed the env LC_MESSAGES and env LANG commands. I also tried running the command with other programmes, such as gedit, but always got the same result: contrary to my expectation, the programme would not open up in the language specified, but in the language I picked at login. This didn't happen on Lucid, and, as I pointed out above, stopped happening on Maverick.

Do other users also experience this problem? Shall we track this in a separate bug report?

Strange. Firefox should be started in English if you set LC_MESSAGES as I suggested. Typo, maybe? ;-)

gedit is another thing. Unlike Firefox gedit recognizes the LANGUAGE env. variable, which overrides both LANG and LC_MESSAGES, so to make gedit be started in English you would need:

env LANGUAGE=en gedit

Yes, if there is a need to track it down, please submit a separate bug.

Thanks again for your suggestions, Gunnar. I copied your commands directly from here to the terminal, so it is not a typo.

Strangely, setting env LANG with firefox or thunderbird actually worked on the last three Ubuntu versions, even though it may be incorrect to run it this way. I was able to run an English instance of Thunderbird, for example, by using env LANG.

I have opened a separate bug report here: https://bugs.launchpad.net/ubuntu/+source/gdm/+bug/792735 and supplied more information to track down the problem. Any suggestions would be welcome!

This still fails on Natty 64 bits. I have defined:

$ cat /etc/default/locale
LANG="es_ES.UTF-8"
LC_ALL=
LC_MESSAGES=POSIX
LC_COLLATE=C

But when I enter in Gnome:
$ locale
LANG=es_ES.UTF-8
LANGUAGE=es_ES:en
LC_CTYPE="es_ES.UTF-8"
LC_NUMERIC="es_ES.UTF-8"
LC_TIME="es_ES.UTF-8"
LC_COLLATE=C
LC_MONETARY="es_ES.UTF-8"
LC_MESSAGES="es_ES.UTF-8"
LC_PAPER="es_ES.UTF-8"
LC_NAME="es_ES.UTF-8"
LC_ADDRESS="es_ES.UTF-8"
LC_TELEPHONE="es_ES.UTF-8"
LC_MEASUREMENT="es_ES.UTF-8"
LC_IDENTIFICATION="es_ES.UTF-8"
LC_ALL=

Only it respects LC_COLLATE but not LC_MESSAGES.

Hi Simon,

/etc/default/locale contains system wide settings that basically affect the startup and the login screen. Once logged in, your personal language settings override the system wide ditto.

Please select English as your user language, either from the language chooser at the bottom of the login screen or from Language Support.

I recommend that you click the "Help" button on the Language Support window for a description of how things are intended to work.

see this also
https://bugs.launchpad.net/ubuntu/+source/unity/+bug/820333