NSS using LDAP+SSL breaks setuid applications like su, sudo, apache2 suexec, and atd

Bug still present on Karmic alpha 5.

Workaround:

It's a bit clunky, but if it is driving you nuts, you can add the user information to the local /etc/passwd file, and the user will be able to use sudo and su.

The catch is that the user_id and the user_name must be the same in both ldap and the local /etc/passwd. It sort of defeats the purpose of ldap, but if you only have one admin user, it works fine.

Apparently still present in RC. As a clarification, users present in /etc/passwd can still sudo just fine. Users present only via ldap cannot.

My suspicion is that the initial reporter had, as Scott G. suggested, the user present both via ldap and /etc/passwd, with the result that hitting the local file first (via compat) made it work, versus hitting ldap first makes it break.

passwd: files ldap
..does not work for me (with the user present only via ldap).
passwd: files compat ldap
..also does not work:
> sudo ls /tmp
sudo: setreuid(ROOT_UID, user_uid): Operation not permitted

Yuck.

Perhaps sudo/sudo-ldap are more appropriate packages ? It appears to be sudo-ldap 1.7.2-2 related (See http://bugs.debian.org/cgi-bin/bugreport.cgi?bug=545414)

Eh. Going back to the Jaunty version of sudo didn't appear to help, so maybe it's not the sudo package itself ?

This did work on Jaunty.

I do not have this problem...

My nsswitch.conf looks like:
passwd: files ldap
group: files ldap
shadow: files ldap

my user only exists in LDAP, but is mentionned in /etc/group (sudo group)

sudoers shows:
%sudo ALL=(ALL) ALL

then:
$ sudo uname -a
Linux dgeos 2.6.31-14-generic #48-Ubuntu SMP Fri Oct 16 14:05:01 UTC 2009 x86_64 GNU/Linux

I did not notice this particular bug before...

I did last upgrade this morning...

I'm also affected by this bug, but in my case it only happens with ldaps (LDAP over SSL) connections.

When I switch from ldaps:// to ldap://, everything begin to work fine again.

Maybe this would help to debug. (I have a self-signed cert in the LDAP server)

I can also make it work by using ldap instead of ldaps. (and am also using a self-signed cert for the ldap server).

I couldn't test ldap vs. ldaps, but sudo was working before my karmic upgrade and it stopped working after. I could do getent operations which did ldap queries, so it wasn't an ldap problem for me.

I was able to ~solve~ this by installing nscd. Not sure why there would be a nscd requirement.

Here's my nsswitch.conf:
passwd: files ldap
group: files ldap
shadow: files ldap
hosts: files mdns4_minimal [NOTFOUND=return] dns mdns4
networks: files
protocols: db files
services: db files
ethers: db files
rpc: db files
netgroup: files ldap

I'm not sure what else to include for debugging...

Had the same problem, installing nscd seems to have solved the issue.

However, I noticed that I had that issue with ldap users, when I was accessing ldap over SSL:

my nsswitch.conf (only usefull lines...):
----------------------------------------
passwd: files ldap [NOTFOUND=return] db
group: files ldap [NOTFOUND=return] db
shadow: files
---------------------------------------

my /etc/ldap.conf :
-----------------------------------------
base ********************
host ********************
ldap_version 3

ssl on
tls_cacertfile /etc/ssl/certs/myCA.crt
tls_checkpeer on

pam_password_prohibit_message Please use Wiki to change your password

bind_policy soft
nss_initgroups_ignoreusers avahi,avahi-autoipd,backup,bin,clamav,couchdb,daemon,dhcpd,freerad,games,gdm,gnats,haldaemon,hplip,irc,kernoops,klog,libuuid,list,lp,mail,man,mcn,messagebus,michael,mysql,news,ntp,ntpd,openldap,pdns,polkituser,postfix,proxy,pulse,root,saned,sftp,snmp,speech-dispatcher,sshd,sync,sys,syslog,uml-net,uucp,vde2-net,www-data
-----------------------------------------

Just changing ssl from 'off' to 'on' made me have the issue.... and reverting it back to 'off' made it work again....

Hope this helps

Same problem here. Only sudoers from the local sudoers file are recognized. Also the setting sin ldap.conf (/etc/ldap.conf and /etc/ldap/ldap.conf were not recognized - sudoers_debug made no change, no debug output).

I fixed a part of the problem by recompiliing sudo-ldap from source:

apt-get build-dep sudo-ldap
apt-get source sudo-ldap
cd sudo-1.7.0
./configure && make && make install

After that the settings in ldap.conf are recognized. I can see the debug output which gives me "sudo: Command allowed". The entries are found and everything should work fine.

But when I'm forced to enter my password sudo tells me that it's wrong.

Any advice?

Made a mistake in my previous post.

I configured sudo with

./configure --with-ldap

Now I configured it with

./configure --with-ldap --with-pam

and everything works fine!!!

Hope it works for you too.

Workaround confirmed on latest karmic x86_64, recompiling sudo-ldap fixed problem.

I think we're talking about two separate problems. I recompiled sudo-ldap as above, and still receive:
sudo: setresuid(user_uid, user_uid, ROOT_UID): Operation not permitted
..when I have ssl enabled in /etc/ldap.conf

In my case, the sudoers file is always local and contains:
<username> ALL=(ALL) ALL # Not literally <username>

When I have:
uri ldaps://192.168.1.1/
ssl on
..in /etc/ldap.conf, the above user will receive the above error.
When I change that to:
uri ldap://192.168.1.1/
ssl off
..it goes back to working again.

Matthias/Philipp - Are you using LDAP over SSL or just vanilla LDAP ? Are the users present in /etc/passwd, or only via LDAP ?

@John: You're right, I'm not using SSL, my solution was just for the "sudo not working with ldap in general" issue. Users are only present in the LDAP directory, not in /etc/passwd!

I use vanilla LDAP and the users are only available via LDAP, too.

I can confirm this bug on a box updated from 9.04 to 9.10, where both sudo and su worked just fine.

The test user is only in LDAP, and locally (in /etc/groups) member of the "admin" group. If LDAP uses ssl, su/sudo fail, and they work as advertised with a plain connection.

An other effect which is apparently related: If I launch evince with LDAP/ssl, it takes very long (~15 seconds) to start, and then spits out the message

(evince:2541): GLib-WARNING **: getpwuid_r(): failed due to: Permission denied.

Switching to LDAP/plain, it comes up immediately. Probably this is a glibc (?) and not a su(do) bug.

And as unencrypted LDAP is a security risk, this bug is IMHO a critical one.

This seems like it may be at least partly related to debian bug #545414

Bug still present as of today.

Changing /etc/slapd on the server from:

SLAPD_SERVICES="ldaps:///"

to:

SLAPD_SERVICES="ldap:/// ldaps:///"

and /etc/ldap.conf on the clients from:

uri ldaps://server.my.lan/
ssl on

to:

uri ldap://server.my.lan/
ssl off

'fixed' the problem for me, where 'fixed' means it is no longer authenticating over ssl and validating the server certificate. Not happy.

Is anyone looking at this?

nsswitch included below

greg@blah:$ cat /etc/nsswitch.conf
# /etc/nsswitch.conf
#
# Example configuration of GNU Name Service Switch functionality.
# If you have the `glibc-doc-reference' and `info' packages installed, try:
# `info libc "Name Service Switch"' for information about this file.

# pre_auth-client-config # passwd: compat
passwd: files ldap
# pre_auth-client-config # group: compat
group: files ldap
# pre_auth-client-config # shadow: compat
shadow: files ldap

hosts: files dns
networks: files

protocols: db files
services: db files
ethers: db files
rpc: db files

# pre_auth-client-config # netgroup: nis
netgroup: nis

Any news on this issue?

I still have the issue, I will try on a different install and will
compile the sudo package from the sources.

Actually, sudo is still failling with the message "sudo:
setreuid(ROOT_UID, user_uid): Operation not permitted" when connecting
to an ldaps. Disabling SSL is the only work arround i found so far.

On Sat, Dec 5, 2009 at 4:12 PM, Boris Devouge
<email address hidden> wrote:
> Any news on this issue?
>
> --
> NSS using LDAP on Karmic breaks 'su' and 'sudo'
> https://bugs.launchpad.net/bugs/423252
> You received this bug notification because you are a direct subscriber
> of the bug.
>
> Status in “glibc” package in Ubuntu: Confirmed
>
> Bug description:
> On Karmic (alpha 4 plus updates), changing the nsswitch.conf 'passwd' field to anything with 'ldap' as the first item breaks the ability to become root using 'su' and 'sudo' as anyone but root.
>
> Default nsswitch.conf:
>
> passwd:         compat
> group:          compat
> shadow:         compat
>
> matt@box:~$ sudo uname -a
> [sudo] password for matt:
> Linux box 2.6.31-9-server #29-Ubuntu SMP Sun Aug 30 18:37:42 UTC 2009 x86_64 GNU/Linux
>
> matt@box:~$ su -
> Password:
> root@box:~#
>
> Modified nsswitch.conf with 'ldap' before 'compat':
>
> passwd:         ldap compat
> group:          ldap compat
> shadow:         ldap compat
>
> matt@box:~$ sudo uname -a
> sudo: setreuid(ROOT_UID, user_uid): Operation not permitted
>
> matt@box:~$ su -
> Password:
> setgid: Operation not permitted
>
> Modified nsswitch.conf with 'ldap' after 'compat':
>
> passwd:         compat ldap
> group:          compat ldap
> shadow:         compat ldap
>
> matt@box:~$ sudo uname -a
> [sudo] password for matt:
> Linux box 2.6.31-9-server #29-Ubuntu SMP Sun Aug 30 18:37:42 UTC 2009 x86_64 GNU/Linux
>
> matt@box:~$ su -
> Password:
> root@box:~#
>
> The same arrangements in nsswitch.conf work as expected in Jaunty and earlier releases.
>
> To unsubscribe from this bug, go to:
> https://bugs.launchpad.net/ubuntu/+source/glibc/+bug/423252/+subscribe
>

Confirmed, still having this issue as well.

The workaround turning ssl off works, however (even if this is not a very fortunate solution).

I worked around it by installing nscd..
Why is it that installing this package removes the problem?

With nscd installed ssl=on works for me.

This is also a problem with fuse on ltsp. I could not mount local devices when logging on as a ldap user, only as a local user.
getent shadow does not return any ldap users.
nscd workaround also helps in this case.

Confirmed, I also have this bug.

I can disable ssl or install nscd and it goes away.

NOTE: This is a regression (or design change) from Jaunty.

Installing nscd gets rid of this error message when trying to sudo as an ldap user
sudo: setreuid(ROOT_UID, user_uid): Operation not permitted

With nscd installed and ldaps and ssl on, I can sudo as an ldap user. In addition, I can su from an ldap user to a local user. However, I cannot su from a local user to an ldap user.
nhavens@cslab04:/homes.all/linux/nhavens$ su ladmin
Password:
ladmin@cslab04:/homes.all/linux/nhavens$ su nhavens
Password:
setgid: Operation not permitted
ladmin@cslab04:/homes.all/linux/nhavens$

In /etc/ldap.conf, with the following combination, I still get "Operation not permitted"
ldaps://**********
ssl off

The "Operation not permitted" error, when suing from local to ldap users only goes away when I change ldaps://***** to ldap://******

Is anyone experiencing this bug running an LDAPS server that does NOT have a self-signed certificate? I'm wondering if the issue might be certificate-related, since using plaintext ldap works.

I was not using a self-signed certificate at the time I reported this
bug.

On Jan 17, 2010, at 5:14 PM, David Tomaschik wrote:

> Is anyone experiencing this bug running an LDAPS server that does NOT
> have a self-signed certificate? I'm wondering if the issue might be
> certificate-related, since using plaintext ldap works.
>
> --
> NSS using LDAP on Karmic breaks 'su' and 'sudo'
> https://bugs.launchpad.net/bugs/423252
> You received this bug notification because you are a direct subscriber
> of the bug.
>
> Status in “glibc” package in Ubuntu: Confirmed
> Status in “sudo” package in Debian: Confirmed
> Status in “sudo” package in Kairos Linux: New
>
> Bug description:
> On Karmic (alpha 4 plus updates), changing the nsswitch.conf
> 'passwd' field to anything with 'ldap' as the first item breaks the
> ability to become root using 'su' and 'sudo' as anyone but root.
>
> Default nsswitch.conf:
>
> passwd: compat
> group: compat
> shadow: compat
>
> matt@box:~$ sudo uname -a
> [sudo] password for matt:
> Linux box 2.6.31-9-server #29-Ubuntu SMP Sun Aug 30 18:37:42 UTC
> 2009 x86_64 GNU/Linux
>
> matt@box:~$ su -
> Password:
> root@box:~#
>
> Modified nsswitch.conf with 'ldap' before 'compat':
>
> passwd: ldap compat
> group: ldap compat
> shadow: ldap compat
>
> matt@box:~$ sudo uname -a
> sudo: setreuid(ROOT_UID, user_uid): Operation not permitted
>
> matt@box:~$ su -
> Password:
> setgid: Operation not permitted
>
> Modified nsswitch.conf with 'ldap' after 'compat':
>
> passwd: compat ldap
> group: compat ldap
> shadow: compat ldap
>
> matt@box:~$ sudo uname -a
> [sudo] password for matt:
> Linux box 2.6.31-9-server #29-Ubuntu SMP Sun Aug 30 18:37:42 UTC
> 2009 x86_64 GNU/Linux
>
> matt@box:~$ su -
> Password:
> root@box:~#
>
> The same arrangements in nsswitch.conf work as expected in Jaunty
> and earlier releases.
>
> To unsubscribe from this bug, go to:
> https://bugs.launchpad.net/ubuntu/+source/glibc/+bug/423252/+subscribe

We use LDAP over TLS via PAM for auth, and use NSSWITCH as well. After upgrade from Hardy -> Jaunty -> Karmic, su no longer functioned, however sudo did work.

Here is what I found. When upgrading to Karmic, keeping our old /etc/pam.d/common-auth failed for su. Putting in the default common-auth from a fresh install of Karmic works.

Old /etc/pam.d/common-auth:
auth sufficient pam_ldap.so debug
auth sufficient pam_unix.so try_first_pass likeauth nullok
auth required pam_deny.so
auth required pam_nologin.so

Baseline Karmic /etc/pam.d/common-auth that works (snippet):
# here are the per-package modules (the "Primary" block)
auth [success=2 default=ignore] pam_unix.so nullok_secure
auth [success=1 default=ignore] pam_ldap.so use_first_pass
# here's the fallback if no module succeeds
auth requisite pam_deny.so
# prime the stack with a positive return value if there isn't one already;
# this avoids us returning an error just because nothing sets a success code
# since the modules above will each just jump around
auth required pam_permit.so

Our relavent nsswitch lines:
passwd: files ldap
group: files ldap
shadow: files ldap

Relevant /etc/ldap.conf lines:
ssl start_tls
tls_checkpeer no

So, after "fixing" the common-auth file, su began working for us. Hope that is helpful.

I have the same problem when using tls with a fresh install. But, installing nscd doesn't solve the problem and I have the original comme-auth file...

I was seeing this problem with sudo/sudo-ldap 1.7.0-1ubuntu3 in Lucid. I didn't have nscd installed. Once I installed it, everything came good.

Any progress with this ?
LDAP integration is crucial to so many sites, this is a major blocker for me at present. Would hate to see this end up also in Lucid.

I am running a 64bit karmic.
Is this restricted to 64bit karmic or also present on 32bit karmic?

On my computer libnss_ldap is provided by ia32-libs:
rune@rune-laptop:~$ dpkg -S libnss_ldap
ia32-libs: /usr/lib32/libnss_ldap.so
ia32-libs: /lib32/libnss_ldap-2.10.1.so
ia32-libs: /lib32/libnss_ldap.so.2

Could it not be this library that is causing the problem?
Does it change anything to install libnss-ldap which provides the 64bit version?

Maybe the bug should be against libnss-ldap?

Same problem here with LDAP over TLS. Enabling nscd solves the problem, but as we have a history of crashing nscd daemons this isn't a reliable solution.

We have the same problem at our company, since Karmic Ubuntu is quiet unusable for us :/
Would really appreciate a fix for Lucid or at least a better workaround then running nscd, which here doesn't help at all :(

Looks like it's still busted with latest 'aptiude upgrade' in karmic 64bit Server.
I tried to work-around by copying /lib/libnss_ldap-2.8.so /lib/libnss_ldap.so.2 and /usr/lib/libnss_ldap.so
from Jaunty host to karmic host (as questioned above) but 'su -' still failed.

from root, 'su -' to a localuser is fine, but if we 'su -' from localuser to ldap user we get errors
[root@karmic64 ~]
# su - localuser
localuser@karmic64:~$ su - ldap_user384
Password:
setgid: Operation not permitted

if we 'su -' from root to an ldap user, it works OK
[root@karmic64 ~]
# su - ldap_user384
No directory, logging in with HOME=/
[ldap_user384@karmic64 /]
$

can also 'su -' from ldap user to local user successfully
[ldap_user384@karmic64 /]
$ su - localuser
Password:
localuser@karmic64:~$

- nscd is running but does not fix
- ldap is running over SSL
- 'ssl off' in /etc/ldap.conf and /etc/ldap/ldap.conf does not fix problem
- nsswitch.conf has "files ldap" for group, shadow and passwd

Nice that others can reproduce it too, but bad that no fix seems to be available. Lucid is coming up soon as LTS and will be completly unusable for any company using LDAP authentification......

It would help to understand if the problem is with eglibc, sudo, or libnss-ldap.

Since 8.10 (Intrepid), the Ubuntu Server Guide section regarding LDAP authentication references the auth-client-config and libnss-ldap packages. However, following these instructions when using LDAP with TLS breaks various authentication functions in 9.10 (Karmic). After determining this issue still exists (nearly 7 months later) in the beta release of 10.04 (Lucid), I began looking for new workarounds. I found a package first available in Karmic, nslcd, which also installs libnss-ldapd, nscd, and several other dependencies. All of the latter packages exist in the 'universe' component. I removed the nscd package for testing purposes. Using the configuration provided in the opening post for this bug report, 'su' and 'sudo' now function properly. Although I'm still unsure as to the exact origin of this issue (e.g., eglibc, libnss-ldap, sudo, etc.), replacing libnss-ldap with libnss-ldapd probably provides the core of this workaround.

Bug still present under the latest lucid beta.

Finally got a chance to revisit this after post #29 above. For that servers config I still had a local /etc/passwd entry for the affected account and so was not triggering the described su and sudo symptoms.

On Karmic with:
libnss-ldap 261-2.1ubuntu4
sudo 1.7.0-1ubuntu2.1
login 1:4.1.4.1-1ubuntu2

Without an /etc/passwd entry and an otherwise working libnss-ldap setup sudo returns

sudo: setreuid(ROOT_UID, user_uid): Operation not permitted

and su fails with

su: Authentication failure

Tests:
With libnss-ldap, su and sudo fail.
With nscd and libnss-ldap, su and sudo work.
With libnss-ldapd, with or without nscd, su and sudo work.
As root, getent returns passwd entries correctly for all the above cases.

I'm also experiencing this issue in the latest Lucid beta. When I started investigating, I found that the system does not obey the setuid bit on /bin/su if I'm running it as a user in the LDAP database. However, running as a local user works as expected.

It seems like this wan't an error in the handling of the setuid bit after all. It seems like the EUID is reset to UID after getpwnam is called. Attaching a simple test program. Compile and change the owner to root and set the setuid bit.

Expected output:
EUID0: 0, EUID1: 0

On systems with broken LDAP support (EUID1 will be set to the UID of the process):
EUID0: 0, EUID1: 1000

@Andreas: wow, that explains a lot, especially why installing nscd seems to help.
Because if nscd is not running, the libnss_ldap function getpwnam() resolves to will run in process, and someone in there (libgnutls ?) does the priviledge drop.
Good catch, let's investigate further.

I worked around the problem by recompiling libldap2 with openssl. It seems there is either a bug in GnuTLS or some kind of interface bug between ldap and gnutls.

This is not the first problem I encountered with openldap and gnutls. Maybe we should link openldap with openssl in the distro?

I agree. gnutls is often the source of strange, unforseen errors. I'd also vote to replace it with the reliable and proven openssl.

So linking libldap2 with openssl fixes this issue?

I was suspecting this was the case, I was meaning to try this myself but got caught up in doing more urgent things (paper deadline). I think liking with OpenSSL is an acceptable solution if this fixes this issue. Anyway this bug is really a show stopper for using Ubuntu in a lot of companies, so it should be fixed prior to the LTS release.

Yes, the problem went away, when I installed libldap, which I compiled with openssl.

apt-get source libldap
Edit openldap-2.4.21/debian/configure.options to change --with-tls=gnutls to --with-tls=openssl
(do not forget to install the libssl-dev package and build-depends for openldap)
debian/rules binary-arch

That solves the problem on Karmic. I hope it is going to be solved on the LTS. I was intending to use that for corporate workstations. The fewer manual modifications the better.

I do not know if this solution would be acceptable for Debian, due to some license concerns.

I can confirm that this is a show stopper for lots of companies. For example in the company where I work we are planning to upgrade lots of machines from Hardy to Lucid, but this won't happen before this bug has been solved. And I don't expect we are going to compile ldap with a custom option on each server.

By the way I don't understand the fun of GnuTLS. It has caused us a lot of trouble in the past. Was licensing the only reason to replace openssl?

Why would openssl be a problem?
It is included in debian http://packages.debian.org/sid/openssl

On Wed, Apr 14, 2010 at 06:35:10AM -0000, Bolesław Tokarski wrote:
>
> I do not know if this solution would be acceptable for Debian, due to
> some license concerns.
>

Ubuntu follows the same principle as Debian in terms of licensing.

Openldap cannot be compiled with openssl due to the conflict between the
license of programs linked to libldap and OpenSSL license:

  program (Program license) <-linked-> libldap <-linked-> libssl (OpenSSL license)

We have programs in the archive which license is incompatible with OpenSSL
license.

--
Mathias Gug
Ubuntu Developer http://www.ubuntu.com

Debugging this one proved to be pretty tricky. GDB has a nasty habit of loosing control over the debuggee (bug #563361). I've managed to find out that setuid is called from nptl/allocatestack.c:__nptl_setxid in eglibc, but I haven't succeeded in getting a stack trace. If I understand things correctly, this routine is called from a signal handler installed by pthreads.

I managed to get a core-dump from a test program (a nice little hack that debugs the test application and core dumps it when it executes the setuid syscall) that reproduces the bug. The following stack trace might be of interest:

#0 __nptl_setxid (cmdp=0x7fff1439ad00) at allocatestack.c:1135
#1 0x00007f41dda052fb in __setuid (uid=<value optimized out>) at ../sysdeps/unix/sysv/linux/setuid.c:26
#2 0x00007f41db005124 in lock_pool (n=<value optimized out>) at secmem.c:296
#3 secmem_init (n=<value optimized out>) at secmem.c:477
#4 0x00007f41db0052da in _gcry_secmem_malloc_internal (size=128) at secmem.c:509
#5 0x00007f41db005368 in _gcry_secmem_malloc (size=128) at secmem.c:544
#6 0x00007f41db00084d in do_malloc (n=1000, flags=1000, mem=0x7fff1439adb8) at global.c:730
#7 0x00007f41db00087c in _gcry_malloc_secure (n=1000) at global.c:769
#8 0x00007f41db0130c0 in md_open (h=0x7fff1439ae28, algo=1, secure=<value optimized out>, hmac=<value optimized out>) at md.c:487
#9 0x00007f41db0131ea in _gcry_md_open (h=0x7fff1439af18, algo=1000, flags=<value optimized out>) at md.c:530
#10 0x00007f41dbd03c0f in wrap_gcry_mac_init (algo=<value optimized out>, ctx=0x3e8) at mac-libgcrypt.c:42
#11 0x00007f41dbcea127 in _gnutls_hmac_init (dig=0x7fff1439af10, algorithm=GNUTLS_MAC_MD5, key=0x10afbc0, keylen=24) at gnutls_hash_int.c:277
#12 0x00007f41dbcfad78 in _gnutls_P_hash (algorithm=<value optimized out>, secret=<value optimized out>, secret_size=<value optimized out>, seed=<value optimized out>, seed_size=<value optimized out>, total_bytes=<value optimized out>, ret=0x7fff1439b170 "\231\376~", <incomplete sequence \316>) at gnutls_state.c:811
#13 0x00007f41dbcfafca in _gnutls_PRF (session=<value optimized out>, secret=<value optimized out>, secret_size=<value optimized out>, label=<value optimized out>, label_size=<value optimized out>, seed=0x7fff1439b570 "K\310\331\346-\364\310*~E%\026\223g\216\323K֜\272^1\270Fn\025\254\307`\235%\rK\310\331\345\267\337\023y\314Tn\262-\277\236S\017\362B\237W\220\017\366H\035\372͟5\204\027\001", seed_size=<value optimized out>, total_bytes=48, ret=0x10b2552) at gnutls_state.c:926
#14 0x00007f41dbce883f in generate_normal_master (session=0x10b2530, keep_premaster=0) at gnutls_kx.c:155
#15 0x00007f41dbcf35bb in _gnutls_connection_state_init (session=0x3e8) at gnutls_constate.c:434
#16 0x00007f41dbce43f8 in _gnutls_send_handshake_final (session=0x10b2530, init=1) at gnutls_handshake.c:2472
#17 0x00007f41dbce45d5 in _gnutls_handshake_common (session=0x10b2530) at gnutls_handshake.c:2700
#18 0x00007f41dbce5c67 in gnutls_handshake (session=0x10b2530) at gnutls_handshake.c:2297
#19 0x00007f41dd3196de in ?? () from /usr/lib/libldap_r-2.4.so.2
#20 0x00007f41dd3184a2 in ?? () from /usr/lib/libldap_r-2.4.so.2
#21 0x00007f41dd318703 in ldap_int_tls_start () from /usr/lib/libldap_r-2.4.so.2
#22 0x00007f41dd5338fc in ?? () from /lib/libnss_ldap.so.2
#23 0x00007f41dd533f29 in ?? () from /lib/libnss_ldap.so.2
#24 0x00007f41dd534832 in ?? () from /lib/libnss_ldap.so.2
#25 0x00007f41dd534bbd in ?? () from /lib/libnss_ldap.so.2
#26 0x00007f41dd5352b7 in _nss_ldap_getpwnam_r () from /lib/libnss_ldap.so.2
#27 0x00007f41dda0345d in __get...

Great find, Andreas. So gnutls is calling gcrypt's secure memory functions. And yet, the gnutls docs say these functions are not used by default, and certainly OpenLDAP does not configure gnutls to use them. Something else in the stack must be setting that behavior.

Regardless of what the root cause turns out to be, you guys really need to switch to libnss-ldapd, which will reliably isolate the user apps from whatever junk is going on inside libldap / gnutls / whatever. (And if you're not using the latest version, which also handles pam_ldap, then you need to update.)

Howard, a quick google didn't show much other than package info for libnss-ldapd. Do you have any links to documentation that might be of use? We're about to do a major ldap rollout for our servers at work and I want to know as much as I can ahead of time.

You can find detailed design docs at its home page http://arthurdejong.org/nss-pam-ldapd/

You can also find my LDAPCon2009 presentation on the subject here http://www.symas.com/ldapcon2009/papers/hyc1.shtml

Howard: I really agree that the libnss-ldapd design is much cleaner and a better alternative in the long run (e.g., doing client certificates with libnss-ldap would be "interesting"). However, the documented way [1] of using ldap for authentication uses libnss-ldap, so this should be supported or the documentation needs to be changed. Besides, I think a lot of organizations would be hesitant to migrate to libnss-ldapd.

https://help.ubuntu.com/9.10/serverguide/C/openldap-server.html

I'd be happy to write a patch for the documentation. And given all of the problems with the design (and implementation) of libnss-ldap, I'd say any analysis will show that libnss-ldapd is still the path of lowest risk and greatest stability. (In particular, when used with OpenLDAP nssov.)

Howard, I have longingly looked at libnss-ldapd for almost 4 years now, and absolutely agree it has a better architecture, cleaner code etc., and is a sensible long-term migration path. (The other possibly being sssd.)
  But multiple test migrations in my LDAP deployments always turned up some show-stopper problem or another. The last of these happend 3-4 months ago with ubuntu workstations, running an up-to-date karmic client-side (actually triggered by trying to work around exactly this bug).
  If the server team decides they want to try migrating for lucid, i'd be the first to offer help testing. But I sure don't see this happening before lucid+1.

Disclaimer: haven't tried the caching slapd with nssov yet, only nslcd, because i need at least an incremental migration path.

Right, given the timing for the Lucid release it's probably way too late. I can't comment on your experience with nslcd as I have never used its code or read it in depth. The stub library and nssov have been pretty well tested internally in Symas; since the stub library is almost entirely cookie-cutter code it's known to be bug-free. At the risk of sounding like a commercial, I should note that Symas is offering standalone packages for free evaluation (our SUUMv4 product, based on nssov). A number of our customers have migrated successfully, it's an easy transition.

In the meantime, for this bug, it looks like gcrypt uses its internal secure malloc function if the app didn't set any overrides. I'm not sure that making libldap override the secure malloc is a good idea, since some apps may still want that secure malloc behavior. And any app that explicitly uses gnutls or libgcrypt may get its preference silently overridden by libldap, or vice versa.

Again, the only safe way to address this bug is by taking libldap/nss_ldap out of the application's address space.

Looking at the gcrypt code, it seems this bug should be reported against that; this whole secmem implementation (1) requires a program to be started as root (setuid) and (2) always drops the root priv when it has initialized its secure memory. These behaviors would certainly interfere with any setuid programs normal behavior. Seems like a design flaw in libgcrypt, as the docs http://www.gnupg.org/documentation/manuals/gcrypt/Initializing-the-library.html#Initializing-the-library state that the application is responsible for controlling this behavior. Apps that are unaware that they are using gcrypt (because it came in implicitly through gnutls, thru libldap, thru nss) are SOL.

mathias, could you clarify if there is anything we can do here pre-Lucid release ?

If I can try to summarize the situation on this bug:
- This is a regression from hardy. This use to work on hardy.
- We document using tls for LDAP authentication in the server guide

The most annoying point IMHO, is that someone who has root access to his server with a LDAP account with sudo, will not be able to get root access after upgrading. If we are not able to fix this before release (which is highly probable considering where we are in the release cycle), I think it worth some words in the release notes.

Updating statuses based on Howard feedback in comment 62.

Lucid release note:

Upgrading systems configured to use ldap via ssl as the first service in the nss stack (in nsswitch.conf) leads to a broken nss resolution afterwards (for example sudo would stop working). A workaround is to configure ldap to be used after the compat service in nsswitch.conf before the upgrade is started.

According to the reporter this configuration used to work in jaunty. libgrypt11 version is 1.4.1-2ubuntu1. It seems that something changed/broke between 1.4.1 and 1.4.4.

Mathias: IIRC, the compat workaround only works if you have all users in the local passwd database. You still won't be able to use sudo, or run any other setuid binary that uses nss-services, for users that don't exist in the local passwd database.

The workarounds available AFAIK is:
- install nscd
- or replace libnss-ldap with libnss-ldapd (and nslcd)

Both workarounds worked for me on karmic.

Rune: The first solution (nscd) is not waterproof as nscd tends to crash quite often (and thus revoking your ability to do su or sudo as ldap user)

2010/4/27 Hark <email address hidden>:
> Rune: The first solution (nscd) is not waterproof as nscd tends to crash
> quite often (and thus revoking your ability to do su or sudo as ldap
> user)
>

i use a combination of LDAP + NSCD + cached credentials (ccreds) and i
can do su or sudo without problems.

--
========================
     ^ ^
     O O
    (_ _)
muzzol(a)muzzol.com
========================
jabber id: muzzol(a)jabber.dk
========================
No atribueixis qualitats humanes als ordinadors.
No els hi agrada.
========================
"El gobierno español sólo habla con terroristas, homosexuales y
catalanes, a ver cuando se decide a hablar con gente normal"
Jiménez Losantos
========================
<echelon spamming>
bomb terrorism bush aznar teletubbies
</echelon spamming>

I read all of the diffs between 1.4.1 and 1.4.4 but didn't find any likely suspects. However, tracing the library initialization in gdb, I found the specific problem.

Ordinarily gnutls will initialize the gcrypt library, if no app has done so already. In the gnutls initialization, it specifically turns gcrypt's secure malloc off, and everything works fine.

However, in my trace on Lucid, libnss-ldap is linked to libldap_r, not libldap. And because libldap_r has to support threads, it is required to initialize libgcrypt's thread callbacks, and it must do this before doing anything else with libgcrypt or gnutls.

http://www.gnupg.org/documentation/manuals/gcrypt/Multi_002dThreading.html#Multi_002dThreading

The problem with that is, once we do this thread initialization, libgcrypt considers itself fully initialized. When we next call gnutls's init function, it checks to see if gcrypt is init'd or not, sees that it is, and skips any further init'ing. So the secure malloc stuff remains enabled.

I guess in this case we could do the initialization that gnutls skips, but that's rather ugly, libldap shouldn't have to know or duplicate the initialization steps inside gnutls_global_init(). Alternatively, libgcrypt could be changed to not call its global_init() right after setting the thread callbacks, since it's obvious that the caller still has other initialization calls that it needs to make. (Frankly I think this is the correct option.)

Finally, gnutls_global_init() could be changed to check for initialization_finished, instead of initialization_started. (i.e., check for GCRYCTL_INITIALIZATION_FINISHED_P, instead of GCRYCTL_ANY_INITIALIZATION_P). But this latter is pretty dicey, gnutls really has no way to know if it should be meddling in a half-initialized libgcrypt or not.

I'm trying really hard not to say "I told you so" again, but I just can't stop myself.

Probably the best fix: don't call global_init when setting the thread callbacks.

There is always the chance of something crashing, no matter how you fix this problem!
Of course having a sudoable (or root) account in the local passwd database would remove a lot of the uncertainty that comes from relying on network for resolving users.

Several has commented that nscd is unstable. However, I see no launchpad bug documenting this.
I have had no problems with nscd and it seems muzzol hasn't either.
Also, I have had no problems with libnss-ldapd.

So both are possible workarounds.
The suggested release note from mathias gug doesn't really provide an acceptable workaround as it essentially disables the use of ldap for resolving users.

Potential gnutls fix: do gcrypt initialization as long it isn't already finished. probably a bad idea.

Rune: just google for nscd problems, it has a long history of stability issues. But on top of the issues caused by poor implementation, it also has problems due to an inherently inadequate design. Some of these issues are outlined in my LDAPCon presentation linked above. All of this is well documented, I don't think it bears repeating in this already-too-long bug report. (Just bringing this report up on my Seamonkey browser drags the browser to its knees.)

The nscd workaround does work for me under Lucid.
I haven't had a problem using nscd under 9.10 either, though my systems have very light traffic.

Added this text to the release notes:

== NSS resolution breaks with LDAP over SSL in Ubuntu Server ==

Upgrading systems configured to use LDAP via SSL as the first service in the NSS stack (in /etc/nsswitch.conf) leads to broken NSS resolution afterwards such that applications like sudo would stop working. A workaround is to configure the "ldap" service to be used after the "compat" service in /etc/nsswitch.conf before the upgrade is started. (Bug:423252)

Please reopen if the text needs fixing to mention nscd instead.

Please use the Release note from the description instead of the one in comment 66.

Updated text with:

== NSS resolution breaks with LDAP over SSL in Ubuntu Server ==

Upgrading systems configured to use LDAP via SSL as the first service in the NSS stack (in /etc/nsswitch.conf) leads to broken NSS resolution afterwards such that `setuid` applications like `sudo` would stop working. To work around this, switch to the libnss-ldapd package instead of libnss-ldap before the upgrade, or use nscd. (Bug:423252)

This bug was introduced in karmic, so upgrade from karmic to lucid shouldn't be affected.
But for instance all upgrades from last LTS release (not only Ubuntu Server) will get affected.

I don't see that this requires any changes to the release notes text, though.

This bug also affects new installations.

Installing nscd does not fully fix the problem as it leaves su unable to su from an ldap user to another ldap user and a local user unable to su to an ldap user. Error is "setgid: Operation not permitted"

libpam-ldapd/libnss-ldapd does not support all the features that libpam-ldap/libnss-ldap does. Primarily all the pam_* configuration options are not supported leaving you unable to support limiting authentication to subsets of user on subsets of computers.

That's unfortunate, I didn't realize libpam-ldapd was so incomplete. You can still use nssov for full pam support.

Your best option for an immediate fix is still the libgcrypt patch I posted. Without that basically all Karmic and Lucid nss-ldap+SSL installations are dead in the water. As a longer term step, the design of libgcrypt and gnutls needs revisiting. Midterm, migrate everyone to nssov.

Could you provide a link to explanation of what nssov is and maybe how to use it?

http://www.openldap.org/devel/cvsweb.cgi/~checkout~/contrib/slapd-modules/nssov/README?rev=1.11

It's an overlay for OpenLDAP slapd which implements all of the nss and pam calls, replacing Arthur deJong's nslcd.

For me both workarounds in the release notes aren't very usable, unfortunately.
I can't use libnss-ldapd as the pam_check_host_attr and pam_check_service_attr options in libnss-ldap are essential for me, but as Ian Gordon wrote libnss-ldapd doesn't support these.
The other solution (nscd) is usable, but only as long as nscd doesn't crash. The past told us nscd isn't the most stable daemon in the world unfortunately.

Also confirming that installing nscd was successful as a workaround for me. Also worth noting that my symptoms of this issue matched those explained above. This is on a fresh 10.04 install.

My experience with libnss-ldap is that nscd allows sudo but not su. Also the GUI authentication doesn't work.

I just got around to testing libnss-ldapd and I can verify that both sudo and su work there.

Ubuntu's gui authentication process does not work though. When you try to perform a privileged action like adding a printer it opens the "Authentication" dialogue and asks for a password. When you enter an ldap password it returns "bad password" and the entire printer config system freezes. The following errors are from auth.log:

May 7 11:53:11 engine cupsd: pam_unix(cups:auth): check pass; user unknown
May 7 11:53:11 engine cupsd: pam_unix(cups:auth): authentication failure; logname= uid=0 euid=0 tty=cups ruser= rhost=localhost
May 7 11:53:11 engine cupsd: pam_unix(cups:account): could not identify user (from getpwnam(lquigley))

I'm sorry, my previous comment didn't hold up to scrutiny. A reboot seems to have gotten rid of those errors....

The GUI authentication dialogue is asking for the root password rather than the password of the logged in user. Does anyone know if this is the intended behavior?

I was a bit disheartened to see this is an actual bug - be it because of X package or not, of the two workstations at my desk, the fedora13 box has no problems (and uses nscd) and the ubuntu one does.

Reading here, I saw the libnss-ldapd suggestion, tried it, worked fine. Regarding Hark's comment about pam_check_host_attr and pam_check_service_attr (comment #87), I would bet you could solve that another way. Solution I already had in place for another reason entirely was the follows, and works just fine for host-based restrictions:

1) create a group in ldap for the host "hostgrp" - and put the users in that group you want.
2) use pam_succeed_if.so to restrict to the group you want.
In common-auth, before pam_ldap.so, put:
auth requisite pam_succeed_if.so user ingroup hostgrp quiet_success

This will cause pam to only allow someone in group hostgrp to log in. Remove them from hostgrp, they can't log in. That's what you're wanting that nslcd/libnss-ldapd isn't allowing, right? Well, PAM has been around longer than nscd/nslcd anyway ;)

BTW, in case it isn't obvious, pam_succeed_if statements can be stacked - the chain goes one to the next, same as when you're using one mod then another.

I have the same symptoms on Lucid:
With nscd I can at least su to root but not to any other LDAP user

Googlebait: This also breaks apache2 suexec, as initgroups() fails, resulting in the error "failed to setgid".

same here (like comment #92). I can su to any local user not just root. Trying to su to any LDAP user results in 'su[***]: initgroups failed for user `atest': Operation not permitted'

nslcd is a fail on lucid for me. Trying to start from upstart fails. Running it by hand in debug mode works but when trying to su from one LDAP user to another it again fails:

# service nslcd start
 * Starting LDAP connection daemon nslcd nslcd: unable to daemonize: No such device

Seems to work in debug mode
# /usr/sbin/nslcd -d
nslcd: DEBUG: add_uri(ldaps://10.xx.xx.xx)
nslcd: DEBUG: add_uri(ldaps://10.xx.xx.xxx)
nslcd: DEBUG: ldap_set_option(LDAP_OPT_X_TLS_REQUIRE_CERT,0)
nslcd: version 0.7.2 starting
nslcd: DEBUG: unlink() of /var/run/nslcd/socket failed (ignored): No such file or directory
nslcd: DEBUG: setgroups(0,NULL) done
nslcd: DEBUG: setgid(126) done
nslcd: DEBUG: setuid(117) done
nslcd: accepting connections

When I try to su to another user however, more fail:
[2]# sudo -u nslcd nslcd -d
nslcd: DEBUG: add_uri(ldaps://10.12.51.165)
nslcd: DEBUG: add_uri(ldaps://10.14.13.250)
nslcd: DEBUG: ldap_set_option(LDAP_OPT_X_TLS_REQUIRE_CERT,0)
nslcd: version 0.7.2 starting
nslcd: DEBUG: unlink() of /var/run/nslcd/socket failed (ignored): No such file or directory
nslcd: cannot setgroups(0,NULL) (ignored): Operation not permitted
nslcd: DEBUG: setgid(126) done
nslcd: DEBUG: setuid(117) done
nslcd: accepting connections
nslcd: [8b4567] DEBUG: connection from pid=13359 uid=0 gid=1000
nslcd: [8b4567] DEBUG: nslcd_passwd_byname(user333)
nslcd: [8b4567] DEBUG: myldap_search(base="ou=HDA,ou=DC,o=FMW", filter="(&(objectClass=posixAccount)(uid=user333))")
nslcd: [8b4567] DEBUG: ldap_initialize(ldaps://10.12.51.165)
nslcd: [8b4567] DEBUG: ldap_set_rebind_proc()
nslcd: [8b4567] DEBUG: ldap_set_option(LDAP_OPT_PROTOCOL_VERSION,3)
nslcd: [8b4567] DEBUG: ldap_set_option(LDAP_OPT_DEREF,0)
nslcd: [8b4567] DEBUG: ldap_set_option(LDAP_OPT_TIMELIMIT,0)
nslcd: [8b4567] DEBUG: ldap_set_option(LDAP_OPT_TIMEOUT,0)
nslcd: [8b4567] DEBUG: ldap_set_option(LDAP_OPT_NETWORK_TIMEOUT,0)
nslcd: [8b4567] DEBUG: ldap_set_option(LDAP_OPT_REFERRALS,LDAP_OPT_ON)
nslcd: [8b4567] DEBUG: ldap_set_option(LDAP_OPT_RESTART,LDAP_OPT_ON)
nslcd: [8b4567] DEBUG: ldap_set_option(LDAP_OPT_X_TLS,LDAP_OPT_X_TLS_HARD)
nslcd: [8b4567] DEBUG: ldap_simple_bind_s(NULL,NULL) (uri="ldaps://10.12.51.165")
nslcd: [8b4567] connected to LDAP server ldaps://10.12.51.165
nslcd: [8b4567] DEBUG: ldap_result(): end of results
nslcd: [7b23c6] DEBUG: connection from pid=13359 uid=0 gid=1000
nslcd: [7b23c6] DEBUG: nslcd_passwd_byname(user333)
nslcd: [7b23c6] DEBUG: myldap_search(base="ou=HDA,ou=DC,o=FMW", filter="(&(objectClass=posixAccount)(uid=user333))")
nslcd: [7b23c6] DEBUG: ldap_initialize(ldaps://10.12.51.165)
nslcd: [7b23c6] DEBUG: ldap_set_rebind_proc()
nslcd: [7b23c6] DEBUG: ldap_set_option(LDAP_OPT_PROTOCOL_VERSION,3)
nslcd: [7b23c6] DEBUG: ldap_set_option(LDAP_OPT_DEREF,0)
nslcd: [7b23c6] DEBUG: ldap_set_option(LDAP_OPT_TIMELIMIT,0)
nslcd: [7b23c6] DEBUG: ldap_set_option(LDAP_OPT_TIMEOUT,0)
nslcd: [7b23c6] DEBUG: ldap_set_option(LDAP_OPT_NETWORK_TIMEOUT,0)
nslcd: [7b23c6] DEBUG: ldap_set_option(LDAP_OPT_REFERRALS,LDAP_OPT_ON)
nslcd: [7b23c6] DEBUG: ldap_set_option(LDAP_OPT_RESTART,LDAP_OPT_ON)
nslcd: [7b23c6] DEBUG: ldap_set_option(LDAP_OPT_X_TLS,LDAP_OPT_X_TLS_HARD)
nslcd: [7b23c...

Regarding the pam_check_host_attr and pam_check_service_attr options of pam_ldap, nslcd has a pam_authz_search option that can replace functionality of those options (and much more). This option has been in nss-pam-ldapd since version 0.7.4.

Any other feature requests (and bug reports) are very much welcome on the nss-pam-ldapd-users mailing list.

Regarding the previous comment, could you file a bug report on the nss-pam-ldapd package if you think the problem is there? From the log it shows that only three lookups for the user "user333" came in. No authentication requests were done for that user (at least not through nslcd).

We had the same problem over here - upgrading from Debian Lenny to Testing...
The problem is caused by a change in the configuration files, which have apparently changed in the different versions of openldap. Reinstall ldap to get the original configuration files. /etc/ldap.conf seems to be the new location for the ldap configuration.

The steps in detail:
apt-get install --reinstall *all ldap-related packages *
When asked if you want to update "/etc/pam.d/common-*", say YES
copy "/etc/libnss-ldap.conf" or "/etc/pam_ldap.conf" to "/etc/ldap.conf"
run "getent passwd"

All bests, skulda

My bug 509734 was marked as a duplicate of this one. This was a special case using the atd job scheduler. At jobs launched by ldap users worked, but at jobs launched by root did *not* work. atd was doing a group lookup, and nss was dropping privileges, thus breaking root-launched at jobs. To work around this, I added the following line to my /etc/ldap.conf:

nss_initgroups_ignoreusers <users>

where <users> is the list of local non-ldap users, particularly root! There is a script that is part of nss-ldap that does this for you:

/usr/sbin/nssldap-update-ignoreusers

Atd is finally working for me now.

On Wed, Sep 22, 2010 at 22:26:31 -0000, greenmoss wrote:
> My bug 509734 was marked as a duplicate of this one. This was a special
> case using the atd job scheduler. At jobs launched by ldap users worked,
> but at jobs launched by root did *not* work. atd was doing a group
> lookup, and nss was dropping privileges, thus breaking root-launched at

Yeah, I found this behavior on in my test machine (where I'm
running Lucid), too.

Based on the discussion earlier in this bug (423252), I did some
testing of the behavior of "atd" with various combinations of
libpam/nss-ldap, nscd, and libpam/nss-ldapd.

As greenmoss found, when I was running with libpam/nss-ldap and
no nscd (and didn't have any of the users in question listed in
the "ignoreusers" line), my "at" commands worked for LDAP users
but not for ones defined in /etc/passwd. (When an LDAP user
attempted to run an "at" command, the following syslog message
would appear:
  atd[<PID>]: Cannot delete saved userids: Operation not permitted

However, I found that when nscd was running... the situation was
reversed: "at" commands did work for LDAP-defined users, but not
for /etc/passwd-defined ones (and attempts to use "at" as one of
those users would cause the same error message as above to show
up in the syslog).

When I had libpam/nss-ldapd installed (with or without nscd),
the "at" command worked fine for both types of users.

> jobs. To work around this, I added the following line to my
> /etc/ldap.conf:
>
> nss_initgroups_ignoreusers <users>
>
> where <users> is the list of local non-ldap users, particularly root!

In the "libpam/nss-ldap, no nscd" case, this also worked in my
tests; listing the /etc/passwd-defined user in the ignoreusers
line did allow "at" to work for that user (and it continued to
work for the LDAP-defined user as well).

Interestingly, this change did NOT help in the "libpam/nss-ldap,
with nscd" case -- even with LDAP username listed in the
ignoreusers line, when I tried to run "at" as the LDAP-defined
user, the command still failed and "atd" still generated the
same syslog error message....

(libpam/nss-ldapd does not reference the /etc/ldapd.conf file,
so the ignoreusers line doesn't affect that test case.)

[For what it's worth, I tested "cron" using those same
combinations of NSS/PAM resolution libraries but didn't find any
situation where it failed...]

      Nathan

On Fri, Sep 24, 2010 at 16:46:25 -0000, Nathan Stratton Treadway wrote:
> As greenmoss found, when I was running with libpam/nss-ldap and
> no nscd (and didn't have any of the users in question listed in
> the "ignoreusers" line), my "at" commands worked for LDAP users
> but not for ones defined in /etc/passwd. (When an LDAP user
> attempted to run an "at" command, the following syslog message
> would appear:
> atd[<PID>]: Cannot delete saved userids: Operation not permitted
>
> However, I found that when nscd was running... the situation was
> reversed: "at" commands did work for LDAP-defined users, but not
> for /etc/passwd-defined ones (and attempts to use "at" as one of
> those users would cause the same error message as above to show
> up in the syslog).

I'm sorry, I seem to have managed to jumble the succeeded/failed
statuses given in those two paragraphs...

Hopefully the following table is more clear:

without ncsd:
  passwd user: failed (and "Cannot delete" syslog message appeared)
  LDAP user: succeeded

with ncsd running (and after restarting the "atd" service):
  passwd user: succeeded
  LDAP user: failed (with same "Cannot delete" syslog message)

Nathan

For completeness' sake, another bug tracker with the same issue

https://bugs.g10code.com/gnupg/issue1181

bug still present in Maverick 10.10

this is the output with debug option enabled:

$ sudo -u news /usr/sbin/fetchnews
LDAP Config Summary
===================
uri ldap://127.0.0.1/
ldap_version 3
sudoers_base ou=sudoers,dc=homelinux,dc=doma
binddn (anonymous)
bindpw (anonymous)
timelimit 30
ssl (no)
===================
sudo: ldap_initialize(ld, ldap://127.0.0.1/)
sudo: ldap_set_option: debug -> 0
sudo: ldap_set_option: ldap_version -> 3
sudo: ldap_set_option: timelimit -> 30
sudo: ldap_sasl_bind_s() ok
sudo: found:cn=defaults,ou=sudoers,dc=homelinux,dc=doma
sudo: ldap sudoOption: 'timestamp_timeout=30'
sudo: ldap sudoOption: 'env_reset'
sudo: setuid(): Operation not permitted

Building the openldap source with openssl instead of gnutls libraries and installing the resulting libldap package works here.

On Wed, Mar 23, 2011 at 08:35:56PM -0000, Peter Matulis wrote:
> Building the openldap source with openssl instead of gnutls libraries
> and installing the resulting libldap package works here.

Yes, but that results in combinations of software that are not
redistributable in the archive because there's GPL software that uses
libldap.

--
Steve Langasek Give me a lever long enough and a Free OS
Debian Developer to set it on, and I can move the world.
Ubuntu Developer http://www.debian.org/
<email address hidden> <email address hidden>

Yeah it was more as confirmation. I'm going to test a patched libgcrypt (comment #73) next.

I just tried Howard's patch from #73 this morning, using the libgcrypt11_1.4.4-5ubuntu2_amd64.deb source files to roll a new libgcrypt11 package. I can now su to root from accounts not in the local password file database, before I could not. That was on a Lucid 10.04.2 LTS vm. Next week sometime we might be able to test Apache2/phpsuexec for a larger base of user accounts.

I also tested a patched libgcrypt11 package according to comment #73. I can now su from a local user to a non-local user and have a non-local user use sudo (sudo-ldap).

Just a follow up to #106. We have been running with the libgcrypt11 patch from #73 with a couple thousand openldap and AD users using Apache2/phpsuexec on Lucid 10.04.2 64 bit for months now with no troubles.

Thanks a lot, works like a charm. I wish i could be of any help to
you, saved me a lot of time.

2011/10/4 cdmiller <email address hidden>:
> Just a follow up to #106.  We have been running with the libgcrypt11
> patch from #73 with a couple thousand openldap and AD users using
> Apache2/phpsuexec on Lucid 10.04.2 64 bit for months now with no
> troubles.
>
> --
> You received this bug notification because you are subscribed to the bug
> report.
> https://bugs.launchpad.net/bugs/423252
>
> Title:
>  NSS using LDAP+SSL breaks setuid applications like su, sudo, apache2
>  suexec, and atd
>
> Status in Release Notes for Ubuntu:
>  Fix Released
> Status in “eglibc” package in Ubuntu:
>  Invalid
> Status in “libgcrypt11” package in Ubuntu:
>  Confirmed
> Status in “libnss-ldap” package in Ubuntu:
>  Invalid
> Status in “sudo” package in Ubuntu:
>  Invalid
> Status in “eglibc” source package in Lucid:
>  Invalid
> Status in “libgcrypt11” source package in Lucid:
>  Confirmed
> Status in “libnss-ldap” source package in Lucid:
>  Invalid
> Status in “sudo” source package in Lucid:
>  Invalid
> Status in “eglibc” source package in Maverick:
>  Invalid
> Status in “libgcrypt11” source package in Maverick:
>  Confirmed
> Status in “libnss-ldap” source package in Maverick:
>  Confirmed
> Status in “sudo” source package in Maverick:
>  Invalid
> Status in “eglibc” source package in Karmic:
>  Invalid
> Status in “libgcrypt11” source package in Karmic:
>  Won't Fix
> Status in “libnss-ldap” source package in Karmic:
>  Invalid
> Status in “sudo” source package in Karmic:
>  Invalid
> Status in “libgcrypt11” package in Debian:
>  Confirmed
> Status in “sudo” package in Debian:
>  Confirmed
> Status in “sudo” package in Kairos Linux:
>  Confirmed
>
> Bug description:
>  On Karmic (alpha 4 plus updates), changing the nsswitch.conf 'passwd'
>  field to anything with 'ldap' as the first item breaks the ability to
>  become root using 'su' and 'sudo' as anyone but root.
>
>  Default nsswitch.conf:
>
>  passwd:         compat
>  group:          compat
>  shadow:         compat
>
>  matt@box:~$ sudo uname -a
>  [sudo] password for matt:
>  Linux box 2.6.31-9-server #29-Ubuntu SMP Sun Aug 30 18:37:42 UTC 2009 x86_64 GNU/Linux
>
>  matt@box:~$ su -
>  Password:
>  root@box:~#
>
>  Modified nsswitch.conf with 'ldap' before 'compat':
>
>  passwd:         ldap compat
>  group:          ldap compat
>  shadow:         ldap compat
>
>  matt@box:~$ sudo uname -a
>  sudo: setreuid(ROOT_UID, user_uid): Operation not permitted
>
>  matt@box:~$ su -
>  Password:
>  setgid: Operation not permitted
>
>  Modified nsswitch.conf with 'ldap' after 'compat':
>
>  passwd:         compat ldap
>  group:          compat ldap
>  shadow:         compat ldap
>
>  matt@box:~$ sudo uname -a
>  [sudo] password for matt:
>  Linux box 2.6.31-9-server #29-Ubuntu SMP Sun Aug 30 18:37:42 UTC 2009 x86_64 GNU/Linux
>
>  matt@box:~$ su -
>  Password:
>  root@box:~#
>
>  The same arrangements in nsswitch.conf work as expected in Jaunty and
>  earlier releases.
>
>  Lucid Release Note:
>
>  == NSS via LDAP+SSL breaks setuid applications like sudo ==
>
>  Upgrading systems configured to use ldap over ssl a...

I tested with Ubuntu 12.04 today and the nscd work-around no longer works. The failure occurs with or without running nscd on Ubuntu 12.04.

PPA for the patch suggested by Howard Chu's in comment #73

https://launchpad.net/~nutznboltz/+archive/howard-chu-libgcrypt11-patch-for-ldap-clients

According to Andreas Metzler
http://lists.debian.org/debian-legal/2011/02/msg00006.html
{{ GnuTLS upstream has added support for different crypto backends in
2.11.x and has chosen nettle as prefered backend (2.10.x is using
libgcrypt). }}

I have started to experiment with using a gnutls26 package with nettle
instead of libgcrypt11 on Ubuntu 12.04.

I have yet to adjust the gnutls26 package dependencies, at this point
I just cheat and install nettle-dev manually:

sudo apt-get install nettle-dev

Then I

apt-get source gnutls26

to fetch the source for gnutls26-2.12.14

chop out
--with-libgcrypt
from debian/rules

bump the package version in debian/changelog to 2.12.14-5ubuntu2.1

and rebuild with

debuild -i -uc -us -b

then I put a checkpoint on the VM and install the package:

dpkg -i libgnutls26_2.12.14-5ubuntu2.1_amd64.deb

but then sudo works on my LDAP+SSL client.

This bug no longer pertains to be as it is opened against libgcrypt11 now and to me this is now a GnuTLS backend selection bug.

I put the patched gnutls into this PPA as my preferred solution.

https://launchpad.net/~nutznboltz/+archive/gnutls26-with-nettle

I meant to type "no longer pertains to me" not "to be".

I am unsubscribing from this bug report.

Rebuilding against nettle is no solution for lucid. This bug is not a duplicate.

This will never be fixed in Lucid and Lucid has the "use nscd" work-around.

I can confirm that the "use nscd" workaround no longer works in the current Precise beta. This will cause anyone updating from the current LTS to the forthcoming LTS to be unable to run su, sudo, apache2 suexec, and atd from LDAP accounts.

Confirmed for Precise beta.

Please, for the sake of corporate and government customers (where LDAP and/or Kerberos is very likely to be deployed), elevate the priority of this bug. I gather that this is of not much interest for the majority of private desktop users, but in more professional environments, this is a showstopper, meaning that such distribution simply cannot be deployed in our corporate networks.

It is probably best to migrate to either nss-pam-ldapd, sssd or nss-pam-ldapd in combination with the nssov slapd overlay.

re #108 (cdmiller)

Is this fix still a workable solution for you and have you run into any issues not yet experienced on this bug?

Thanks

Re #120 (adam-stokes)

The best workable solution for me would be working official packages for Lucid and Pangolin. Working LDAP authn/z over TLS is baseline functionality for us (servers and academic computer labs).

I've had no problems with the patch from #73 thus far on our Lucid servers. Most traffic is Apache php/suexec. Day to day use is sudo/su for sysadmins. Have not noticed any side effects. We've been running this way since 2011-04-11.

Currently planning to test nutznbotz #113 gnutls using nettle and adejong #119 nss-pam-ldapd, but not until summer when we test Pangolin for production.

Thanks canonical folks and patch contributors for all the great work on this.

Just to add something that has nothing to do directly with this bug, but is related: we have been using SSSD for quite a while now, using Timo Aaltonen's PPA <https://launchpad.net/~sssd/+archive/updates> and could not be happier. In my opinion SSSD is the superior solution for all things authn/authz, tying together LDAP, Kerberos and PAM. There are still some minor Ubuntu-related issues with the ordering of the involved PAM modules but everything works very well, apart from that.

I think Ubuntu would do well to properly integrate and incorporate SSSD, especially in the Server edition.

Hi all,

this bug has been brought to my attention by my boss today.
If I understand the situation correctly, the problem is:

• OpenLDAP links against GnuTLS (gnutls26)
• gnutls26 links against gcrypt, which has the bug
• gnutls28 links against nettle, but also gmp which is LGPLv3+
• OpenLDAP thus can’t link against gnutls28, as it has reverse
  dependencies that are not LGPLv3-/GPLv3-compatible
• the package affected is libnss-ldap though

For some reason, neither nscd nor unscd seem to be able to
work around this bug, so it has become rather critical (e.g.
for use in company networks).

Why not do a readline and provide *two* versions of the
OpenLDAP client libraries, keep libldap-2.4-2 linked
against gnutls26 and add another shared library plus
development package (with at least the two shared library
packages coïnstallable) to link against gnutls28 and build
these BOTH from the SAME source package at the SAME time,
so an upload of OpenLDAP will not need another package to
be (re-)built to stay in sync.

Did anyone think of it already and will shoot this idea
down immediately? Or could it work?

bye,
//mirabilos • <email address hidden>
--
tarent solutions GmbH
Rochusstraße 2-4, D-53123 Bonn • http://www.tarent.de/
Tel: +49 228 54881-393 • Fax: +49 228 54881-314
HRB AG Bonn 5168 • USt-ID (VAT): DE122264941
Geschäftsführer: Boris Esser, Elmar Geese

Thorsten Glaser <email address hidden> writes:
> Why not do a readline and provide *two* versions of the
> OpenLDAP client libraries, keep libldap-2.4-2 linked
> against gnutls26 and add another shared library plus
> development package (with at least the two shared library
> packages coïnstallable) to link against gnutls28 and build
> these BOTH from the SAME source package at the SAME time,
> so an upload of OpenLDAP will not need another package to
> be (re-)built to stay in sync.
>
> Did anyone think of it already and will shoot this idea
> down immediately? Or could it work?

I proposed this with openssl in Debian before (#579647), but it looks
like it was merged with my original bug report in gcrypt later and then
got lost.

Regards,
Ansgar

Ah right, there would be the usecase of LDAP with SSL used by non-GPL-compatible programmes.
So the proper fix is to have three sets of LDAP (client) libraries. The rest of the packages (server
and utilities) can then be built against whatever of those the maintainers see best fit.

3 sets of LDAP client libraries? That sounds like a terrible solution.

Fwiw, I wrote a version of OpenLDAP's TLS support that could use any/all of OpenSSL, GnuTLS, and MozillaNSS simultaneously, and never released it, because it seemed that would be too confusing if separate apps had different expectations of TLS config options. But it would certainly be possible to add libltdl support in, and make libldap dynamically load a single TLS implementation. I still don't see any technical merit in supporting anything besides OpenSSL.

On Wed, Apr 25, 2012 at 09:14:58AM -0000, Howard Chu wrote:
> I still don't see any technical merit in supporting anything besides
> OpenSSL.

As soon as someone provides an OpenSSL that it's legal for us to link to in
a Linux OS product, instead of with a wink and a nod to the GPL, we'd be
elated to discuss the technical merits of the SSL implementations.

In the meantime, I agree that we don't want to try to address this with
multiple LDAP client libraries. Unmarking openldap as affected.

My point being, if you want to accommodate multiple TLS libraries simultaneously with only a single libldap, that code is still available in the OpenLDAP git repo. The relevant changes are between a225b02f17fe79f6680d5d31db37320981e24774..4dff3e6807fb3451405373c2b85e02ccf27b882f

Let me point out my non-technical, management-like point of view to this:

For my company it would have been a discussable way to put Ubuntu LTS with paid support to a row of our Desktops. But with this issue it is a complete nogo...
Rating this as a "high" issue isn't going far enough, for enterprises this is a major blocker aka. showstopper.
I would have thought that this would be the utmost interest for Canonical, 'cause supportcontracts is what their business is build upon.

I'm not technically skilled to point to a solution for this and I respect the licensing problem and that nobody wants multiple sets of ldap client libraries. All I want to say is: There must be a solution to this kind of issues to get enterprises to buy support for Ubuntu.

I fully agree with kibe. While the whole licensing issues certainly lead to a big mess, but this bug is an even bigger issue in any enterprise / medium to large scale environment (at least those not using Kerberos). There has to be a some viable solution. How do other Linux distributions handle this? Do they all just ignore the licensing issues?

In any case the solution is not to leave this bug open for over two years and ignore the numerous suggestions that have been made. Especially now that the nscd workaround is broken in 12.04 LTS.

A quick workaround that solved my problem with this:

First install libnss-ldap
Configure ldap stuff, test if you can login but can't use setuid apps.
Then install nslcd and configure.

After these steps my system is working, and allowing me to login with ldap account and use sudo.

This bug is pretty serious, any enterprise who has more than one linux box, is using ldap to authenticate.
And this is a show stopper......

This should be fixed ASAP.

I forgot to mention that I used 12.04 and 10.04 with the same results.

Sonic, did you do any special config?
With nslcd we are running into the next problem:

# /etc/init.d/nslcd start
 * Starting LDAP connection daemon nslcd
nslcd: Warning: /lib/x86_64-linux-gnu/libnss_ldap.so.2: undefined symbol: _nss_ldap_enablelookups (probably older NSS module loaded)
wich leeds us straight to
https://bugs.launchpad.net/ubuntu/+source/nss-pam-ldapd/+bug/917208
and still not getting groups etc.

If you are seeing something like:
  Warning: /lib/x86_64-linux-gnu/libnss_ldap.so.2: undefined symbol: _nss_ldap_enablelookups (probably older NSS module loaded)
It means that you probably have libnss-ldap installed instead of libnss-ldapd (note the extra d). Using nslcd works best with libnss-ldapd and libpam-ldapd.

This is busted in 12.04 and we cant use sudo with LDAP and SSL.

Kind of a show stopper when it comes to enterprise deployments

I

I my recent experience, installing libnss-ldapd and libpam-ldapd (which in turn install nslcd and uninstall libnss-ldap and libpam-ldap) fixes the problem on 11.10 and 12.04.

-sbi

See here's the deal...

Yes, the problem seems to occur with starttls being enabled but given the choice...

- libnss-ldap
- libpam-ldap
- nscd

or

- libnss-ldapd
- libpam-ldapd
- nslcd

The first one fails if starttls is used to connect to the ldap server
The second one fails to respect 'pam_check_host_attr yes' setting

So I have problems no matter what I do. It's time for Ubuntu to clean this up as I was able to function w/ Lucid (libnss-ldap/libpam-ldap/nscd) but have only 2 lousy choices in Precise

You can replace "pam_check_host_attr yes" with
  pam_authz_search (&(objectClass=posixAccount)(uid=$username)(|(host=$hostname)(host=$fqdn)(host=\\*)))
See the nslcd.conf manual page for more details (the 0.7 series doesn't have the fqdn value yet).

Btw, you can use libpam-ldap fine together with libnss-ldapd if you prefer.

Also note that nslcd is no replacement for nscd. nslcd doesn't do much caching and nscd (or unscd) can still be used to reduce the load on your LDAP server.

The only real things that are missing in nss-pam-ldapd are nested groups and LDAP password policies. Patches are welcome ;)

adejong@138

Put that entry into /etc/nslcd.conf as you suggested (and as the man page suggests) and removed my 'host' attribute which should prevent me from logging into my upgraded 12.04 system but it didn't. (NSCD & NSLCD running, libnss-ldapd and libpam-ldapd installed & configured)

Obviously I can't expect the pam_check_host_attr in /etc/ldap.conf to work with this configuration but I did hope that the nslcd.conf would work and prevent me from being able to login - it didn't. This was after a reboot so I'm reasonably sure that nothing was cached in nslcd or nscd that would have impacted.

craig-white@139

Please file a bug against nslcd to track the problem with pam_authz_search.

Also, in general, bug reports for any missing features are welcome.

adejong@139

https://bugs.launchpad.net/ubuntu/+source/nss-pam-ldapd/+bug/992737

Thanks

This bug was fixed in the package libgcrypt11 - 1.5.0-3ubuntu1

---------------
libgcrypt11 (1.5.0-3ubuntu1) quantal; urgency=low

  * Do not call global_init when setting thread callbacks (LP: #423252)
 -- Adam Stokes <email address hidden> Tue, 15 May 2012 13:56:17 -0400

Precise debdiff for SRU

Uploaded your debdiff to precise-proposed with two small changes:
 - Targeted precise-proposed instead or precise
 - Changed version to -0ubuntu0.1 instead of -0ubuntu1

natty-proposed debdiff

Hello Matt, or anyone else affected,

Accepted libgcrypt11 into precise-proposed. The package will build now and be available in a few hours. Please test and give feedback here. See https://wiki.ubuntu.com/Testing/EnableProposed for documentation how to enable and use -proposed. Thank you in advance!

Hello Matt, or anyone else affected,

Accepted libgcrypt11 into oneiric-proposed. The package will build now and be available in a few hours. Please test and give feedback here. See https://wiki.ubuntu.com/Testing/EnableProposed for documentation how to enable and use -proposed. Thank you in advance!

Hello Matt, or anyone else affected,

Accepted libgcrypt11 into natty-proposed. The package will build now and be available in a few hours. Please test and give feedback here. See https://wiki.ubuntu.com/Testing/EnableProposed for documentation how to enable and use -proposed. Thank you in advance!

Hello, Martin.

Any chance of getting that for lucid?

Thank you in advance!

Yes, it'll be SRUed for lucid too, I'm just waiting for Adam to finish preparing the debdiff and I'll review and upload it to lucid-proposed.

Hello Martin,

I was affected by the bug ann I confirm that using the latest packages from proposed solves the problem with my company's setup for LDAP.

Thanks a lot.
Regards.

Hi mcguire.

Which release of Ubuntu do the updated packages solve the problem on for you? There are fixes available for natty, oneiric, and precise.

Thanks!

Ubuntu 12.04 Precise

Lucid-proposed debiff

not "committed" until it's available in -proposed

Hello Matt, or anyone else affected,

Accepted libgcrypt11 into lucid-proposed. The package will build now and be available in a few hours. Please test and give feedback here. See https://wiki.ubuntu.com/Testing/EnableProposed for documentation how to enable and use -proposed. Thank you in advance!

This bug was fixed in the package libgcrypt11 - 1.5.0-3ubuntu0.1

---------------
libgcrypt11 (1.5.0-3ubuntu0.1) precise-proposed; urgency=low

  * Do not call global_init when setting thread callbacks (LP: #423252)
 -- Adam Stokes <email address hidden> Wed, 16 May 2012 13:35:06 -0400

Hey, raof,

I have tested the lucid-proposed version and it works fine. sudo no longer segfaults.

Thanks for the package! I am waiting for it to arrive at lucid-updates!

Any chance of it getting to lucid-updates anytime soon?

This bug was fixed in the package libgcrypt11 - 1.4.4-5ubuntu2.1

---------------
libgcrypt11 (1.4.4-5ubuntu2.1) lucid-proposed; urgency=low

  * Do not call global_init when setting thread callbacks (LP: #423252)
 -- Adam Stokes <email address hidden> Thu, 24 May 2012 16:31:52 -0400

Thank you!

The patch applied to libgcrypt breaks other software: https://bugs.launchpad.net/ubuntu/+source/libgcrypt11/+bug/1013798

After reviewing this bug again I've outlined a possible course of action:

a) Revert global_init patch in all supported distro's
b) Lucid users continue to use the nscd workaround.
c) Precise, Quantal, etc - rebuild gnutls without --with-libgcrypt in order to make use of nettle.

Unfortunately, the version of gnutls in Lucid is older than the required gnutls version for nettle support. So I do not believe there will be much that we can do as far as a supportable option is concerned.

I will need to speak with some other engineers about getting option c pushed through[1].

[1] http://pad.lv/926350

Forcing use of nscd is a non-starter at many sites. Aside from cache staleness issues, and nscd's well known instability, there's also the issue that nscd doesn't intercept get*ent enumerations so things will still crash depending on which nsswitch functions an app calls.

It would make sense to use nettle on the newer releases that support it, but keep the patch in place otherwise.

This additional patch fixes the crash in bug#1013798.

Oops. The attachment in comment#166 includes the patch in #73, it is not incremental.

Thanks Howard,

I'll get this reviewed and tested this week.

Adam

Are there any testers for Natty, Oneiric for this issue? I'd like to try and get this one closed out.

Thanks
Adam

The fix for this bug has been awaiting testing feedback in the -proposed repository for oneiric for more than 90 days. Please test this fix and update the bug appropriately with the results. In the event that the fix for this bug is still not verified 15 days from now, the package will be removed from the -proposed repository.

See http://bugs.debian.org/368297 http://bugs.debian.org/658896 for an alternative patch to fix this bug that don't introduced any regression (LP: #1013798).

The patch applied to libgcrypt (no-global-init-thread-callbacks.diff) should be reverted and the following patch http://bugs.debian.org/cgi-bin/bugreport.cgi?msg=135;filename=fix-dropping-privileges-by-libgcrypt-secmem.diff;att=2;bug=368297 should be applied to OpenLDAP.

Actual comment on the Debian bug history with the patch and the explanation: http://bugs.debian.org/658896#104

Changes made above as per eol https://wiki.ubuntu.com/Releases

The libcrypt11 package was released for precise and lucid and marked won't fix for the others, so removing the verification-needed tag.