process limit unlimited (regression)

Bug #391761 reported by Kees Cook
274
This bug affects 4 people
Affects Status Importance Assigned to Milestone
pam (Ubuntu)
Fix Released
Low
Kees Cook

Bug Description

AFAIK, RLIMIT_NPROC is determined by the kernel based on available physical memory. In my 512M VMs, I see the following results of "ulimit -u":

dapper: unlimited
hardy, intrepid: 4095
jaunty, karmic: unlimited

The intention is to have this set to in an attempt to reasonably mitigate fork-bombs without getting in the way of intentionally big process collections. Jaunty and Karmic appear to have regressed. I am assuming this is a PAM bug, but it may be a kernel issue. Tracking RLIMIT setting has always eluded me. :)

Related branches

Revision history for this message
Kees Cook (kees) wrote :
Changed in pam (Ubuntu):
importance: Undecided → High
status: New → Confirmed
Kees Cook (kees)
Changed in pam (Ubuntu):
importance: High → Low
security vulnerability: no → yes
tags: added: regression-release
Revision history for this message
Steve Langasek (vorlon) wrote :

Right, side-effect of having fixed the 027_pam_limits_better_init_allow_explicit_root patch so that it's operational again.

I guess we need to duplicate the kernel's logic here for setting the NPROC default.

Changed in pam (Ubuntu):
status: Confirmed → Triaged
Revision history for this message
dronus (paul-geisler) wrote :

Maybe just deliver a reasonably large static limit in /etc/security/limits.conf for all users? That would quick fix the security concerns till this bug is fixed and obsoletes it.

Revision history for this message
dronus (paul-geisler) wrote :

Please switch this to importance high! This isn't only a small issue with fork bombs:

I just stopped my system just by accidently holding down "enter" for a second while a video file in nautilus was highlighted. This spawned a vast amount of VLC players which in turn results in excessive swapping.
Recovery took about 1/2h just to login on the text console prompt and killall them. Thus any normal user is easily capable bringing down the whole system without proper limits.

Kees Cook (kees)
Changed in pam (Ubuntu):
assignee: nobody → Kees Cook (kees)
status: Triaged → Fix Committed
Revision history for this message
astrostl (astrostl) wrote :

ETA or process for fix release? We're using workarounds in an HPC environment.

Revision history for this message
Steve Langasek (vorlon) wrote :

For oneiric this will be fixed soon with an updated pam package. For existing Ubuntu releases, we have not committed to providing an update. A workaround for this is to add a line such as the following to /etc/security/limits.conf:

* hard nproc 4096

(you may wish to adjust the value of the limit according to the resources of your environment)

Or you can disable pam_limits entirely in /etc/pam.d if you aren't otherwise using it, which would cause the system to fall back to the kernel defaults.

Revision history for this message
Steve Langasek (vorlon) wrote :

Fixed in oneiric with the merge of pam 1.1.2-3. Changelog:

pam (1.1.2-3) unstable; urgency=low

  [ Kees Cook ]
  * 027_pam_limits_better_init_allow_explicit_root: load rlimit defaults
    from the kernel (via /proc/1/limits), instead of continuing to hardcode
    the settings internally. Fall back to internal defaults when the kernel
    rlimits are not found. Closes: #620302. (LP: #746655, #391761)

  * Updated debconf translations:
    - Vietnamese, thanks to Clytie Siddall <email address hidden>
      (closes: #601197)
    - Dutch, thanks to Eric Spreen <email address hidden> (closes: #605592)
    - Danish, thanks to Joe Dalton <email address hidden> (closes: #606739)
    - Catalan, thanks to Innocent De Marchi <email address hidden>
      (closes: #622786)

Changed in pam (Ubuntu):
status: Fix Committed → Fix Released
Revision history for this message
astrostl (astrostl) wrote :

Lucid LTS?

Revision history for this message
Jan Rathmann (kaiserclaudius) wrote :

I just did the "fork bomb test" (entering :(){ :|:&};: in a terminal) on my Oneiric installation, and it still crashes the system.
This should not happen if I get this report right, so perhaps the report should be re-opened?

A workaround for me is adding lines like

* hard nproc 4096

to /etc/security/limits.conf (as suggested above by Steve Langasek).

Revision history for this message
Steve Langasek (vorlon) wrote : Re: [Bug 391761] Re: process limit unlimited (regression)

On Mon, Nov 28, 2011 at 11:03:45AM -0000, Jan Rathmann wrote:
> I just did the "fork bomb test" (entering :(){ :|:&};: in a terminal) on
> my Oneiric installation, and it still crashes the system.

How does it "crash"?

What does 'ulimit -u' show from a terminal on this system?

> This should not happen if I get this report right, so perhaps the report
> should be re-opened?

Probably not. PAM is now correctly getting its defaults from the kernel; if
one of those defaults is inadequate, there should be a new bug filed against
the kernel.

--
Steve Langasek Give me a lever long enough and a Free OS
Debian Developer to set it on, and I can move the world.
Ubuntu Developer http://www.debian.org/
<email address hidden> <email address hidden>

Revision history for this message
Jan Rathmann (kaiserclaudius) wrote :

Am 28.11.2011 17:17, schrieb Steve Langasek:
> How does it "crash"?
I apologize if the term "crash" was a bit unprecise. It does not crash
in sense of hard-locking (where only Reset switch will revive it again),
but it becomes totally unresponsive due to excessive swapping.

Yesterday I let the fork bomb run for ~1 hour to give the system time to
reach the process limit (the default one, nothing was specified in
limits.conf) and become responsive again, but this did not happen in
this time period. However, it still reacted to Alt+Print+e, which means
all tasks are terminated (and all unsaved work is being lost).

Tomorrow I made an attempt to log into TTY1 (Ctrl+Alt+F1) while the fork
bomb was running for ~10 minutes. The TTY appeared after 1-2 minutes,
but actually logging was not finished after ~5-10 minutes when I lost
patience and pressed Alt+Print+e again.

So the system does not "crash" literally but becomes effectively
unusable, with all unsaved work being probably lost.

>
> What does 'ulimit -u' show from a terminal on this system?

31517 is the output. My system has 4 GB of RAM.

Do you think it is appropriate for this case to open a new bug report
against the kernel?

Kind regards,
Jan

Revision history for this message
Steve Langasek (vorlon) wrote :

On Tue, Nov 29, 2011 at 07:52:52AM -0000, Jan Rathmann wrote:
> > What does 'ulimit -u' show from a terminal on this system?

> 31517 is the output. My system has 4 GB of RAM.

> Do you think it is appropriate for this case to open a new bug report
> against the kernel?

Yes, please. If the limit calculation needs to be changed, this should be
done in the kernel.

From what I see, 31517 processes in 4GB comes out to an average of 133K per
process. That seems like an unrealistically low amount of per-process
memory for common usage, and I think the process ulimit should probably be
lowered.

--
Steve Langasek Give me a lever long enough and a Free OS
Debian Developer to set it on, and I can move the world.
Ubuntu Developer http://www.debian.org/
<email address hidden> <email address hidden>

To post a comment you must log in.
This report contains Public Security information  
Everyone can see this security related information.

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.