Too many repositories "dynamic mmap ran out of room"

Bug #24626 reported by Karl Hegbloom
280
This bug affects 2 people
Affects Status Importance Assigned to Milestone
apt (Debian)
Fix Released
Unknown
apt (Ubuntu)
Fix Released
Low
Michael Vogt

Bug Description

The "dynamic mmap" error happens when there are too many repositories, or more
accurately, too many packages total over all repositories. I tried to have more
than one Ubuntu mirror in the list. It went to update package listings, and
then gave that error and hung, not refreshing it's display. I killed its
window, and now it fails to run when I try to run it again so I can edit the
sources.list and remove some. I'll have to edit sources.list myself using "sudo
vi /etc/apt/sources.list" and comment off the extra repositories.
Alternatively, I could set the size of the "dynamic mmap", whatever that is...
;-) I think that the GUI tool should do that for me or at least explain WTF it
means and attempt to correct this likely to be common error condition. It
should at least let me run the sources configuration dialog agian.

The reason I wanted two mirrors was to have it get them from the fastest one...

Revision history for this message
Douglas Green (douglas-s-green) wrote :

I had a similar error, but it seemed in my case to be due to a specific repository having issues with the availability of AMD64 software, rather than "too many repositories" in general. Here was my error for comparison:

E: Dynamic MMAP ran out of room.
E: Error occurred while processing kerberos4kth-kdc (NewVersion1).
E: Problem with MergeList /var/lib/apt/lists
archive.ubuntu.com_ubuntu_dists_hoary_universe_binary-amd64_Packages
E: The package lists or status file could not be parsed or opened

After that error occurred, it would not display packages. But removing that repository fixed the issue for me. Note: I am running Dapper Drake Flight 3 on a recently fresh install.

Michael Vogt (mvo)
Changed in apt:
status: Unconfirmed → Confirmed
Revision history for this message
Michael Vogt (mvo) wrote :

You can use the "APT::Cache-Limit" option to increase the mmap cache limit that apt uses. It is currently set to "12*1024*1024" (12582912).

Try:
$ sudo apt-get update -o APT::Cache-Limit=25165824

Cheers,
 Michael

Revision history for this message
Someone (s4910321931-deactivatedaccount) wrote :

Then apt tools should provide a hint to increase the APT::Cache-Limit

Revision history for this message
Thomas Schwinge (tschwinge) wrote :

I was also just hit by that. Why is there a need at all to have such a fixed limit for apt?

Revision history for this message
Loye Young (loyeyoung) wrote :

This is an old, known, upstream issue. The reason for the limit is to prevent the cache from getting so large that it crowds out more important files on whatever partition /var is on. The Debian developers tend to be very conservative and figure that it's better to have a low limit and protect low resource systems than have a high limit and end up with an unbootable system. There have been several discussions about removing the limit, but the conventional wisdom still seems to be to keep it.

Personally, I think that a 12 MB limit is ludicrously low, even for low-resource systems. (It would be better to put limits on log files than on the apt cache.) When I run into the issue, I raise the limit to a rather large number, like 120MB. Granted, it's unlikely that the cache should ever get that big, but that's the point: the limit should prevent a runaway cache size but not ever get in the way of anything useful.

A better, more permanent way to fix the issue is to create a cache file in /etc/apt/apt.conf.d/, like this:
# echo 'APT::Cache-Limit "125829120" ' > /etc/apt/apt.conf.d/30cache

The only issue for releasing this as a patch is to ensure that the update doesn't cause problems for corner cases where the sysadmin knew about and relied on the cache limit. A possible solution is to have the upgrade script calculate a limit programmatically to the lower of:
(1) an arbitrary percentage (say, 5%) of the total disk space on the partition where /var resides;
(2) an arbitrary percentage (say 25%) of the total free disk space on the partition where /var resides; and
(3) an arbitrary, but high, absolute limit (say, 125829120).

Using the example limits I have listed, the limit would be set lower on machines where the /var directory is on a partition of less than 2 GB or where the free space remaining on that partition is less than approximately 500 MB.

Happy Trails,

Loye Young
Isaac & Young Computer Company
Laredo, Texas
http://www.iycc.biz

Revision history for this message
Ray Ward (ray-ward4) wrote :

This is the same as Bug #113424. If anybody is stuck repairing this follow the bug number there is an explanation on how to repair it.

Revision history for this message
dpkg-install (akqjt98) wrote :

My problem is different I believed,

This is cross compiled with arm-gcc-4.1.1, using it apt-0.7.6 code,

When I run
>apt-get update
Ign file: ./ Release.gpg
Ign file: ./ Release
Reading package lists... Error!
E: Couldn't make mmap of 16777216 bytes - mmap (22 Invalid argument)
W: Unable to munmap
E: The package lists or status file could not be parsed or opened.

after changed cache-limit to
>apt-get update -o Apt::Cache-Limit=33554432
Ign file: ./ Release.gpg
Ign file: ./ Release
Reading package lists... Error!
E: Couldn't make mmap of 33554432 bytes - mmap (22 Invalid argument)
W: Unable to munmap
E: The package lists or status file could not be parsed or opened.

here is the df -h
 df -h
Filesystem Size Used Available Use% Mounted on
/dev/mtdblock2 48.0M 15.1M 32.9M 31%
192.168.1.22:/local 255.3G 200.8G 55.6G 22% /usr/local

the package is mounted through the nfs, to /usr/local

I haven't put the dpkg to the system yet? Any clue,

Thanks in advance

Revision history for this message
Otonte (oegenia-sterogen) wrote :

Hi everyone :)

Related issue to qa-hardy desktop, Douglas Green and Michael Vogt is am confronted with the following issue:

Synaptic Downloader reports:
'E:Dynamic MMap ran out of room,
E:Error occurred while processing libmyth-0.20 (NewVersion1),
E:Problem with MergeList /var/lib/apt/lists/gb.archive.ubuntu.com_ubuntu_dists_gutsy_multiverse_binary-i386_Packages,
E:The package lists or status file could not be parsed or opened.'

I was helped by Bhavani and other similar comments for the few first steps:

     sudo apt-get update -o APT::Cache-Limit=23000000; (Yes, I pushed it to 23 000 000!!!)

     The # sign was missing before the line "deb cdrom:[Ubuntu 7.10 _Gutsy Gibbon_ - Release i386 (20071016)]/ gutsy main restricted"
     I added it: "#deb cdrom:[Ubuntu 7.10 _Gutsy Gibbon_ - Release i386 (20071016)]/ gutsy main restricted" in /etc/apt/sources.list

     I unchecked everything in System->Administration->Software Sources->Third Party Software.

but still face the above error report.
[note: My processor is an AMD64... I am a bit confused with the i386 release: is this processor related in any way?]

PS: attached are screenshots of the various stages I completed.

I am still investigating, but earlier comments seem more enlightening than my current understanding.

Revision history for this message
Otonte (oegenia-sterogen) wrote :
Revision history for this message
Otonte (oegenia-sterogen) wrote :

I think this BUG should be noted with a High importance:

1) Where the Headline states it resulted from too many mirrors being listed, this was not the case for me.

2) The Synaptic downloader but ALSO the Upgrade manager and ANY trial to install packages falling within a reasonable size lead to the same report (as shown above).

3) Expanding indefinitelly the Cache leads to a real slow down in terms of efficiency as it is overtaking other partition.

4) This Bug is affecting a lot of people and cannot be resolved from the user side.

5) It also freezes any upgrade which means users will consider other distros rather than re-installing an unfixed Ubuntu version.

Revision history for this message
Loye Young (loyeyoung) wrote :

If you are still having trouble after raising the cache limit, try one or both of the following:

1) Clean the package cache, viz:
# aptitude clean

2) Clean up your repository list:

   2.1) Ensure that your repository lists in /etc/apt/sources.list refer to not more than one release (i.e., one of dapper, edgy, feisty, gutsy, hardy, etc.).

    2.2) Ensure that your repository lists in /etc/apt/sources.list refer to a limited number of full repositories. Don't include several mirrors of the same repository, and don't include competing repositories (i.e., ubuntu OR debian OR mepis, etc., but not more than one).

    2.3) update your package lists, viz:
# aptitude update

3) As always, report back whether these suggestions helped, so that others can benefit.

Happy Trails,

Loye Young
Isaac & Young Computer Company
Laredo, Texas
http://www.iycc.net

Revision history for this message
Runaway1956 (runaway1956) wrote :

Oh-kay now.

When my problem popped up, I was unable to update, upgrade, or anything of the sort. Adept and synaptic both failed to start, and apt-get commands terminated with errors. Errors included the phrase The package lists or status file could not be parsed or opened.

I did a search on that phrase, and found several pages. Almost all of those that I opened referred to a cache limit. Rather than adjust the cache limit, I just edited my /etc/apt/apt.conf.d/30cache with a comment (#) in front of the single line found there.

Adept is now updating for me.

It seems pretty obvious that my system COULD BE overwhelmed by the monster cache from hell - but the likelihood of that happening seems pretty slim. ;)

If I run into problems, I will come back to report that my unsupported fix didnt work so well.........

Revision history for this message
Denis Rut'kov (dendron2000) wrote :

I just ran into this problem. However, it may be a bit different, the problem appears only when I use not standard LOCALE (LANG=UTF8.RU_ru in my case). When I use LANG=C the problem disappears.

Revision history for this message
Karl Hegbloom (karl.hegbloom) wrote : Re: [Bug 24626] Re: Too many repositories "dynamic mmap ran out of room"

On Sun, 2009-08-09 at 12:07 +0000, Denis Rut'kov wrote:
> I just ran into this problem. However, it may be a bit different, the
> problem appears only when I use not standard LOCALE (LANG=UTF8.RU_ru in
> my case). When I use LANG=C the problem disappears.

I think that's probably because of the LANG=UTF8.RU_ru characters taking
up more than one byte each, where LANG=C characters are essentially 1
byte ASCII. So, for the same text in the package descriptions, etc, it's
using roughly twice as much memory, so no wonder it runs out of room.

--
Karl Hegbloom <email address hidden>

Changed in apt (Debian):
status: New → Fix Released
Revision history for this message
rusivi2 (rusivi2-deactivatedaccount) wrote :

Thank you for posting this bug.

As per upstream bug this is fixed. Marking as such.

Changed in apt (Ubuntu):
status: Confirmed → Fix Released
Revision history for this message
Philip Muškovac (yofel) wrote :

Confirmed, it's fixed in maverick.

Revision history for this message
C de-Avillez (hggdh2) wrote :

This was fixed on 0.7.26~exp12ubuntu1 on Maverick.

To post a comment you must log in.
This report contains Public information  
Everyone can see this information.

Other bug subscribers

Related questions

Remote bug watches

Bug watches keep track of this bug in other bug trackers.