[i945] Running Out of Memory with composited managers (UXA bug)

@Jonathan, for all of these tests you use the 2.6.30 kernel from karmic right? If so, does the leak also repro using the jaunty kernel? Also, I'd like to know exactly how do you measure this "cached memory", i.e. what command do you run and what number are you looking at?

Yes I use right now the kernel from Karmic as KMS is not present in 2.6.28. The leak appears with the default kernel too but as the performances are worst for me with UXA, I had switched back to EXA and thought the UXA was too young for me. But KMS is only available with UXA so I've a little more investigated and with no solution, open this bug.

I measure the cached memory with the system monitor of the gnome panel on one side and using the "free" command on the other side. When the memory is said filled with cache (about 800M of 1G) the swap is beeing used.

Typically:

             total used free shared buffers cached
Mem: 1004576 962184 42392 0 3396 820128
-/+ buffers/cache: 138660 865916
Swap: 1959920 161820 1798100

That with only compiz and firefox and gajim opened (and gnome-terminal).
The "real" cache has been cleared i.e. running:
echo 3 | sudo tee /proc/sys/vm/drop_caches

Is all this answering your questions?

I'm just running a live Jaunty, architecture i386 (my installed system is amd64).

This is the xorg.conf I need to reproduce the problem:

Section "Device"
        Identifier "Configured Video Device"
        Option "AccelMethod" "UXA"
EndSection

Section "Monitor"
        Identifier "Configured Monitor"
        Option "PreferredMode" "1920x1200"
EndSection

Section "Screen"
        Identifier "Default Screen"
        Monitor "Configured Monitor"
        Device "Configured Video Device"
EndSection

I've run a little my bash loop and now free shows:
$ free
             total used free shared buffers cached
Mem: 1018452 960456 57996 0 2936 714888
-/+ buffers/cache: 242632 775820
Swap: 1959920 410200 1549720

One could see that the swap is largely used.
I'm running compiz with the default settings (so changing nothing to the defaults as compiz is automatically launched).
The PreferredMode line makes my display show 1920x1200 as I've an external display and the default would use a resolution supported by both screens (1024×768).

So it's related neither to architecture nor to kernel.

Maybe you could check if it necessary to have the external monitor to reproduce the bug?

It seems that you have been doing a good job in documenting this. Would you mind reporting this upstream? http://intellinuxgraphics.org/how_to_report_bug.html

Results of the day:

1) The external monitor plays nothing here:
I've booted the live Jaunty VGA disconnected, switched to UXA; same symptoms.

2) The leak seems related to the screen resolution i.e. 1920×1200 window open/close fills my memory quicker than with 1280×800 screen.

I'm now going to report this upstream as suggested.

Upstream report done.
https://bugs.freedesktop.org/show_bug.cgi?id=21770

Thanks for reporting this bug upstream. I have subscribed to it. I think they like to have the necessary information attached to the bug, at least Xorg.0.log. Could you upload (upstream) the Xorg.0.log you get with the 2.7.99 driver and no external monitor (to keep it simple)?

Okay, I've uploaded it upstream. But something I don't exactly understand is the relationship between these two bug reports. Is the one upstream to make devels know the problem exists and where to find informations about it or is it a standalone duplicate of the one here at launchpad? If so, why should I fill a field called URL (on the other one)? Yes, I'm quite new to bug reporting.

The upstream report is supposed to be complete, so that the upstream devs don't have do fiddle around too much to find the necessary information. Of course, there should be a link to the ubuntu bug report (if there is one, it is also possible to report directly upstream if you know what you're doing) for cross-referencing.

That being said, the bug reports are not really duplicates. At the ubuntu level, we know what versions of the driver are included in different releases (upstream doesn't), what kind of tweaking has been done to specific packages, etc. We know (well, think we know) what information upstream requires for different kinds of bugs, and if some newer packages need to be tested before reporting upstream. Usually, a ubuntu bug report will be much more messy than the upstream report. We then forward the relevant information upstream when we think we have all we need.

In your case you pretty much got everything right in the first round (including testing with newest development snapshots), so it could more or less be upstreamed directly.

Btw, here is some more information about the getting the ubuntu bug reports complete (triaging) and upstreaming:
General triaging for ubuntu: https://wiki.ubuntu.com/Bugs/HowToTriage
Triaging X bugs: https://wiki.ubuntu.com/X/Triaging

We could always use some more people to help with triaging and you are welcome to help.

Thanks for all these very interesting informations. I really didn't see the
things clearly. Is this said somewhere in the general bug reporting process that
I missed?
Of course I would be glad to help triaging but for now I'm in the middle of exams.

And about this particular bug report, the upstream report I made is not very
good reading your links… I hope it will be understood tough as this is a very
annoying problem. Btw, I'm wondering why nobody else experiences this. One
couldn't use UXA and compiz without notice the problem!

PPAs have been updated today to 2.7.99.1+git20090519.09beee37-0ubuntu0sarvatt~jaunty but no change on this issue, still happening.

The PPA is updated quite often (every few days), since it is taken directly from the master branch of the development tree. It is unlikely (but possible) that it will go away automatically before your upstream bug is addressed.

I'm not sure if there is currently any links to the wiki pages during the bug reporting process. The one that should probably be referenced if possible is https://wiki.ubuntu.com/X/Reporting . I actually assumed you had read those documents since you filed a very good bug report here and already knew about ubuntu-x-swat and xorg-edgers.

I'm affected by another bug that lock all when using fullscreen on mplayer but it seems to have been corrected in the 6th rc on the kernel so I'm waiting for that before reporting more. I just thought It could be the same on this issue here.

About the bug reporting process, no I hadn't read any page but what comes when opening a new bug. It was said to use "ubuntu-bug" and so did I. I don't thing your links are given but it may be a good idea to put them somewhere.
About ubuntu-x-swat and xorg-edgers, I've just found them when searching around before reporting as I'm interested in all kind of aspect of my system and always try to learn more when the occasion is here.

Can't other people reading that try compiz+UXA and see if the memory get filled? I can't believe to be the only facing this issue. Intel cards are well widespread AFAIK. (in case I've something wrong on my laptop)

> Can't other people reading that try compiz+UXA and see if the memory get filled? I can't believe to be the only facing this issue. Intel cards are well widespread AFAIK. (in case I've something wrong on my laptop)

There are small differences between computer models using the same
intel chipsets. For example, I have a 965GM chipset, but I am not
affected by the critical bug 359392 at all. Someone with the exact
same laptop model as you have should be able to reproduce it (what is
your model?). I don't think a lot of people have experimented with KMS
yet. I haven't and kudos to those who have.

Okay I see. The problem is I have no "standard" laptop but one assembled, based
on a barebone and customized. The barebone I have is of course no longer
available and I don't have any clue on what it is except the informations given
by lspci et al. If useful, the company that sold me the laptop is called
Transtec, based in Germany. May a special BIOS or customization from them alter
something and lead to this problem??? I don't think so.

Just to remember: this issue reproduces on a live Jaunty too (no KMS, no hacking
from me)…

I'm experiencing the same "cache grows" issue with driver 2.7.99 and kernels 2.6.30, but a i915 card. So the problematic code must be common.

It took me some time to figure that was not a kernel issue, but a video driver one. I can confirm caches take about half of my 500 MB of RAM, arent' dropped by issuing "echo 3 > /proc/sys/vm/drop_caches", and don't decrease even when the system is dying by lack of memory.

Thanks for comment, I'm not alone!

To notify: xorg-edges have just published the development xserver:
1.6.1.901+git20090523+server-1.6-branch.5cd5a012-0ubuntu0sarvatt2
The issue is still here (but KMS is now instantaneous :))

New rc6 of 2.6.30 kernel is packaged, no advance for this bug too (but corrected indeed my other bug).

I have the same problem too.

I have an Intel 945GM. The problem occurs for me while using both Jaunty's and Karmic's kernel. Upgrading xorg-xserver-video-intel to 2.7.1 from ubuntu-x-swat doesn't solve the problem either. I use the 32-bit version of Ubuntu with the generic kernel.

Bryce: For the record, I'm using composited Metacity here, and also seeing the bug.

The upstream bug report (https://bugs.freedesktop.org/show_bug.cgi?id=21770 comment #10) mentions a possible duplicate (https://bugs.freedesktop.org/show_bug.cgi?id=20704) which has been fixed on the 15th of June via commit d027e8feff7d38cccadc6aaccc0454b21ce4dca0 (as mentioned in comment #54 of the second bug). That commit has arrived in xorg-edgers by now. For me [1] the issue is fixed, I get no more leaking memory using the above test. Note that the patch is for the mesa component [2].

[1] 2.6.30 kernel from mainline, xorg-edgers (20090617), compiz on Jaunty, with the GM965/GL960 chip.
[2] git://git.freedesktop.org/git/mesa/mesa

Are you sure it has entered mesa master? The comments only say the patch is good, but no trace of inclusion. Please confirm so that we can give that information upstream.

Anyway, I'm still seeing this bug, which is not the very same as the one that has been fixed. Using mesa from xorg-edgers, 2009-06-18.

I've checked again, yes I believe it's in the master branch of mesa:
http://cgit.freedesktop.org/mesa/mesa/commit/?id=d027e8feff7d38cccadc6aaccc0454b21ce4dca0
(The comment of the first bug said the patch is in review, while the comment on the second bug specifically mentions it has been commited).

The ChangeLog file (copy of 'git log') of "mesa 7.5.0~git20090619+mesa-7-5-branch.abfd56c2-0ubuntu0sarvatt" (via apt-get source mesa) also mentions this patch.
I've checked the actual source to be sure and the patch is in there too.

The problem specifically on my system was that with 500MB out 2GB of RAM filled, the swap space started growing until it was filled completely(2.5GB), apart from that I run the same versions and such as the original bug reporter. With the patch this issue doesn't appear anymore. (I forget to mention the details of the memory problem I see before and I do here to ensure I'm not talking about a different issue).

Hope this is helpful.

Yes the fixes are in mesa 7.5 branch

http://cgit.freedesktop.org/mesa/mesa/commit/?h=mesa_7_5_branch&id=9dfce365c7f35ddea6d81b7f595ddcd6d35382a5

http://cgit.freedesktop.org/mesa/mesa/commit/?h=mesa_7_5_branch&id=d027e8feff7d38cccadc6aaccc0454b21ce4dca0

I dont see the glXDestroyGLXPixmap fix in mesa master (7.6) yet though so the 7.6 on edgers for karmic is still affected until the next 7.5 branch merge.

This bug could well be duplicate of 360319
I.e. you probably don't need 3d composition effects to trigger and reproduce it.

Please open a new bug if you are having different problems, this bug is very specific to using GL compositing managers under UXA only. If you are having memory leaks outside of using compiz/mutter/kwin 3d compositing it is most likely a problem elsewhere that needs to be addressed.

xorg-edgers has been updated so mesa for both jaunty and karmic include the upstream fixes now if you can test it Jonathan. I am getting ~400MB gem objects after 27 hours uptime now and the fixes look good to me, I would wrap around to a negative number after 1.5gb or so in 12 hours previously. I have been using metacity's xrender compositing for the past month and a half that I've had this problem and gem objects didnt go over 220MB even after weeks of uptime using that.

Happy news! I'll test that as soon as I return home, so on thuesday the
25th only. Sorry for the delay but I'm 1000km away and have only ssh
acces that isn't enough.

Tested just now with all updates from xorg-edgers. But in fact, I can't test as I'm suffering from another bug:
http://bugs.freedesktop.org/show_bug.cgi?id=22428
that just make crash any 3D application, including of course compiz.
Now I've no 3D more. That's very annoying. Any idea ?

Okay, bug is effectively corrected :)
Good work !