Comment 11 for bug 720095

Revision history for this message
Stefan Bader (smb) wrote :

So this is what I found out so far: whenever a client connects to vsftpd, it forks of a process to handle the connection. This is done in a way that also duplicates the network namespace (beside of the process namespace). This can actually be observed by the fact that every time that happens there is a message about "lo: Disabled Privacy Extensions" (which is slightly stupid to note as that is the default for lo). Anyway, so cloning the network namespace also sets up the snmp mib structures and those are allocated by pcpu.

The main problem seems to be that cleaning up those structures is done in Lucid for each interface on its own by putting it onto a work queue. This seems to be rather slow, so while the test case is running its seems the system is too busy with creating new namespaces than it is able to clean them up. Even after stopping the test this takes a while and because the way that vmalloc pcpu areas are handled can potentially stick even longer (the areas are not exclusively used by network namespace, something else may use parts of the area and the area cannot be cleaned up until the last user is gone).

Between 2.6.32 and 2.6.35, there was a series of changes that allowed to batch the cleanup of network namespace. The comment on one of those indicated that 4000 namespaces would have taken more than 7 minutes before and could be reduced to 44 seconds (at the price of an increased cpu load). I was able to backport all required changes and this seems to avoid the build up of the vmalloc area (I am not sure about that but it felt like the speed of connects and disconnects was lower). I am still reluctant to go that road because the required changes were somewhat big and the more gets changed, the higher chance to pick up some regression. Also the question is whether the test case models a realistic usage. To clarify, this is not a real leakage. It is a combination of specially allocated memory and slowness / complicated policy to free that allocations.