kswapd0 freaks out if rsync runs on an USB disk

I forgot to mention: From time to time, also two "kworker" kernel threads use quite a bit CPU in this state, but it's dwarfed by the processing time uswed by kswapd0.

The following patch from Andrea Arcangeli sound as if it may fix this problem:

http://www.spinics.net/lists/stable-commits/msg11136.html

"mm: compaction: prevent kswapd compacting memory to reduce CPU usage"

Grek KH added it to the proposed 2.6.28-stable patchset a week ago.

Can we expect to see this patch in the Ubuntu kernel patch set?

This patch is in latest Natty kernel recently uploaded/available (2.6.38-8.40), can you update and report here if you still have the issue?

With kernel linux-image-2.6.38-8-generic 2.6.38-8.40, kswapd does not use significant CPU time any more.

However, now one or two kworker(s) sit(s) on top of the process list, constantly using between 20 and 40% CPU, and the system still gets unresponsive (freezing GUI, including mouse) from time to time while rsync is running.

It's definitely better than before, but still not really usable and definitely much worse than with Karmic's kernel.

It's not just USB access---it's anything that hits a filesystem hard.

I can confirm that 2.6.38-7-generic on AMD64 freaks out exactly the same way: kswapd0 goes
to 100% if driven by 'nc -l 1234 | tar xvfp -' to ext4 (no crypto, no LVM, no RAID, just an ordinary
partition) on an ordinary SATA 2TB disk when the other end sends large files via gigabit. This
ruins throughput by 2x and drives the CPU fan to max on my 6-core 2.8GHz AMD64. I haven't
yet tried 2.6.38-8 because I'm doing my testing with a daily build from 3/30 via a LiveUSB and
I've learned the hard way that doing update-manager on a persistent LiveUSB leads to a broken
system. The current daily build of 4/5 still only has 2.6.38-7, according to the manifest. (When
can we expect 2.6.38-8 in the daily build? Tomorrow? I'm looking at
http://cdimage.ubuntu.com/daily-live/current/)

If I direct the nc into /dev/null instead of tar, kswapd0 stays dormant, of course.

I am now in the extremely unpalatable situation of having -terrible- (6x) slowdowns on small-file
access if they're on top of LUKS in Maverick AMD64 (see bug #731340) or -terrible- large-file
performance in Natty while trying to build a new server and migrate a 2TB dirvish vault with
many of each from 8.10 or thereabouts, where all this worked just fine. This seems like a
showstopper; neither one will work half as well as one from over 2 years ago on this simple
task.

Oh yuck, sorry for the horrible formatting of the previous;
I usually yank in text from Emacs, but this time I typed it
directly into the box. Apparently it's hard-wrapped at 80
columns, but the box is about double that width. Yet another
bug in Launchpad, but I've learned that reporting those doesn't
yield action in years. *sigh* I'll report it anyway...

To follow up: Trying 2.6.38-8 (installed the beta and let it update to whatever was current as of 4/9) fixed the problem. kswapd0 no longer hogs CPU, and unlike the OP's report of 4/5, I don't see kworker chewing up time and I see no freezes or other issues. For me, it Just Works. Note that my test was using an "nc | tar" pipeline to push as much stuff to the disk over gigabit as fast as I could, to stress-test things as much as possible. I haven't tried rsync, but see no reason that would act differently. Thanks!

natty has no proposed candidates ergo, this cannot be a regression-proposed bug. Removing the tag.

~JFo

I have the same problem on an AMD E-350 system (HP dm1 laptop) and kernel 2.6.38. Downgrading to 2.6.37 or lower fixes the problem, but it is still there with 2.6.39 from the kernel-ppa.

Stefan

I also have a similar problem with an HP dm1 laptop and kernel 2.6.38-9-generic. For me the problem is triggered by rsync, especially when copy very large files (greater than the laptop physical memory). The system eventually hangs in my case.
However, I tried the -server kernel, and the problem disappeared. Not the best option for a laptop, but at least it doesn't hang...

On my system, rsyncing loads of data to an external usb drive or an iscsi lun does not work properly. The system always hangs at some point! I am running kernel 2.6.37-11-server. When I tar the same data to the iscsi lun, everything works...

Due to the fact that the system hangs and I have to reset it by pressing the power button for a few secs, I cannot say if the system remains responsive. Trying to ping or ssh it from another pc fails. Unfortunately, I cannot find anything suspicious in the logs...

So... how can I provide more information?

I made a typo... my kernel is 2.6.38-11-server (Natty, 64bit)

I also can confirm that the problem did not exist before I started to use natty.

This bug was filed against a series that is no longer supported and so is being marked as Won't Fix. If this issue still exists in a supported series, please file a new bug.

This change has been made by an automated script, maintained by the Ubuntu Kernel Team.