Comment 2 for bug 402657

So the basic issue here is that _get_remaining_record_stream does lots of individual _get_block calls.

I'm working on a patch to add some batching, initial indications are looking good but I'll do more thorough testing on Monday. It attempts to batch up the _get_block calls until there's at least 64k of data to request. It certainly cuts down the number tiny HTTP readv requests made when fetching Launchpad.

There's a selftest failure it somehow introduces too, so more work required. But this bug is definitely In Progress.