Comment 3 for bug 402657

If we are 'optimally packed' then each group is between 2-4MB in size. (A single content will expand to 4MB, mixed content grows to 2MB.)

That is quite a bit bigger than Andrew's current 64kB group request size.

I think it is worthwhile to figure out the buffering/pre-read issue. But the reason it wasn't done earlier is that we guessed it wouldn't be as common to have small groups.

Because of stuff like the fragmentation bug (bug #402645) it turns out to be more common. Also because of the 'fetch from a mixed source creates an unoptimal single pack.

I also ran into a case that I'd like to exploit the packing heuristics. Namely keeping the 'current' content slightly looser packed than 'older' content. This was a moderate win for stuff like 'ls -r -1 --recurse', which also effects the time to build a checkout.

When I was testing it, the lack of wide reads caused a lot more round-trips to read the more minimal data, so having a bit of buffering in the middle would have been helpful.