Comment 4 for bug 368717

Revision history for this message
John A Meinel (jameinel) wrote :

Can you give the version of bzr you were testing? While I agree with Robert that we probably won't spend a huge amount of time optimizing lightweight checkouts for high-latency connections (consider something more like stacked-branches with minimal history, etc.)

Anyway, there *was* some work done recently by me to change the lightweight checkout fetch to batch things up a bit more.

The change was the 'group_keys_for_io' change, which was around bzr.dev @ 4039.3.7, landed in 4051, which should be in both 1.13 and 1.14.

Though reading the log file, it looks like at least some batching is going on. There are only 91 readv commands issued.

You are also in a somewhat lucky position, in that you have a lot of bandwidth, such as:
26.404 result: ('readv',)
30.315 4352629 body bytes read

^- 3.9s to download 4.15MiB == 1.06MiB/s

But fairly high latency:
26.148 hpss call w/readv: 'readv', '/~ubuntu-core-doc/ubuntu-doc/ubuntu-karmic/.bzr/repository/packs/2115d5e1ee8917c83744105928120b0f.pack'
26.149 13 bytes in readv request
26.232 result: ('readv',)
26.233 221 body bytes read

^- it takes 85ms to get a single request made.
So in that 85ms you can download almost 90KiB.

However, 91 round trips * 0.085s == 7.7s, which doesn't quite account for your 30s difference (not to mention that you are downloading far less content.)

Anyway, the new format should be a bit better about lightweight checkouts, especially with a patch like this:
  http://bzr.arbash-meinel.com/branches/bzr/brisbane/split_pack

(it changes the compressor logic to cluster all 'new' texts into their own set of groups)

Regardless that branch, it would be interesting to see how the new format changes this effect.