Comment 19 for bug 602614

Revision history for this message
John A Meinel (jameinel) wrote :

I just associated a branch which might help. The idea is that if it gets a text which is large, rather than creating a delta index which overlays every byte, it 'skips' some. This should reduce the best-possible compression it can compute, but should lower peak memory as well. I tested it with MAX_NUM_ENTRIES = 1000 (which would mean that texts >16kB would be under-sampled). And the resulting bzr repository was 80,488kB rather than 79,892kB.

I don't have a great feeling for where the best cutoff is, etc.

If you want to test it, I suggest you copy your repository to a temp location, build this bzr, and then run:
 test/bzr pack temp/repo -Dmemory

That should give some information about whether this is worthwhile or not.

Note, the change is not well tested, but I did do some sanity tests. As such, I wouldn't run it on your live repository, but it certainly would help give information about whether it is worth pursuing this further.

The other workaround is to just switch to a 64-bit machine and run "bzr pack" there. It should only need to be run on a single machine one time, and from then the clients should be ok with the one-large-pack for a while.