Comment 11 for bug 256757

Revision history for this message
John A Meinel (jameinel) wrote : Re: [Bug 256757] Re: upgrade from <RepositoryFormatKnit1> to the newest format eats all my ram

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Robert Collins wrote:
> On Mon, 2008-08-18 at 18:35 +0000, John A Meinel wrote:
>> The attached patch seems to correct a logic bug. Specifically the
>> variable is "fetch_uses_deltas" but the variable passed is
>> "include_delta_closure". Which actually means "and transmit all history
>> and use full-texts". Which "fetch_uses_deltas" seems to have the
>> opposite meaning.
>>
>> With the 'not' fix, memory consumption drops to 100MB. (Instead of 700+MB).
>> However, this only applies to bzr.dev, so it shouldn't actually be a regression in 1.6.
>> Also, the target repository is still grossly bloated (140MB).
>>
>> If I change the fetch order for pack_repo to also be "topological" then
>> I get the expected target size of 35MB. I'm not as convinced that is the
>> right fix, but it does make the fetch correct.
>
> VF shouldn't be converting texts to full-text willy-nilly. It can't with
> the uses_deltas set appropriately anyway; it has to have the basis text
> locally to be able to convert via a fulltext.
>
> -Rob

So the specific issue was that I wasn't setting the _use_deltas for all
objects. I was setting it for .texts, but I missed the .inventories. And, as
it turns out with this repo, the .inventories was bloating more than the rest.
 (This repo is about 6k small files, so a large inventory causes more bloat
than extra file fulltexts.)

Which is why my final fix set it to 'unordered'. (Rather than 'unsorted',
because get_record_stream() isn't supposed to know what 'unsorted' means. :)

John
=:->
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.6 (GNU/Linux)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org

iD8DBQFIqfpLJdeBCYSNAAMRAmiiAJ9qKJ/IwCuWWf5W7i3q5Br/R8s/dgCdFYXQ
fsvtq+YJ15gqFDw++AGTjdg=
=Yml+
-----END PGP SIGNATURE-----