Comment 14 for bug 393366

Revision history for this message
John A Meinel (jameinel) wrote :

Digging deeper it seems that _group_keys_for_io is designed to preserve the key ordering that we got from 'present_keys', which means that it does preserve the topological sorting (as long as there are no non-local keys)

If we have a non-local key, we first have put that into a set() which destroys the ordering.
We then end up with 2 queues, one with the properly sorted 'local' keys, and another with randomly ordered 'nonlocal' keys.

Now in the common case, these won't be interleaved. However in the real world, I think it could happen. Consider:

(time goes down)
 #- fallback repo
    #- stacked repo
 A
 |\
 | B
 |/|
 C |
  \|
   E

In this case, the revisions A B C will be present in the fallback, and the revisions B & E will be present in the stacked repo.

The proper topological sorting would be:
 A B C E

However, because the lists are now split, we have:
 B E (correct) set([A, C])

Now, we hit this problem much more often than the above would indicate, *because* the code in _VFContentManager does:

   missing_keys = set(nonlocal_keys)
   # Read from remote versioned file instances and provide to our caller.
   for source in self.vf._fallback_vfs:
       if not missing_keys:
           break
       # Loop over fallback repositories asking them for texts - ignore
       # any missing from a particular fallback.
       for record in source.get_record_stream(missing_keys,
           'unordered', True):

           ^^^^^^^^^^- always an unordered fetch from fallbacks

The latter is pretty easy to fix, so I'm writing up something for that. I expect it to handle 90% of the cases.

Going further, --2a formats handle this case properly *today*, so I'm loathe to spend a lot of time fixing the old one. I'll open up a new bug on that.