Digging deeper it seems that _group_keys_for_io is designed to preserve the key ordering that we got from 'present_keys', which means that it does preserve the topological sorting (as long as there are no non-local keys)
If we have a non-local key, we first have put that into a set() which destroys the ordering.
We then end up with 2 queues, one with the properly sorted 'local' keys, and another with randomly ordered 'nonlocal' keys.
Now in the common case, these won't be interleaved. However in the real world, I think it could happen. Consider:
(time goes down)
#- fallback repo
#- stacked repo
A
|\
| B
|/|
C |
\|
E
In this case, the revisions A B C will be present in the fallback, and the revisions B & E will be present in the stacked repo.
The proper topological sorting would be:
A B C E
However, because the lists are now split, we have:
B E (correct) set([A, C])
Now, we hit this problem much more often than the above would indicate, *because* the code in _VFContentManager does:
missing_keys = set(nonlocal_keys)
# Read from remote versioned file instances and provide to our caller.
for source in self.vf._fallback_vfs:
if not missing_keys:
break
# Loop over fallback repositories asking them for texts - ignore
# any missing from a particular fallback.
for record in source.get_record_stream(missing_keys, 'unordered', True):
^^^^^^^^^^- always an unordered fetch from fallbacks
The latter is pretty easy to fix, so I'm writing up something for that. I expect it to handle 90% of the cases.
Going further, --2a formats handle this case properly *today*, so I'm loathe to spend a lot of time fixing the old one. I'll open up a new bug on that.
Digging deeper it seems that _group_keys_for_io is designed to preserve the key ordering that we got from 'present_keys', which means that it does preserve the topological sorting (as long as there are no non-local keys)
If we have a non-local key, we first have put that into a set() which destroys the ordering.
We then end up with 2 queues, one with the properly sorted 'local' keys, and another with randomly ordered 'nonlocal' keys.
Now in the common case, these won't be interleaved. However in the real world, I think it could happen. Consider:
(time goes down)
#- fallback repo
#- stacked repo
A
|\
| B
|/|
C |
\|
E
In this case, the revisions A B C will be present in the fallback, and the revisions B & E will be present in the stacked repo.
The proper topological sorting would be:
A B C E
However, because the lists are now split, we have:
B E (correct) set([A, C])
Now, we hit this problem much more often than the above would indicate, *because* the code in _VFContentManager does:
missing_keys = set(nonlocal_keys) _fallback_ vfs: get_record_ stream( missing_ keys,
'unordered' , True):
# Read from remote versioned file instances and provide to our caller.
for source in self.vf.
if not missing_keys:
break
# Loop over fallback repositories asking them for texts - ignore
# any missing from a particular fallback.
for record in source.
The latter is pretty easy to fix, so I'm writing up something for that. I expect it to handle 90% of the cases.
Going further, --2a formats handle this case properly *today*, so I'm loathe to spend a lot of time fixing the old one. I'll open up a new bug on that.