>>>>> "Andrew" == Andrew Bennetts <email address hidden> writes:
Andrew> Public bug reported:
Andrew> I have a bzr branch at 2977
Andrew> <email address hidden> in packs-0.92 format.
Andrew> I have a checkout (in a separate repository) of current bzr.dev (3041
Andrew> <email address hidden>)
Andrew> When using current bzr.dev to update the 2977 branch from http://bazaar-
Andrew> vcs.org/bzr/bzr.dev/ I got this error:
Andrew> ShortReadvError: readv() read unknown bytes rather than unknown bytes at
Andrew> unknown for "http://bazaar-
Andrew> vcs.org/bzr/bzr.dev/.bzr/repository/packs/a3069b4df97462558e8666ff0cc72386.pack":
Andrew> Server aborted the request
Andrew> Also, it took 38m32s wall time!
Ouch.
Andrew> This is repeatable. (It's slightly exacerbated by the fact that bzr is
Andrew> uninterruptible by Ctrl-C or Ctrl-\ during this, apparently due to being
Andrew> inside pycurl; I think there's already a bug about that).
I don't remember a bug about this, but even, we can hardly fix
it, the workaround is to use urllib instead.
Andrew> At Robert's suggestion, I bypassed my home network's squid proxy. This
Andrew> avoids the problem; the pull succeeds in slightly over 3 minutes.
Robert, what made you think about that work-around ?
Andrew> Here is the full .bzr.log contents for this invocation:
Andrew> bzr arguments: [u'pull']
Andrew> looking for plugins in /home/andrew/.bazaar/plugins
Andrew> looking for plugins in /tmp/bzr.packs/bzrlib/plugins
Andrew> Plugin name __init__ already loaded
Andrew> Plugin name __init__ already loaded
Andrew> encoding stdout as sys.stdout encoding 'UTF-8'
Andrew> opening working tree '/home/andrew/code/bzr'
Andrew> Using fetch logic to copy between KnitPackRepository('http://bazaar-vcs.org/bzr/bzr.dev/.bzr/repository/')(<RepositoryFormatKnitPack1>) and KnitPackRepository('file:///home/andrew/warthogs/bzr/.bzr/repository/')(<RepositoryFormatKnitPack1>)
This is pycurl detecting a short read, to say the truth I never
encountered it myself, the code was ready for it though. packs
are really making life hard for our http implementations ;-)
Did your squid logs shed some light on why it occurred ?
Apparently the 169 ranges request was accepted so I wonder why it
decided to interrupt the transfer of the body part...
Anyway, that confirmed just what I was afraid of, it means readv
has to keep track of which offsets have been already transferred
to avoid requesting them again.
>>>>> "Andrew" == Andrew Bennetts <email address hidden> writes:
Andrew> Public bug reported:
Andrew> I have a bzr branch at 2977
Andrew> <email address hidden> in packs-0.92 format.
Andrew> I have a checkout (in a separate repository) of current bzr.dev (3041
Andrew> <email address hidden>)
Andrew> When using current bzr.dev to update the 2977 branch from http:// bazaar- bzr/bzr. dev/ I got this error:
Andrew> vcs.org/
Andrew> ShortReadvError: readv() read unknown bytes rather than unknown bytes at bazaar- bzr/bzr. dev/.bzr/ repository/ packs/a3069b4df 97462558e8666ff 0cc72386. pack":
Andrew> unknown for "http://
Andrew> vcs.org/
Andrew> Server aborted the request
Andrew> Also, it took 38m32s wall time!
Ouch.
Andrew> This is repeatable. (It's slightly exacerbated by the fact that bzr is
Andrew> uninterruptible by Ctrl-C or Ctrl-\ during this, apparently due to being
Andrew> inside pycurl; I think there's already a bug about that).
I don't remember a bug about this, but even, we can hardly fix
it, the workaround is to use urllib instead.
Andrew> At Robert's suggestion, I bypassed my home network's squid proxy. This
Andrew> avoids the problem; the pull succeeds in slightly over 3 minutes.
Robert, what made you think about that work-around ?
Andrew> Here is the full .bzr.log contents for this invocation:
Andrew> bzr arguments: [u'pull'] .bazaar/ plugins packs/bzrlib/ plugins andrew/ code/bzr' ory('http:// bazaar- vcs.org/ bzr/bzr. dev/.bzr/ repository/ ')(<RepositoryForm atKnitPack1> ) and KnitPackReposit ory('file: ///home/ andrew/ warthogs/ bzr/.bzr/ repository/ ')(<RepositoryF ormatKnitPack1> )
Andrew> looking for plugins in /home/andrew/
Andrew> looking for plugins in /tmp/bzr.
Andrew> Plugin name __init__ already loaded
Andrew> Plugin name __init__ already loaded
Andrew> encoding stdout as sys.stdout encoding 'UTF-8'
Andrew> opening working tree '/home/
Andrew> Using fetch logic to copy between KnitPackReposit
<snip/> 58e8666ff0cc723 86.pack offsets => 649 collapsed 149 bazaar- vcs.org/ bzr/bzr. dev/.bzr/ repository/ packs/a3069b4df 97462558e8666ff 0cc72386. pack
Andrew> http readv of a3069b4df974625
Andrew> got pycurl error: 18, transfer closed with 162685 bytes remaining to read, (18, 'transfer closed with 162685 bytes remaining to read'), url: http://
This is pycurl detecting a short read, to say the truth I never
encountered it myself, the code was ready for it though. packs
are really making life hard for our http implementations ;-)
Did your squid logs shed some light on why it occurred ?
Apparently the 169 ranges request was accepted so I wonder why it
decided to interrupt the transfer of the body part...
Anyway, that confirmed just what I was afraid of, it means readv
has to keep track of which offsets have been already transferred
to avoid requesting them again.
<snip/>
Andrew> File "/tmp/bzr. packs/bzrlib/ transport/ http/_pycurl. py", line 304, in _curl_perform bazaar- vcs.org/ bzr/bzr. dev/.bzr/ repository/ packs/a3069b4df 97462558e8666ff 0cc72386. pack": Server aborted the request
Andrew> extra='Server aborted the request')
Andrew> ShortReadvError: readv() read unknown bytes rather than unknown bytes at unknown for "http://
And unfortunately pycurl do not tell us where we failed.
For that, can you retry with pycurl but with line
curl. setopt( pycurl. VERBOSE, 1)
uncommented out (in _pycurl.py around line 267) ?
That will display the headers sent and received and help understanding what happened.