Comment 5 for bug 409615

Martin Pool wrote:
> 2009/12/1 David Muir <email address hidden>:
>
>> I can give it a shot, but I may need a bit of hand-holding :-)
>>
>> Where should I start looking in regards to fixing this bug?
>>
>
> So the way I think i would tackle it is: look (by setting -Derror or
> in .bzr.log) at the traceback when this error occurs. This will tell
> you some code that's trying to append to a file on the ftp server.
> The belief is that this code doesn't really need to append to an
> existing file (since current formats don't require that behaviour) and
> it can easily be changed to just write a file. Post the traceback to
> the bug report so others can help.
>
> You may not actually straightforwardly get a traceback, in which case
> you probably need to insert a
>
> import pdb;pdb.set_trace()
>
> line to pop the debugger at that point. You could just put this into
> ftp.py/ append. Then use 'bt' to get a traceback.
>
> The code that's originally doing the append should be changed to use
> open_write_stream().
>
> hope that helps, don't hesitate to ask,
>
>

Got the traceback:

Traceback (most recent call last):
  File "/usr/lib/python2.6/dist-packages/bzrlib/commands.py", line 842,
in exception_to_return_code
    return the_callable(*args, **kwargs)
  File "/usr/lib/python2.6/dist-packages/bzrlib/commands.py", line 1037,
in run_bzr
    ret = run(*run_argv)
  File "/usr/lib/python2.6/dist-packages/bzrlib/commands.py", line 654,
in run_argv_aliases
    return self.run(**all_cmd_args)
  File "/usr/lib/python2.6/dist-packages/bzrlib/builtins.py", line 1156,
in run
    use_existing_dir=use_existing_dir)
  File "/usr/lib/python2.6/dist-packages/bzrlib/push.py", line 128, in
_show_push_branch
    remember, create_prefix)
  File "/usr/lib/python2.6/dist-packages/bzrlib/bzrdir.py", line 1263,
in push_branch
    repository_to.fetch(source.repository, revision_id=revision_id)
  File "/usr/lib/python2.6/dist-packages/bzrlib/repository.py", line
1695, in fetch
    find_ghosts=find_ghosts, fetch_spec=fetch_spec)
  File "/usr/lib/python2.6/dist-packages/bzrlib/decorators.py", line
192, in write_locked
    result = unbound(self, *args, **kwargs)
  File "/usr/lib/python2.6/dist-packages/bzrlib/repository.py", line
3413, in fetch
    pb=pb, find_ghosts=find_ghosts)
  File "/usr/lib/python2.6/dist-packages/bzrlib/fetch.py", line 81, in
__init__
    self.__fetch()
  File "/usr/lib/python2.6/dist-packages/bzrlib/fetch.py", line 107, in
__fetch
    self._fetch_everything_for_search(search)
  File "/usr/lib/python2.6/dist-packages/bzrlib/fetch.py", line 135, in
_fetch_everything_for_search
    stream, from_format, [])
  File "/usr/lib/python2.6/dist-packages/bzrlib/repository.py", line
4234, in insert_stream
    self.target_repo.start_write_group()
  File "/usr/lib/python2.6/dist-packages/bzrlib/repository.py", line
1770, in start_write_group
    self._start_write_group()
  File "/usr/lib/python2.6/dist-packages/bzrlib/repofmt/pack_repo.py",
line 2265, in _start_write_group
    self._pack_collection._start_write_group()
  File "/usr/lib/python2.6/dist-packages/bzrlib/repofmt/pack_repo.py",
line 2021, in _start_write_group
    file_mode=self.repo.bzrdir._get_file_mode())
  File
"/usr/lib/python2.6/dist-packages/bzrlib/repofmt/groupcompress_repo.py",
line 154, in __init__
    self._writer.begin()
  File "/usr/lib/python2.6/dist-packages/bzrlib/pack.py", line 129, in begin
    self.write_func(self._serialiser.begin())
  File "/usr/lib/python2.6/dist-packages/bzrlib/pack.py", line 132, in
write_func
    self._write_func(bytes)
  File
"/usr/lib/python2.6/dist-packages/bzrlib/repofmt/groupcompress_repo.py",
line 147, in _write_data
    _write(bytes)
  File "/usr/lib/python2.6/dist-packages/bzrlib/transport/__init__.py",
line 267, in write
    self.transport.append_bytes(self.relpath, bytes)
  File "/usr/lib/python2.6/dist-packages/bzrlib/transport/__init__.py",
line 1011, in append_bytes
    return self.append_file(relpath, StringIO(bytes), mode=mode)
  File
"/usr/lib/python2.6/dist-packages/bzrlib/transport/ftp/__init__.py",
line 404, in append_file
    self._try_append(relpath, text, mode)
  File
"/usr/lib/python2.6/dist-packages/bzrlib/transport/ftp/__init__.py",
line 444, in _try_append
    self._try_append(relpath, text, mode, retries+1)
  File
"/usr/lib/python2.6/dist-packages/bzrlib/transport/ftp/__init__.py",
line 444, in _try_append
    self._try_append(relpath, text, mode, retries+1)
  File
"/usr/lib/python2.6/dist-packages/bzrlib/transport/ftp/__init__.py",
line 444, in _try_append
    self._try_append(relpath, text, mode, retries+1)
  File
"/usr/lib/python2.6/dist-packages/bzrlib/transport/ftp/__init__.py",
line 440, in _try_append
    "Aborting." % abspath, orig_error=e)

Looking through the code, it looks like the append fallback relies on
the server returning a permanent error rather than a temporary error.
The ftp server I'm connecting to is nice enough to give me a temporary
error, so the fallback is never triggered.

I'm a little confused by terminology... and Python in general...
IIUC, the transport assumes append based file streams (because of
previous limitations?). So even though the method is called append_file,
having it put the file instead is ok?

Cheers,
David