'out of memory' sending empty patch on SunOS

Under Linux, with beefier hardware (AMD, 8G mem), using same repository, the results are:

$ bzr send -o test.patch .
bzr: ERROR: Not a branch: "/home/foo/tmp/test.bzr/".

$ uname -a
Linux 2.6.32-trunk-amd64 #1 SMP Sun Jan 10 22:40:40 UTC 2010 x86_64 GNU/Linux

$ bzr --version
Bazaar (bzr) 2.0.3

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Jari Aalto wrote:
> Public bug reported:
>
> A simple command causes: OUT OF MEMORY error:
>
> $ cd test.bzr/
> $ bzr send -o test.patch .
>
> The used test repository was almost empty: only two test files with only
> one line text in each. See attached bzr repository.
>
> $ uname -a
> SunOS 5.9 Generic_122300-47 sun4u sparc
>

I would guess there is probably a 'malloc(0)' that is confusing us. "bzr
send -o test.patch ." will actually contain an empty content patch
because it is comparing the branch with itself.

John
=:->
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.9 (Cygwin)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/

iEYEARECAAYFAktZ3l4ACgkQJdeBCYSNAAOtVQCgqV23tfV4BvfvvHqzaDyQvGmK
pYQAoLP+GnvHmQJpdXz4NCttuYxzmWi6
=IzC6
-----END PGP SIGNATURE-----

Thanks for the report. If this is the problem John suspects, and especially if it's sun-specific, a traceback for the error would help a lot. bzr -Derror blah should do it.

[Backtrace on SUnOS]

$ bzr -Derror send -o test.patch .

bzr: ERROR: exceptions.MemoryError:

Traceback (most recent call last):
  File "/usr/local/lib/python/bzrlib/commands.py", line 853, in exception_to_return_code
    return the_callable(*args, **kwargs)
  File "/usr/local/lib/python/bzrlib/commands.py", line 1055, in run_bzr
    ret = run(*run_argv)
  File "/usr/local/lib/python/bzrlib/commands.py", line 661, in run_argv_aliases
    return self.run_direct(**all_cmd_args)
  File "/usr/local/lib/python/bzrlib/commands.py", line 665, in run_direct
    return self._operation.run_simple(*args, **kwargs)
  File "/usr/local/lib/python/bzrlib/cleanup.py", line 122, in run_simple
    self.cleanups, self.func, *args, **kwargs)
  File "/usr/local/lib/python/bzrlib/cleanup.py", line 156, in _do_with_cleanups
    result = func(*args, **kwargs)
  File "/usr/local/lib/python/bzrlib/builtins.py", line 5146, in run
    strict=strict)
  File "/usr/local/lib/python/bzrlib/send.py", line 133, in send
    public_branch, no_patch, no_bundle, message, base_revision_id)
  File "/usr/local/lib/python/bzrlib/send.py", line 158, in _send_4
    base_revision_id=base_revision_id)
  File "/usr/local/lib/python/bzrlib/merge_directive.py", line 571, in from_objects
    ancestor_id).encode('base-64')
  File "/usr/local/lib/python/bzrlib/merge_directive.py", line 197, in _generate_bundle
    ancestor_id, s)
  File "/usr/local/lib/python/bzrlib/bundle/serializer/__init__.py", line 122, in write_bundle
    base_revision_id, out)
  File "/usr/local/lib/python/bzrlib/bundle/serializer/v4.py", line 244, in write_bundle
    write_op = BundleWriteOperation(base, target, repository, fileobj)
  File "/usr/local/lib/python/bzrlib/bundle/serializer/v4.py", line 273, in __init__
    bundle = BundleWriter(fileobj)
  File "/usr/local/lib/python/bzrlib/bundle/serializer/v4.py", line 51, in __init__
    self._compressor = bz2.BZ2Compressor()

Thanks.

That looks a bit like a problem in your Python installation. What does this command do?

python -c 'import bz2;bz2.BZ2Compressor()'

$ python -c 'import bz2;bz2.BZ2Compressor()'
<no output; no errors>

$ python -V
Python 2.5.1

$ bzr --version
Bazaar (bzr) 2.1.0rc1
  Python interpreter: /usr/local/bin/python 2.5.1
  Python standard library: /usr/local/lib/python2.5
  Platform: Solaris-2.9-sun4u-sparc-32bit-ELF
  bzrlib: /usr/local/lib/python/bzrlib
  Bazaar configuration: /home/foo/.bazaar
  Bazaar log file: /home/foo/.bzr.log

I don't know what this is then. We might be close to being out of memory and constructing this object tips it over the edge but that seems unlikely. It might help to look at it under a C debugger.

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Jari Aalto wrote:
> $ python -c 'import bz2;bz2.BZ2Compressor()'
> <no output; no errors>
>
> $ python -V
> Python 2.5.1
>
> $ bzr --version
> Bazaar (bzr) 2.1.0rc1
> Python interpreter: /usr/local/bin/python 2.5.1
> Python standard library: /usr/local/lib/python2.5
> Platform: Solaris-2.9-sun4u-sparc-32bit-ELF
> bzrlib: /usr/local/lib/python/bzrlib
> Bazaar configuration: /home/foo/.bazaar
> Bazaar log file: /home/foo/.bzr.log
>

That would hint that perhaps we have a function raising an exception,
and another function suppressing that exception. And that the
BZ2Compressor() is then seeing an exception left in the queue, and
raising it at an odd time.

Alternatively, something we are doing is triggering BZ2Compressor() to
fail an internal malloc while...

John
=:->

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.9 (Cygwin)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/

iEYEARECAAYFAkthxKUACgkQJdeBCYSNAANw+QCfZ057fcnfdRMG+BqChajOC/8D
O80AoK/8JWcRSCfXMF2uk7MqBFqbY84U
=nOy4
-----END PGP SIGNATURE-----

> It might help to look at it under a C debugger.

Let me know how to dot that for bzr Python code, and I'll try to provide more information.

http://wiki.python.org/moin/DebuggingWithGdb

-Rob

Oh, it needs debug enabled Python (*-dbg), which is not unfortunately available here.

Can you get a dtrace/truss/similar trace that might give us an idea
what is failing?

FWIW, a quick skim of _patience_diff_c.c suggests that malloc(0) can occur when the second set of lines is empty (when allocating self->backpointers in PatienceSequenceMatcher_new). It's probably not the only place in our C extensions that does this. It'd be good to figure out a way to catch this reliably on linux, and set a up buildbot slave or similar for it.

Quick grep suggests the only other C (or Pyrex) extension with a likely issue with malloc directly is:

bzrlib/_bencode_pyx.pyx:263: p = <char*>malloc(maxsize)

So probably we should try making minimal fixes to both _bencode_pyx.pyx and _patience_diff_c.c and see if that fixes this particular bug.

But we also need to be careful with things that call malloc indirectly, and I don't know of a good way to find those. Perhaps an LD_PRELOAD hack?

On 4 February 2010 03:43, Andrew Bennetts <email address hidden> wrote:
> Quick grep suggests the only other C (or Pyrex) extension with a likely
> issue with malloc directly is:
>
> bzrlib/_bencode_pyx.pyx:263:        p = <char*>malloc(maxsize)
>
> So probably we should try making minimal fixes to both _bencode_pyx.pyx
> and _patience_diff_c.c and see if that fixes this particular bug.
>
> But we also need to be careful with things that call malloc indirectly,
> and I don't know of a good way to find those.  Perhaps an LD_PRELOAD
> hack?

We could have a macro that aborts if you try to allocate 0 bytes. It
looks like this code will do something strange if it does allocate 0
bytes. But perhaps we should just change it and review more carefully
for the 0 element case.

--
Martin <http://launchpad.net/~mbp/>

This has been seen on aix a while ago too: bug 331095.