Misleading "please run connect() first" error when SMTP server is down

Can you include the traceback from ~/.bzr.log? I realize it may be hard to reproduce now.

But I'm trying to figure out which line is failing. We specifically do:

smtp = smtplib.SMTP()
smtp.connect(server)

try:
    # The world can always use a little bit more encryption :)
    smtp.starttls()
except smtplib.SMTPException:
    pass

if smtp_username is not None:
    if smtp_password is None:
        smtp_password = bzrlib.ui.ui_factory.get_password(
            'Please enter SMTP password for %(host)s', host=server)
    try:
        smtp.login(smtp_username, smtp_password)
    except smtplib.SMTPHeloError, e:
        raise BzrCommandError('SMTP server refused HELO: %d %s'
                              % (e.smtp_code, e.smtp_error))
    except smtplib.SMTPAuthenticationError, e:
        raise BzrCommandError('SMTP server refused authentication: %d %s'
                              % (e.smtp_code, e.smtp_error))
    except smtplib.SMTPException, e:
        raise BzrCommandError(str(e))

Which looks like we should be doing the right thing. It would seem that 'connect()' should be failing.

I suppose it is possible that we are running into the final

    except smtplib.SMTPException, e:
        raise BzrCommandError(str(e))

Which means that we need to trap another specific exception at this point.

I can see this happening:
>>> s.connect('mynonexistantserver.com')
Traceback (most recent call last):
  File "<stdin>", line 1, in ?
  File "/usr/lib/python2.4/smtplib.py", line 292, in connect
    for res in socket.getaddrinfo(host, port, 0, socket.SOCK_STREAM):
socket.gaierror: (-2, 'Name or service not known')

But that is if the domain doesn't resolve.

If I stop the local mail server I get:
>>> s.connect('localhost')
Traceback (most recent call last):
  File "<stdin>", line 1, in ?
  File "/usr/lib/python2.4/smtplib.py", line 306, in connect
    raise socket.error, msg
socket.error: (111, 'Connection refused')

Which again seems reasonable.

Can you reproduce this?

smtp.myrealbox.com is temperamental, so I should be able to reproduce the bug sometime in the next few months and attach ~/.bzr.log.

Oops, didn't realize .bzr.log was cumulative. Anyway, just reproduced the bug:

load plugin (None, '/home/mpt/.bazaar/plugins/pqm-submit', ('', '', 5))
registered plugin command pqm-submit
loaded succesfully
encoding stdout as sys.stdout encoding 'UTF-8'
got branch format Bazaar-NG Metadir branch format 5
ssh implementation is OpenSSH
got branch format Bazaar-NG Metadir branch format 5
** Env var TTY empty, cannot set GPG_TTY. Is TTY exported?
Traceback (most recent call last):
  File "/usr/lib/python2.4/site-packages/bzrlib/commands.py", line 626, in run_bzr_catch_errors
    return run_bzr(argv)
  File "/usr/lib/python2.4/site-packages/bzrlib/commands.py", line 588, in run_bzr
    ret = run(*run_argv)
  File "/usr/lib/python2.4/site-packages/bzrlib/commands.py", line 292, in run_argv_aliases
    return self.run(**all_cmd_args)
  File "/home/mpt/.bazaar/plugins/pqm/__init__.py", line 81, in run
  File "/home/mpt/.bazaar/plugins/pqm-submit/pqm_submit.py", line 140, in submit
    raise BzrCommandError(str(e))
BzrCommandError: please run connect() first

return code 3

My guess is that the problem code is:

    try:
        # The world can always use a little bit more encryption :)
        smtp.starttls()
    except smtplib.SMTPException:
        pass

Looking at the code in smtp.starttls(), normal server errors should be indicated in the return value. By the look of it, the only SMTPException that can get raised is SMTPServerDisconnected, which we certainly don't want to ignore.

Matthew: could you try changing the "pass" bit to "raise" in the above code, and see if this hypothesis is correct? If it is correct, then you should get an SMTPServerDisconnected traceback instead of the "please run connect() first" error message.

Well, I did check with "a.mx.mail.yahoo.com", and it does indeed only return (502, 'Command Unimplemented')

I have run into Windows virus scanners in the past, that cause problems when you connect to port 25, and try to 'starttls()'. They do something weird to the connection so that it fails, because they want to be able to scan all emails for viruses,etc.

Which is why I had the try/except there. But I'm okay with removing that, and just letting the return code fail.

John
=:->

(I haven't forgotten about testing James' suggestion. The server just went down again, but came up before I could finish changing pqm_submit.py.)

I have now experienced this half a dozen times:
1. got the "please run connect() first" error
2. changed "pass" to "raise" in pqm_submit.py, to test James' hypothesis
3. tried to pqm-submit again, but instead of getting a traceback, the pqm-submit succeeded

Is this just dumb luck from the server coming up between steps 1 and 3, or is it even remotely possible that pqm-submit might work more reliably with "raise" than it does with "pass"?

I'm 95% positive it is just dumb luck. As there is no external try/except wrapper that would recover from a 'raise'.

I'm willing to just change it, though.

Ultimately I would like to switch over to the work I did for 'bzr-email' which provides better mailing functionality (proper unicode support, etc)

Though I don't know if pqm would actually handle properly encoded unicode emails.

We probably do want to switch to your new email code at some point.

If the branch ~c3po/pqm/bundles gets merged, PQM would be able to accept bundles as part of the merge request. We'd need to be able to construct MIME encoded messages in that case.

Interesting. I haven't heard of c3po's work before. I know Alexander (bialix) has wanted to be able to do that for a while.

df

Preceding comment is spam.

My e-mail provider no longer has a temperamental SMTP server, so further testing of this bug will require someone with their own SMTP server.

However, it is still wrong for bzr to say "bzr: ERROR: please run connect() first", because bzr still does not have a "connect" command.

I assume this is fixed with James' recent work to properly support connecting to an smtp server, and requesting starttls if it is available.