python-botocore in xenial-updates can no longer make S3 requests due to some incompatibility with python-openssl

Bug #1905069 reported by Marcin Sulikowski
8
This bug affects 1 person
Affects Status Importance Assigned to Milestone
python-botocore (Ubuntu)
New
Undecided
Unassigned

Bug Description

I found python-botocore 1.16.19+repack-1ubuntu0.16.04.2 unable to make S3 requests. Minimal reproducer:

    import os
    import boto3
    os.environ["AWS_ACCESS_KEY_ID"] = "ABCABCABCABCABCABCAB"
    os.environ["AWS_SECRET_ACCESS_KEY"] = "YWJjYWJjYWJjYWJjYWJjYWJjYWJjYWJjYWJhYmMK"
    os.environ["AWS_DEFAULT_REGION"] = "us-east-1"
    boto3.client("s3").list_objects(Bucket="abc")

The error I get with python-botocore 1.16.19+repack-1ubuntu0.16.04.2 from xenial-updates/universe is:

$ python repro.py
Traceback (most recent call last):
  File "repro.py", line 6, in <module>
    boto3.client("s3").list_objects(Bucket="abc")
  File "/usr/lib/python2.7/dist-packages/botocore/client.py", line 316, in _api_call
    return self._make_api_call(operation_name, kwargs)
  File "/usr/lib/python2.7/dist-packages/botocore/client.py", line 622, in _make_api_call
    operation_model, request_dict, request_context)
  File "/usr/lib/python2.7/dist-packages/botocore/client.py", line 641, in _make_request
    return self._endpoint.make_request(operation_model, request_dict)
  File "/usr/lib/python2.7/dist-packages/botocore/endpoint.py", line 102, in make_request
    return self._send_request(request_dict, operation_model)
  File "/usr/lib/python2.7/dist-packages/botocore/endpoint.py", line 137, in _send_request
    success_response, exception):
  File "/usr/lib/python2.7/dist-packages/botocore/endpoint.py", line 256, in _needs_retry
    caught_exception=caught_exception, request_dict=request_dict)
  File "/usr/lib/python2.7/dist-packages/botocore/hooks.py", line 356, in emit
    return self._emitter.emit(aliased_event_name, **kwargs)
  File "/usr/lib/python2.7/dist-packages/botocore/hooks.py", line 228, in emit
    return self._emit(event_name, kwargs)
  File "/usr/lib/python2.7/dist-packages/botocore/hooks.py", line 211, in _emit
    response = handler(**kwargs)
  File "/usr/lib/python2.7/dist-packages/botocore/retryhandler.py", line 183, in __call__
    if self._checker(attempts, response, caught_exception):
  File "/usr/lib/python2.7/dist-packages/botocore/retryhandler.py", line 251, in __call__
    caught_exception)
  File "/usr/lib/python2.7/dist-packages/botocore/retryhandler.py", line 269, in _should_retry
    return self._checker(attempt_number, response, caught_exception)
  File "/usr/lib/python2.7/dist-packages/botocore/retryhandler.py", line 317, in __call__
    caught_exception)
  File "/usr/lib/python2.7/dist-packages/botocore/retryhandler.py", line 223, in __call__
    attempt_number, caught_exception)
  File "/usr/lib/python2.7/dist-packages/botocore/retryhandler.py", line 359, in _check_caught_exception
    raise caught_exception
botocore.exceptions.HTTPClientError: An HTTP Client raised and unhandled exception: name must be a byte string

Downgrading it to python-botocore 1.3.23-1 from xenial/universe solves the problem (the error changes to "The AWS Access Key Id you provided does not exist in our records" but this is expected).

The error exception "name must be a byte string" originates from pyopnessl (https://github.com/pyca/pyopenssl/issues/515) so my understanding is that the latest version of botocore in xenial-updates is incompatible with xenial's ssl libraries.

Tags: patch
Revision history for this message
Marcin Sulikowski (msulikowski) wrote :

Please find attached a patch which fixes the backport of python-botocore 1.16.19 to Ubuntu 16.04. The patch contains a more detailed description of what's exactly broken with 1.16.19+repack-1ubuntu0.16.04.2.

Revision history for this message
Ubuntu Foundations Team Bug Bot (crichton) wrote :

The attachment "0003-Dont-pass-unicode-urls-to-urllib.patch" seems to be a patch. If it isn't, please remove the "patch" flag from the attachment, remove the "patch" tag, and if you are a member of the ~ubuntu-reviewers, unsubscribe the team.

[This is an automated message performed by a Launchpad user owned by ~brian-murray, for any issues please contact him.]

tags: added: patch
To post a comment you must log in.
This report contains Public information  
Everyone can see this information.

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.