Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Issue with running s3cmd info? #1354

Closed
bryansray opened this issue Nov 16, 2023 · 2 comments
Closed

Issue with running s3cmd info? #1354

bryansray opened this issue Nov 16, 2023 · 2 comments

Comments

@bryansray
Copy link

I'm just setting up s3cmd and running into this error ... does anyone happen to know what I'm doing wrong?

!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!

Invoked as: /opt/homebrew/bin/s3cmd info s3://kinoo-assets
Problem: <class 'TypeError: sequence item 1: expected str instance, bytes found
S3cmd:   2.3.0
python:   3.12.0 (main, Oct  2 2023, 12:03:24) [Clang 15.0.0 (clang-1500.0.40.1)]
environment LANG=en_US.UTF-8

Traceback (most recent call last):
  File "/opt/homebrew/bin/s3cmd", line 3286, in <module>
    rc = main()
         ^^^^^^
  File "/opt/homebrew/bin/s3cmd", line 3183, in main
    rc = cmd_func(args)
         ^^^^^^^^^^^^^^
  File "/opt/homebrew/bin/s3cmd", line 1020, in cmd_info
    info = s3.bucket_info(uri)
           ^^^^^^^^^^^^^^^^^^^
  File "/opt/homebrew/Cellar/s3cmd/2.3.0/libexec/lib/python3.12/site-packages/S3/S3.py", line 486, in bucket_info
    response['bucket-location'] = self.get_bucket_location(uri)
                                  ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/homebrew/Cellar/s3cmd/2.3.0/libexec/lib/python3.12/site-packages/S3/S3.py", line 466, in get_bucket_location
    location = getTextFromXml(response['data'], "LocationConstraint")
               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/homebrew/Cellar/s3cmd/2.3.0/libexec/lib/python3.12/site-packages/S3/BaseUtils.py", line 306, in getTextFromXml
    tree = getTreeFromXml(xml)
           ^^^^^^^^^^^^^^^^^^^
  File "/opt/homebrew/Cellar/s3cmd/2.3.0/libexec/lib/python3.12/site-packages/S3/BaseUtils.py", line 266, in getTreeFromXml
    xml, xmlns = stripNameSpace(encode_to_s3(xml))
                 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/homebrew/Cellar/s3cmd/2.3.0/libexec/lib/python3.12/site-packages/S3/BaseUtils.py", line 258, in stripNameSpace
    xml = RE_XML_NAMESPACE.sub("\\1\\2", xml, 1)
          ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
TypeError: sequence item 1: expected str instance, bytes found

It looks like on line 258 it is trying to do a regex on a byte array and it wants a sting instance instead?

Do I have a mismatch on a python version or perhaps something else weird on my machine?

Any help would be greatly appreciated.
@bryansray
Copy link
Author

Adding in some debug output for more help ...

DEBUG: ConfigParser: website_index->index.html
DEBUG: Updating Config.Config cache_file ->
DEBUG: Updating Config.Config follow_symlinks -> False
DEBUG: Updating Config.Config verbosity -> 10
DEBUG: Command: info
DEBUG: CreateRequest: resource[uri]=/
DEBUG: ===== SEND Inner request to determine the bucket region =====
DEBUG: CreateRequest: resource[uri]=/
DEBUG: Using signature v4
DEBUG: get_hostname(kinoo-assets): s3.amazonaws.com
DEBUG: canonical_headers = host:s3.amazonaws.com
x-amz-content-sha256:e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855
x-amz-date:20231117T002755Z

DEBUG: Canonical Request:
GET
/kinoo-assets/
location=
host:s3.amazonaws.com
x-amz-content-sha256:e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855
x-amz-date:20231117T002755Z

host;x-amz-content-sha256;x-amz-date
e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855
----------------------
DEBUG: signature-v4 headers: {'x-amz-date': '20231117T002755Z', 'Authorization': 'AWS4-HMAC-SHA256 Credential=[Redacted]/20231117/us-east-1/s3/aws4_request,SignedHeaders=host;x-amz-content-sha256;x-amz-date,Signature=b8d81d905902024af08621212d6f8bb6703655237f20603fc6d4f3ab12587427', 'x-amz-content-sha256': 'e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855'}
DEBUG: Processing request, please wait...
DEBUG: get_hostname(kinoo-assets): s3.amazonaws.com
DEBUG: ConnMan.get(): creating new connection: https://s3.amazonaws.com
DEBUG: Using ca_certs_file None
DEBUG: Using ssl_client_cert_file None
DEBUG: Using ssl_client_key_file None
DEBUG: httplib.HTTPSConnection() has only context
DEBUG: non-proxied HTTPSConnection(s3.amazonaws.com, None)
DEBUG: format_uri(): /kinoo-assets/?location
DEBUG: Sending request method_string='GET', uri='/kinoo-assets/?location', headers={'x-amz-date': '20231117T002755Z', 'Authorization': 'AWS4-HMAC-SHA256 Credential=[Redacted]/20231117/us-east-1/s3/aws4_request,SignedHeaders=host;x-amz-content-sha256;x-amz-date,Signature=b8d81d905902024af08621212d6f8bb6703655237f20603fc6d4f3ab12587427', 'x-amz-content-sha256': 'e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855'}, body=(0 bytes)
DEBUG: ConnMan.put(): connection put back to pool (https://s3.amazonaws.com#1)
DEBUG: Response:
{'data': b'<?xml version="1.0" encoding="UTF-8"?>\n<LocationConstraint xmlns'
         b'="http://s3.amazonaws.com/doc/2006-03-01/">us-east-2</LocationConstr'
         b'aint>',
 'headers': {'content-type': 'application/xml',
             'date': 'Fri, 17 Nov 2023 00:27:56 GMT',
             'server': 'AmazonS3',
             'transfer-encoding': 'chunked',
             'x-amz-id-2': 'iRXggj3TZhlzJ/E0xJzLxesrWLfKoIjsm3K6XbVz2H18IYkWEnO2QnriAPnl7qVNqKqU1fSClJw=',
             'x-amz-request-id': '3RH1KSCHVJRAGVJP'},
 'reason': 'OK',
 'status': 200}
DEBUG: getlocation inner request failure reason: sequence item 1: expected str instance, bytes found
DEBUG: ===== FAILED Inner request to determine the bucket region =====
DEBUG: Using signature v4
DEBUG: get_hostname(kinoo-assets): kinoo-assets.s3.amazonaws.com
DEBUG: canonical_headers = host:kinoo-assets.s3.amazonaws.com
x-amz-content-sha256:e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855
x-amz-date:20231117T002755Z

DEBUG: Canonical Request:
GET
/
location=
host:kinoo-assets.s3.amazonaws.com
x-amz-content-sha256:e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855
x-amz-date:20231117T002755Z

host;x-amz-content-sha256;x-amz-date
e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855
----------------------
DEBUG: signature-v4 headers: {'x-amz-date': '20231117T002755Z', 'Authorization': 'AWS4-HMAC-SHA256 Credential=[Redacted]/20231117/US/s3/aws4_request,SignedHeaders=host;x-amz-content-sha256;x-amz-date,Signature=4fff4cb75640ead01e25ada090d70bd421debf108233e9e644feab7755094dcc', 'x-amz-content-sha256': 'e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855'}
DEBUG: Processing request, please wait...
DEBUG: get_hostname(kinoo-assets): kinoo-assets.s3.amazonaws.com
DEBUG: ConnMan.get(): creating new connection: https://kinoo-assets.s3.amazonaws.com
DEBUG: httplib.HTTPSConnection() has only context
DEBUG: non-proxied HTTPSConnection(kinoo-assets.s3.amazonaws.com, None)
DEBUG: format_uri(): /?location
DEBUG: Sending request method_string='GET', uri='/?location', headers={'x-amz-date': '20231117T002755Z', 'Authorization': 'AWS4-HMAC-SHA256 Credential=[Redacted]/20231117/US/s3/aws4_request,SignedHeaders=host;x-amz-content-sha256;x-amz-date,Signature=4fff4cb75640ead01e25ada090d70bd421debf108233e9e644feab7755094dcc', 'x-amz-content-sha256': 'e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855'}, body=(0 bytes)
DEBUG: ConnMan.put(): connection put back to pool (https://kinoo-assets.s3.amazonaws.com#1)
DEBUG: Response:
{'data': b'<?xml version="1.0" encoding="UTF-8"?>\n<Error><Code>Authorizatio'
         b'nHeaderMalformed</Code><Message>The authorization header is malforme'
         b"d; the region 'US' is wrong; expecting 'us-east-2'</Message><Region>"
         b'us-east-2</Region><RequestId>ZG1HMX409XPCCXA2</RequestId><HostId>jyG'
         b'ttjyVsI41N6d6AdjwED16AAMd9lX15PJyJ6SSETFpZqbFrLs6znq06OwbV/FNcgmrsgB'
         b'7Mkk=</HostId></Error>',
 'headers': {'connection': 'close',
             'content-type': 'application/xml',
             'date': 'Fri, 17 Nov 2023 00:27:56 GMT',
             'server': 'AmazonS3',
             'transfer-encoding': 'chunked',
             'x-amz-id-2': 'jyGttjyVsI41N6d6AdjwED16AAMd9lX15PJyJ6SSETFpZqbFrLs6znq06OwbV/FNcgmrsgB7Mkk=',
             'x-amz-request-id': 'ZG1HMX409XPCCXA2'},
 'reason': 'Bad Request',
 'status': 400}
INFO: Forwarding request to us-east-2
DEBUG: Using signature v4
DEBUG: get_hostname(kinoo-assets): kinoo-assets.s3.amazonaws.com
DEBUG: canonical_headers = host:kinoo-assets.s3.amazonaws.com
x-amz-content-sha256:e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855
x-amz-date:20231117T002756Z

DEBUG: Canonical Request:
GET
/
location=
host:kinoo-assets.s3.amazonaws.com
x-amz-content-sha256:e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855
x-amz-date:20231117T002756Z

host;x-amz-content-sha256;x-amz-date
e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855
----------------------
DEBUG: signature-v4 headers: {'x-amz-date': '20231117T002756Z', 'Authorization': 'AWS4-HMAC-SHA256 Credential=[Redacted]/20231117/us-east-2/s3/aws4_request,SignedHeaders=host;x-amz-content-sha256;x-amz-date,Signature=45cbdabd0ccbdfd2e14cdf8fb16677eae559b495190f6f4694e2becd328cf0fc', 'x-amz-content-sha256': 'e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855'}
DEBUG: Processing request, please wait...
DEBUG: get_hostname(kinoo-assets): kinoo-assets.s3.amazonaws.com
DEBUG: ConnMan.get(): re-using connection: https://kinoo-assets.s3.amazonaws.com#1
DEBUG: format_uri(): /?location
DEBUG: Sending request method_string='GET', uri='/?location', headers={'x-amz-date': '20231117T002756Z', 'Authorization': 'AWS4-HMAC-SHA256 Credential=[Redacted]/20231117/us-east-2/s3/aws4_request,SignedHeaders=host;x-amz-content-sha256;x-amz-date,Signature=45cbdabd0ccbdfd2e14cdf8fb16677eae559b495190f6f4694e2becd328cf0fc', 'x-amz-content-sha256': 'e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855'}, body=(0 bytes)
DEBUG: ConnMan.put(): connection put back to pool (https://kinoo-assets.s3.amazonaws.com#2)
DEBUG: Response:
{'data': b'<?xml version="1.0" encoding="UTF-8"?>\n<LocationConstraint xmlns'
         b'="http://s3.amazonaws.com/doc/2006-03-01/">us-east-2</LocationConstr'
         b'aint>',
 'headers': {'content-type': 'application/xml',
             'date': 'Fri, 17 Nov 2023 00:27:57 GMT',
             'server': 'AmazonS3',
             'transfer-encoding': 'chunked',
             'x-amz-id-2': 'Q6xa3YeQoaC+XwWwDrVll0oXmDCXtU+87NEIv91X4LNzfdXHYuTTR739gRsFw2LvaioEBLp+VpA=',
             'x-amz-request-id': 'ZG1JBZ7R7SQF0TZY'},
 'reason': 'OK',
 'status': 200}

!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!
    An unexpected error has occurred.
  Please try reproducing the error using
  the latest s3cmd code from the git master
  branch found at:
    https://github.com/s3tools/s3cmd
  and have a look at the known issues list:
    https://github.com/s3tools/s3cmd/wiki/Common-known-issues-and-their-solutions-(FAQ)
  If the error persists, please report the
  following lines (removing any private
  info as necessary) to:
   s3tools-bugs@lists.sourceforge.net


!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!

Invoked as: /opt/homebrew/bin/s3cmd --debug info s3://kinoo-assets
Problem: <class 'TypeError: sequence item 1: expected str instance, bytes found
S3cmd:   2.3.0
python:   3.12.0 (main, Oct  2 2023, 12:03:24) [Clang 15.0.0 (clang-1500.0.40.1)]
environment LANG=en_US.UTF-8

Traceback (most recent call last):
  File "/opt/homebrew/bin/s3cmd", line 3286, in <module>
    rc = main()
         ^^^^^^
  File "/opt/homebrew/bin/s3cmd", line 3183, in main
    rc = cmd_func(args)
         ^^^^^^^^^^^^^^
  File "/opt/homebrew/bin/s3cmd", line 1020, in cmd_info
    info = s3.bucket_info(uri)
           ^^^^^^^^^^^^^^^^^^^
  File "/opt/homebrew/Cellar/s3cmd/2.3.0/libexec/lib/python3.12/site-packages/S3/S3.py", line 486, in bucket_info
    response['bucket-location'] = self.get_bucket_location(uri)
                                  ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/homebrew/Cellar/s3cmd/2.3.0/libexec/lib/python3.12/site-packages/S3/S3.py", line 466, in get_bucket_location
    location = getTextFromXml(response['data'], "LocationConstraint")
               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/homebrew/Cellar/s3cmd/2.3.0/libexec/lib/python3.12/site-packages/S3/BaseUtils.py", line 303, in getTextFromXml
    tree = getTreeFromXml(xml)
           ^^^^^^^^^^^^^^^^^^^
  File "/opt/homebrew/Cellar/s3cmd/2.3.0/libexec/lib/python3.12/site-packages/S3/BaseUtils.py", line 263, in getTreeFromXml
    xml, xmlns = stripNameSpace(encode_to_s3(xml))
                 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/homebrew/Cellar/s3cmd/2.3.0/libexec/lib/python3.12/site-packages/S3/BaseUtils.py", line 255, in stripNameSpace
    xml = RE_XML_NAMESPACE.sub("\\1\\2", xml, 1)
          ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
TypeError: sequence item 1: expected str instance, bytes found

!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!
    An unexpected error has occurred.
  Please try reproducing the error using
  the latest s3cmd code from the git master
  branch found at:
    https://github.com/s3tools/s3cmd
  and have a look at the known issues list:
    https://github.com/s3tools/s3cmd/wiki/Common-known-issues-and-their-solutions-(FAQ)
  If the error persists, please report the
  above lines (removing any private
  info as necessary) to:
   s3tools-bugs@lists.sourceforge.net
!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!

@bryansray
Copy link
Author

Duplicate of #1343 ... apologies.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
1 participant
@bryansray