s3cmd info does not work with ceph nautilus radosgw

[mistur]

Hello,

I have a new fresh ceph cluster in Nautilus and I cannot get s3cmd info on buckets. It works on my older cluster in Luminous. I only change URL endpoint, access_key and secret_key in the .s3cfg file.

Is this issue on s3cmd or ceph side ?

$ s3cmd -v info s3://image-net
s3://image-net/ (bucket):
   Location:  us-east-1
   Payer:     None

!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!
    An unexpected error has occurred.
  Please try reproducing the error using
  the latest s3cmd code from the git master
  branch found at:
    https://github.com/s3tools/s3cmd
  and have a look at the known issues list:
    https://github.com/s3tools/s3cmd/wiki/Common-known-issues-and-their-solutions
  If the error persists, please report the
  following lines (removing any private
  info as necessary) to:
   s3tools-bugs@lists.sourceforge.net


!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!

Invoked as: /usr/local/bin/s3cmd -v info s3://image-net
Problem: <class 'TypeError: must be str, not NoneType
S3cmd:   2.0.2
python:   3.6.9 (default, Nov  7 2019, 10:44:02) 
[GCC 8.3.0]
environment LANG=fr_CH.UTF-8

Traceback (most recent call last):
  File "/usr/local/lib/python3.6/dist-packages/s3cmd-2.0.2-py3.6.egg/EGG-INFO/scripts/s3cmd", line 3092, in <module>
  File "/usr/local/lib/python3.6/dist-packages/s3cmd-2.0.2-py3.6.egg/EGG-INFO/scripts/s3cmd", line 3001, in main
  File "/usr/local/lib/python3.6/dist-packages/s3cmd-2.0.2-py3.6.egg/EGG-INFO/scripts/s3cmd", line 944, in cmd_info
TypeError: must be str, not NoneType

!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!
    An unexpected error has occurred.
  Please try reproducing the error using
  the latest s3cmd code from the git master
  branch found at:
    https://github.com/s3tools/s3cmd
  and have a look at the known issues list:
    https://github.com/s3tools/s3cmd/wiki/Common-known-issues-and-their-solutions
  If the error persists, please report the
  above lines (removing any private
  info as necessary) to:
   s3tools-bugs@lists.sourceforge.net
!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!

Thanks for your help.

Best regards,

Yoann

[fviard]

Hi, this is curious.
Can you run you command also with the -d flag and post here the log?
thanks

[mistur]

Hello,

here the result of the command on the 2 clusters :

Ceph Nautilus : image-net_KO.log
Ceph Luminous : image-net_OK.log

[mistur]

It was a misconfiguration on the ceph side, I close this issue.
Thanks.
Yoann

[fviard]

Thanks for the update, I was about to reply to you.
Would you mind to share what kind of configuration you did change?
My goal is to understand what kind of things trigger that with Ceph and make s3cmd more resilient.

To better understand, thanks to your debug logs, we can see that,
In your Luminous instance:

• When asking for the lifecycle policy info (ie myobject?lifecycle), the reply was 404, that would mean that no policy was set (or that it is not supported by the server)
  In the Nautilus instance:
• When asking the same thing, this time there is a reply with content. BUT, instead of being an expected "LifecycleConfiguration" response, it is the standard "listObject" response like if the "?lifecycle" part was ignored.

So, what I'm curious to understand is:

• Did your fix was just to disable the lifecycle policy for the object. And in that case, maybe lifecycle replies of Ceph are not working.
• Or is it that now it is working correctly? (Than can be seen in the debug log policy, if you look at the reply of a request to an uri like "object?lifecycle").

Thanks

[mistur]

the "rgw dns name" entry was not set, the rgw didn't manage vhost correctly.
cf https://tracker.ceph.com/issues/45099
Best,
Yoann

[fviard]

Ok, I see, it is just a side effect of something unrelated.
If you want to follow up on the Ceph issue, you can transmit my remark that:
despite the configuration issue redirecting the request to RGWListBuckets, for safety, it should detect that a sub-ressource was asked, and that it does not make any sense in the context of RGWListBuckets and so to throw any error: 404, 500, ...

[fviard]

Despite not being a client issue, I just pushed a fix to be more safe/resilient if ever it was to happening again.