Aws::S3::Client#get_bucket_lifecycle_configuration raise NotImplementedError #1764

inokappa · 2018-04-21T10:11:19Z

Issue description

When I test the following code, Aws::S3::Client#get_bucket_lifecycle_configuration, it raises NotImplementedError instead of getting lifecycle_configurations.

require 'pp'
require 'aws-sdk-s3'

s3 = Aws::S3::Client.new(
  region: 'ap-northeast-1',
  profile: 'my-profile'
)

bcl = s3.get_bucket_lifecycle_configuration(bucket: ARGV[0])
pp bcl

I confirmed that this phenomenon occurs only when filter.and.tags is set in lifecycle configuration of Bucket. Please see "Code snippets / steps to reproduce".

Gem name ('aws-sdk', 'aws-sdk-resources' or service gems like 'aws-sdk-s3') and its version

aws-partitions (1.80.0)
aws-sdk-core (3.19.0)
aws-sdk-kms (1.5.0)
aws-sdk-s3 (1.9.1)
aws-sigv4 (1.0.2)
bundler (1.16.1)
coderay (1.1.2)
jmespath (1.4.0)
method_source (0.9.0)
pry (0.11.3)

Version of Ruby, OS environment

$ ruby --version
ruby 2.5.0p0 (2017-12-25 revision 61468) [x86_64-darwin15]

Code snippets / steps to reproduce

Bucket lifecycle configuration

# OK Bucket lifecycle configuration
$ aws s3api get-bucket-lifecycle-configuration --bucket ok-bucket
{
    "Rules": [
        {
            "Status": "Enabled",
            "NoncurrentVersionExpiration": {
                "NoncurrentDays": 1
            },
            "Filter": {
                "Prefix": ""
            },
            "Expiration": {
                "Days": 4
            },
            "Transitions": [
                {
                    "Days": 3,
                    "StorageClass": "GLACIER"
                }
            ],
            "ID": "ok-rule"
        }
    ]
}

# NG Bucket lifecycle configuration
$ aws s3api get-bucket-lifecycle-configuration --bucket ng-bucket
{
    "Rules": [
        {
            "Status": "Enabled",
            "NoncurrentVersionExpiration": {
                "NoncurrentDays": 366
            },
            "NoncurrentVersionTransitions": [
                {
                    "NoncurrentDays": 1,
                    "StorageClass": "GLACIER"
                }
            ],
            "Filter": {
                "And": {
                    "Prefix": "dev",
                    "Tags": [
                        {
                            "Value": "",
                            "Key": "dev"
                        },
                        {
                            "Value": "hoge",
                            "Key": "foobar"
                        }
                    ]
                }
            },
            "Expiration": {
                "Days": 395
            },
            "Transitions": [
                {
                    "Days": 30,
                    "StorageClass": "STANDARD_IA"
                }
            ],
            "ID": "ng-rule"
        }
    ]
}

Run Test

$ bundle exec ruby s3_test.rb ng-bucket

backtrace will be like following.

Traceback (most recent call last):
        32: from s3_test.rb:9:in `<main>'
        31: from /Users/OreOre/s3_check/vendor/bundle/ruby/2.5.0/gems/aws-sdk-s3-1.9.1/lib/aws-sdk-s3/client.rb:1834:in `get_bucket_lifecycle_configuration'
        30: from /Users/OreOre/s3_check/vendor/bundle/ruby/2.5.0/gems/aws-sdk-core-3.19.0/lib/seahorse/client/request.rb:70:in `send_request'
        29: from /Users/OreOre/s3_check/vendor/bundle/ruby/2.5.0/gems/aws-sdk-core-3.19.0/lib/seahorse/client/plugins/response_target.rb:23:in `call'
        28: from /Users/OreOre/s3_check/vendor/bundle/ruby/2.5.0/gems/aws-sdk-core-3.19.0/lib/aws-sdk-core/plugins/response_paging.rb:10:in `call'
        27: from /Users/OreOre/s3_check/vendor/bundle/ruby/2.5.0/gems/aws-sdk-core-3.19.0/lib/aws-sdk-core/plugins/param_converter.rb:24:in `call'
        26: from /Users/OreOre/s3_check/vendor/bundle/ruby/2.5.0/gems/aws-sdk-core-3.19.0/lib/aws-sdk-core/plugins/idempotency_token.rb:17:in `call'
        25: from /Users/OreOre/s3_check/vendor/bundle/ruby/2.5.0/gems/aws-sdk-core-3.19.0/lib/aws-sdk-core/plugins/jsonvalue_converter.rb:20:in `call'
        24: from /Users/OreOre/s3_check/vendor/bundle/ruby/2.5.0/gems/aws-sdk-s3-1.9.1/lib/aws-sdk-s3/plugins/accelerate.rb:35:in `call'
        23: from /Users/OreOre/s3_check/vendor/bundle/ruby/2.5.0/gems/aws-sdk-s3-1.9.1/lib/aws-sdk-s3/plugins/dualstack.rb:26:in `call'
        22: from /Users/OreOre/s3_check/vendor/bundle/ruby/2.5.0/gems/aws-sdk-s3-1.9.1/lib/aws-sdk-s3/plugins/sse_cpk.rb:22:in `call'
        21: from /Users/OreOre/s3_check/vendor/bundle/ruby/2.5.0/gems/aws-sdk-core-3.19.0/lib/seahorse/client/plugins/raise_response_errors.rb:14:in `call'
        20: from /Users/OreOre/s3_check/vendor/bundle/ruby/2.5.0/gems/aws-sdk-core-3.19.0/lib/aws-sdk-core/plugins/param_validator.rb:24:in `call'
        19: from /Users/OreOre/s3_check/vendor/bundle/ruby/2.5.0/gems/aws-sdk-core-3.19.0/lib/seahorse/client/plugins/endpoint.rb:45:in `call'
        18: from /Users/OreOre/s3_check/vendor/bundle/ruby/2.5.0/gems/aws-sdk-core-3.19.0/lib/aws-sdk-core/plugins/user_agent.rb:13:in `call'
        17: from /Users/OreOre/s3_check/vendor/bundle/ruby/2.5.0/gems/aws-sdk-core-3.19.0/lib/aws-sdk-core/rest/handler.rb:9:in `call'
        16: from /Users/OreOre/s3_check/vendor/bundle/ruby/2.5.0/gems/aws-sdk-core-3.19.0/lib/seahorse/client/response.rb:44:in `on'
        15: from /Users/OreOre/s3_check/vendor/bundle/ruby/2.5.0/gems/aws-sdk-core-3.19.0/lib/seahorse/client/http/response.rb:137:in `on_success'
        14: from /Users/OreOre/s3_check/vendor/bundle/ruby/2.5.0/gems/aws-sdk-core-3.19.0/lib/seahorse/client/http/response.rb:130:in `on_done'
        13: from /Users/OreOre/s3_check/vendor/bundle/ruby/2.5.0/gems/aws-sdk-core-3.19.0/lib/seahorse/client/http/response.rb:166:in `block in listener'
        12: from /Users/OreOre/s3_check/vendor/bundle/ruby/2.5.0/gems/aws-sdk-core-3.19.0/lib/seahorse/client/http/response.rb:139:in `block in on_success'
        11: from /Users/OreOre/s3_check/vendor/bundle/ruby/2.5.0/gems/aws-sdk-core-3.19.0/lib/seahorse/client/response.rb:45:in `block in on'
        10: from /Users/OreOre/s3_check/vendor/bundle/ruby/2.5.0/gems/aws-sdk-core-3.19.0/lib/aws-sdk-core/rest/handler.rb:9:in `block in call'
         9: from /Users/OreOre/s3_check/vendor/bundle/ruby/2.5.0/gems/aws-sdk-core-3.19.0/lib/aws-sdk-core/rest/response/parser.rb:12:in `apply'
         8: from /Users/OreOre/s3_check/vendor/bundle/ruby/2.5.0/gems/aws-sdk-core-3.19.0/lib/aws-sdk-core/rest/response/parser.rb:32:in `extract_body'
         7: from /Users/OreOre/s3_check/vendor/bundle/ruby/2.5.0/gems/aws-sdk-core-3.19.0/lib/aws-sdk-core/rest/response/body.rb:23:in `apply'
         6: from /Users/OreOre/s3_check/vendor/bundle/ruby/2.5.0/gems/aws-sdk-core-3.19.0/lib/aws-sdk-core/rest/response/body.rb:37:in `parse'
         5: from /Users/OreOre/s3_check/vendor/bundle/ruby/2.5.0/gems/aws-sdk-core-3.19.0/lib/aws-sdk-core/xml/parser.rb:34:in `parse'
         4: from /Users/OreOre/s3_check/vendor/bundle/ruby/2.5.0/gems/aws-sdk-core-3.19.0/lib/aws-sdk-core/xml/parser/engines/rexml.rb:19:in `parse'
         3: from /Users/OreOre/.rbenv/versions/2.5.0/lib/ruby/2.5.0/rexml/parsers/streamparser.rb:34:in `parse'
         2: from /Users/OreOre/s3_check/vendor/bundle/ruby/2.5.0/gems/aws-sdk-core-3.19.0/lib/aws-sdk-core/xml/parser/engines/rexml.rb:27:in `tag_start'
         1: from /Users/OreOre/s3_check/vendor/bundle/ruby/2.5.0/gems/aws-sdk-core-3.19.0/lib/aws-sdk-core/xml/parser/stack.rb:18:in `start_element'
/Users/OreOre/s3_check/vendor/bundle/ruby/2.5.0/gems/aws-sdk-core-3.19.0/lib/aws-sdk-core/xml/parser/frame.rb:150:in `child_frame': NotImplementedError (NotImplementedError)

Please confirm.

The text was updated successfully, but these errors were encountered:

awood45 · 2018-04-23T16:45:20Z

Can you share your Gemfile.lock? Just want to check something on that.

cjyclaire · 2018-04-23T17:46:24Z

Just s soft update, I can reproduce this, looks like an issue when our xml parser is expecting 'Tag' while provided with a 'Key', still investigating

cjyclaire · 2018-04-23T20:25:37Z

Just figured out what the issue is, so TagSet shape was modeled to be flattened at shape reference instead of shape itself, thus, our parser failed to detect it wants to be flattened.

inokappa · 2018-04-23T21:51:50Z

@awood45

Thank you for your response.

Can you share your Gemfile.lock? Just want to check something on that.

Please confirm following.

GEM
  remote: https://rubygems.org/
  specs:
    aws-partitions (1.80.0)
    aws-sdk-core (3.19.0)
      aws-partitions (~> 1.0)
      aws-sigv4 (~> 1.0)
      jmespath (~> 1.0)
    aws-sdk-kms (1.5.0)
      aws-sdk-core (~> 3)
      aws-sigv4 (~> 1.0)
    aws-sdk-s3 (1.9.1)
      aws-sdk-core (~> 3)
      aws-sdk-kms (~> 1)
      aws-sigv4 (~> 1.0)
    aws-sigv4 (1.0.2)
    coderay (1.1.2)
    jmespath (1.4.0)
    method_source (0.9.0)
    pry (0.11.3)
      coderay (~> 1.1.0)
      method_source (~> 0.9.0)

PLATFORMS
  ruby

DEPENDENCIES
  aws-sdk-s3 (~> 1)
  pry

BUNDLED WITH
   1.16.1

Regards,
Yohei

cjyclaire · 2018-04-24T00:00:49Z

PR #1766 addressing the fix is opened for review

cjyclaire · 2018-04-24T18:09:07Z

PR has been merged, will be out in the next release

cjyclaire · 2018-04-24T19:56:42Z

aws-sdk-core Release 3.20.1 containing the fix is out, closing, feel free to reopen with further questions or issues

inokappa mentioned this issue Apr 21, 2018

"prefix" of s3_bucket have_lifecycle_rule does not work k1LoW/awspec#365

Closed

cjyclaire added the bug label Apr 23, 2018

cjyclaire mentioned this issue Apr 24, 2018

Fix XML parsing flattened shape from shape ref #1766

Merged

cjyclaire added pr-in-progress and removed pr-in-progress labels Apr 24, 2018

cjyclaire closed this as completed Apr 24, 2018

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Aws::S3::Client#get_bucket_lifecycle_configuration raise NotImplementedError #1764

Aws::S3::Client#get_bucket_lifecycle_configuration raise NotImplementedError #1764

inokappa commented Apr 21, 2018 •

edited

Loading

awood45 commented Apr 23, 2018

cjyclaire commented Apr 23, 2018

cjyclaire commented Apr 23, 2018

inokappa commented Apr 23, 2018

cjyclaire commented Apr 24, 2018

cjyclaire commented Apr 24, 2018

cjyclaire commented Apr 24, 2018

Aws::S3::Client#get_bucket_lifecycle_configuration raise NotImplementedError #1764

Aws::S3::Client#get_bucket_lifecycle_configuration raise NotImplementedError #1764

Comments

inokappa commented Apr 21, 2018 • edited Loading

Issue description

Gem name ('aws-sdk', 'aws-sdk-resources' or service gems like 'aws-sdk-s3') and its version

Version of Ruby, OS environment

Code snippets / steps to reproduce

Bucket lifecycle configuration

Run Test

awood45 commented Apr 23, 2018

cjyclaire commented Apr 23, 2018

cjyclaire commented Apr 23, 2018

inokappa commented Apr 23, 2018

cjyclaire commented Apr 24, 2018

cjyclaire commented Apr 24, 2018

cjyclaire commented Apr 24, 2018

inokappa commented Apr 21, 2018 •

edited

Loading