Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Allow recursive copy of objects in S3 bucket #1379

Closed
1 task done
jsabater opened this issue Feb 20, 2023 · 1 comment · Fixed by #1608
Closed
1 task done

Allow recursive copy of objects in S3 bucket #1379

jsabater opened this issue Feb 20, 2023 · 1 comment · Fixed by #1608
Assignees
@abikouo
Labels
feature This issue/PR relates to a feature request WIP Work in progress

Comments

@jsabater
Copy link

Summary

Implement recursive copy of all objects from (source) bucket to (destination) bucket and from local to bucket. Example on Amazon AWS:

aws s3 cp localfolder s3://dstbucket/ --recursive

Example using MinIO console client:

mcli cp --recursive minio1/srcbucket/ minio1/dstbucket/

Context: Right now, when a new website is created via a playbook, srcbucket contains the default list of objects (images) to be copied to the newly created bucket. For that I am executing the example above console command of mcli because feeding the list of images taken using s3_object in list mode to s3_object in copy mode is just too slow.

Issue Type

Feature Idea

Component Name

amazon.aws.s3_object

Additional Information

Using current examples from the documenation, a proposal could be as follows:

- name: Copy all objects already stored in another bucket
  amazon.aws.s3_object:
    endpoint_url: <my-endpoint>
    access_key: <my-access-key>
    secret_key: <my-secret-key>
    bucket: dstbucket
    object: / # Or 'all' or something similar. Could even not be required given the value of 'copy_src.object' below
    mode: copy
    copy_src:
        bucket: srcbucket
        object: / # Or 'all' or something similar

Code of Conduct

  • I agree to follow the Ansible Code of Conduct
@abikouo abikouo added waiting_on_contributor Needs help. Feel free to engage to get things unblocked feature This issue/PR relates to a feature request labels Feb 21, 2023
reparepo pushed a commit to reparepo/amazon.aws that referenced this issue Mar 17, 2023
@wp-davisona @goneri
route53: Restore support for zero weighted DNS records (ansible-colle…
d1c04ae 
…ctions#1379)

route53: Restore support for zero weighted DNS records

SUMMARY
In ansible-collections#1117 (comment) and https://github.com/ansible-collections/community.aws/pull/1117/files#r869391659 this line was recommended to be simplified, but not any will also return true if weight_in has a value of 0, not only when it is None
Fixes ansible-collections#1378
ISSUE TYPE

Bugfix Pull Request

COMPONENT NAME
route53
ADDITIONAL INFORMATION

Previously it was possible to create weighted records with a weight of 0. Currently the playbook below returns the error:
You have specified identifier which makes sense only if you specify one of: weight, region, geo_location or failover.

- name: Bug demo
  hosts: localhost
  tasks:
    - name: Set 0 weight for old env
      route53:
        wait: yes
        ttl: '5'
        type: 'CNAME'
        identifier: old
        overwrite: yes
        record: 'record.example.com.'
        zone: 'example.com.'
        value: 'record-old.example.com.'
        weight: '0'
        state: present

Reviewed-by: Mark Chappell <None>

This commit was initially merged in https://github.com/ansible-collections/community.aws
See: ansible-collections/community.aws@9195021
@ansibullbot
Copy link

Files identified in the description:
None

If these files are inaccurate, please update the component name section of the description or use the !component bot command.

click here for bot help

@abikouo abikouo self-assigned this Jun 12, 2023
@abikouo abikouo added WIP Work in progress and removed waiting_on_contributor Needs help. Feel free to engage to get things unblocked labels Jun 12, 2023
@abikouo abikouo mentioned this issue Jun 12, 2023
Merged
softwarefactory-project-zuul bot pushed a commit that referenced this issue Jul 3, 2023
@abikouo
s3_object - allow recursive copy of all objects in S3 bucket (#1608)
3594dec 
s3_object - allow recursive copy of all objects in S3 bucket

SUMMARY

Add support to copy recursively all objects from one bucket to another one, user can set prefix to limit the object to copy.
closes #1379

ISSUE TYPE


Feature Pull Request

COMPONENT NAME

s3_object

Reviewed-by: Helen Bailey <hebailey@redhat.com>
Reviewed-by: Bikouo Aubin
abikouo pushed a commit to abikouo/amazon.aws that referenced this issue Sep 18, 2023
@wp-davisona
route53: Restore support for zero weighted DNS records (ansible-colle…
825f10d 
…ctions#1379)

route53: Restore support for zero weighted DNS records

SUMMARY
In ansible-collections#1117 (comment) and https://github.com/ansible-collections/community.aws/pull/1117/files#r869391659 this line was recommended to be simplified, but not any will also return true if weight_in has a value of 0, not only when it is None
Fixes ansible-collections#1378
ISSUE TYPE


Bugfix Pull Request

COMPONENT NAME
route53
ADDITIONAL INFORMATION


Previously it was possible to create weighted records with a weight of 0. Currently the playbook below returns the error:
You have specified identifier which makes sense only if you specify one of: weight, region, geo_location or failover.


- name: Bug demo
  hosts: localhost
  tasks:
    - name: Set 0 weight for old env
      route53:
        wait: yes
        ttl: '5'
        type: 'CNAME'
        identifier: old
        overwrite: yes
        record: 'record.example.com.'
        zone: 'example.com.'
        value: 'record-old.example.com.'
        weight: '0'
        state: present

Reviewed-by: Mark Chappell <None>
abikouo pushed a commit to abikouo/amazon.aws that referenced this issue Sep 18, 2023
@wp-davisona @abikouo
route53: Restore support for zero weighted DNS records (ansible-colle…
5a505b3 
…ctions#1379)

route53: Restore support for zero weighted DNS records

SUMMARY
In ansible-collections#1117 (comment) and https://github.com/ansible-collections/community.aws/pull/1117/files#r869391659 this line was recommended to be simplified, but not any will also return true if weight_in has a value of 0, not only when it is None
Fixes ansible-collections#1378
ISSUE TYPE


Bugfix Pull Request

COMPONENT NAME
route53
ADDITIONAL INFORMATION


Previously it was possible to create weighted records with a weight of 0. Currently the playbook below returns the error:
You have specified identifier which makes sense only if you specify one of: weight, region, geo_location or failover.


- name: Bug demo
  hosts: localhost
  tasks:
    - name: Set 0 weight for old env
      route53:
        wait: yes
        ttl: '5'
        type: 'CNAME'
        identifier: old
        overwrite: yes
        record: 'record.example.com.'
        zone: 'example.com.'
        value: 'record-old.example.com.'
        weight: '0'
        state: present

Reviewed-by: Mark Chappell <None>
abikouo pushed a commit to abikouo/amazon.aws that referenced this issue Oct 24, 2023
@wp-davisona
route53: Restore support for zero weighted DNS records (ansible-colle…
9195021 
…ctions#1379)

route53: Restore support for zero weighted DNS records

SUMMARY
In ansible-collections#1117 (comment) and https://github.com/ansible-collections/community.aws/pull/1117/files#r869391659 this line was recommended to be simplified, but not any will also return true if weight_in has a value of 0, not only when it is None
Fixes ansible-collections#1378
ISSUE TYPE


Bugfix Pull Request

COMPONENT NAME
route53
ADDITIONAL INFORMATION


Previously it was possible to create weighted records with a weight of 0. Currently the playbook below returns the error:
You have specified identifier which makes sense only if you specify one of: weight, region, geo_location or failover.


- name: Bug demo
  hosts: localhost
  tasks:
    - name: Set 0 weight for old env
      route53:
        wait: yes
        ttl: '5'
        type: 'CNAME'
        identifier: old
        overwrite: yes
        record: 'record.example.com.'
        zone: 'example.com.'
        value: 'record-old.example.com.'
        weight: '0'
        state: present

Reviewed-by: Mark Chappell <None>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@abikouo abikouo

Labels
feature This issue/PR relates to a feature request WIP Work in progress
Projects
None yet
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

s3_object - allow recursive copy of all objects in S3 bucket abikouo/amazon.aws
3 participants
@jsabater @ansibullbot @abikouo