sync and cp does not work as expected

[S0uLHun43r]

The same problem here, awscli==1.16.186
aws s3 cp and aws s3 sync happen same .

I have 1217 files.but sync and cp all give 1000 files.

but when i use s3cmd,it works . @JordonPhillips please open this issue

[justnance]

@S0uLHun43r - Thanks for reporting this issue. It appears this is related to issue #3087. I not able to reproduce the same problem after testing with both the s3 cp and s3 sync command, the latest version of the CLI, and with a bucket with 1200 items of various file types. With the screen shot provided, I see there is a 3rd party endpoint in use:

aws s3 --endpoint=http://ceph.whlocal.com:8080 sync s3://ums-tmp/umpic/JPG/...
aws s3 --endpoint=http://ceph.whlocal.com:8080 ls s3://ums-tmp/umpic/JPG/...

It appears the return of "1000 files" instead of all 1217 is because the 3rd party endpoint is not working with the CLI pagination where it is not returning the next page token. If you add the --debug option on the end of the command it may show if there is an error around the continuation-token. It is possible the next page token may not be support by Ceph. As such, this behavior is not controlled by the CLI.

[no-response]

This issue has been automatically closed because there has been no response to our request for more information from the original author. With only the information that is currently in the issue, we don't have enough information to take action. Please reach out if you have or find the answers we need so that we can investigate further.