aws s3 sync returns non-zero code when all is fine

[lukaszbudnik]

Hi awscli team,

I'm using awscli from pip:

$ aws --version aws-cli/1.7.4 Python/2.7.8 Darwin/13.3.0

When I execute sync between bucket and local folder like this:

aws s3 sync "s3://${env}-yyy" "${env}-yyy" > output.txt 2>&1

Sync executes fine ${env} is substituted with the correct env variable. In test bucket I have some test files added: max file size is around 2 kb and the number of files is around 60. All files are fetched fine, but when I check the return code the return code is always 2.

When I execute my sync command with --debug and do this:

grep 'HTTP Response Code’ output.txt

all files were downloaded OK:

Thread-13 - awscli.errorhandler - DEBUG - HTTP Response Code: 200

doing:

grep 'Received print task' output.txt

gives for all files:

error=False, total_parts=None, warning=None

aws s3 sync is a part of much larger bash script for disaster recovery and I really need to distinguish when sync failed or not (error reporting is very important to me).

Could you take a look at it?

[lukaszbudnik]

On my production Ubuntu machines I also get return code 2 for a successful sync. awscli used:

aws-cli/1.7.1 Python/2.7.3 Linux/3.2.0-69-virtual

and

aws-cli/1.7.4 Python/2.7.3 Linux/3.2.0-69-virtual

[kyleknap]

So a return code of 2 means that some files were skipped in your sync but all of the files that were transferred succeeded. A return code of 1 means that at least one of your transfers failed. The skipped files are printed to standard error (even when you do not add --debug). So if you could figure out what files are being skipped I can provide you with more information why. If I were to guess, the message you are getting is somewhat related to this issue: #1082

[jzt]

One thing that can cause a return code of 2 is when you do something like:

aws s3 sync s3://mybucket ./newdirectory

The first thing you will see in the output is:
warning: Skipping file /path/to/cwd/new_directory/. File does not exist.

It will create /path/to/cwd/new_directory for you, the sync will complete successfully, and it will return 2.

So for @lukaszbudnik , does the ${env}-yyy directory exist before the sync is executed? If not, that is probably your problem.

Side question: are these return codes documented anywhere? It would be nice to have an idea how to handle these cases.

Edit: Sorry, just clicked on the link to #1082 and noticed that everything I said is spelled out there. Sorry for the noise.

[lukaszbudnik]

hi guys,

Indeed I see this warning in the --debug output.

I added creating this directory before synchronising and now the return code is 0.

Feel free to close this issue.

thanks,
Łukasz

[jamesls]

Good to hear, thanks for the update.