--user-data is broken if file contains non-7-bit characters

[cperciva]

Error message is "'ascii' codec can't decode byte 0x## in position #: ordinal not in range(128)" where the ## and # match the file.

[cperciva]

Running with --debug:

2014-05-05 18:09:36,275 - awscli.clidriver - DEBUG - Exception caught in main()
Traceback (most recent call last):
File "/usr/local/lib/python2.7/site-packages/awscli/clidriver.py", line 187, in main
return command_table[parsed_args.command](remaining, parsed_args)
File "/usr/local/lib/python2.7/site-packages/awscli/clidriver.py", line 332, in call
return command_table[parsed_args.operation](remaining, parsed_globals)
File "/usr/local/lib/python2.7/site-packages/awscli/clidriver.py", line 437, in call
self.arg_table)
File "/usr/local/lib/python2.7/site-packages/awscli/clidriver.py", line 463, in _build_call_parameters
arg_object.add_to_params(service_params, parsed_args[py_name])
File "/usr/local/lib/python2.7/site-packages/awscli/arguments.py", line 361, in add_to_params
unpacked = self._unpack_argument(value)
File "/usr/local/lib/python2.7/site-packages/awscli/arguments.py", line 387, in _unpack_argument
return unpack_cli_arg(self.argument_object, value)
File "/usr/local/lib/python2.7/site-packages/awscli/argprocess.py", line 370, in unpack_cli_arg
return unpack_scalar_cli_arg(parameter, value)
File "/usr/local/lib/python2.7/site-packages/awscli/argprocess.py", line 423, in unpack_scalar_cli_arg
return six.text_type(value)
UnicodeDecodeError: 'ascii' codec can't decode byte 0x8b in position 1: ordinal not in range(128)

[cperciva]

I think the "return six.text_type(value)" at the end of argprocess.py should probably be "return six.binary_type(value)" to avoid this error, but I don't know what other code paths use this code.

[quiver]

Confirmed using command like this:

$ aws ec2 run-instances --image-id ami-a1bec3a0 --count 1 --instance-type t1.micro --key-name amzn-linux --user-data file://userdata.sh

I looked into this a bit.

When user-data is passed as file(--user-data file://userdata.sh), awscli/argprocess.py::unpack_scalar_cli_arg function's six.text_type tries to unicode the file body without specifying encoding, which results in UnicodeDecodeError.

When user-data are passed as argument(--user-data "echo 'こんにちは'"), parameter is converted into unicode using terminal encoding, so six.text_type succeeds.
But user-data needs to be base64 encoded.
When botocore/parameter.py::BlobParameter tries to base64 parameter, base64.b64encode(six.b(value)) fails, because:
a) base64 can only encode byte strings, and
b) six.b basically does nothing for py2 (https://bitbucket.org/bdarnell/six/src/c47e429277ad73db61afcfe333fbc197d2f48688/six.py?at=default#cl-272), so base64.b64encode(six.b(uni_char)) is base64.b64encode(uni_char)

[jamesls]

Confirmed, taking a look. Thanks for the debug info.

[jamesls]

This has been fixed and is now in the 1.3.11 release. Thanks for the report.