Gax pubsub ignores custom service account credentials and fetchs ADC instead

[ernestoalejo]

Pubsub with GRPC ignores custom service account credentials passed to the client and fetchs Application Default Credentials instead, raising an exception because it can't found them (obviously).

1. OS type and version: google/debian:jessie (container)
2. Python version and virtual environment information python --version: Python 2.7.9
3. google-cloud-python version pip show google-cloud, pip show google-<service> or pip freeze. This is the freeze output removing some Django-related lines:

cffi==1.8.3
chardet==2.3.0
click==6.6
colorama==0.3.2
configparser==3.5.0
contextlib2==0.5.4
cryptography==1.5.2
enum34==1.1.6
first==2.0.1
functools32==3.2.3.post2
future==0.15.2
futures==3.0.5
gapic-google-logging-v2==0.10.1
gapic-google-pubsub-v1==0.10.1
google-cloud==0.20.0
google-cloud-bigquery==0.20.0
google-cloud-bigtable==0.20.0
google-cloud-core==0.20.0
google-cloud-datastore==0.20.1
google-cloud-dns==0.20.0
google-cloud-error-reporting==0.20.0
google-cloud-language==0.20.0
google-cloud-logging==0.20.0
google-cloud-monitoring==0.20.0
google-cloud-pubsub==0.20.0
google-cloud-resource-manager==0.20.0
google-cloud-storage==0.20.0
google-cloud-translate==0.20.0
google-cloud-vision==0.20.0
google-gax==0.14.1
googleapis-common-protos==1.3.5
grpc-google-iam-v1==0.10.1
grpc-google-logging-v2==0.10.1
grpc-google-pubsub-v1==0.10.1
grpcio==1.0.0
html5lib==0.999
httplib2==0.9.2
idna==2.1
ipaddress==1.0.17
jsonschema==2.5.1
lxml==3.6.4
mccabe==0.5.2
ndg-httpsclient==0.4.2
oauth2client==3.0.0
pip-tools==1.7.0
ply==3.8
protobuf==3.1.0.post1
pyasn1==0.1.9
pyasn1-modules==0.0.8
pycodestyle==2.0.0
pycparser==2.16
pycrypto==2.6.1
pyflakes==1.3.0
pyOpenSSL==16.2.0
rsa==3.4.2
six==1.10.0
urllib3==1.9.1

1. Stacktrace if available:

  File "/usr/local/lib/python2.7/dist-packages/google/cloud/pubsub/topic.py", line 179, in exists
    api = client.publisher_api
  File "/usr/local/lib/python2.7/dist-packages/google/cloud/pubsub/client.py", line 78, in publisher_api
    generated = make_gax_publisher_api(self.connection)
  File "/usr/local/lib/python2.7/dist-packages/google/cloud/pubsub/_gax.py", line 526, in make_gax_publisher_api
    return PublisherApi(channel=channel)
  File "/usr/local/lib/python2.7/dist-packages/google/cloud/gapic/pubsub/v1/publisher_api.py", line 195, in __init__
    scopes=scopes)
  File "/usr/local/lib/python2.7/dist-packages/google/gax/grpc.py", line 119, in create_stub
    metadata_transformer = auth.make_auth_func(scopes)
  File "/usr/local/lib/python2.7/dist-packages/google/gax/auth.py", line 39, in make_auth_func
    google_creds = auth_client.GoogleCredentials.get_application_default()
  File "/usr/local/lib/python2.7/dist-packages/oauth2client/client.py", line 1288, in get_application_default
    return GoogleCredentials._get_implicit_credentials()
  File "/usr/local/lib/python2.7/dist-packages/oauth2client/client.py", line 1278, in _get_implicit_credentials
    raise ApplicationDefaultCredentialsError(ADC_HELP_MSG)
oauth2client.client.ApplicationDefaultCredentialsError: The Application Default Credentials are not available. They are available if running in Google Compute Engine. Otherwise, the environment variable GOOGLE_APPLICATION_CREDENTIALS must be defined pointing to a file defining the credentials. See https://developers.google.com/accounts/docs/application-default-credentials for more information.

1. Code example

from google.cloud import pubsub

from oauth2client.service_account import ServiceAccountCredentials


service_account = {
  "type": "service_account",
  # .....
}


def run():
  credentials = ServiceAccountCredentials.from_json_keyfile_dict(service_account)
  client = pubsub.Client(project='myprojectid', credentials=credentials)
  client.topic('mytopic').exists()

run()

[dhermes]

@ernestoalejo Thanks for filing! @daspecster and I just noticed this earlier this week and a fix is needed ASAP.

[ernestoalejo]

In GCE/GKE the error message sounded like an IAM problem with the service account. (StatusCode.PERMISSION_DENIED, Request had insufficient authentication scopes.)

If someone here needs a quick workaround this will probably be enough until a new version gets released with the fix:

import os

os.environ['GOOGLE_CLOUD_DISABLE_GRPC'] = 'true'

[dhermes]

@ernestoalejo That won't help, since os.environ['GOOGLE_CLOUD_DISABLE_GRPC'] is checked at import time.

[ernestoalejo]

Oh, sorry I forgot to mention that; I changed the env var before importing any other library. Good point. Will probably be better to define it directly in the container.

[theacodes]

@bjwatson I'm going to fix this in google-auth, but by all means fix it in gax.

[bjwatson]

Ok, thanks @jonparrott.

[theacodes]

This is partially addressed by googleapis/gax-python#135 which allows credentials to be passed in.