UnicodeError for Dataset(tablename).download() if credentials are not set properly

[andy-esch]

I'm not sure what is going on, but I'm getting UnicodeError for pretty standard use of Dataset:

Dataset('tablename').download()

Gives this:

---------------------------------------------------------------------------
UnicodeError                              Traceback (most recent call last)
~/.local/share/virtualenvs/landing-page-QW9sxmZW/bin/../lib/python3.7/encodings/idna.py in encode(self, input, errors)
    164                 if not (0 < len(label) < 64):
--> 165                     raise UnicodeError("label empty or too long")
    166             if len(labels[-1]) >= 64:

UnicodeError: label empty or too long

The above exception was the direct cause of the following exception:

UnicodeError                              Traceback (most recent call last)
~/.local/share/virtualenvs/landing-page-QW9sxmZW/lib/python3.7/site-packages/carto/auth.py in send(self, relative_path, http_method, **requests_args)
    146 
--> 147             response = super(APIKeyAuthClient, self).send(relative_path, http_method, **requests_args)
    148         except Exception as e:

~/.local/share/virtualenvs/landing-page-QW9sxmZW/lib/python3.7/site-packages/pyrestcli/auth.py in send(self, relative_path, http_method, **requests_args)
     35 
---> 36         return self.session.request(http_method, url, **requests_args)
     37 

~/.local/share/virtualenvs/landing-page-QW9sxmZW/lib/python3.7/site-packages/requests/sessions.py in request(self, method, url, params, data, headers, cookies, files, auth, timeout, allow_redirects, proxies, hooks, stream, verify, cert, json)
    523         settings = self.merge_environment_settings(
--> 524             prep.url, proxies, stream, verify, cert
    525         )

~/.local/share/virtualenvs/landing-page-QW9sxmZW/lib/python3.7/site-packages/requests/sessions.py in merge_environment_settings(self, url, proxies, stream, verify, cert)
    699             no_proxy = proxies.get('no_proxy') if proxies is not None else None
--> 700             env_proxies = get_environ_proxies(url, no_proxy=no_proxy)
    701             for (k, v) in env_proxies.items():

~/.local/share/virtualenvs/landing-page-QW9sxmZW/lib/python3.7/site-packages/requests/utils.py in get_environ_proxies(url, no_proxy)
    760     """
--> 761     if should_bypass_proxies(url, no_proxy=no_proxy):
    762         return {}

~/.local/share/virtualenvs/landing-page-QW9sxmZW/lib/python3.7/site-packages/requests/utils.py in should_bypass_proxies(url, no_proxy)
    744         try:
--> 745             bypass = proxy_bypass(parsed.hostname)
    746         except (TypeError, socket.gaierror):

/usr/local/Cellar/python/3.7.4_1/Frameworks/Python.framework/Versions/3.7/lib/python3.7/urllib/request.py in proxy_bypass(host)
   2609         else:
-> 2610             return proxy_bypass_macosx_sysconf(host)
   2611 

/usr/local/Cellar/python/3.7.4_1/Frameworks/Python.framework/Versions/3.7/lib/python3.7/urllib/request.py in proxy_bypass_macosx_sysconf(host)
   2586         proxy_settings = _get_proxy_settings()
-> 2587         return _proxy_bypass_macosx_sysconf(host, proxy_settings)
   2588 

/usr/local/Cellar/python/3.7.4_1/Frameworks/Python.framework/Versions/3.7/lib/python3.7/urllib/request.py in _proxy_bypass_macosx_sysconf(host, proxy_settings)
   2559                 try:
-> 2560                     hostIP = socket.gethostbyname(hostonly)
   2561                     hostIP = ip2num(hostIP)

UnicodeError: encoding with 'idna' codec failed (UnicodeError: label empty or too long)

During handling of the above exception, another exception occurred:

CartoException                            Traceback (most recent call last)
~/.local/share/virtualenvs/landing-page-QW9sxmZW/lib/python3.7/site-packages/carto/sql.py in send(self, sql, parse_json, do_post, format, **request_args)
    105                                              'GET',
--> 106                                              params=params)
    107             else:

~/.local/share/virtualenvs/landing-page-QW9sxmZW/lib/python3.7/site-packages/carto/auth.py in send(self, relative_path, http_method, **requests_args)
    148         except Exception as e:
--> 149             raise CartoException(e)
    150 

CartoException: encoding with 'idna' codec failed (UnicodeError: label empty or too long)

During handling of the above exception, another exception occurred:

CartoException                            Traceback (most recent call last)
<ipython-input-12-051050de5296> in <module>
----> 1 Dataset(station_points).download(limit=2)

~/.local/share/virtualenvs/landing-page-QW9sxmZW/lib/python3.7/site-packages/cartoframes/data/dataset/dataset.py in __init__(self, data, credentials, schema)
     93     def __init__(self, data, credentials=None, schema=None):
     94         self._registry = self._get_strategies_registry()
---> 95         self._strategy = self._init_strategy(data, credentials, schema)
     96 
     97     def _init_strategy(self, data, credentials=None, schema=None):

~/.local/share/virtualenvs/landing-page-QW9sxmZW/lib/python3.7/site-packages/cartoframes/data/dataset/dataset.py in _init_strategy(self, data, credentials, schema)
     99         for strategy in self._registry.get_strategies():
    100             if strategy.can_work_with(data):
--> 101                 return strategy.create(data, credentials, schema)
    102 
    103         raise ValueError('We can not detect the Dataset type')

~/.local/share/virtualenvs/landing-page-QW9sxmZW/lib/python3.7/site-packages/cartoframes/data/dataset/registry/table_dataset.py in create(cls, data, credentials, schema)
     24     @classmethod
     25     def create(cls, data, credentials, schema=None):
---> 26         return cls(data, credentials, schema)
     27 
     28     def download(self, limit, decode_geom, retry_times):

~/.local/share/virtualenvs/landing-page-QW9sxmZW/lib/python3.7/site-packages/cartoframes/data/dataset/registry/table_dataset.py in __init__(self, data, credentials, schema)
     13 
     14         self._table_name = normalize_name(data)
---> 15         self._schema = schema or self._get_schema()
     16 
     17         if self._table_name != data:

~/.local/share/virtualenvs/landing-page-QW9sxmZW/lib/python3.7/site-packages/cartoframes/data/dataset/registry/base_dataset.py in _get_schema(self)
    192     def _get_schema(self):
    193         if self._credentials:
--> 194             self._schema = self._context.get_schema()
    195             return self._schema
    196         else:

~/.local/share/virtualenvs/landing-page-QW9sxmZW/lib/python3.7/site-packages/cartoframes/lib/context/api_context.py in get_schema(self)
     52         """Get user schema from current credentials"""
     53         query = 'select current_schema()'
---> 54         result = self.execute_query(query, do_post=False)
     55         return result['rows'][0]['current_schema']

~/.local/share/virtualenvs/landing-page-QW9sxmZW/lib/python3.7/site-packages/cartoframes/lib/context/api_context.py in execute_query(self, query, parse_json, do_post, format, **request_args)
     44 
     45     def execute_query(self, query, parse_json=True, do_post=True, format=None, **request_args):
---> 46         return self.sql_client.send(query.strip(), parse_json, do_post, format, **request_args)
     47 
     48     def execute_long_running_query(self, query):

~/.local/share/virtualenvs/landing-page-QW9sxmZW/lib/python3.7/site-packages/carto/sql.py in send(self, sql, parse_json, do_post, format, **request_args)
    112             raise e
    113         except Exception as e:
--> 114             raise CartoException(e)
    115 
    116 

CartoException: encoding with 'idna' codec failed (UnicodeError: label empty or too long)

[andy-esch]

@jgoizueta could this be related to your work on character encodings from the last few days?

[andy-esch]

It turns out that I was using the Credentials functionality incorrectly:

Loading JSONs doesn't work and causes that UnicodeError for some reason.

set_default_credentials(Credentials.from_file('./webinar-credentials.json'))

That works.

I guess this issue is about:

1. A better error about incorrect credentials
2. Supporting taking credentials in set_default_credentials from a file

[Jesus89]

The first point is related to #776.

Is the second point about enabling set_default_credentials('./webinar-credentials.json')?

[Jesus89]

Btw, the bug in the description is related to use './credentials.json' as a username, and that's why it raises the CartoException. I have already implemented set_default_credentials('./filepath.json)` :)

[andy-esch]

Ah ok, looks like we're ok then!

[Jesus89]

Sorry, I mean its implemented in my branch :)

[Jesus89]

Closed via #1175