nominating warning

[darribas]

From the test suite:

tests/test_ctx.py::test_plot_map

  /home/travis/build/darribas/contextily/miniconda/envs/TEST/lib/python3.5/site-packages/geopy/geocoders/osm.py:138: 

UserWarning: Using Nominatim with the default "geopy/1.16.0" `user_agent` is strongly discouraged, as it violates Nominatim's ToS https://operations.osmfoundation.org/policies/nominatim/ and may possibly cause 403 and 429 HTTP errors. 

Please specify a custom `user_agent` with `Nominatim(user_agent="my-application")` or by overriding the default `user_agent`: `geopy.geocoders.options.default_user_agent = "my-application"`. In geopy 2.0 this will become an exception.

    UserWarning

Maybe for @choldgraf to have a look?

[choldgraf]

whoah - weird, I don't think we're setting anything like this, just using the defaults: https://github.com/darribas/contextily/blob/3fba3a3cc099dc5cc6782c40b7e270b916478d6e/contextily/place.py#L60

[darribas]

Right, I think reading through it they might have changed their approach. Should we encourage (or at least give the option to) the user to pass their own user_agent? I think this should be resolved with a user ID, which I imagine will be free. Can our users currently pass that to the Place call?

[choldgraf]

hmmm - that's a good question, I'd be +1 on letting users do this. Does this mean geopy will expect people to have their own user identity before doing geotagging?!

[darribas]

This one is similar to #75. We should probably extend the policy adopted over there to geopy as well.

[jorisvandenbossche]

Yes, and for the tests we can maybe specify a contextily-testing user agent or something like that.

[jorisvandenbossche]

I read your sentence wrongly. In this case, I am not sure if we should copy the same logic as we did for the tile providers. The terms of services can be much more restricted here, I think (actually, I didn't look at those of the tile providers, but I know for geocoding services they are somewhat limited in rate etc). See some discussion in geopandas as well (geopandas/geopandas#907, geopandas/geopandas#975), where in the end we switched to geocodefarm as the default but with a rate limit performed by geopandas, and a pointer to the docs how to set another geocoding service / user agent.

[darribas]

Could we have a similar approach here? geocodefarm at least for running the tests?

[jorisvandenbossche]

A good start is probably switching to geocodefarm as default. Then the question is if we want the user be able to specify the geocoding service (but in this case, you won't run into rate limits that quickly, since we are also making plots of it, not en masse geocoding a full dataframe as in the case of geopandas. So it is certainly less pressing as in geopandas I think)

[darribas]

Yeah, maybe a switch to geocodefarm would be good. I think, if possible, allowing the user to pass arguments on Place to be directly passed on to geopy would be good.

[APaganini]

Hi, I'd just like to mention that I hit the same issue by running

nightlights = cx.providers.NASAGIBS.ViirsEarthAtNight2012
ireland = cx.Place("Ireland", source=nightlights)

from the documentation.
Instead of a Warning, this now triggers a ConfigurationError:

~/Documents/FIREDRAKE/firedrake/lib/python3.7/site-packages/geopy/geocoders/nominatim.py in __init__(self, timeout, proxies, domain, scheme, user_agent, ssl_context, adapter_factory)
       112                 'overriding the default `user_agent`: '
       113                 '`geopy.geocoders.options.default_user_agent = "my-application"`.'
   --> 114                 % self.headers['User-Agent']
       115             )
       116 

ConfigurationError: Using Nominatim with default or sample `user_agent` "geopy/2.0.0" is strongly discouraged, as it violates Nominatim's ToS https://operations.osmfoundation.org/policies/nominatim/ and may possibly cause 403 and 429 HTTP errors. Please specify a custom `user_agent` with `Nominatim(user_agent="my-application")` or by overriding the default `user_agent`: `geopy.geocoders.options.default_user_agent = "my-application"`.

Does it mean that cx.Place cannot work at present?

[darribas]

It looks like the geocoder is hard-coded:

https://github.com/geopandas/contextily/blob/master/contextily/place.py#L96

We should definitely let the user pick the geocoder (for example, set an argument like geocoder=gp.geocoders.Nominatim().geocode by default) so there is flexibility on this. PRs very welcome on this ;-)

However, for the problem @APaganini raises, note that, following the error's message, you can change the user before calling contextily:

import geopy
geopy.geocoders.options.default_user_agent = "test_user"
nightlights = cx.providers.NASAGIBS.ViirsEarthAtNight2012
ireland = cx.Place("Ireland", source=nightlights)

That worked for me.

Cleverer suggestions in the form of PRs also very welcome!

[darribas]

Fixed in #164