use socrata connector, auth for ingest/library calls

[fvankrieken]

Meant to close #1311

Seems that some dot datasets went private, meaning we can't simply download them without auth

2 commits just to make changes easier to see (1 just moves some logic, 2 makes needed changes)

passing run here

[codecov]

Codecov Report

Attention: Patch coverage is 66.66667% with 15 lines in your changes missing coverage. Please review.

Project coverage is 70.51%. Comparing base (e2f01fc) to head (c589be0).
Report is 1 commits behind head on main.

Files with missing lines	Patch %	Lines
dcpy/connectors/socrata/publish.py	18.18%	9 Missing ⚠️
dcpy/library/config.py	69.23%	3 Missing and 1 partial ⚠️
dcpy/connectors/socrata/utils.py	85.71%	1 Missing and 1 partial ⚠️

Additional details and impacted files

@@            Coverage Diff             @@
##             main    #1312      +/-   ##
==========================================
+ Coverage   70.41%   70.51%   +0.09%     
==========================================
  Files         114      115       +1     
  Lines        6129     6129              
  Branches      700      700              
==========================================
+ Hits         4316     4322       +6     
+ Misses       1669     1662       -7     
- Partials      144      145       +1     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

[sf-dcp]

can we move this and _socrata_request() to some utils ile?

[fvankrieken]

Yup

[sf-dcp]

LGTM! The only rec is to move helper functions out of the init file

[alexrichey]

I would think we'd want to throw an exception if _simple_auth() is called without these vars present. Any reason not to? (IIRC the Socrata request would fail in that case, even for public resources)

[fvankrieken]

Going with no auth is fine for get requests for things like metadata or pulling datasets

[fvankrieken]

I think it makes sense to try to use auth when user/pass is defined, and let failures occur when a resource isn't allowed for a given user (or None auth).

Arguably then, maybe we only call simple_auth if those are defined. But I thought this was simple - get optional auth credentials if available.

[alexrichey]

gotcha, so you've tested that this works for public resources when there are no creds defined? (for some reason I remember that failing, but that was a long time ago.) Maybe it makes sense to just log a warning?

[fvankrieken]

Yup - just ran locally with no issue

[alexrichey]

cool - had one question, but this looks good