Retry GitHub download failures

[leahwicz]

resolves #3546

Description

We are seeing dbt deps fail intermittently when trying to reach out to GitHub. We do retry logic for when we reach out to the registry hub so we want to try the same retry logic for GitHub to see if it improves the success rate of dbt deps.

Checklist

• I have signed the CLA
• I have run this code in development and it appears to resolve the stated issue
• This PR includes tests, or tests are not required/relevant for this PR
• I have updated the CHANGELOG.md and added information about my change to the "dbt next" section.

[leahwicz]

Does this even look in the ballpark? My Python is not good 😄

[jtcohen6]

Hooray!! Glad we're reusing + extending the existing decorator. The code looks pretty good to me, but I should not be the arbiter :)

Also interested to hear folks' thoughts about how best to test - perhaps mocking a package with a nonexistent tarball URL? I know test_registry_get_request_exception added some unneeded baggage to our unit testing.

[nathaniel-may]

The python checks out. However, on a personal note, I find this wrapping pattern to be particularly opaque.

Here is an example of how I might suggest refactoring this:

# no juggling of indexes which is extremely error prone
def _exception_retry(lambda_fn, retries):
    if retries <= 0:
        raise RegistryException('Unable to connect to registry hub')
    else: 
        try:
            return lambda_fn()
        except (requests.exceptions.ConnectionError, requests.exceptions.Timeout):
            time.sleep(1)
            _exception_retry(lambda_fn, retries - 1)

# example call:
# accepting a lambda function as a parameter allows the outer 
# function to run it regardless of what parameters it takes
_exception_retry(lambda: 1+2, 5)

(the original exception is lost here, but that can be put back. I didn't write that part because it obfuscated the point I'm trying to make in the review)

In both ways of writing this, I would suggest the following tests:

1. pass a function that always throws these exceptions and check that you get the right exception back
2. pass a function that throws twice then returns correctly and check that you get the correct response