Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

As a user, I don't want to run into errors when updating the application when offline #669

Open
christophertubbs opened this issue Jun 27, 2024 · 2 comments
Assignees

Comments

@christophertubbs
Copy link
Contributor

This pertains to ./scripts/update_package.sh

My internet went down this morning and I needed to update my codebase so all my changes were in place so I could test.

With everything already available locally (including all remote sources), I ran into issues like:

WARNING: Retrying (Retry(total=1, connect=None, read=None, redirect=None, status=None)) after connection broken by 'NewConnectionError('<pip._vendor.urllib3.connection.HTTPSConnection object at 0x1068942e0>: Failed to establish a new connection: [Errno 8] nodename nor servname provided, or not known')': /simple/dmod-hydrofabric/
WARNING: Retrying (Retry(total=0, connect=None, read=None, redirect=None, status=None)) after connection broken by 'NewConnectionError('<pip._vendor.urllib3.connection.HTTPSConnection object at 0x1068944c0>: Failed to establish a new connection: [Errno 8] nodename nor servname provided, or not known')': /simple/dmod-hydrofabric/
Processing ./python/lib/hydrofabric/dist/dmod_hydrofabric-0.0.1-py3-none-any.whl
WARNING: Retrying (Retry(total=4, connect=None, read=None, redirect=None, status=None)) after connection broken by 'NewConnectionError('<pip._vendor.urllib3.connection.HTTPSConnection object at 0x106882f70>: Failed to establish a new connection: [Errno 8] nodename nor servname provided, or not known')': /simple/dmod-metrics/
WARNING: Retrying (Retry(total=3, connect=None, read=None, redirect=None, status=None)) after connection broken by 'NewConnectionError('<pip._vendor.urllib3.connection.HTTPSConnection object at 0x106882d00>: Failed to establish a new connection: [Errno 8] nodename nor servname provided, or not known')': /simple/dmod-metrics/
WARNING: Retrying (Retry(total=2, connect=None, read=None, redirect=None, status=None)) after connection broken by 'NewConnectionError('<pip._vendor.urllib3.connection.HTTPSConnection object at 0x106882a60>: Failed to establish a new connection: [Errno 8] nodename nor servname provided, or not known')': /simple/dmod-metrics/
WARNING: Retrying (Retry(total=1, connect=None, read=None, redirect=None, status=None)) after connection broken by 'NewConnectionError('<pip._vendor.urllib3.connection.HTTPSConnection object at 0x106882910>: Failed to establish a new connection: [Errno 8] nodename nor servname provided, or not known')': /simple/dmod-metrics/
WARNING: Retrying (Retry(total=0, connect=None, read=None, redirect=None, status=None)) after connection broken by 'NewConnectionError('<pip._vendor.urllib3.connection.HTTPSConnection object at 0x106880b50>: Failed to establish a new connection: [Errno 8] nodename nor servname provided, or not known')': /simple/dmod-metrics/
Processing ./python/lib/metrics/dist/dmod_metrics-0.2.0-py3-none-any.whl
WARNING: Retrying (Retry(total=4, connect=None, read=None, redirect=None, status=None)) after connection broken by 'NewConnectionError('<pip._vendor.urllib3.connection.HTTPSConnection object at 0x106894cd0>: Failed to establish a new connection: [Errno 8] nodename nor servname provided, or not known')': /simple/dmod-modeldata/
WARNING: Retrying (Retry(total=3, connect=None, read=None, redirect=None, status=None)) after connection broken by 'NewConnectionError('<pip._vendor.urllib3.connection.HTTPSConnection object at 0x106894eb0>: Failed to establish a new connection: [Errno 8] nodename nor servname provided, or not known')': /simple/dmod-modeldata/
WARNING: Retrying (Retry(total=2, connect=None, read=None, redirect=None, status=None)) after connection broken by 'NewConnectionError('<pip._vendor.urllib3.connection.HTTPSConnection object at 0x106894fa0>: Failed to establish a new connection: [Errno 8] nodename nor servname provided, or not known')': /simple/dmod-modeldata/
WARNING: Retrying (Retry(total=1, connect=None, read=None, redirect=None, status=None)) after connection broken by 'NewConnectionError('<pip._vendor.urllib3.connection.HTTPSConnection object at 0x1068a21f0>: Failed to establish a new connection: [Errno 8] nodename nor servname provided, or not known')': /simple/dmod-modeldata/
WARNING: Retrying (Retry(total=0, connect=None, read=None, redirect=None, status=None)) after connection broken by 'NewConnectionError('<pip._vendor.urllib3.connection.HTTPSConnection object at 0x1068a23a0>: Failed to establish a new connection: [Errno 8] nodename nor servname provided, or not known')': /simple/dmod-modeldata/
Processing ./python/lib/modeldata/dist/dmod_modeldata-0.12.0-py3-none-any.whl

An internet connection should not be required 100% of the time just to develop.

Either handling needs to be added to avoid having to call pip whenever moving code into the virtual environment or update_script.sh needs to be updated to 'install' in edit mode where packages don't need to be in the virtual environment to begin with.

@robertbartel
Copy link
Contributor

It may be worth discussing whether we should move to editable mode as a general expectation for deployments, development or otherwise. I'm not sure I like it, but it would make certain things simpler.

Assuming we don't expect that for every deployment, I prefer not to use editable mode with DMOD's current multi-package setup. I think it's better to require ourselves to have the packages we use installed the same way for development as they will be for deployments. I see it as worth the small extra overhead, though I'm open to alternative ideas if there's a more convenient way to ensure we have this kind of environmental consistency for testing.

Regardless, I agree we should support at least some development activity without an active Internet connection.
Probably the simplest solution here is to add an option to scripts/update_package.sh that causes pip's --no-deps flag to be used.

This may be another sign that we need to (eventually) not have a bunch of individual packages like this all within the same repository, so we can potentially have that conversation as well (though perhaps in a different issue).

@aaraney
Copy link
Member

aaraney commented Jun 27, 2024

Im fairly certain this is unavoidable because of how we specify / need to specify several dependencies. To my best knowledge, pip will re-pull any dependencies that are specified using a url like syntax (e.g. git+https://github.com/noaa-owp/hypy@master#egg=hypy&subdirectory=python) b.c. pip cannot be certain of the version. Like, @robertbartel mentioned, I think the best you can do is use pip's --no-deps flag in this case. If noaa-owp python repos were uploaded to pypi this would not be an issue.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

3 participants