Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

remote build: do not ignore 'Cancel build? negative response #4929

Closed
popey opened this issue Jul 19, 2024 · 4 comments · Fixed by #5081
Closed

remote build: do not ignore 'Cancel build? negative response #4929

popey opened this issue Jul 19, 2024 · 4 comments · Fixed by #5081
Labels
bug Actual bad behavior that don't fall into maintenance or documentation triaged

Comments

@popey
Copy link
Contributor

popey commented Jul 19, 2024

Bug Description

snapcraft remote-build supports starting a remote-build from a client, killing snapcraft, then recovering/continuing the build from a different place.

In the past pressing ctrl-c on a remote build would just drop to the prompt. Now, you get asked to 'cleanup' with y/n options. Choose 'no' and it does the cleanup anyway, and terminates the remote build. This seems counterintuitive.

To Reproduce

  1. Create a snap/snapcraft.yaml
  2. Run  snapcraft remote-build --launchpad-accept-public-upload
  3. Wait for the process starts building in launchpad (once it says "Stopped: amd64 | (12.2s)" or similar)
  4. Press `CTRL+C'
  5. At the Cancel builds? [Y/n]: prompt, say 'n'

Snapcraft will say 'Cleaning up' and while doing so, the build will be killed in launchpad.

Environment

 $ snapcraft --version
snapcraft 8.3.1
 $ lsb_release -a
No LSB modules are available.
Distributor ID: Ubuntu
Description:    Ubuntu 24.04 LTS
Release:        24.04
Codename:       noble

snapcraft.yaml

name: emoj
summary: emoj
version: latest
description: |
  Find relevant emoji from text on the command-line.
base: core24
grade: stable
confinement: strict

apps:
  emoj:
    command: bin/npm run emoj
    plugs: [network, x11]

parts:
  emoj:
    plugin: npm
    source: https://github.com/sindresorhus/emoj.git
    source-type: git
    npm-include-node: true
    npm-node-version: "22.5.1"
    stage-packages:
      - xsel


### Relevant log output

```shell
Cancel builds? [Y/n] n
Cleaning up

Additional context

We discussed this on 2024-07-19 at the snapcraft clinic.
@popey
Copy link
Contributor Author

popey commented Jul 19, 2024

BONUS almost certainly a related bug.

If you answer 'Y' to the Cancel build? [Y/n], it also fails! :D

  ~/Source/anchore/grype-snap/ > snapcraft remote-build --launchpad-accept-public-upload
remote-build is experimental and is subject to change. Use with caution.                                                                        
Stopped: amd64,arm64,armhf,ppc64el,s390x / (3232.4s)                                                                                            Stopped: amd64,arm64,armhf,ppc64el,s390x / (5571.9s)                                                                                           
Cancel builds? [Y/n]: y
snapcraft internal error: HTTPError()

Here's the log.

2024-07-19 16:47:11.589 Emitter: Resuming control of the terminal
2024-07-19 16:47:11.589 Cancelling builds.
2024-07-19 16:47:11.591 snapcraft internal error: HTTPError()
2024-07-19 16:47:11.607 Traceback (most recent call last):
2024-07-19 16:47:11.607   File "/snap/snapcraft/12136/lib/python3.10/site-packages/snapcraft/commands/remote.py", line 234, in _run
2024-07-19 16:47:11.607     returncode = self._monitor_and_complete(build_id, builds)
2024-07-19 16:47:11.607   File "/snap/snapcraft/12136/lib/python3.10/site-packages/snapcraft/commands/remote.py", line 254, in _monitor_and_complete
2024-07-19 16:47:11.607     for states in builder.monitor_builds():
2024-07-19 16:47:11.607   File "/snap/snapcraft/12136/lib/python3.10/site-packages/craft_application/services/remotebuild.py", line 165, in monitor_builds
2024-07-19 16:47:11.607     states = self._get_build_states()
2024-07-19 16:47:11.607   File "/snap/snapcraft/12136/lib/python3.10/site-packages/craft_application/services/remotebuild.py", line 368, in _get_build_states
2024-07-19 16:47:11.607     self._refresh_builds()
2024-07-19 16:47:11.607   File "/snap/snapcraft/12136/lib/python3.10/site-packages/craft_application/services/remotebuild.py", line 374, in _refresh_builds
2024-07-19 16:47:11.607     build.lp_refresh()
2024-07-19 16:47:11.607   File "/snap/snapcraft/12136/lib/python3.10/site-packages/craft_application/launchpad/models/base.py", line 184, in lp_refresh
2024-07-19 16:47:11.607     self._obj.lp_refresh()
2024-07-19 16:47:11.607   File "/snap/snapcraft/12136/lib/python3.10/site-packages/lazr/restfulclient/resource.py", line 811, in lp_refresh
2024-07-19 16:47:11.607     super(Entry, self).lp_refresh(new_url, etag)
2024-07-19 16:47:11.607   File "/snap/snapcraft/12136/lib/python3.10/site-packages/lazr/restfulclient/resource.py", line 335, in lp_refresh
2024-07-19 16:47:11.607     representation = self._root._browser.get(
2024-07-19 16:47:11.607   File "/snap/snapcraft/12136/lib/python3.10/site-packages/lazr/restfulclient/_browser.py", line 494, in get
2024-07-19 16:47:11.607     response, content = self._request(url, extra_headers=headers)
2024-07-19 16:47:11.607   File "/snap/snapcraft/12136/lib/python3.10/site-packages/lazr/restfulclient/_browser.py", line 441, in _request
2024-07-19 16:47:11.607     response, content = self._request_and_retry(
2024-07-19 16:47:11.607   File "/snap/snapcraft/12136/lib/python3.10/site-packages/lazr/restfulclient/_browser.py", line 400, in _request_and_retry
2024-07-19 16:47:11.607     response, content = self._connection.request(
2024-07-19 16:47:11.607   File "/snap/snapcraft/12136/lib/python3.10/site-packages/httplib2/__init__.py", line 1692, in request
2024-07-19 16:47:11.607     (response, new_content) = self._request(
2024-07-19 16:47:11.607   File "/snap/snapcraft/12136/lib/python3.10/site-packages/launchpadlib/launchpad.py", line 144, in _request
2024-07-19 16:47:11.607     response, content = super(LaunchpadOAuthAwareHttp, self)._request(
2024-07-19 16:47:11.607   File "/snap/snapcraft/12136/lib/python3.10/site-packages/lazr/restfulclient/_browser.py", line 204, in _request
2024-07-19 16:47:11.607     return super(RestfulHttp, self)._request(
2024-07-19 16:47:11.607   File "/snap/snapcraft/12136/lib/python3.10/site-packages/httplib2/__init__.py", line 1444, in _request
2024-07-19 16:47:11.607     (response, content) = self._conn_request(conn, request_uri, method, body, headers)
2024-07-19 16:47:11.607   File "/snap/snapcraft/12136/lib/python3.10/site-packages/httplib2/__init__.py", line 1396, in _conn_request
2024-07-19 16:47:11.607     response = conn.getresponse()
2024-07-19 16:47:11.607   File "/snap/snapcraft/12136/usr/lib/python3.10/http/client.py", line 1375, in getresponse
2024-07-19 16:47:11.607     response.begin()
2024-07-19 16:47:11.607   File "/snap/snapcraft/12136/usr/lib/python3.10/http/client.py", line 318, in begin
2024-07-19 16:47:11.607     version, status, reason = self._read_status()
2024-07-19 16:47:11.607   File "/snap/snapcraft/12136/usr/lib/python3.10/http/client.py", line 279, in _read_status
2024-07-19 16:47:11.607     line = str(self.fp.readline(_MAXLINE + 1), "iso-8859-1")
2024-07-19 16:47:11.607   File "/snap/snapcraft/12136/usr/lib/python3.10/socket.py", line 705, in readinto
2024-07-19 16:47:11.607     return self._sock.recv_into(b)
2024-07-19 16:47:11.607   File "/snap/snapcraft/12136/usr/lib/python3.10/ssl.py", line 1303, in recv_into
2024-07-19 16:47:11.607     return self.read(nbytes, buffer)
2024-07-19 16:47:11.607   File "/snap/snapcraft/12136/usr/lib/python3.10/ssl.py", line 1159, in read
2024-07-19 16:47:11.607     return self._sslobj.read(len, buffer)
2024-07-19 16:47:11.607 KeyboardInterrupt
2024-07-19 16:47:11.607 
2024-07-19 16:47:11.607 During handling of the above exception, another exception occurred:
2024-07-19 16:47:11.607 Traceback (most recent call last):
2024-07-19 16:47:11.607   File "/snap/snapcraft/12136/lib/python3.10/site-packages/craft_application/application.py", line 530, in run
2024-07-19 16:47:11.607     return_code = dispatcher.run() or 0
2024-07-19 16:47:11.607   File "/snap/snapcraft/12136/lib/python3.10/site-packages/craft_cli/dispatcher.py", line 487, in run
2024-07-19 16:47:11.607     return self._loaded_command.run(self._parsed_command_args)
2024-07-19 16:47:11.607   File "/snap/snapcraft/12136/lib/python3.10/site-packages/craft_application/commands/base.py", line 196, in run
2024-07-19 16:47:11.607     result = self._run(parsed_args, **kwargs) or result
2024-07-19 16:47:11.607   File "/snap/snapcraft/12136/lib/python3.10/site-packages/snapcraft/commands/remote.py", line 238, in _run
2024-07-19 16:47:11.607     builder.cancel_builds()
2024-07-19 16:47:11.607   File "/snap/snapcraft/12136/lib/python3.10/site-packages/craft_application/services/remotebuild.py", line 219, in cancel_builds
2024-07-19 16:47:11.607     build.cancel()
2024-07-19 16:47:11.607   File "/snap/snapcraft/12136/lib/python3.10/site-packages/craft_application/launchpad/models/build.py", line 127, in cancel
2024-07-19 16:47:11.607     self._obj.cancel()
2024-07-19 16:47:11.607   File "/snap/snapcraft/12136/lib/python3.10/site-packages/lazr/restfulclient/resource.py", line 642, in __call__
2024-07-19 16:47:11.607     response, content = self.root._browser._request(
2024-07-19 16:47:11.607   File "/snap/snapcraft/12136/lib/python3.10/site-packages/lazr/restfulclient/_browser.py", line 465, in _request
2024-07-19 16:47:11.607     raise HTTPError(response, content)
2024-07-19 16:47:11.607 lazr.restfulclient.errors.HTTPError: HTTP Error 304: Not Modified
2024-07-19 16:47:11.607 Response headers:
2024-07-19 16:47:11.607 ---
2024-07-19 16:47:11.607 date: Fri, 19 Jul 2024 15:47:10 GMT
2024-07-19 16:47:11.607 etag: "9d289065e4626431521fd8b5197bea6fe86c3001-38332cb4d8dcd449cf45c3aa7d48937a74e75f5d"
2024-07-19 16:47:11.607 server: gunicorn
2024-07-19 16:47:11.607 status: 304
2024-07-19 16:47:11.607 vary: Accept
2024-07-19 16:47:11.607 ---
2024-07-19 16:47:11.607 Response body:
2024-07-19 16:47:11.607 ---
2024-07-19 16:47:11.607 b''
2024-07-19 16:47:11.607 ---
2024-07-19 16:47:11.608 Full execution log: '/home/alan/.local/state/snapcraft/log/snapcraft-20240719-150929.404271.log'

@mr-cal
Copy link
Collaborator

mr-cal commented Jul 22, 2024

I can confirm this is a bug with this code block:

snapcraft/snapcraft/commands/remote.py

Lines 233 to 245 in 3325af0

try:
returncode = self._monitor_and_complete(build_id, builds)
except KeyboardInterrupt:
if confirm_with_user("Cancel builds?", default=True):
emit.progress("Cancelling builds.")
builder.cancel_builds()
returncode = 0
except Exception:
returncode = 1 # General error on any other exception
if returncode != 75: # TimeoutError
emit.progress("Cleaning up")
builder.cleanup()
return returncode

In addition to fixing this bug, using exit code constants from the os module would make that code more readable.

@mr-cal mr-cal added bug Actual bad behavior that don't fall into maintenance or documentation triaged labels Jul 22, 2024
@syncronize-issues-to-jira Syncronize issues to Jira
Copy link

Thank you for reporting us your feedback!

The internal ticket has been created: https://warthogs.atlassian.net/browse/CRAFT-3140.

This message was autogenerated

@mr-cal mr-cal changed the title bug: snapcraft remote-build ignores 'Cancel build? negative response remote build: do not ignore 'Cancel build? negative response Aug 21, 2024
@mr-cal mr-cal mentioned this issue Sep 30, 2024
Merged
4 tasks
mr-cal added a commit that referenced this issue Oct 2, 2024
@mr-cal
fix(remotebuild): do not auto clean interrupted builds (#5081)
caa0757 
Fixes a bug where the remote builder would ignore the user and always
clean the launchpad project.

Also drops some magic values in favor of constants.

Fixes #4929

Signed-off-by: Callahan Kovacs <callahan.kovacs@canonical.com>
@mr-cal
Copy link
Collaborator

mr-cal commented Oct 2, 2024

Fixed via #5081

@mr-cal mr-cal closed this as completed Oct 2, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
bug Actual bad behavior that don't fall into maintenance or documentation triaged
Projects
None yet
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

fix(remotebuild): do not auto clean interrupted builds canonical/snapcraft
2 participants
@popey @mr-cal