Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

composer: propagate component startup errors #1833

Closed
ethanoroshiba opened this issue Nov 22, 2024 · 1 comment · Fixed by #1838
Closed

composer: propagate component startup errors #1833

ethanoroshiba opened this issue Nov 22, 2024 · 1 comment · Fixed by #1838
Assignees
Labels
code-quality composer pertaining to composer stale

Comments

@ethanoroshiba
Copy link
Contributor

ethanoroshiba commented Nov 22, 2024

Currently, the Composer run_until_stopped() method has two loops which wait on executor and the collectors to be ready before continuing. These checks do a fine job at exiting early if these components fail during startup, but do not propagate the errors which caused the failure. For example, if the chain ID check fails, the following error chain is propagated, not including executor source error chain:

executor failed to become ready
executor failed while waiting for it to become ready
channel closed

We should find a way to propagate these startup errors such that they will end up in the error chain in the composer JoinHandle.

Relevant Code:

// wait for collectors and executor to come online
wait_for_collectors(&geth_collector_statuses, &mut composer_status_sender)
.await
.wrap_err("geth collectors failed to become ready")?;
wait_for_executor(executor_status, &mut composer_status_sender)
.await
.wrap_err("executor failed to become ready")?;

┆Issue Number: ENG-1026

@joroshiba
Copy link
Member

This issue is stale because it has been open 45 days with no activity. Remove stale label or this issue
be closed in 7 days.

github-merge-queue bot pushed a commit that referenced this issue Jan 23, 2025
## Summary
Propagate errors which occur while composer is starting up and/or
running so that the will be returned by the Composer's handle.

## Background
Previously, composer would only exit with an error if the collectors' or
executor's status channels were closed, and then the error message did
not provide detailed information about the error that occurred.
Additionally, if either of the `wait_for_*` loops failed, the composer
would not shut down gracefully. This change is meant to expose the first
eyre report which causes the composer to shut down, and gracefully shut
down in all circumstances.

## Changes
- Started collector and executor `wait_for_ready` loops concurrently,
and continue with graceful shutdown even if these fail.
- Store the first error composer encounters, and return it after
graceful shutdown. If waiting for collectors or executor fails, Composer
continues so that it can ascertain the underlying error from the task
which caused it.

## Testing
Passing all tests

## Changelogs
Changelog updated

## Related Issues
closes #1833
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
code-quality composer pertaining to composer stale
Projects
None yet
Development

Successfully merging a pull request may close this issue.

2 participants