Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Feature: Force stop. Pipeline status should be stopped, not degraded #1879

Closed
raulb opened this issue Oct 4, 2024 · 2 comments
Closed

Feature: Force stop. Pipeline status should be stopped, not degraded #1879

raulb opened this issue Oct 4, 2024 · 2 comments
Labels
feature New feature or request triage Needs to be triaged

Comments

@raulb
Copy link
Member

raulb commented Oct 4, 2024

Feature description

When a user tries to force stop a pipeline, the result is that the status of the pipeline ends up being degraded and not stopped.

I think it would make sense that it's stopped instead.

Steps to reproduce

  1. Run a pipeline
  2. Force stop it: curl -X POST localhost:8080/v1/pipelines/my-pipeline/stop -d '{"force": true}'
  3. Fetch status pipeline: curl 'http://localhost:8080/v1/pipelines/my-pipeline' | jq .state.status

Expected

STATUS_STOPPED

Actual

STATUS_DEGRADED

@raulb raulb added feature New feature or request triage Needs to be triaged labels Oct 4, 2024
@github-project-automation github-project-automation bot moved this to Triage in Conduit Main Oct 4, 2024
@lovromazgon
Copy link
Member

This is what the statuses mean right now:

  • STATUS_STOPPED - the pipeline gracefully stopped, any in-flight records were flushed, acks were delivered back to the source, no error was encountered while stopping the pipeline.
  • STATUS_DEGRADED - an error caused the pipeline to stop. Any in-flight records were dropped, not all acks might have been delivered back to the source. Starting a degraded pipeline again could potentially result in duplicated data (depending on how the destination connector handles records), since some data might get re-delivered.

So given that force stopping a pipeline results in the second behavior (i.e. nodes stop immediately without flushing in-flight records) I'm not sure it would be correct to set the pipeline to "stopped".

@raulb
Copy link
Member Author

raulb commented Oct 21, 2024

@lovromazgon Good point. I think it's important to make this distinction between gracefully stopped vs not. This is being documented on ConduitIO/conduit-site#168.

@raulb raulb closed this as completed Oct 21, 2024
@github-project-automation github-project-automation bot moved this from Triage to Done in Conduit Main Oct 21, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
feature New feature or request triage Needs to be triaged
Projects
Status: Done
Development

No branches or pull requests

2 participants