Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Importer(s) are not communicating import done correctly #26046

Closed
Tracked by #26058
ChrisKujawa opened this issue Dec 13, 2024 · 1 comment · Fixed by #26124
Closed
Tracked by #26058

Importer(s) are not communicating import done correctly #26046

ChrisKujawa opened this issue Dec 13, 2024 · 1 comment · Fixed by #26124
Assignees
@EuroLew
Labels
chaos-day Marks an issue or PR as related to an experiment performed during a chaos day component/zeebe Related to the Zeebe component/team kind/bug Categorizes an issue or PR as a bug severity/high Marks a bug as having a noticeable impact on the user with no known workaround version:8.7.0-alpha3

Comments

@ChrisKujawa
Copy link
Member

Describe the bug

When running a benchmark, we observed that the Camunda Exporter failed after restart, because the Importer doesn't seem to be completed (where it should be).

INFO 2024-12-12T21:20:50.478899335Z Waiting for importers to finish, cached record with key 2251799814997670 but did not flush

The importer seems to not complete all documents.

import-completed
@ChrisKujawa ChrisKujawa added kind/bug Categorizes an issue or PR as a bug component/zeebe Related to the Zeebe component/team labels Dec 13, 2024
@ChrisKujawa ChrisKujawa added the severity/high Marks a bug as having a noticeable impact on the user with no known workaround label Dec 13, 2024
@ChrisKujawa ChrisKujawa mentioned this issue Dec 13, 2024
Closed
@ChrisKujawa
Copy link
Member Author

https://zeebe-io.github.io/zeebe-chaos/2024/12/12/News-from-Camunda-Exporter-project

@ChrisKujawa ChrisKujawa added the chaos-day Marks an issue or PR as related to an experiment performed during a chaos day label Dec 13, 2024
@ChrisKujawa ChrisKujawa mentioned this issue Dec 13, 2024
Open
@EuroLew EuroLew mentioned this issue Dec 16, 2024
Merged
1 task
github-merge-queue bot pushed a commit that referenced this issue Dec 16, 2024
@ChrisKujawa
Importers not communicated completion (#26124)
61079ca 
## Description

The importers were not correctly marking their import position documents
as completed to signify the record readers were done. This was due to
missing zeebe indices causing a `NoSuchIndex` exception which means the
path through which record reader completion was checked and set was
missed.

The fix involves treating a no such index exception as an empty batch
with regards to record reader completion check as they signify the same
thing (no records to export).

## Checklist

- [x] Given missing zeebe indices record readers can still get marked as
completed.

## Related issues

closes #26046
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@EuroLew EuroLew

Labels
chaos-day Marks an issue or PR as related to an experiment performed during a chaos day component/zeebe Related to the Zeebe component/team kind/bug Categorizes an issue or PR as a bug severity/high Marks a bug as having a noticeable impact on the user with no known workaround version:8.7.0-alpha3
Projects
None yet
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

Importers not communicated completion camunda/camunda
3 participants
@ChrisKujawa @camundait @EuroLew