-
Notifications
You must be signed in to change notification settings - Fork 261
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Ansible-runner: list artifacts one by one #237
Conversation
so IIUC this means that we might have missed an artifact before and now we'll get all artifacts in order - how does it solve the problem that we get the same output more than once? |
we didn't miss it, we just processed it out of order and sometimes twice
now we won't process events twice so we won't have same output twice |
/ost |
would you be so kind to explain how we may end up in this state (of processing something twice)? is it because the lastEventId was not set to the latest one in the sorted list of events? I have quite a few comments on the code itself, but it would help if I'll better understand the root cause which is the more important part here first |
When running an async task in ansible runner it generates the log after it is finished. The code before had a counter of already processed files and the counter was used as an ID prefix of the log files. So if we would have a playbook that would create normal 3 logs and the async task would be in the middle the logs would look like
If you comments to the patch itself please do comment anything, if you mean the ansible runner in engine, please checkout my refactoring patch #261 |
ah ok, I thought that in this case you do lastEventId = 3 and therefore miss '2'
I see that you did one of the things I wanted to suggest there, but not everything - some relates to this patch |
...es/bll/src/main/java/org/ovirt/engine/core/common/utils/ansible/AnsibleRunnerHttpClient.java
Show resolved
Hide resolved
...es/bll/src/main/java/org/ovirt/engine/core/common/utils/ansible/AnsibleRunnerHttpClient.java
Outdated
Show resolved
Hide resolved
...es/bll/src/main/java/org/ovirt/engine/core/common/utils/ansible/AnsibleRunnerHttpClient.java
Outdated
Show resolved
Hide resolved
AnsibleRunnerHttpClient was listing all new artifacts in artifacts dir becuase ansible-runner does not create the artifacts one by one. When we are logging we need to wait for the proper artifacts to be generated and then we can log it.
Signed-off-by: Arik Hadas <ahadas@redhat.com>
Signed-off-by: Arik Hadas <ahadas@redhat.com>
Signed-off-by: Arik Hadas <ahadas@redhat.com>
/ost |
The artifacts created by ansible-runner have the id prefix, which we used to gather the events.
We were listing new artifacts by counting the number of previously processed artifacts and checking the new artifacts from that number as a prefix id. This approach causes multiple issues with async execution where the artifacts are not created in order but there could be a bit of time delay between them. For example, artifact 4 was created before 3.
The new approach fixes this issue by waiting for each file with an id prefix until the playbook finishes or times out.