[incubator-kie-issues-1444-weekly] drools.weekly-deploy jobs frequent…

[tkobayas]

…ly fail with Request Timeout (408) (#6056)

• Disable deployAtEnd

Issue:

• drools.weekly-deploy jobs frequently fail with Request Timeout (408) incubator-kie-issues#1444

See apache/incubator-kie-issues#1444 (comment)

#6056 aether.connector.basic.parallelPut=false seems to have a positive effect, but this PR explores further improvement.

[tkobayas]

When we enable deployAtEnd, retryFailedDeploymentCount retries uploading for all artifacts regardless of the artifact was successfully deployed at the previous attempt or not. It results in the pretty long "retry". (One attempt is around 35 minutes)

By disabling deployAtEnd, uploading/retrying will be done per-artifact basis. So the retrying would give quicker and stable result.

One downside is that if a sub module in the drools repo fails to build, we cannot rollback the already uploaded artifacts. But is it a serious issue? If we fail to upload artifacts in the middle, it would result in the same situation (= only some artifacts are uploaded).

[jstastny-cz]

Can you please provide evidence for your findings? Because from what I've seen the retry was only for artifacts that actually failed.

[jstastny-cz]

@tkobayas the problem is not failed upload in the middle - but we would be uploading artifacts that might turn out later as faulty - resulting in broken snapshots and projects impossible to build.

[tkobayas]

@jstastny-cz Thank you for the comment!

Can you please provide evidence for your findings? Because from what I've seen the retry was only for artifacts that actually failed.

1. I created debug jars of maven-resolver-connector-basic-1.9.18.jar and maven-resolver-transport-http-1.9.18.jar. It throws an Exception when simple-pojo05 jar is about to be uploaded. Placed the debug jars into maven lib directory : https://gist.github.com/tkobayas/d74b103eb98d62b1300e6d905699486a
2. Running Nexus locally
3. I build the project to deploy multiple artifacts : https://github.com/tkobayas/maven-experiment/tree/main/multi-artifacts

mvn -s ./settings.xml -fae -ntp -Dfull clean deploy -DdeployAtEnd -DretryFailedDeploymentCount=3  -Dfull -Dmaven.test.failure.ignore=true -DskipTests=false

Here is the result debug log : https://gist.github.com/tkobayas/12c4379d5eb0fbe17facbc14ea47cee8

[INFO] --- deploy:3.1.1:deploy (default-deploy) @ simple-pojo06 ---
...
[INFO] execute PUT http://localhost:8081/repository/maven-snapshots/org/example/deploy/simple-pojo01/1.0.0-SNAPSHOT/simple-pojo01-1.0.0-20240909.083542-8.jar HTTP/1.1
...
[INFO] execute done PUT http://localhost:8081/repository/maven-snapshots/org/example/deploy/simple-pojo01/1.0.0-SNAPSHOT/simple-pojo01-1.0.0-20240909.083542-8.jar HTTP/1.1
...
[INFO] execute PUT http://localhost:8081/repository/maven-snapshots/org/example/deploy/simple-pojo05/1.0.0-SNAPSHOT/simple-pojo05-1.0.0-20240909.083542-8.jar HTTP/1.1
...
[WARNING] Encountered issue during deployment: Failed to deploy artifacts: Could not transfer artifact org.example.deploy:simple-pojo05:jar:1.0.0-20240909.083542-8 from/to nexus-snapshots (http://localhost:8081/repository/maven-snapshots/): DEBUG
[INFO] Retrying deployment attempt 2 of 3
...
[INFO] execute PUT http://localhost:8081/repository/maven-snapshots/org/example/deploy/simple-pojo01/1.0.0-SNAPSHOT/simple-pojo01-1.0.0-20240909.083542-8.jar HTTP/1.1
...
[INFO] execute done PUT http://localhost:8081/repository/maven-snapshots/org/example/deploy/simple-pojo01/1.0.0-SNAPSHOT/simple-pojo01-1.0.0-20240909.083542-8.jar HTTP/1.1

You can see that simple-pojo01-1.0.0-20240909.083542-8.jar was successfully deployed, but it was deployed again after retrying (caused by simeple-pojo05).

resulting in broken snapshots and projects impossible to build.

Agreed that it's a more important issue than the uploading failure.

Also sorry that I missed that deployAtEnd is an agreed decision. I'm not willing to push this solution. I'm closing this PR.

[jstastny-cz]

Mentioning also here - -DdeployAtEnd was deploy configuration used across KIE projects to unify the method - this would deviate from that goal again, please discuss on mailing list, original thread was https://lists.apache.org/thread/d6oxh6qtm6mm4hc2zv1pwcqqb2kfmv70 .