Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[BUG] nightly spark303 deploy pulling spark301 aggregator #4356

Closed
pxLi opened this issue Dec 14, 2021 · 3 comments
Closed

[BUG] nightly spark303 deploy pulling spark301 aggregator #4356

pxLi opened this issue Dec 14, 2021 · 3 comments
Labels
bug Something isn't working P0 Must have for release

Comments

@pxLi
Copy link
Collaborator

pxLi commented Dec 14, 2021

Describe the bug
This cannot be reproduced locally, only in nightly pipeline

mvn -B -Dbuildver=303 deploy -pl '!tools,!dist' -s jenkins/settings.xml -P mirror-apache-to-urm -Dmaven.repo.local=/home/jenkins/agent/workspace/jenkins-rapids_nightly-dev-github-410/.m2 -Dcuda.version=cuda11 -DskipTests

09:52:52  [INFO] ----------------< com.nvidia:rapids-4-spark-tests_2.12 >----------------
09:52:52  [INFO] Building RAPIDS Accelerator for Apache Spark Tests 22.02.0-SNAPSHOT [9/11]
09:52:52  [INFO] --------------------------------[ jar ]---------------------------------
09:52:52  [INFO] Downloading from snapshots: https://urm.nvidia.com/artifactory/sw-spark-maven/com/nvidia/rapids-4-spark-aggregator_2.12/22.02.0-SNAPSHOT/rapids-4-spark-aggregator_2.12-22.02.0-20211211.165958-291-spark301.jar
09:52:52  [INFO] Downloaded from snapshots: https://urm.nvidia.com/artifactory/sw-spark-maven/com/nvidia/rapids-4-spark-aggregator_2.12/22.02.0-SNAPSHOT/rapids-4-spark-aggregator_2.12-22.02.0-20211211.165958-291-spark301.jar (11 MB at 42 MB/s)
@pxLi pxLi added bug Something isn't working ? - Needs Triage Need team to review and classify labels Dec 14, 2021
@pxLi
Copy link
Collaborator Author

pxLi commented Dec 14, 2021

intermittently it could be reproduced locally.
I would guess some commands in maven plugin do work consistently in different ENVs,
if we use nightly image (<urm_url>/sw-spark-docker/plugin:dev-ubuntu18-cuda11.0-blossom-dev), it could always reproduce the issue

@jlowe
Copy link
Contributor

jlowe commented Dec 14, 2021

@tgravescs and I believe this is triggered by the dist jar depending upon the aggregator jar. See #4347

@Salonijain27 Salonijain27 added P0 Must have for release and removed ? - Needs Triage Need team to review and classify labels Dec 14, 2021
@pxLi
Copy link
Collaborator Author

pxLi commented Dec 16, 2021

fixed in #4355

@pxLi pxLi closed this as completed Dec 16, 2021
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working P0 Must have for release
Projects
None yet
Development

No branches or pull requests

3 participants