[JENKINS-51183] - Introduce Integration testing flow with BOM #20

oleg-nenashev · 2018-05-07T17:28:46Z

Reference implementation for jenkins-infra/pipeline-library#57 . It does not do anything exactly useful outside AWS environment, and the artifact-manager-s3.enabled system property should be set for it.

On Jenkins Infra (which is on Azure) it will just run some bits with an Existing Artifact manager and new APIs. So it will smoke-tests plugins for the case when an ArtifactManager is present but not enabled. Maybe useful, but not that much.

But it is a good testing playground for @jglick's Incrementals auto-update logic (see TODO in BOM)

https://issues.jenkins-ci.org/browse/JENKINS-51183

@reviewbybees @jglick @carlossg

jglick · 2018-05-07T17:33:45Z

Jenkinsfile

@@ -1,6 +1,9 @@
 if (infra.isRunningOnJenkinsInfra()) {
    // ci.jenkins.io
    buildPlugin(platforms: ['linux'])
+
+    // Integration tests
+    essentialsTest(baseDir: "src/test/it")


I guess this baseDir could just be the default?

Not sure. In some cases essentials.yml is located in the root (e.g. Core or Git Plugin). CC @raul-arabaolaza . Currently I use root dir as a default

I also use root dir as the default (and is the default location used by pipeline-library) but there is no strong technical reason for that, note that in any case any path can be used as is a param of the step itself

jglick · 2018-05-07T17:34:51Z

src/test/it/bom.yml

+    # Needed to make PCT happy
+    groupId: org.jenkins-ci.main
+    artifactId: jenkins-war
+    version: 2.118-artifact-manager-s3-SNAPSHOT


Why do we need to specify a version, anyway? Could the packager not just pick something arbitrary (999-SNAPSHOT), or generate a version based on the spec.core.version?

Current BOM specification seems to require this field. jenkins-infra/pipeline-library#57 actually runs without it, but it will set 1.0-SNAPSHOT as a default now. Makes sense to switch to 999-SNAPSHOT as you propose

jglick · 2018-05-07T17:35:36Z

src/test/it/bom.yml

+  plugins:
+    - groupId: "org.jenkins-ci.plugins.workflow"
+      artifactId: "workflow-api"
+      version:  2.28-rc333.0675e9e9cb4c


jenkinsci/jep#92 (comment) FTR:

branch: jglick:VirtualFile-JENKINS-49635

would be useful here in addition to the version.

Only for cosmetic reasons, right? When version is set, Custom WAR Packager will ignore other fields anyway

jglick · 2018-05-07T17:46:19Z

src/test/it/bom.yml

+  environments:
+    - name: aws
+      plugins:
+        #TODO: BOM does not support FileSystem REFs, so somebody needs to add BOM auto-update logic here


Uh, no, updating the BOM would not make sense here, because presumably you wish for the integration tests to be run as part of this plugin’s PR builds. So to verify some change in the plugin against these other tests, you would need to commit the change, then update the version here and commit again, before filing the PR! Does not make sense.

We could allow ${changelist} to be substituted so you would have

version: 1.0-alpha-1${changelist}

but this would need to be updated after every MRP usage, which is still awkward. Filesystem path references could help, but it needs some exploration what they would look like. One option is

versionFrom: ../../../.flattened-pom.xml

assuming the local process-resources phase has been run. Alternately,

versionFrom: ../../..

where the version is inferred from the output of

mvn -Dset.changelist -Dexpression=project.version -Doutput=version help:evaluate version=`cat version` rm version echo $version

Yes, whatever works. I could easily do it in the custom WAR packager spec, but BOM would require a whatever standardized definition in jenkinsci/jep#92

jglick · 2018-05-07T17:48:52Z

src/test/it/bom.yml

+    version: 2.118
+  plugins:
+    - groupId: "org.jenkins-ci.plugins.workflow"
+      artifactId: "workflow-api"


So this is duplicating information: the pom.xml already specifies a dependency on workflow-api, and specifies the version. We should not have that twice in the same repo.

I think this is a fundamental flaw in the idea of using a BOM in individual repos for integration tests (as opposed to the overall jenkins-infra/evergreen/configuration/essentials.yaml): there is no clear proposal for how to reconcile duplicated configuration. In jenkinsci/jep#92 (comment) @carlossg claims that bom.yml is the “source of truth” and that pom.xml would be updated by “the tooling” but I do not see how this is going to work. Is pom.xml not going to be versioned any longer? That seems impractical. So, you would need to change version numbers in both places and commit them at once? What is enforcing that they stay aligned?

To me it seems more comfortable for the POM to specify details of artifacts and their versions, and for the BOM to express additional information, if any—perhaps just delete the bom.yml altogether, since the essentials.yml here seems to have all the interesting configuration. For purposes of the custom WAR packager, you can just bundle up the jenkins.war and Jenkins plugins in Maven’s test scope. This trick works fine.

And that would also bypass the “filesystem ref” problem mentioned below.

I could create a "POM to BOM" tool which dumps a YAML with all HPI dependencies in the plugin and adds them to bom.yml (would make sense to rename it to bom-template.yml then).

would make sense to rename it to bom-template.yml then

My point is that there would be nothing in such a template! essentials.yml would have all the information needed for running tests.

Of course we can do it if needed, in the most of the cases

…51183

oleg-nenashev · 2018-05-21T21:32:02Z

The change requires fixes in runATH() (https://issues.jenkins-ci.org/browse/JENKINS-51306). CC @raul-arabaolaza

raul-arabaolaza · 2018-05-24T08:09:03Z

@oleg-nenashev jenkins-infra/pipeline-library#65 should unblock the ATH run here

…51183

oleg-nenashev · 2018-05-24T23:08:14Z

src/test/it/bom.yml

+      version: 2.20
+    - groupId: "org.jenkins-ci.plugins.workflow"
+      artifactId: "workflow-basic-steps"
+      version: 2.8-rc351.c6608322f479


can be removed, because it comes from dependencies (need to solve upper bounds properly tho) => APPEND pom.xml deps here unless overwritten?

All of the plugins would better have their version be inherited from pom.xml. For that matter the whole section could be deleted I think and Maven test-scoped classpath used (with copyartifact added if this is in fact in active use).

There is a task for that: https://issues.jenkins-ci.org/browse/JENKINS-51574
But cannot commit on any ETA

oleg-nenashev · 2018-05-24T23:11:58Z

Caused by: org.eclipse.aether.transfer.ArtifactNotFoundException: Could not find artifact io.jenkins.tools.custom-war-packager:custom-war-packager-cli:jar:0.1-alpha-6-20180524.212000-3 in azure (https://repo.azure.jenkins.io/public/)

Hmm... It's either glitch or something is messed up in https://github.com/jenkins-infra/pipeline-library/blob/master/resources/settings-azure.xml . @raul-arabaolaza Should not it explicitly allow snapshots

jglick · 2018-05-25T13:45:19Z

src/test/it/Makefile

+.PHONY: all
+
+ARTIFACT_ID = jenkins-war
+VERSION = 2.121-artifact-manager-s3-SNAPSHOT


Best to avoid encoding the current core baseline version.

jglick · 2018-05-25T13:46:03Z

src/test/it/Makefile

+		-e ARTIFACT_ID=artifact-manager-s3 \
+		-e INSTALL_BUNDLED_SNAPSHOTS=true \
+		jenkins/pct \
+		-mavenProperties jenkins-test-harness.version=2.38:jth.jenkins-war.path=/pct/jenkins.war


Should not be overriding JTH version.

https://issues.jenkins-ci.org/browse/JENKINS-51575
Currently it's needed to run PCT properly against the WAR produced by CWP.

jglick · 2018-05-25T13:47:46Z

src/test/it/bom.yml

+      version: 2.20
+    - groupId: "org.jenkins-ci.plugins.workflow"
+      artifactId: "workflow-basic-steps"
+      version: 2.8-rc351.c6608322f479


All of the plugins would better have their version be inherited from pom.xml. For that matter the whole section could be deleted I think and Maven test-scoped classpath used (with copyartifact added if this is in fact in active use).

jglick · 2018-05-25T13:47:53Z

src/test/it/bom.yml

+      version:  2.28-rc337.8abe7c5204d9
+    - groupId: "org.jenkins-ci.plugins"
+      artifactId: "copyartifact"
+      version: 1.40-rc298.60da56cfabea


jglick · 2018-05-25T13:47:59Z

src/test/it/bom.yml

+      version: 1.14
+    - groupId: "org.jenkins-ci.plugins.workflow"
+      artifactId: "workflow-step-api"
+      version: 2.15-rc308.607bdefd8c6a


jglick · 2018-05-25T13:48:30Z

src/test/it/bom.yml

+      version: 2.8-rc351.c6608322f479
+    - groupId: "org.jenkins-ci.plugins.workflow"
+      artifactId: "workflow-support"
+      version: 2.18


Currently have an incremental version of this one actually.

jglick · 2018-05-25T13:48:49Z

src/test/it/essentials.yml

+pct:
+  useLocalSnapshots: false
+  jth:
+    version: 2.38


jglick · 2018-05-25T13:49:21Z

src/test/it/essentials.yml

+          dir: initScripts
+ath:
+  # TODO: ATH test dependency issues
+  disabled: true


Why does this attribute need to exist? Why not just comment out the whole ath section?

Yes, you can do that if you want

jglick · 2019-03-27T16:10:42Z

I believe this is obsolete.

[JENKINS-51183] - Introduce Integration testing flow with BOM

f5d5968

oleg-nenashev requested review from jglick and carlossg May 7, 2018 17:29

oleg-nenashev mentioned this pull request May 7, 2018

[INFRA-1617, INFRA-1595] - Draft implementation of a library method for integration testing jenkins-infra/pipeline-library#57

Merged

jglick reviewed May 7, 2018

View reviewed changes

jglick mentioned this pull request May 7, 2018

Bill of Materials jenkinsci/jep#92

Merged

oleg-nenashev added 4 commits May 11, 2018 11:44

[JENKINS-51183] - Retrigger CI

c9eee6c

[JENKINS-51183] - Fix the Init Scripts location

76c5e0c

Merge commit 'c9eee6c8f7c3b676f30dee9185d25a77753e915c' into JENKINS-…

11bdec7

…51183

[JENKINS-51183] - Enable ATH and PCT

410f446

jglick mentioned this pull request May 21, 2018

Fixing JCloudsApiExtensionPoint.getProvider #26

Merged

oleg-nenashev added the work-in-progress label May 21, 2018

oleg-nenashev added 3 commits May 24, 2018 08:22

Merge branch 'master' into JENKINS-51183

42771f3

Archive artifacts, disable ATH

3dc7ccc

[JENKINS-51183] - reenable the flag

fc7af4b

oleg-nenashev mentioned this pull request May 24, 2018

[JENKINS-51183] - Support Directory references in BOM jenkinsci/custom-war-packager#29

Merged

oleg-nenashev added 5 commits May 24, 2018 23:25

[JENKINS-51183] - Use relative path in the BOM

a8a9bdc

Merge commit 'cb859635d11ca7c3a2822ee6608588bb3e9b1159' into JENKINS-…

7efa8fd

…51183

Minor diagnostics fix for missing provider

3159cd4

[JENKINS-51183] - Update all plugins to the recent versions + fix groovy

772444c

[JENKINS-51183] - Add a Makefile for manual build testing

7356475

oleg-nenashev commented May 24, 2018

View reviewed changes

[JENKINS-51183] - Use the released CWP version

23084cf

jglick reviewed May 25, 2018

View reviewed changes

kuisathaverat mentioned this pull request Jun 19, 2018

Custom war test deploy #60

Closed

oleg-nenashev mentioned this pull request Mar 3, 2019

BOM example jenkinsci/custom-war-packager#73

Open

jglick closed this Mar 27, 2019

ghost mentioned this pull request May 7, 2019

Neither Jenkins core nor Jenkins war have been defined by configuration file or BOM/POM jenkinsci/custom-war-packager#94

Closed

jglick mentioned this pull request Nov 16, 2022

Mark JEP-309 as withdrawn jenkinsci/jep#395

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[JENKINS-51183] - Introduce Integration testing flow with BOM #20

[JENKINS-51183] - Introduce Integration testing flow with BOM #20

oleg-nenashev commented May 7, 2018 •

edited

Loading

jglick May 7, 2018

oleg-nenashev May 7, 2018

raul-arabaolaza May 8, 2018

jglick May 7, 2018

oleg-nenashev May 7, 2018

jglick May 7, 2018

oleg-nenashev May 14, 2018

jglick May 7, 2018

oleg-nenashev May 7, 2018

jglick May 7, 2018

oleg-nenashev May 7, 2018 •

edited

Loading

jglick May 7, 2018

oleg-nenashev May 7, 2018

oleg-nenashev commented May 21, 2018

raul-arabaolaza commented May 24, 2018

oleg-nenashev May 24, 2018

jglick May 25, 2018

oleg-nenashev May 29, 2018

oleg-nenashev commented May 24, 2018

jglick May 25, 2018

jglick May 25, 2018

oleg-nenashev May 29, 2018

jglick May 25, 2018

jglick May 25, 2018

jglick May 25, 2018

jglick May 25, 2018

jglick May 25, 2018

jglick May 25, 2018

oleg-nenashev May 29, 2018

jglick commented Mar 27, 2019

[JENKINS-51183] - Introduce Integration testing flow with BOM #20

[JENKINS-51183] - Introduce Integration testing flow with BOM #20

Conversation

oleg-nenashev commented May 7, 2018 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

oleg-nenashev May 7, 2018 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

oleg-nenashev commented May 21, 2018

raul-arabaolaza commented May 24, 2018

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

oleg-nenashev commented May 24, 2018

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

jglick commented Mar 27, 2019

oleg-nenashev commented May 7, 2018 •

edited

Loading

oleg-nenashev May 7, 2018 •

edited

Loading