Implement Pull Request CI builds #590

felicio · 2024-10-03T10:38:42Z

what

Build pull request only for https://github.com/status-im/status-web/tree/main/apps/connector directory

why

Allow other teams to test desktop integration independently

how (for example)

https://github.com/status-im/status-browser-extension/issues/20

jakubgs · 2024-10-03T10:40:06Z

We already had builds for the desktop extension, but my understanding is that it was merged into this repo:
https://ci.status.im/job/status-browser-extension/

felicio · 2024-10-03T10:42:13Z

We already had builds for the desktop extension, but my understanding is that it was merged into this repo: https://ci.status.im/job/status-browser-extension/

Correct.

markoburcul · 2024-10-10T06:55:59Z

I've created the multibranch pipeline on Jenkins under status-web/prs/linux: https://ci.status.im/job/status-web/job/prs/job/linux/ ,
if it is possible could someone from your team test it @felicio ?

felicio · 2024-10-10T07:43:31Z

@markoburcul, tried creating #594 PR to see if the status-im-auto's comment with builds would show up for changes in https://github.com/status-im/status-web/tree/main/apps/connector directory. So far nothing.

Since nix shell function from jenkins librarz uses WORKSPACE env variable to find shell.nix we need to override it for steps using nix shell. For all of the steps I'm using dir directive to change cwd to the apps/connector. Referenced issue: #590 Signed-off-by: markoburcul <marko@status.im>

Use newest jenkins lib tag which adds the entryPoint as an argument to the nix shell function. Referenced issue: #590 Signed-off-by: markoburcul <marko@status.im>

* jenkinsfile: fix paths for all steps Use newest jenkins lib tag which adds the entryPoint as an argument to the nix shell function. Referenced issue: #590 Signed-off-by: markoburcul <marko@status.im>

markoburcul · 2024-10-10T15:50:50Z

The pipeline is successfully up and running. I've enabled status-web repo on ghcmgr and I did some modifications on jenkins-lib in order to make nix shell steps look cleaner.

felicio · 2024-10-10T15:58:41Z

@markoburcul why didn't this #600 PR get built?

markoburcul · 2024-10-10T16:27:19Z

@markoburcul why didn't this #600 PR get built?

I've forgot to add webhook for this repo. It is now added and working fine, feel free to test it!

felicio · 2024-10-10T19:53:06Z

@markoburcul thank you very much. One more thing, those builds should really just be for that directory and not for #602 (comment), for example.

markoburcul · 2024-10-11T10:59:43Z

@markoburcul thank you very much. One more thing, those builds should really just be for that directory and not for #602 (comment), for example.

Jenkins by default doesn't support filtering of build triggers based on path in the repository where change occured. I've tried on the stage level but it won't work first time the PR is opened which is not acceptable so that option is not possible.

Why do you need comments from status-im-auto on the PR? Are you using it for Q/A? If not, there is also a check in the bottom of the PR which reports if build has succeeded or failed.

Ideally we would want to trigger Jenkins build based on the path where change occurred. Unfortunately this is not supported by Jenkins. As a workaround we are conditionally executing steps if the change occured in the apps/connector/ path. Prior to this we need to fetch the changelog diff between current branch and target branch to understand if there was a change present in the target dir and if we need to run all stages. The population of changelog is needed just the first time the PR is opened and in the subsequent steps we don't need this. Referenced issue: #590 Signed-off-by: markoburcul <marko@status.im>

felicio · 2024-10-11T14:01:47Z

@markoburcul was about to just reply before you commented and now deleted your message 🙂

I've tried on the stage level but it won't work first time the PR

Finishing early wouldn't work?

pipeline {
    agent any
    stages {
        stage('Check Changed Files') {
            steps {
                script {
                    // Get the list of changed files
                    def changedFiles = sh(
                        script: "git diff --name-only origin/main...HEAD",
                        returnStdout: true
                    ).trim()

                    // Define paths or files that should trigger the build
                    def relevantPaths = ['src/', 'config/']

                    // Check if the changes are relevant
                    def isRelevant = changedFiles.split('\n').any { file ->
                        relevantPaths.any { path -> file.startsWith(path) }
                    }

                    // If no relevant changes, skip the build
                    if (!isRelevant) {
                        echo 'No relevant changes, skipping the job...'
                        currentBuild.result = 'SUCCESS'
                        return // Exit the pipeline early
                    } else {
                        echo 'Relevant changes detected, proceeding with the build...'
                    }
                }
            }
        }
...

– source, chatgpt

Why do you need comments from status-im-auto on the PR?

For consistency with other repos like desktop and mobile.

Are you using it for Q/A?

For example.

markoburcul · 2024-10-11T14:06:33Z

@markoburcul was about to just reply before you commented and now deleted your message 🙂

Sorry for that, I realized my solution wasn't working as expected!

Finishing early wouldn't work?

It would but for other people with opened PR's the check down below would be failed. In the case of no changes I'm affraid the only option is to skip stages after.

For consistency with other repos like desktop and mobile.

Sure.

felicio · 2024-10-11T14:12:00Z

It would but for other people with opened PR's the check down below would be failed.

You mean those PRs already open and only within this repo – as result of all of this testing? That'd be alright, as long as the new PRs would work as expected (i.e. /apps/connector completes Jenkins jobs and comments while changes to rest don't).

I'm affraid the only option is to skip stages after.

That's what I meant by finishing early and not executing reaming steps from the Jenkins pipeline (i.e. build, zip, comment).

markoburcul · 2024-10-11T14:31:45Z

Here is the PR that does the trick, thanks for the suggestion!

felicio · 2024-10-11T15:06:39Z

#615

Thank you @markoburcul

felicio · 2024-10-17T07:48:38Z

@markoburcul could you please check what is triggering these builds in #600 (comment) if that PR itself didn't have any new pushes/commits itself?

markoburcul · 2024-10-17T08:18:58Z

@markoburcul could you please check what is triggering these builds in #600 (comment) if that PR itself didn't have any new pushes/commits itself?

The file changed is within the apps/connector directory, which is the directory where we are checking if any change happened. Could it be that you rebased the branch or merged changes from main?

felicio · 2024-10-17T08:21:32Z

@markoburcul

Could it be that you rebased the branch or merged changes from main?

Nope. GitHub would show. Something else gotta be triggering those kind of builds, which eventually produces those detached/not searchable commits

Merging remotes/origin/main commit 1b05730c3d282f20cab0a147bde0207a944fc29d into PR head commit 152e00381c99cdcea498d5edb5083fa34cd4d34e
Merge succeeded, producing 570a13c7173b24d00b827c3e49adcb26af282c5f

https://ci.status.im/job/status-web/job/prs/job/linux/job/PR-600/4/console

markoburcul · 2024-10-17T08:23:22Z

Nope. GitHub would show. Something else gotta be triggering those kind of builds, which eventually produces those detached/not searchable commits
Merging remotes/origin/main commit 1b05730c3d282f20cab0a147bde0207a944fc29d into PR head commit 152e00381c99cdcea498d5edb5083fa34cd4d34e
Merge succeeded, producing 570a13c7173b24d00b827c3e49adcb26af282c5f
https://ci.status.im/job/status-web/job/prs/job/linux/job/PR-600/4/console

Is this the only one where it happened or there is more?

felicio · 2024-10-17T08:25:06Z

PRs? There hasn't been any other for that directory open. Builds? All but the first one.

markoburcul · 2024-10-17T08:33:29Z

PRs? There hasn't been any other for that directory open. Builds? All but the first one.

* https://ci.status.im/job/status-web/job/prs/job/linux/job/PR-600/2/

* https://ci.status.im/job/status-web/job/prs/job/linux/job/PR-600/3/

* https://ci.status.im/job/status-web/job/prs/job/linux/job/PR-600/4/

I've changed one of the settings for the pipeline. You could test it by opening new pr with a change in apps/connector dir, but you should wait for any new change in the main branch and see if it would trigger a new build on the pr.

felicio · 2024-10-17T08:35:49Z

You could test it by opening new pr with a change in apps/connector dir

Did you 😉?

markoburcul · 2024-10-17T08:48:07Z

You could test it by opening new pr with a change in apps/connector dir

Did you 😉?

I've opened the pr and now I'll wait for any change in the main

felicio · 2024-10-17T08:50:10Z

Merged a commit into main.

felicio · 2024-10-17T08:51:53Z

But I see your PR wasn't even built #620.

markoburcul · 2024-10-17T09:20:43Z

But I see your PR wasn't even built #620.

In my PR I explained the behaviour #622

markoburcul · 2024-10-17T10:00:07Z

@felicio could you take a look at my PR so we can merge it? Since the changing of the strategy in Jenkins settings will result in all other PR's that don't have this Jenkinsfile change to fail as here

markoburcul · 2024-10-17T13:38:43Z

@felicio as you can see here on this PR which was created after we merged this commit there are no build comments and additionally in Jenkins UI the build wasn't triggered after this commit was pushed to main branch.
So we managed to get desired behaviour:

the PR gets comment from autobot only when the change happens in the apps/connector directory
builds are issued only when change occurs on the PR

felicio · 2024-10-17T13:43:11Z

Created #626 to see if it will build only once and remain so overtime as per the original issue.

markoburcul · 2024-10-17T14:00:19Z

Created #626 to see if it will build only once and remain so overtime as per the original issue.

As you can see with this commit we have proven that PR builds won't be triggered by pushes to main.

markoburcul · 2024-10-18T10:42:04Z

Created #626 to see if it will build only once and remain so overtime as per the original issue.

Could you check my last comment? I believe that we achieved the desired behavior.

felicio · 2024-10-18T14:09:17Z

No new unrelated builds in #600 (comment) and #626 (comment).

Closing. Thanks again @markoburcul .

jakubgs assigned markoburcul Oct 4, 2024

markoburcul mentioned this issue Oct 10, 2024

jenkinsfile: fix paths for all steps #597

Merged

felicio mentioned this issue Oct 10, 2024

Update README.md #594

Closed

markoburcul added a commit that referenced this issue Oct 10, 2024

jenkinsfile: use newest status jenkins lib

f1bcde6

Use newest jenkins lib tag which adds the entryPoint as an argument to the nix shell function. Referenced issue: #590 Signed-off-by: markoburcul <marko@status.im>

markoburcul added a commit that referenced this issue Oct 10, 2024

jenkinsfile: use newest status jenkins lib

90e33d2

Use newest jenkins lib tag which adds the entryPoint as an argument to the nix shell function. Referenced issue: #590 Signed-off-by: markoburcul <marko@status.im>

markoburcul closed this as completed Oct 10, 2024

felicio reopened this Oct 10, 2024

markoburcul mentioned this issue Oct 11, 2024

jenkinsfile: filter stages based on path #612

Closed

felicio closed this as completed Oct 11, 2024

felicio reopened this Oct 17, 2024

felicio closed this as completed Oct 18, 2024

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Implement Pull Request CI builds #590

Implement Pull Request CI builds #590

felicio commented Oct 3, 2024 •

edited

Loading

jakubgs commented Oct 3, 2024

felicio commented Oct 3, 2024

markoburcul commented Oct 10, 2024

felicio commented Oct 10, 2024

markoburcul commented Oct 10, 2024

felicio commented Oct 10, 2024

markoburcul commented Oct 10, 2024

felicio commented Oct 10, 2024

markoburcul commented Oct 11, 2024

felicio commented Oct 11, 2024 •

edited

Loading

markoburcul commented Oct 11, 2024

felicio commented Oct 11, 2024

markoburcul commented Oct 11, 2024

felicio commented Oct 11, 2024 •

edited

Loading

felicio commented Oct 17, 2024

markoburcul commented Oct 17, 2024

felicio commented Oct 17, 2024

markoburcul commented Oct 17, 2024

felicio commented Oct 17, 2024

markoburcul commented Oct 17, 2024

felicio commented Oct 17, 2024

markoburcul commented Oct 17, 2024

felicio commented Oct 17, 2024

felicio commented Oct 17, 2024

markoburcul commented Oct 17, 2024

markoburcul commented Oct 17, 2024

markoburcul commented Oct 17, 2024

felicio commented Oct 17, 2024

markoburcul commented Oct 17, 2024

markoburcul commented Oct 18, 2024

felicio commented Oct 18, 2024

Implement Pull Request CI builds #590

Implement Pull Request CI builds #590

Comments

felicio commented Oct 3, 2024 • edited Loading

what

why

how (for example)

jakubgs commented Oct 3, 2024

felicio commented Oct 3, 2024

markoburcul commented Oct 10, 2024

felicio commented Oct 10, 2024

markoburcul commented Oct 10, 2024

felicio commented Oct 10, 2024

markoburcul commented Oct 10, 2024

felicio commented Oct 10, 2024

markoburcul commented Oct 11, 2024

felicio commented Oct 11, 2024 • edited Loading

markoburcul commented Oct 11, 2024

felicio commented Oct 11, 2024

markoburcul commented Oct 11, 2024

felicio commented Oct 11, 2024 • edited Loading

felicio commented Oct 17, 2024

markoburcul commented Oct 17, 2024

felicio commented Oct 17, 2024

markoburcul commented Oct 17, 2024

felicio commented Oct 17, 2024

markoburcul commented Oct 17, 2024

felicio commented Oct 17, 2024

markoburcul commented Oct 17, 2024

felicio commented Oct 17, 2024

felicio commented Oct 17, 2024

markoburcul commented Oct 17, 2024

markoburcul commented Oct 17, 2024

markoburcul commented Oct 17, 2024

felicio commented Oct 17, 2024

markoburcul commented Oct 17, 2024

markoburcul commented Oct 18, 2024

felicio commented Oct 18, 2024

felicio commented Oct 3, 2024 •

edited

Loading

felicio commented Oct 11, 2024 •

edited

Loading

felicio commented Oct 11, 2024 •

edited

Loading