fix: Do not unnecessarily update apply check if it doesn't exist yet

[lukemassa]

what

When determining how to update commit status, if there are plans that still need to be applied, do not update the apply status at all (as opposed to now where it is updated to PendingCommitStatus).

why

#3378 introduced a change to the logic on updating plans, which includes a pre-emptive update to the "apply" check. The intention here was, for VCSs that have an apply check "in progress", to set it to "successful", to allow tools that auto merge to not have to run a full atlantis apply if there are no changes.

In gitlab, this works fine for plans with no changes. However, the logic as is sets the apply check to "pending" if there are changes. In gitlab, this creates a new pipeline on the MR, that is then stuck in "Running" state (See #3725).

I don't know how other VCSs work, but in gitlab, calling UpdateStatus creates/updates an "external" pipeline with whatever name you give it, which makes it seem like it's "still happening" instead of just waiting for the next command. This change therefore prevents the pipeline for being stuck.

I can't imagine this PR would cause an issue to other VCSs, because this returns us, in this particular case, to the "pre 3378" world, where we didn't update the apply check at all in the plan command. Even if the apply check exists, there's no need for plan to "remind" apply that it's in progress (any more than there was a need to do so before 3378).

I could potentially see an argument for likewise returning early if numErrored > 0 instead of updating apply with a FailedCommitStatus, but that isn't hurting anything so happy to keep it.

tests

A plan with changes does not run an apply pipeline:

However a plan with no changes correctly "runs" (and successfully completes) an apply pipeline:

references

closes #3725

[lukemassa]

@chroju (author of #3378) if you're available to weigh in and/or make sure my understanding of the current logic is correct?

[lukemassa]

The CI failure above is I believe unrelated to my change, I addressed it separately here: #3748

[lukemassa]

OK now this round of CI issues are mine, I will look into them :)

[chroju]

@lukemassa
Based on my understanding, for the plan with any changes, the intention is not to set the apply status as Pending, but rather not to set any status. I think this aligns with the content of my pull request.

I'm sorry for not being familiar with the specifications of GitLab since I don't use it (In fact, this behavior posed no issues with GitHub).

[lukemassa]

@chroju great thanks! Yeah I think github deals with these checks very differently than gitlab, so it takes some tweaking to get logic to work for both.

I also updated the CI slightly in a way I think makes sense, still open to feedback there though.

[lukemassa]

OK I've cleaned up CI, ready for review!

[X-Guardian]

Hi guys, I've tested this code on GitLab and GitHub, and it is working successfully. GitLab no longer has a pending atlantis/apply pipeline on every plan, and both GitLab and GitHub create a successful atlantis/apply pipeline if all the plan jobs have no changes.

@GenPage / @jamengual, can we get this reviewed and merged for the next release, as v0.25.0 and the current unreleased code in the main branch is not usable for us on GitLab with this issue.

Thanks!