Base commit on main stuck to some months ago on Tanstack

[Balastrong]

Describe the bug
On TanStack Query and Form all coverage reports are compared against really old base commits leading to wrong results when the Github Action writes comments in the PR (also makes the PR checks fail if below coverage threshold!)

Query -> Base commit is 2fe16e5
Form -> Base commit is 4e54c09

To Reproduce
Unknown

Expected behavior
Base commit is up to date, if a branch is opened from main's latest commit that's the expected base.

Screenshots

[drazisil-codecov]

Hi @Balastrong ,

It looks like, for https://app.codecov.io/gh/TanStack/form, the most recent SHA on the main branch, 4e54c0940b751e44a0a4a1b944fbfcccc206c761, was indeed months ago. Do you have a newer SHA, and possibly the CL output from the coverage upload step?

For https://app.codecov.io/gh/TanStack/query, the most recent commit on main is https://app.codecov.io/gh/TanStack/query/commit/4a2e838fc09b910656aee6b214ab37557c7064ef, which was 5 days ago. This was compared to https://app.codecov.io/gh/TanStack/query/commit/0df28830a756c0d8b7545e7a6f75c1eacfb2df54, also from 5 days ao.

Can you further detail what you are seeing?

[Balastrong]

Hey @drazisil-codecov, thanks for the answer!

For form let's use this comment as example.

As per Codecov's comment "Report is 167 commits behind head on main." but that branch has been created on main's head and immediately merged back. I was expecting Codecov to compare the PR with this commit instead that is the base commit for that branch.

[drazisil-codecov]

Hi @Balastrong It looks like TanStack/form@04238b1 didn't upload coverage. Is it possible that https://app.codecov.io/gh/TanStack/form/commit/5473bb8be2e046dccc49a26c3f2ed85491aabb60 is the nst recent commit on main where coverage was uoloaded?

[Balastrong]

We do have an action with a step that uploads the coverage but now that you mentioned that I had a closer look

https://github.com/TanStack/form/actions/runs/11920712618/job/33223266275#step:8:66

info - 2024-11-19 19:59:09,748 -- Process Upload complete
error - 2024-11-19 19:59:09,749 -- Upload failed: {"message":"Token required because branch is protected"}

The step is marked as successful but this log seems to indicate it wasn't, I'm now learning about the existence of the fail_ci_if_error flag :D

This is how we upload the result

      - name: Upload coverage to Codecov
        uses: codecov/codecov-action@v4
        with:
          directory: packages
        env:
          CODECOV_TOKEN: ${{ secrets.CODECOV_TOKEN }}

I'll check if we have a problem with the token from the secrets, thanks!

[Balastrong]

Hey @drazisil-codecov, we checked and Form indeed was missing the valid token and it is not set correctly, thanks again!
However we're still facing some issues and now the same on both Form and Query.

This is the last succesful upload on Query (7 days ago) but the following run of the exact same action (on the next commit) even if succeeded shows some errors on the codecov page and since that day all uploads lead to this error.

It seems pretty random, can you point us in the right direction?

[drazisil-codecov]

Hi @Balastrong

For https://app.codecov.io/github/TanStack/query/commit/fadfbde04548a0b21e7af7f6b9798d87f57ad984, at least, the error message should link to a docs page explaining how to fix. Does it not?

[TkDodo]

@drazisil-codecov I’m not sure I understand the error:

Upload exceeds the max age of 12h. Please download and review your report or turn off the age check by visiting expired reports.

Isn’t the upload happening right when the PR gets merged?

@Balastrong we can try and set max_report_age: off in our config if that helps?

[drazisil-codecov]

@TkDodo It's usually the result of a cached coverage report, which is why we alert on it.

For https://app.codecov.io/github/TanStack/query/commit/fadfbde04548a0b21e7af7f6b9798d87f57ad984, you uploaded at roughly 18 Nov, 12:22 PM EST. You sent us a report named packages/query-codemods/coverage/clover.xml that has a very broken generated value of Friday, May 18, 2249 8:11:57.315 PM EST

Not sure what caused that, but that's a diff of greater then 12 hours by....... a lot.

# path=packages/query-codemods/coverage/clover.xml
<?xml version="1.0" encoding="UTF-8"?>
<coverage generated="8816343117315" clover="3.2.0">

[TkDodo]

@drazisil-codecov where are you seeing that?

I downloaded the report from the url you provided ( https://app.codecov.io/github/TanStack/query/commit/fadfbde04548a0b21e7af7f6b9798d87f57ad984) - it points towards https://api.codecov.io/upload/gh/TanStack/query/download?path=shelter/github/TanStack::::query/fadfbde04548a0b21e7af7f6b9798d87f57ad984/717e16ee-5e3f-426a-b91a-41be95f045a3.txt

and in there, all the timestamps seem okay?

# path=packages/query-codemods/coverage/clover.xml
<?xml version="1.0" encoding="UTF-8"?>
<coverage generated="1731588163431" clover="3.2.0">
  <project timestamp="1731588163431" name="All files">

1731588163431 = Thu Nov 14 2024 12:42:43.431 UTC

[drazisil-codecov]

/Users/drazisil-codecov/Downloads/717e16ee-5e3f-426a-b91a-41be95f045a3.txt <- downloaded yesterday

/Users/drazisil-codecov/Downloads/717e16ee-5e3f-426a-b91a-41be95f045a3 (1).txt <- downloaded just now.

I'm.....

...

Can I get back to you? 😅

[drazisil-codecov]

I think I must have somehow changed the value. My sincere apologies. The correct timestamp, as you said, is 14 November. You uploaded to Codecov 18 November.

[TkDodo]

okay, could this be because of nx caching? Like, we create a report on 14th on the PR, then it takes us some time to merge the PR, then we re-run it on main, but it just reads the cached report?

if that’s the case, we could either:

• turn off caching this task (not sure why we should, the output is expected to be “the same”, except newer timestamps maybe)
• allow older reports

[drazisil-codecov]

I don't have enough experience with nx to answer that, but it sounds plausible. If you are sure the older reports are still good, I'd just disable the age check. Sorry for the initial confusion.

[TkDodo]

Thanks, we’ve disabled the setting:

• ci: disable max report age in codecov TanStack/query#8354

Let’s see if things improve 👍