Add option `tools: linked` for `tools` input of init action. #2281

NlightNFotis · 2024-05-09T13:53:49Z

Description

The init action supports an input tools, which can get a range of values, including a special value latest that is supposed to force the use of the bundled version of CodeQL for the action.

The use of the word latest, however, might hint to users that this is really fetching the newest version of CodeQL, rather than the one associated with the action when it was released.

This PR is adding an option tools: linked that is functionally equivalent to tools: latest, but is supposed to guide user expectations as to the version that's actually used.

Review Guidance

The basic mechanism that supports the new input option has been added in 80f8aff, in the file src/setup-codeql.ts.
Tests for the new option, and the old option working in an equivalent manner have been added in 80f8aff, in the file src/setup-codeql.test.ts
Tests for the logging of the version being present in the logs generated by the program have been added in 3a2da5f

Still outstanding

Will need to check whether readme or changelog need to be edited.
Will need to add documentation for the new option, but I want to do this as a different PR associated with Clarify documentation for the tools field in the init action #1327
Need to document some information transfer from Henry that unblocked me while trying to write a stub using Sinon.
- Added https://github.com/github/codeql-core/pull/4293

Merge / deployment checklist

Confirm this change is backwards compatible with existing workflows.
Confirm the readme has been updated if necessary.
Confirm the changelog has been updated if necessary.

henrymercer

Very nice! Could you update the docs for tools in init/action.yml too?

henrymercer · 2024-05-09T14:51:36Z

src/setup-codeql.ts

+  logger.info(
+    `Using CodeQL CLI version ${source.toolsVersion} from ${source.sourceType}.`,
+  );


There are a few routes where we don't actually know the CLI version at this point, for example if the tools were specified using a local path or if an old bundle URL was specified like https://github.com/github/codeql-action/releases/download/codeql-bundle-20230317/codeql-bundle-linux64.tar.gz. In both cases toolsVersion is a bit opaque (for the first case it will be local, for the second it will be 0.0.0-20230317), and I think we want to avoid showing this to users to avoid confusion.

What do you think about pushing this message into getCodeQLSource and returning more specific messages in each case, for instance:

Local could say something like "Using CodeQL CLI from local path $path"

Old bundle URL could say something like "Using CodeQL CLI from URL $url"

Cases where we know the CLI version could say what we have here "Using CodeQL CLI version $version from $source."

Ah, I see - I wasn't aware that we actually can be in a position where we don't have all the information.

I was hoping that by putting the logging into the getCodeQLBundle, after getCodeQLSource has returned, that I could get away with logging at just one point, and at a time where the dust had settled and a decision on what/where to get codeql from had been made, instead of trying to chase the various return paths in getCodeQLSource.

I will revise the approach.

src/setup-codeql.ts

henrymercer · 2024-05-09T14:55:51Z

src/setup-codeql.ts

+        `"tools: linked" or "tools: latest" was requested. The version shipped with the Action is ` +
+        `${defaultCliVersion.cliVersion}.`,


Couple of suggestions:

Say something like "Overriding the version of the CodeQL tools by , the version shipped with the Action"

Use tools: ${toolsInput} to simplify the message

If latest is deprecated, we should also create a warning if we see it.

I wanted to ask actually about the deprecation process for the tool. I'll take we're happy to just log.warn?

My 2¢: it's fine for customers who have already set up tools: latest to continue using it. I don't think it's worth removing support for this for the forseeable future.

henrymercer · 2024-05-09T15:00:01Z

CHANGELOG.md

@@ -6,7 +6,7 @@ Note that the only difference between `v2` and `v3` of the CodeQL Action is the

 ## [UNRELEASED]

-No user facing changes.
+- Add `tools: linked` option for input of `init` action. [#2281](https://github.com/github/codeql-action/pull/2281)


Suggested change

- Add `tools: linked` option for input of `init` action. [#2281](https://github.com/github/codeql-action/pull/2281)

- The `tools: latest` input to the `init` Action has been renamed to `tools: linked`. This option specifies that the Action should use the tools shipped at the same time as the Action. The old name will continue to work for backwards compatibility, but we recommend that new workflows use the new name. [#2281](https://github.com/github/codeql-action/pull/2281)

aeisenberg · 2024-05-09T19:43:24Z

Once this is done, we will also need to update the documentation in docs-internal. Though, I took a quick look and I can't find any reference to the tools input. Is this even documented there? The init action file contains a link to some documentation, but that page doesn't mention tools anywhere.

https://github.com/github/codeql-action/blob/main/init/action.yml#L17

For now, maybe we just need to update this input description with the new change and remove the link.

Then later, we should update the docs-internal with a full description of tools.

NlightNFotis · 2024-05-10T13:50:36Z

Very nice! Could you update the docs for tools in init/action.yml too?

Yes, this is my plan before the end of the ticket as a whole. I was originally thinking of doing this as part of a seperate PR that's designed to specifically address the issue in #1327.

Will raise this soon - unless there's a preference to address this in this PR? I was thinking that a separate PR will be cleaner, and can afford being more focused on drafting a good general documentation for tools, without being subjected to the review cycle/scope of this PR. Any thoughts?

henrymercer · 2024-05-10T14:29:14Z

It's not currently documented, so I'm happy with a separate PR if you prefer!

NlightNFotis · 2024-05-10T15:45:20Z

Apologies, I had to force push a rebase because of some issues with the CHANGELOG. What has changed in 5a08657:

I moved the logging to the downstream function getCodeQLSource, with more specialised log messages at each return point.
Added a deprecation notice for the tools: latest input.
Added tests for the logging remaining valid for more code paths of the getCodeQLSource when called with different toolsInput arguments from setupCodeQLBundle function.

henrymercer

Nice! Just a couple of final comments.

henrymercer · 2024-05-10T16:29:10Z

src/setup-codeql.ts

+    !CODEQL_BUNDLE_VERSION_ALIAS.includes(toolsInput) &&
+    !toolsInput.startsWith("http")
+  ) {
+    logger.info("Using CodeQL CLI from local path $path");


Suggested change

logger.info("Using CodeQL CLI from local path $path");

logger.info(`Using CodeQL CLI from local path ${toolsInput}`);

henrymercer · 2024-05-10T16:30:15Z

src/setup-codeql.ts

+    if (toolsInput === "latest") {
+      logger.warning(
+        "The 'latest' alias for the CodeQL tools has been deprecated. Please use 'linked' instead.",
+      );
+    }


See my previous comment — I'm not sure it's worth creating work for customers to change their workflows to use "linked".

Hmm, thinking out loud here:

Does it make sense to keep the deprecation warning, but soften the wording a bit, to indicate that the preferred option going forward is linked, but that latest remains compatible?

I'm thinking that if we don't have this bit of documentation (via the warning), from the user's perspective there's a chance of looking at the other bits of documentation we have, which indicate linked to be used, compare with their action containing latest, and wonder what happened to latest and why there's a discrepancy between our documentation and their current version, without any warning.

I believe that raising a warning here at least alerts a user to the fact that this has changed, which is better user experience than shadow changes, but I don't have any strong feelings on the subject.

Any thoughts?

That's a good point. We could have an info message like "tools: latest has been renamed to tools: linked, but the previous name is still supported for backwards compatibility. No action is necessary.".

henrymercer · 2024-05-10T16:34:31Z

src/setup-codeql.ts

+    const version = cliVersion ?? humanReadableVersion;
+    logger.info(
+      `Using CodeQL CLI version ${version} from toolcache at ${codeqlFolder}`,
+    );


humanReadableVersion can still contain something like 0.0.0-<bundle version number> (it could probably have a better name as it's mainly for telemetry and debug logs). How about only printing the version ${cliVersion} bit if cliVersion is defined?

henrymercer · 2024-05-10T16:35:06Z

src/setup-codeql.ts

@@ -452,12 +474,16 @@ export async function getCodeQLSource(
    url = await getCodeQLBundleDownloadURL(tagName!, apiDetails, logger);
  }

+  const toolsVersion = cliVersion ?? humanReadableVersion;
+  logger.info(
+    `Using CodeQL CLI version ${toolsVersion} downloaded from ${url}.`,


Likewise here. Also we might want to rephrase this slightly so it doesn't sound like the CLI has already been downloaded.

NlightNFotis · 2024-05-13T13:22:55Z

Hi @henrymercer, this is now ready for another round of reviews.

Also ensure that option latest remains compatible, and add tests for the two options.

…gs from some locations

See [1]. [1]: github/codeql-action#2281 Signed-off-by: Sebastian Schuberth <sebastian@doubleopen.org>

NlightNFotis requested a review from a team as a code owner May 9, 2024 13:53

NlightNFotis self-assigned this May 9, 2024

NlightNFotis force-pushed the tools_latest_improvement branch from a294209 to e4ee1da Compare May 9, 2024 14:21

henrymercer reviewed May 9, 2024

View reviewed changes

NlightNFotis force-pushed the tools_latest_improvement branch from e4ee1da to 5a08657 Compare May 10, 2024 15:41

NlightNFotis mentioned this pull request May 10, 2024

Add documentation for the tools option in the various workflow files #2284

Merged

3 tasks

henrymercer reviewed May 10, 2024

View reviewed changes

NlightNFotis requested a review from henrymercer May 13, 2024 10:40

henrymercer approved these changes May 13, 2024

View reviewed changes

NlightNFotis added 8 commits May 13, 2024 14:51

Add option tools: linked for init action.

cbe408d

Also ensure that option latest remains compatible, and add tests for the two options.

Add logging for CodeQL CLI Version and add tests to the logger for it.

cff15d7

Add test for CodeQL version appearing in log.

c92bbd4

Change overriding log message to make default version more explicit.

998a4c7

[SYNC] Sync files in lib/ as requested by CI check that is failing

4522b08

Add new input option tools: linked to CHANGELOG

1796f54

Move logging messages to downstream function and add deprecation notice

bcc1365

Soften deprecation message wording and remove unhelpful version strin…

df4819e

…gs from some locations

NlightNFotis force-pushed the tools_latest_improvement branch from b24e695 to df4819e Compare May 13, 2024 13:51

NlightNFotis merged commit 5a14b05 into main May 13, 2024
320 checks passed

NlightNFotis deleted the tools_latest_improvement branch May 13, 2024 14:07

NlightNFotis mentioned this pull request May 15, 2024

Update default bundle to 2.17.3 #2295

Merged

This was referenced May 20, 2024

Merge main into releases/v3 #2300

Merged

Merge releases/v3 into releases/v2 #2302

Merged

sschuberth added a commit to oss-review-toolkit/ort that referenced this pull request May 21, 2024

ci(github): Use a non-deprecated value to configure CodeQL tools

3644e72

See [1]. [1]: github/codeql-action#2281 Signed-off-by: Sebastian Schuberth <sebastian@doubleopen.org>

sschuberth mentioned this pull request May 21, 2024

ci(github): Use a non-deprecated value to configure CodeQL tools oss-review-toolkit/ort#8666

Merged

sschuberth added a commit to oss-review-toolkit/ort that referenced this pull request May 21, 2024

ci(github): Use a non-deprecated value to configure CodeQL tools

6ae4429

See [1]. [1]: github/codeql-action#2281 Signed-off-by: Sebastian Schuberth <sebastian@doubleopen.org>

angelapwen mentioned this pull request May 31, 2024

PR Checks: Use tools: linked rather than tools: latest #2320

Merged

3 tasks

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Add option `tools: linked` for `tools` input of init action. #2281

Add option `tools: linked` for `tools` input of init action. #2281

NlightNFotis commented May 9, 2024 •

edited

Loading

henrymercer left a comment

henrymercer May 9, 2024 •

edited

Loading

NlightNFotis May 10, 2024

henrymercer May 9, 2024

aeisenberg May 9, 2024

NlightNFotis May 10, 2024

henrymercer May 10, 2024

henrymercer May 9, 2024

aeisenberg commented May 9, 2024

NlightNFotis commented May 10, 2024

henrymercer commented May 10, 2024

NlightNFotis commented May 10, 2024

henrymercer left a comment

henrymercer May 10, 2024

henrymercer May 10, 2024

NlightNFotis May 13, 2024 •

edited

Loading

henrymercer May 13, 2024 •

edited

Loading

henrymercer May 10, 2024

henrymercer May 10, 2024

NlightNFotis commented May 13, 2024

		`"tools: linked" or "tools: latest" was requested. The version shipped with the Action is ` +
		`${defaultCliVersion.cliVersion}.`,

	- Add `tools: linked` option for input of `init` action. [#2281](https://github.com/github/codeql-action/pull/2281)
	- The `tools: latest` input to the `init` Action has been renamed to `tools: linked`. This option specifies that the Action should use the tools shipped at the same time as the Action. The old name will continue to work for backwards compatibility, but we recommend that new workflows use the new name. [#2281](https://github.com/github/codeql-action/pull/2281)

	logger.info("Using CodeQL CLI from local path $path");
	logger.info(`Using CodeQL CLI from local path ${toolsInput}`);

Add option tools: linked for tools input of init action. #2281

Add option tools: linked for tools input of init action. #2281

Conversation

NlightNFotis commented May 9, 2024 • edited Loading

Description

Review Guidance

Still outstanding

Merge / deployment checklist

henrymercer left a comment

Choose a reason for hiding this comment

henrymercer May 9, 2024 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

aeisenberg commented May 9, 2024

NlightNFotis commented May 10, 2024

henrymercer commented May 10, 2024

NlightNFotis commented May 10, 2024

henrymercer left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

NlightNFotis May 13, 2024 • edited Loading

Choose a reason for hiding this comment

henrymercer May 13, 2024 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

NlightNFotis commented May 13, 2024

Add option `tools: linked` for `tools` input of init action. #2281

Add option `tools: linked` for `tools` input of init action. #2281

NlightNFotis commented May 9, 2024 •

edited

Loading

henrymercer May 9, 2024 •

edited

Loading

NlightNFotis May 13, 2024 •

edited

Loading

henrymercer May 13, 2024 •

edited

Loading