Add Scorecard

[gabibguti]

Resolves #3090

This PR adds Scorecard security tool. Thanks for trying the tool out! And let me know if you have any feedback.

[daxpedda]

Is there a reason why we aren't just using the published versions?

[gabibguti]

One of the security best practices Scorecard recommends is using commit SHAs to reference dependency versions instead of, in this case, branches like main or tags like v4. The reason is that branches and tags get automatically the latest updates and with that it can get a malicious commit without notice.

[daxpedda]

As far as I can see OSFF doesn't even support Cargo projects, is that correct?
In any case, seeing that this is a library pinning dependencies is imo a terrible idea.

I feel similarly about GitHub Actions dependencies, they should definitely not be pinned in a public project, where zero-day exploits can be used against us by anybody who can trigger the CI workflow, which is more or less everybody.

[gabibguti]

As far as I can see OSFF doesn't even support Cargo projects, is that correct?

Yes, true. The Scorecard Pinned-Dependencies checks does not identify Cargo dependencies. I will report that for Scorcard and suggest the documentation is improved to inform which languages/tools are supported.

In any case, seeing that this is a library pinning dependencies is imo a terrible idea.

I partially agree. I think there are pros and cons. My current view is that pinning build and release dependencies is still "safer" for these processes. But pinning the library dependencies and test dependencies is not something I agree with too. So, pinning GitHub actions dependencies in this workflow would still make sense for me but I understand it comes with a maintainance cost.

[daxpedda]

Can we also add the badge here or can we do this only as a follow-up?

[daxpedda]

I'm looking at the results here: https://securityscorecards.dev/viewer/?uri=github.com/rustwasm/wasm-bindgen.

Is there some way to add exceptions to binary artifacts? Currently it includes stuff from examples and benchmarks.
SAST is also an issue, we do use Clippy, but as far as I can see OSFF doesn't support detecting that.

[daxpedda]

LGTM to me, would only like to see the GitHub Actions dependencies unpinned, this is a security practice I don't really agree is appropriate for wasm-bindgen, I elaborated on this above.

OSFF doesn't really support Cargo project, so I'm starting to be a bit skeptical of it's usefulness here. Overall, I'm still fine with this.

@alexcrichton I feel like you should give your stamp of approval here as well.

[alexcrichton]

I'd personally be a bit wary of this given the amount of configuration required in this repository in addition to some of the results. Some of the aspects on the security card I don't understand (e.g. I haven't heard of SAST before) and some aren't really that applicable (fuzzing for this repo, pinned dependencies, wasms-interpreted-as-binary, signed releases, etc).

All that being said I'm not too too involved in wasm-bindgen any more so I don't think I should count as a "final say" of sorts, I think it's fine to add this if you'd like to @daxpedda

[daxpedda]

@hamza1311 @Liamolucko, you guys have any opinion on that?
I'm currently not against it but not for it as well, pretty much for the same reasons Alex Crichton pointed out: the result doesn't seem very representative to me for wasm-bindgen.

So unless there is any further approval I'm gonna close this.

[gabibguti]

I'm looking at the results here: https://securityscorecards.dev/viewer/?uri=github.com/rustwasm/wasm-bindgen.

Is there some way to add exceptions to binary artifacts? Currently it includes stuff from examples and benchmarks. SAST is also an issue, we do use Clippy, but as far as I can see OSFF doesn't support detecting that.

No, there's currently no way of adding exceptions, but Scorecard plans on working on such a feature. And, yes, Scorecard does not support Clippy as far as I know, I will report that too.

[gabibguti]

In any case, thanks for the valuable feedback!

[ranile]

The Scorecard Pinned-Dependencies checks does not identify Cargo dependencies

This reduces the usefulness by a lot.

Also, from the docs

Risk: Critical (vulnerable to repository compromise)

This check determines whether the project's GitHub Action workflows has dangerous code patterns.

Dangerous-Workflow (critical risk)

no dangerous workflow patterns detected

This makes me question the effectiveness of the product. It reports dangerous workflow as critical risk but then says it didn't find any dangerous workflow patterns?

At this point:

• it can't detect supply chain vulnerabilities
• it can't filter results to actual library code
• it can't properly report the issues (see above)
• static code analysis does not use clippy -- what does it use then? if it's better than clippy, then effort should be put to making clippy better. We already run clippy so it's not too important for us

I'm with @daxpedda on closing the PR

[daxpedda]

Alright, as there are no further approvals, I'm going to close this.

Thanks @gabibguti nonetheless.