-
Notifications
You must be signed in to change notification settings - Fork 30k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
doc: add req for sec wg review in some cases #21896
Conversation
Pushed button a bit early, running tests now, tests complete and ok. |
Some recent dicsussions point to it being a good idea to require reviews from the security-wg for security related PRs. See nodejs#21766 Add this requirement to the collaborator guide.
COLLABORATOR_GUIDE.md
Outdated
@@ -378,6 +379,16 @@ multiple commits. Commit metadata and the reason for the revert should be | |||
appended. Commit message rules about line length and subsystem can be ignored. | |||
A Pull Request should be raised and approved like any other change. | |||
|
|||
### Additions to the Cryptography and Security APIs | |||
|
|||
Semver-minor commits that add or change cryptograpy/security APIs should be |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
cryptograpy -> cryptography
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I'm sorry to register opposition to this approach but I don't think this text is the way we want to go.
-
It's not clear to me what a "security API" is. Is that something like
Buffer.from()
. If not, what is a "security API"? -
I'm opposed to adding a new complication to landing stuff. If we want to make it more difficult for problems to slip through the cracks, let's simplify the rules rather than add yet another rule that will be ignored. (We break our policies every single day and I do a lot of biting my tongue about it.) A simplification might be that
semver-minor
requires two TSC approvals the waysemver-major
does. If that's not a simplification, at least it does not add new complications. Another simplification might be to simply require all PRs to be open for 72 hours (except for fast-track PRs). No more figuring out if it's a "weekend" or not (time zones anyone?). That will give people more time to go through new PRs that might be of interest, and more time for people to @-mention interested parties and stakeholders. -
I'd be 💯 on a guideline that strongly recommended notifying Security WG via @-mention. And if we go that route, then we don't have to clarify what a security API is. Not sure something is a security API? Well, when in doubt, @-mention Security WG and let them figure out if it's relevant to their concerns or not.
I don't feel that this adds any complication, exactly. Rather, it just adds an extra step to the process that can be paraphrased as "ensure that it's been seen by the people who are subject matter experts on the kind of changes being made". That seems like a pretty essential part of the process to me, considering the long-term impact of getting it wrong. We're still, as an ecosystem, collectively dealing with the fallout of
That's a reasonable question. This is probably always going to remain fuzzy to some degree, but I think a good first approximation is "changes where getting it wrong could have a security impact somehow", to be determined by those reviewing the changes. To relate this back to your Possibly we should reword the change to "cryptography APIs and changes with potential security impact", or a wordier variation of that? I agree that "cryptography/security APIs" is rather vague, and will probably be misinterpreted. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
As this appears to expand the charter of the security-wg I feel like this is something that should also land in the charter and be approved by the TSC
I share your concerns about already breaking our policies and that we should try to see why those are broken and fix them.
It might be a good idea to give more time for review. One more idea for solving the «we don't wait enough time» — a bot that would ensure that we wait appropriate time via GitHub checks API should be great. Having that, we might also install more complex rules, e.g. wait more time for semver-minor and semver-major requests, enforce TSC approval, etc.
+1 on this one, we have enough people in the TSC for that. Also could be enforced by a bot, I believe. That said, I would still prefer something among these lines to be adopted (for security reasons), and I ultimately believe that solving the «too hard to keep track of our rules» problem is orthogonal and should be solved by other means. I believe that the evidience we have justifies that.
What @joepie91 said above («changes where getting it wrong could have a security impact somehow») is good, but part of the problem is that detemining that is hard. Perhaps that can include something like «any semver-minor or semver-major changes to the «Any semver-minor or semver-major changes to the
Yes, that is also a great idea and could be done independently in a broader range of situations. But that alone wouldn't suffice without a strict requirement, imo. |
Can that be resolved by allowing to override that rule with a TSC consensus or vote instead? All what remains could fall under «Recommend security improvements for the core Node.js project» and «Define and maintain security policies and procedures for the core Node.js project» which are already in the charter. We could refine that, but I don't view this as a blocker. |
I agree that this is something we should have the TSC discuss before progressing. |
Agreed that this needs TSC discussion. I had thought I'd added a @nodejs/tsc ping but looks like I missed that. Will track conversation and then if makes sense add to agenda for next week. |
Reading through the specific comments.
|
FWIW, IMO this sort of thing is not orthogonal to our failure to keep track of and follow our policies. It is a root cause of it. Creating different requirements for different APIs is going to naturally and perhaps inevitably result in those sorts of problems. This is especially true when some of the APIs (like "security APIs") are difficult to define in practical terms. It seems like a necessarily "I know it when I see it" kind of thing. (Lots of things are that way necessarily.) |
Working Groups often go through periods of inactivity or even effectively stop existing altogether. If there is to be WG veto power on code changes, it needs to be an active veto. To stop something, they need to do something. The proposal here invests a WG with the power to veto something by doing nothing. That seems unwise to me. Here's a multi-pronged counter-proposal:
|
Oh, another one...
|
So at the risk of being pedantic... It is possible that improper crypto can lead to security vulnerabilities, but afaik while the security-wg maty have crypto experts it is not a requirement of being part of the group. I would much rather see something to empower the security-wg to have the ability to flag things that are security concerns... this may be crypto, it may be http2, it may be the debugger (e.g. how we expose ports). edit: FWIW I think we should explore spinning up a crypto team to work on a long term plan for ongoing support of crypto in core to help avoid security concerns... such a group could eventually be chartered by the TSC. edit 2: it is also possible that cryptography could be a team under the security wg, with limited membership. Similar to what we currently do with Release / releasers / LTS. all crypto team and in sec-wg but not all sec-wg members are on crypto team |
@Trott: We have basically two failure modes to pick from here:
I would argue that the first failure mode is far preferable where potentially-security-impacting changes are concerned. There's no review model that is entirely without downsides, but requiring active review at least does not leave difficult-to-resolve ecosystem damage on something as critical as security. |
@joepie91 The failure modes argument is, of course, a great point. I'm not sure it tips the scales for me--although it might--but I certainly won't try to dispute it. AFAICT, this whole thing is mostly due to #20816. So I do want to point out that that PR did get a ✅from a Security WG member, so this policy change wouldn't have prevented that. (Of course, for some, that may argue for a stronger version of this policy.) |
@MylesBorins you make some good points about things being potentially broader. I think it would be good though to see if we can get consensus on something more limited (for example crypto additions) as a first step, and once we are comfortable with that then look at other ways to improve/have a better safety net overall on the security front. |
As to the discussion the failure mode, I'm definitely in favor of erring on the side of slowing things down a bit. That would not be the case in other areas, but for security I think its justified. |
Oh, another suggestion along the tools-not-rules line of thinking:
This and the GitHub bot suggestion have the advantage of being things that can't be forgotten about. |
I like the suggestions of having node-core-utils warn, but not sure that removes the reason/benefit of having requirement for additional reviewers. |
@nodejs/tsc would be good to have a few more voices in the discussion. |
COLLABORATOR_GUIDE.md
Outdated
usage. | ||
|
||
These commits must have an approval from at least one member from the | ||
[security working group](https://github.com/nodejs/security-wg). |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Like others, I am fine with the recommendation around taking increased care, but I'm not sure if the Security-WG is the correct venue. The @nodejs/cryto team already exists for coverage of the crypto and openssl related items, and the @nodejs/security team already handles Node.js core security related items. Perhaps an alternative here would be to require approval from at least one member of the Security WG or either @nodejs/crypto or @nodejs/security?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I was under the impression that the Security WG and @nodejs/security were one and the same thing. Is this not the case?
Other than that, requiring approval from either the crypto or security team seems like a good idea.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I was under the impression that the Security WG and @nodejs/security were one and the same thing. Is this not the case?
@joepie91 Not really, there is @nodejs/security and @nodejs/security-wg.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I was under the impression that the Security WG and @nodejs/security were one and the same thing. Is this not the case?
They are not the same thing. I suggested not naming the WG "Security" to avoid exactly this kind of confusion. I don't know what I would name it though. Maybe "Security-and-Something-Else WG" or "Something-Else and Security WG".
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Oh, I guess I should explain the difference. Or at least provide links. security-wg
is the Security Working Group consisting largely of interested parties but not necessarily (or even largely) people who do a lot of work themselves on Node.js core. More details are at https://github.com/nodejs/security-wg.
Meanwhile, security
is the team briefly described at https://github.com/nodejs/security-wg/blob/master/processes/security_team_membership_policy.md. That team existed before security-wg
.
COLLABORATOR_GUIDE.md
Outdated
@@ -378,6 +379,16 @@ multiple commits. Commit metadata and the reason for the revert should be | |||
appended. Commit message rules about line length and subsystem can be ignored. | |||
A Pull Request should be raised and approved like any other change. | |||
|
|||
### Additions to the Cryptography and Security APIs | |||
|
|||
Semver-minor commits that add or change cryptograpy/security APIs should be |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Is semver-major ⊂ semver-minor? Otherwise, I don't see a reason not to include semver-major here.
An alternative idea for discussion: |
(I hasten to add that there are people on the Security WG now who are extremely knowledgable about security and Node.js. Didn't mean to imply that it consists primarily of people who aren't or anything like that!) |
@Trott I completely agree, and even if we assume that there is a group of "security experts", that doesn't mean that they are also "crypto experts".
@joepie91 That would be the security WG (as far as I know, not entirely sure anymore), but being a member of that WG still doesn't require you to be a professional in that area.
I think there are some differences, though I might be wrong here. To review the API design, one doesn't need to fully understand all the native code, OpenSSL routines etc., but one should probably be familiar with security, cryptography, JavaScript, the Node.js ecosystem and established conventions. Side note: We are effectively already kind of requiring reviews from the people in @nodejs/crypto since they are more or less the only people who frequently review crypto PRs, and semver-major PRs also need to be approved by the TSC. |
It seems that non-working-group teams are typically used as groups to ping on specific issues and PRs related to their specialties. In this case, I think if the process is modified to require the approval of folks on specific teams, then it should be the non-working-group teams (i.e. @nodejs/crypto and/or @nodejs/security) rather than @nodejs/security-wg, since the WG's purpose is primarily defining security policy, rather than being a collective of expertise with respect to security-related code. |
@joepie91 (cc @Trott)
Of those, I don't think that this falls into We need more outside expertise on this, especially from people that are dealing with ecosystem stuff, e.g. which analyze issues in ecosystem packages and deal with ecosystem reports. I would rely on @Trott, questioning
Everything said in that citation is both obvious and can be applied to any other group. That is why it's not an argument (imo). What matters is, in my opinion:
Upd: Re: |
@Trott (cc @MylesBorins) Do I understand your concerns correct that the main part of the problem is blocking against the Security WG review and that it's not entirely clear who exactly and how exactly does that? Perhaps a somewhat more flexible way that would not require indefinite blocks would be to:
@joepie91 @Trott @MylesBorins @mhdawson Thoughts? |
@ChALkeR That's roughly what I suggested in #21896 (comment), @joepie91 believes this might lead to things landing without being properly reviewed by the security WG since we aren't blocking on it. |
@tniessen @joepie91 Ok, I see the concern here. The problem is that we don't want potential security-affecting changes to slip by fast without proper review. The other problem is that people could ignore non-important changes and avoid giving reviews becaue they don't want to bother with testing out the code, for example, and we don't want to block indefinitely in that situation. Perhaps we could require "tagged+cc for 72h + approval from security-wg member OR tagged+cc for 120h + approval from a TSC member" then? That should both give more time, not block indefinitely, and encourage pinging security-wg more actively. |
@ChALkeR Previous conversation in this pull request made it clear (IMO anyway) that it was not obvious to everyone that there is no requirement that members of the Security WG have expertise in security.
Agreed. I wrote: "In and of itself, that might not be a reason to oppose this proposal. But hopefully exposing different understandings and assumptions can help clarify issues and achieve a resolution sooner." I didn't expect the membership stuff to persuade anyone of anything as much as I thought it might facilitate clearer communication between the two points of view as there seemed to be different assumptions. Another thing that is obvious to a lot of us but maybe not obvious to everyone: Currently, no other group has special review authority as is being proposed here. Again, not an argument against it: You can make the case that security concerns deserve special treatment that doesn't apply elsewhere in the code base. But I think the fact that this is unprecedented may explain some of the pushback. |
I recall there was some discussion some time ago about streams PRs requiring sign off from nodejs/streams. It was never formalized -- I don't recall why. Edit: Proposal: Require Streams WG to sign off on streams PRs |
I agree with all of @ChALkeR arguments, and believe that we should also follow the same with |
@Trott I'll point you to: https://github.com/nodejs/node/blob/master/doc/guides/adding-new-napi-api.md There seem to be 3 different discussions:
It may make sense to tackle the discussion in that order as they each depend on agreement on the previous one. My personal feeling is that extra review is warranted for SemVer minor security PRs and I'd be in favor or a broader versus narrower review. In my mind we want to get as many eyes on the PR as opposed to arguing if a particular group is qualified or not. We generally trust people to review in the areas they are competent in (whether that be API design, implementation etc.) So if there are multiple potential groups/team then we should require the all be notified and even possibly require some approval from them. When we talk about 72hours for review I'm fine for most PRs, but I think we can/should take a bit more time in some cases and this is one of them. Otherwise its just too easy to miss out on what may be very valuable input. |
@mhdawson So, subtle but crucial ambiguity: It's not clear to me if that n-api doc and this security/crypto proposal here require sign-off from WG/team members who are also Collaborators or if sign-off from any member of the team is sufficient. At face value, it would seem to be any member, but sign-off from non-Collaborators is sufficiently novel that it should probably be made explicit if that is the intention. |
Regarding the n-api doc: #21877 (comment) |
@Trott I assume you mean "core" collaborators. It was not intended to be limited to people who were both core collaborators and members of the security-wg or team. I agree its good to clarify, before I do that do you see that as more problematic? I'd like to figure out if we are trending towards agreement that we should have additional review or we still don't have any agreement on that front. |
Adhereing to the standard review and landing process for node PRs (at least one approval from core collaborator; x hours before landing; weekend considerations etc.) + additional review from members of WG when required, where they may not be a core collaborator looks good to me. By leveraging WG member's expertise and insights, we are only strengthening our PRs. |
@gireeshpunathil I see your point for sure, but how many variations on the standard review and landing process before it's too unwieldy and everyone routinely ignores the variations? If this lands and if one accepts the N-API one as binding policy, that means we now have three different requirements depending on what kind of changes you're making. I would argue that three may already be too many in that people will forget about the policy and violate it. (Conceivably, any issues there could be solved via tooling. If we get the commit queue together, it can enforce restrictions on what can land, although it's a bit difficult to imagine how the requirements in this particular proposal could be followed algorithmically. What constitutes a "security API" is not easy to define with precision.) |
@mhdawson I'm going to punt and say let's see what other folks who have expressed concerns (specifically about Security WG review authority) are thinking: @tniessen @jasnell @MylesBorins @bengl |
@Trott - acknowledge on the issues at ground (per-PR policy eneforcement). But given the intent (tightening the security posture of our project) how about we move forward with this and improve over time? I see you already ping WGs for PRs based on perceived security implications. |
I share @Trott's concern that we are fragmenting our policies here, and I am still not sure whether this is the best way to go. I won't object to this decision and suggest we try it out. If it does not improve the situation or blocks PRs due to lack of approval, we can still revert the decision. |
Got some input from @MylesBorins today. Will take a cut at an updated version when I get back from holiday next week. |
@MylesBorins updated based on our discussion, please take a look. |
For Semver-minor commits changing cryptography APIs, they must have | ||
an approval from at least one member from the crypto team. | ||
|
||
For Semver-minor commits changing Security API's(other than those |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
API's
-> APIs
like it is elsewhere?
For Semver-minor commits changing cryptography APIs, they must have | ||
an approval from at least one member from the crypto team. | ||
|
||
For Semver-minor commits changing Security API's(other than those |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
this seems extremely broad. What are our security APIs?
@@ -378,6 +379,24 @@ multiple commits. Commit metadata and the reason for the revert should be | |||
appended. Commit message rules about line length and subsystem can be ignored. | |||
A Pull Request should be raised and approved like any other change. | |||
|
|||
### Additions to the Cryptography and Security APIs | |||
|
|||
Semver-minor commits that add or change cryptograpy APIs and Security APIs |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Does tls
fall under "security" or "crypto", and if it's "crypto" then are there any "security APIs" at all?
What is the status on this? |
I'm going to close this one and label it stalled, but feel free to comment or re-open if you intend to continue discussion to make the case for it. |
Some recent dicsussions point to it being
a good idea to require reviews from the security-wg
for security related PRs. See #21766
Add this requirement to the collaborator guide.
Checklist
make -j4 test
(UNIX), orvcbuild test
(Windows) passes