SPDX and Non SPDX License Support (Technical Steering Committee) #3949
Comments
To reiterate some of what @othiym23 said on the original issue: the Node TSC does not control the npm CLI project. The Node project is a downstream consumer of the CLI, similar to our relationship with the V8, c-ares, and libuv projects. While we can make requests of those upstream projects, those requests are in no way binding, and the maintainers of those projects are free to ignore them. In short, @othiym23 is the person you have to convince; I do not think it is the TSC's place to convince him on your behalf. I'll leave this issue open for 24 hours, and if no other contributors object in that time, I'll close it. |
|
@chrisdickinson ... While you are correct about this being an upstream issue, I for one would appreciate giving the @nodejs/ctc an opportunity to review and discuss the issue before closing. |
|
@chrisdickinson I disagree. The package.json is integral to node and this impacts upon the experience of all node developers. If would hope that you would be willing to engage on such matters when such collaboration and discussion are warranted. Further, the technical committee is comprised of members including those that produce proprietary software in addition to open sources – whose own code is impacted by these recent decisions. The current solution excludes many forms of licensing directly from the metadata. We are all interested in appropriate solutions that work for the community. There are times when reason is not being heard. For this, neutral committees can be an appropriate forum to engage, even to recommend on an issue with reaching impact. No one is suggesting you control NPM or can force any changes. I am suggesting that you give committee members an opportunity to make their own mind up on whether they feel this ought to be debated before prematurely ending this. Please debate the issue, form a consensus and provide a recommendation is what I am suggesting – but I urge you to act on this since it will a long term impact on search and discovery for developers and tooling. |
|
@chrisdickinson @othiym23 ... perhaps either of you would be willing to at least give the CTC an overview of the changes that were made here and the reasoning behind them? We don't necessarily need to bring it up on the next CTC meeting, it would just be good to make sure everyone is on the same page. |
|
Also for context: https://github.com/npm/npm/releases/tag/v2.10.0 cc @jasnell |
|
Hey @kemitchell ! Thanks for jumping in! I know the SPDX stuff has been in for a while, just want to make sure everyone is aware of the context before we close out the issue, just in case there are questions. I added a few links for context in #3949 (comment). The main thing I'd like to see before closing this is just a quick overview of what the changes basically were and why. As @chrisdickinson said, there's really nothing we can or should do here but it's worth having things documented since there's obviously some question about it. |
|
Here is some background:
I do encourage you to read through those threads – there's a lot of back and forth there, and it took us a little while to get everything right, which was kind of painful for teams, like StrongLoop's, who were doing the right thing but maybe moved a little faster than the npm team was expecting. My motivation for accepting Kyle's changes was pretty simple: there was no meaningful validation of that field before, which made it very difficult to write automated license checkers and validators for My primary reason for turning down further requests for changing this behavior is because I do believe that the current tradeoffs imposed by the field are acceptable, and I really don't want to cause more churn around what's a very small corner of npm's functionality, given that the current format has only just now established itself. I also believe that I've done my utmost to give everyone's concerns and feedback a fair hearing, and that this is an area where, as a product manager, I need to be able to draw a line and say, this work is done for now, and the team (and community it serves) has more important issues to address. |
|
Also, for documentation of how the feature works in practice, here are the docs that @kemitchell wrote for how the field works. |
|
@othiym23 ... awesome, thank you. That's likely enough context to make sure everyone is up to speed. As you know the whole commercial license thing is important to me so I wanted to make sure that this was at least discussed here so that we'd have somewhere to point if the conversation came up again. @chrisdickinson ... no objections to getting this closed whenever you feel it's necessary unless anyone else feels need to weigh in. |
|
@jasnell Cool, I'll plan on closing this tomorrow around ~4PM PST if there are no objections from contributors. |
|
The core concern here is that SEE LICENSE IN defeats the capability of discovering anything other than SPDX licenses in metadata since other forms of licensing are fully excluded from the license property. Further, this is backwards incompatible to more than 5 years worth of data. SPDX is a subset of open licenses, there are many more forms of licensing than those included in SPDX that will now be excluded. If someone could tell me how this is helpful, I am prepared to listen. |
|
@scriptjs Understood. But keep in mind that this has been included in npm since May/June and has shipped in the LTS v4.2 and Stable v5.0 as well as prior versions of io.js. Given the fact that this is in v4.2, we are committed to continue supporting the current mechanism until at least April 2017. There's really not much we're going to be able to do about it here. @chrisdickinson is right, you have to try to convince npm to make the change at that level. Now that you've documented the issue here and the issue is going to be kept open for a bit, all @nodejs/collaborators will have the opportunity to see your concerns and weigh in if they'd like (and I'd encourage them to do so if this is an issue they feel strongly about). Thank you for bringing the issue up! |
|
@jasnell I am not certain it is the case for LTS v4.2 since this shipped with npm 2.x.x as far as I know. It has been a while since I used 4.x, perhaps @othiym23 could verify this. I believe this became an issue around 3.1.x on node 5. I recall this only appeared in warnings in the past couple of months when I became concerned about this. |
|
As @kemitchell and @jasnell pointed out above, the current license validation support was added in |
mscdex
added
the
npm
|
@othiym23 Can you clarify whether changes were made post Node 4 LTS. I believe you removed the UNLICENSED capability already. |
|
I second @othiym23's summary, with two notes: I can't sleep on any summary that doesn't mention:
Then me. My contribution was disproportionate only in that I wrangled Git and GitHub. God bless StrongLoop. It's absolutely true they paid a price in inconvenience for moving fast in support of the first validation PR. So grateful for their feedback. |
|
@kemitchell You do realize StrongLoop did not include both licenses in their dual license supported packages in the license property. They have included a single license of two, the second being proprietary but absent. This demonstrates this is broken. That said, I cannot think there is an expression that would include both without A link for reference: https://github.com/strongloop/strong-remoting/blob/master/package.json In their case one is SPDX and one not. Both would be excluded from discovery. I fail to see this as helpful. I don't think there is a single person saying that licenses should not or cannot be validated against SPDX. But there is a flaw when this is being done in a way that excludes every other license type in exchange for references that mean nothing to people and machines for discovery in metadata. Surely the Linux Foundation can appreciate that the full scope of licensing goes beyond the open licenses included in the subset. This does not make these licenses any less relevant. It would be helpful to recommend solutions to this particular problem than to agree that SPDX is a good start for a license registry that can be used for validation. |
|
|
|
@scriptjs: All: |
|
I have determined a solution in the last hour that will validate without throwing warnings and allows a non-SPDX license within the metadata without conflicting with the SPDX validation scheme. If we can agree on this solution and commit to a decent discussion about how this might evolve for future, I would be satisfied. The solution is as follows and requires no changes to NPM or to the validation scheme. Examples This alleviates the primary concern that non SPDX licenses cannot be included in the package.json metadata for developers and machines to discover. It will still validate free form and what is within brackets could be harvested by tools. Can we agree on this as a recommended solution. ie. You may optionally include the non-SPDX license(s) in brackets following the reference. This solution is not a pretty as first, but it will solve the issue without changes in the long term support. |
|
On the basis of the unwillingness of NPM to compromise on this issue npm/npm#10479, regardless of the impact upon developers and discovery of metadata, you can safely close this issue. It is frustrating that there can be no consultation on this issue at the TSC level since this has been carved out as upstream issue. NPM has a direct impact on the user experience of every node user. To have this dependency for all node users yet for the TSC to have no control over the user experience is terrible frankly. How did we ever get to this point? I will only say that I feel NPM is becoming the elephant in the room. This has been the case for some time. As node was developing, we were plagued by issues and held hostage as it was repaired or scaled. There was the fiasco with funding and now with respect, it is a private enterprise dictating the future of our software and interaction with packages without community consultation. I'll add this is now creeping into a series of new terms for use of our own modules that are being implemented without notice to anyone. https://github.com/npm/policies/commits/master/open-source-terms.md I am no doubt happy that NPM has become more stable due to the efforts of many over time. We are all grateful for that. That said, I believe we have the wrong model here and it is time to change the channel for the future. What is needed is a decentralized approach to module distribution – not one that is centralized and beyond reach as an upstream. I believe this is the issue that should come before the committee for the future of node so that a different future can emerge that puts the community at the center of determining and applying metadata standards that serve the interests of developers. Further, decentralization is the fabric of the Web and offers considerably more safety at scale. Thus, the mandatory requirements of a package.json should not be determined by NPM but by the community. I believe multiple players can and should be involved in delivering modules according to general standards determined by the community to drive the best user experience – but one that remains developer driven. |
|
Closing this now, per my comment here. |
I am referencing dialog and a possible solution for the package.json's license property. Recent changes to the requirements of the package.json imposed by NPM require the use of SDPX licenses while excluding non-SPDX licenses from the manifest. This has hit a nerve with a number of developers in the references identified below.
Alternatives have been proposed (including what follows below). The proposed solutions ensure the inclusion of the license type in the metadata in a way that is agnostic while enabling SPDX validation.
At this point, if a non-SPDX license is used it must be excluded from the package.json with a reference that is of no help for developers or search tools. License discovery is an important issue for all developers that work with node and for tooling that extends beyond NPM.
Further, the form of language NPM has put forward to manage non-SPDX licenses is ambiguous – even to people that speak english as a first language. Consider the following:
NPM does not appear interested in evolving to a solution that is agnostic to the type of license. This feels unilateral. We are presently forced to produce license strings that degrade the quality of the metadata.
The solution put in place by NPM for the license property is inadequate. It forces developers to look outside the manifest to when a license is not classified within SPDX. To be clear, several open and closed source licenses fall outside SPDX. This is not a debate about open or closed licenses or license preferences.
This issue was opened as #10479 npm/npm#10479 and closed again without an appropriate resolution.
I am requesting the Node Technical Steering Committee fully examine this issue to bring an appropriate focus and solution that is neutral and properly considered.
The last issue filed concerning the license field and SPDX follows:
I am opening this issue that was closed while discussion was ongoing for an appropriate solution to #8918. Discussion has been ongoing for months over this in #8918 as well as #8291, #8557, #8773, #8795 so it has touched a nerve. I urge NPM to listen and collaborate for an appropriately considered solution that will work for everyone.
The latest solution recommended is as follows that has the following benefits:
Valid SPDX licenses
Non SPDX licenses
May Emit Warning
Backwards compatible but a SPDX License.
My recommendation is to inform NPM users of the change of the license property and give module developers some time before driving everyone crazy with SPDX warnings as has been done when you imposed it. Perhaps blog about the change first to allow voluntary revisions until a certain date where warnings could be emitted. One way or the other, I urge you to engage users before disturbing software and build systems with noise.
I have not heard anyone come out against SPDX, only the way you have chosen to implement it that is not backwards compatible to about 5 years of data, excludes non SPDX licenses from package metadata, and creates a non standard SPDX description of "SEE LICENSE IN" that makes the language of the metadata awkward. ie.
Metadata is a source of truth and these type of phrases are meaningless and only require more investigation into a repo or package.
The text was updated successfully, but these errors were encountered: