Output is actively misrepresenting the vulnerability.

[ghost]

Type II error, False negative.

Lets say my company has a private package, iconic-spoon-collection this package is not supposed to be public.
This tool would say everything is probably fine if iconic-spoon-collection existed in the public repository.

If the tool knew that iconic-spoon-collection was a private only package it would know it should not public, and the fact that this package is public is a sign that you have been breached.

[joohoi]

Yeah, there's a few things that needs to get done for the tool: providing better guidance how to interpret the results, and to potentially separate the privately scoped packages in the output for manual checking.

Unfortunately the contents of the public repositories cannot be inspected, so this contextual knowledge will be left with the user.

[ghost]

A way forward would be to change the output, for each step of the program we write out the result of our check per package.

Maybe something like.

  iconic-spoon-collection is public

On further reflection maybe generate a csv or similar report, allowing developers to go through each package and the result of the lookup.

Package,{repository}
iconic-spoon-collection,exists

[joohoi]

I added a pull request that tries to clarify the tools use case and how the results should be interpreted. In case you have improvement ideas, feel free to point them out at #9

What comes to the actual runtime output of the tool, I think the clarifications in the documentation linked above should be sufficient for the developers to act on.

There is no way of an external tool like confused to be able to determine if a package that is found in public repositories is published by the developer, their team, company, malicious actor or a well trusted package author.

It simply reports all the packages that aren't publicly accessible through public repositories.

[ghost]

In my example above.

The fact that private package iconic-spoon-collection is present in a public repository is an indicator that you have been attacked, to which the tool says: [*] All packages seem to be available in the public repositories. Dependency confusion should not be possible.

This is actively deceptive, if you are a security engineer running this on your developers repositories you are told in black and white that everything is fine.

You are actively misrepresenting the what the vulnerability is and what it means, conversely you are saying everything is not fine when the packages are not present in the public repository.

[joohoi]

You are right for the response text. Your argument stands valid for the cases where the namespaces are not controlled by the developer. I'll change the response text to underline that in scope of the PR linked above.

I believe a text like:

[*] All packages seem to be available in the public repositories. 

In case your application uses private repositories please make sure that those namespaces in 
public repositories are controlled by a trusted party.

should do the trick.

[ghost]

You are missing the point.

Let say iconic-spoon-collection and exciting-cake-presentation are two private packages.
iconic-spoon-collection is pwned by someone, and exciting-cake-presentation is not so its still private.

the output of the program will then be.

Issues found, the following packages are not available in public package repositories:
exciting-cake-presentation

The nature of the vulnerability is that someone has publicly claimed a private package and is using that to exploit your system.

[joohoi]

The nature of the vulnerability is that someone has publicly claimed a private package and is using that to exploit your system.

Yes, and that's why the suggested output now instructs the developer to check that they actually control the namespaces.

A completely valid way to mitigate or prevent the issue altogether is to register the private repository namespace entries to public repositories.

[ghost]

I'll just double check, the fact that your program actively tells the user that private package = bad is by design

[joohoi]

I'll just double check, the fact that your program actively tells the user that private package = bad is by design

Yes, that's right in the case when the name is not also squatted in public repositories. The vulnerability is two-fold, and before the resolve order for the package management tools themselves is properly fixed, there remains a high potential of compromise if the package namespaces in public repositories are left unclaimed.

This tool does not check if you have already been exploited, but if your project has potential for it. If you were already exploited, you would have seen build failures related to the malicious packages not exposing the functionalities that are expected from the private package at very least.

[joohoi]

This tool does not check if you have already been exploited, but if your project has potential for it. If you were already exploited, you would have seen build failures related to the malicious packages not exposing the functionalities that are expected from the private package at very least.

Also this is not possible to check externally, but requires manual inspection, just like it's now proposed with the new output text

[ghost]

https://media.giphy.com/media/E3L5goMMSoAAo/giphy.gif

[joohoi]

https://media.giphy.com/media/E3L5goMMSoAAo/giphy.gif

Not very constructive now, eh?

I believe you have misunderstood what the tool is used for or the nature of the vulnerability. I have tried to clarify it in the pull request linked above, as well as the comments here.

I'll reiterate once more:

• It is for checking if your application is currently potentially vulnerable
• It is not for checking if your application has already been exploited, as this needs manual inspection. The output has been refined to clarify this as well.

I'll close this issue as I see it mitigated by the pull request #9

[ghost]

1. I don't expect the program to perform package inspection.
2. You are showing private packages who are still private as issues.

If you want to help the user of the applications tell them what is private and what is public dont tell them i found your problem