Support Normalization of VersionRange

[keshav-space]

Why Normalize VersionRange?

A VersionRange represents set of segments of package versions. Different VersionRange can represent the same set of segments.

For example, consider the package pkg:npm/foobar with the following versions: ["1.0", "2.1", "2.2", "3.0", "3.1", "5.0"].
We can represent certain segments using the vers expression vers:npm/<=2.2|>=3.0|<3.1|5.0.
This can also be represented as:

• vers:npm/>=1.0|<3.0|>=3.0|<3.1|5.0
• vers:npm/<3.1|5.0
• vers:npm/>=1.0|<=2.2|>=3.0|<3.1|5.0

These different representations make it difficult to validate whether two different VersionRange represent the
same versions of a package.

To effectively compare these ranges, we need to normalize them. The idea is that all the vers referring to
the same set of package versions should normalize to an identical VersionRange.
To achieve this, we take a vers along with all the versions of the package and generate a new vers
such that it contains not arbitrary version segments but only the longest contiguous segments of versions, leading to the same VersionRange expression which is identical for a particular set of versions.

If we apply this normalization to the above example of pkg:npm/foobar:

• vers:npm/<=2.2|>=3.0|<3.1|5.0 would normalize to vers:npm/>=1.0|<=3.0|5.0

The same normalization applies to other VersionRange since they all refer to the same set of versions,
they will all normalize to the exact same VersionRange:

• vers:npm/>=1.0|<3.0|>=3.0|<3.1|5.0 => vers:npm/>=1.0|<=3.0|5.0
• vers:npm/<3.1|5.0 => vers:npm/>=1.0|<=3.0|5.0
• vers:npm/>=1.0|<=2.2|>=3.0|<3.1|5.0 => vers:npm/>=1.0|<=3.0|5.0

---

• Support normalization of vers range expression using all the available package version.
• Support VersionRange from snyk advisory and discrete version.
• Various bug fix in from_gitlab_native from_gitlab_native can't properly parse range expressions for Composer, Maven and NuGet #136
• Bug fix Incorrect __contains__ resolution in VersionRange #137
• Fixes Incorrect affected package in composer advisories vulnerablecode#1516

[pombredanne]

Thanks
LGTM overall! we have likely to find a bettr way than using the native intbitset of at least have a fallback to plain Python, I will review further later this WE!

[pombredanne]

I think we could:

1. extract Span as its own mini library also reused in ScanCode? May be this is overakill though
2. have a fallback to a plain builtin set when intbitset is not installed... and have this also in SCTK

[keshav-space]

I think we could:

1. extract Span as its own mini library also reused in ScanCode? May be this is overakill though
2. have a fallback to a plain builtin set when intbitset is not installed... and have this also in SCTK

Added fallback to builtin set

[pombredanne]

Thanks! See some/many comments for your consideration!

[pombredanne]

Thanks. See review comments.

[pombredanne]

Thank. Looking fine. There are a few nits for your review. And please merge then!