New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[WIP] Add support for NormalizedVersionRanges #108

Draft

keshav-space wants to merge 4 commits into nexB:main from keshav-space:range_normalization

Member

keshav-space commented Mar 14, 2023

support normalization of range expression from GitHub, Snyk, GitLab
Discrete range normalization for OSV, DEPS, VulerableCode

keshav-space marked this pull request as draft

March 14, 2023 15:03

keshav-space force-pushed the range_normalization branch from 97c2de4 to e001ca7 Compare

March 14, 2023 15:14

keshav-space marked this pull request as ready for review

March 15, 2023 12:08

keshav-space force-pushed the range_normalization branch from 1018732 to b9664ca Compare

March 15, 2023 12:11

keshav-space mentioned this pull request

[WIP] Support Advisory Comparison in VulnTotal nexB/vulnerablecode#1151

Draft

pombredanne reviewed

View reviewed changes

Member

pombredanne left a comment

Thanks
LGTM overall! we have likely to find a bettr way than using the native intbitset of at least have a fallback to plain Python, I will review further later this WE!

pombredanne requested changes

View reviewed changes

Member

pombredanne left a comment

I think we could:

extract Span as its own mini library also reused in ScanCode? May be this is overakill though
have a fallback to a plain builtin set when intbitset is not installed... and have this also in SCTK

keshav-space force-pushed the range_normalization branch from cda6c25 to e387ecb Compare

April 3, 2023 12:51

Member Author

keshav-space commented Apr 3, 2023

I think we could:

extract Span as its own mini library also reused in ScanCode? May be this is overakill though

have a fallback to a plain builtin set when intbitset is not installed... and have this also in SCTK

Added fallback to builtin set

keshav-space added 4 commits

April 5, 2023 15:35


          Add support for NormalizedVersionRanges

218c61a

- support normalization of range expression from GitHub, Snyk, GitLab
- Discrete range normalization for OSV, DEPS, VulerableCode

Signed-off-by: Keshav Priyadarshi <git@keshav.space>


          Test NormalizedVersionRanges

1a69d07

Signed-off-by: Keshav Priyadarshi <git@keshav.space>


          Add ABOUT and LICENSE file for spans.py

3b3570c

Signed-off-by: Keshav Priyadarshi <git@keshav.space>


          Fallback to builtin set when intbitset is not installed

d88b80c

Signed-off-by: Keshav Priyadarshi <git@keshav.space>

keshav-space force-pushed the range_normalization branch from e387ecb to d88b80c Compare

April 5, 2023 10:07

keshav-space requested a review from pombredanne

April 5, 2023 13:34

pombredanne requested changes

View reviewed changes

Member

pombredanne left a comment

Thanks! See some/many comments for your consideration!

src/univers/normalized_range.py Show resolved Hide resolved

src/univers/normalized_range.py Show resolved Hide resolved

src/univers/normalized_range.py Show resolved Hide resolved

src/univers/normalized_range.py Show resolved Hide resolved

src/univers/normalized_range.py

+                      if resolved_operator == operator.lt:
+                          return Span(1, index - 1)
+                      if resolved_operator == operator.gt:
+                          return Span(index + 1, len(version_map))

Member

pombredanne Apr 10, 2023

May be you could create a variable for this?

src/univers/normalized_range.py

+                      Return dict mapping version to integer.
+                      """
+                      if purl_type not in VERSIONS_BY_PACKAGE_TYPE:
+                          return

Member

pombredanne Apr 10, 2023

We should error out with an exception IMHO

src/univers/normalized_range.py

+                      sorted_versions = sorted([version_type(i) for i in versions])
+                      sorted_versions = [version.string for version in sorted_versions]
+                      index = list(range(1, len(sorted_versions) + 1, 1))
+                      return dict(zip(sorted_versions, index))

Member

pombredanne Apr 10, 2023

May be {v: i for i, v in enumerate(versions, 1)}

src/univers/normalized_range.py

+                          return
+                      version_type = VERSIONS_BY_PACKAGE_TYPE.get(purl_type)
+                      sorted_versions = sorted([version_type(i) for i in versions])

Member

pombredanne Apr 10, 2023

What about sorting using a key instead? May be something like this?

# sort using the version class of the purl type
sorted_versions = sorted(versions, key=version_type)

src/univers/normalized_range.py

+                      return dict(zip(sorted_versions, index))
+                  @classmethod
+                  def from_github(cls, range_expression: Union[str, List], purl_type: str, all_versions: List):

Member

pombredanne Apr 10, 2023

This is not GitHub but a GitHub Advisory and same for Gitlab: ghsa, github_advisory gitlab_advisory

src/univers/normalized_range.py

		return version[1:]


		VERSIONS_BY_PACKAGE_TYPE = {

Member

pombredanne Apr 10, 2023

This does double duty with existing code

keshav-space marked this pull request as draft

August 31, 2023 16:21

keshav-space changed the title ~~Add support for NormalizedVersionRanges~~ [WIP] Add support for NormalizedVersionRanges

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment