Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Adds new catalogmetadata package #393

Merged
merged 5 commits into from
Sep 12, 2023

Conversation

m1kola
Copy link
Member

@m1kola m1kola commented Sep 4, 2023

Description

Adding a new package to deal with catalog metadata.

We will be using it to replace existing enttiy soruces. At the moment catalogmetadata package contains:

  • Types which embed declcfg.Package, declcfg.Channel and declcfg.Bundle.
    Later we will be adding more methods to them (for unmarshalling properties, for example). This PR already extends Bundle type.
  • A function to unmarshal catalog metadata into Package, Channel or Bundle structs.

catalogmetadata/filter subpackage:

  • Filter function which can filter all these 3 types using a given predicate
  • And, Or and Not composite predicates to be used with Filter
  • A set of predicates for bundles

catalogmetadata/client subpackage:

  • A client for fetching bundles (today we get info from kube API server, but later we will switch implementation to use catalogd HTTP server).

catalogmetadata/sort subpackage:

  • Functions to sort catalogmetadata structs (e.g. byndles by version).

This will eventually replace the following:

  • internal/resolution/entities
  • internal/resolution/entitysources
  • internal/resolution/util/predicates
  • internal/resolution/util/sort (most likely)
  • cmd/resolutioncli/entity_source.go

Reviewer Checklist

  • API Go Documentation
  • Tests: Unit Tests (and E2E Tests, if appropriate)
  • Comprehensive Commit Messages
  • Links to related GitHub Issue(s)

@openshift-ci openshift-ci bot added the do-not-merge/work-in-progress Indicates that a PR should not merge because it is a work in progress. label Sep 4, 2023
@m1kola m1kola mentioned this pull request Sep 4, 2023
4 tasks
@codecov
Copy link

codecov bot commented Sep 4, 2023

Codecov Report

Patch coverage: 95.18% and project coverage change: +2.00% 🎉

Comparison is base (1ec4a23) 81.64% compared to head (d555222) 83.65%.
Report is 4 commits behind head on main.

Additional details and impacted files
@@            Coverage Diff             @@
##             main     #393      +/-   ##
==========================================
+ Coverage   81.64%   83.65%   +2.00%     
==========================================
  Files          21       27       +6     
  Lines         937     1101     +164     
==========================================
+ Hits          765      921     +156     
- Misses        119      123       +4     
- Partials       53       57       +4     
Flag Coverage Δ
e2e 61.92% <ø> (+0.13%) ⬆️
unit 80.29% <95.18%> (+3.05%) ⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

Files Changed Coverage Δ
internal/resolution/entities/bundle_entity.go 98.27% <ø> (-0.03%) ⬇️
...ternal/catalogmetadata/filter/bundle_predicates.go 89.18% <89.18%> (ø)
internal/catalogmetadata/client/client.go 89.47% <89.47%> (ø)
internal/catalogmetadata/filter/filter.go 100.00% <100.00%> (ø)
internal/catalogmetadata/sort/sort.go 100.00% <100.00%> (ø)
internal/catalogmetadata/types.go 100.00% <100.00%> (ø)
internal/catalogmetadata/unmarshal.go 100.00% <100.00%> (ø)

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

@m1kola m1kola force-pushed the catalogmetadata branch 4 times, most recently from 9534267 to a39d3fc Compare September 5, 2023 09:37
@stevekuznetsov
Copy link
Member

Is there a quick summary of why this change is being proposed?

@m1kola
Copy link
Member Author

m1kola commented Sep 5, 2023

@stevekuznetsov we are looking into ways to remove entities and entity sources from operator-controller. This is just me playing with how we might interact with the catalog data. See #375 for actual removal (still WIP).

@m1kola m1kola force-pushed the catalogmetadata branch 6 times, most recently from 482542a to b2174a4 Compare September 8, 2023 13:30
@m1kola m1kola marked this pull request as ready for review September 8, 2023 16:22
@m1kola m1kola requested a review from a team as a code owner September 8, 2023 16:22
@openshift-ci openshift-ci bot removed the do-not-merge/work-in-progress Indicates that a PR should not merge because it is a work in progress. label Sep 8, 2023

// ByVersion is a sort "less" function that orders bundles
// in inverse version order (higher versions on top).
func ByVersion(b1, b2 *catalogmetadata.Bundle) bool {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should we not also sort by package name as we did before with entities? If we don't, what I'm imagining is our list of:

- packageName: a-package
  version: 1.0.0
- packageName: b-package
  version: 2.0.0
- packageName: a-package
  version: 3.0.0

would get sorted to:

- packageName: a-package
  version: 3.0.0
- packageName: b-package
  version: 2.0.0
- packageName: a-package
  version: 1.0.0

instead of getting grouped up by name followed by version:

- packageName: a-package
  version: 3.0.0
- packageName: a-package
  version: 1.0.0
- packageName: b-package
  version: 2.0.0

Copy link
Member Author

@m1kola m1kola Sep 11, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think ordering by package is not important. We have the following use cases for old bundle entity based sort.ByChannelAndVersion at the moment:

  • RequiredPackageVariableSource - here we always filter by pacakge name so all bundles in sorting will have the same package name. For package name filtering we set a predicate here.
  • Twice in InstalledPackageVariableSource - on the first occasion we call it on a slice filtered by bundle image (so most likely same package) and on the second occasion we call it on a slice filtered by replaces which should also be within the same package
  • BundlesAndDepsVariableSource - here is the use case where we might sort different packages. A bundle might require multiple GVKs from different pacakge. But, I think, as long as dependencies are sorted by version from newest to oldest we should be ok.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Awesome that makes perfect sense, thanks for the explanation!

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm not 100% that this is correct, but let's go with it and fix it if it doesn't work. Trying to not overcomplicate it :)

At the moment it contains:
* Types which embed `declcfg.Package`, `declcfg.Channel` and `declcfg.Bundle`.
  Later we will be adding more methods to them (for unmarshalling properties, for example)
* A function to fetch metadata by scheme & catalog name
* `Filter` function which can filter all these 3 types using a given predicate
* `And`, `Or` and `Not` composite predicates to be used with `Filter`

Later we might expand/change this package to be more convinient in use with variable sources.

Signed-off-by: Mikalai Radchuk <mradchuk@redhat.com>
Signed-off-by: Mikalai Radchuk <mradchuk@redhat.com>
Signed-off-by: dtfranz <dfranz@redhat.com>
Signed-off-by: Mikalai Radchuk <mradchuk@redhat.com>
Signed-off-by: Mikalai Radchuk <mradchuk@redhat.com>
Signed-off-by: Mikalai Radchuk <mradchuk@redhat.com>
Copy link
Contributor

@dtfranz dtfranz left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM!

@m1kola m1kola added this pull request to the merge queue Sep 12, 2023
Merged via the queue into operator-framework:main with commit 2ddd741 Sep 12, 2023
12 checks passed
@m1kola m1kola deleted the catalogmetadata branch September 12, 2023 09:37
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

3 participants