Implement upgrade edges support #275

jmprusi · 2023-06-27T10:23:44Z

Description

Adds upgrade edges support to operator-controller

Annotates bundleDeployments with the original operator information (packageName, Channel, Version)
Adds a new constraint that uses the packages "replaces" value to ensure that the upgrade path between the previous version and the new one is valid.

Closes #268
Closes #267

Reviewer Checklist

API Go Documentation
Tests: Unit Tests (and E2E Tests, if appropriate)
Comprehensive Commit Messages
Links to related GitHub Issue(s)

codecov · 2023-07-06T12:18:41Z

Codecov Report

Merging #275 (552f129) into main (949c06a) will decrease coverage by 18.12%.
The diff coverage is 72.34%.

@@             Coverage Diff             @@
##             main     #275       +/-   ##
===========================================
- Coverage   80.69%   62.58%   -18.12%     
===========================================
  Files          18       21        +3     
  Lines         803      890       +87     
===========================================
- Hits          648      557       -91     
- Misses        112      241      +129     
- Partials       43       92       +49

Flag	Coverage Δ
e2e	`62.58% <72.34%> (+6.79%)`	⬆️
unit	`?`

Flags with carried forward coverage won't be shown. Click here to find out more.

Files Changed	Coverage Δ
internal/resolution/variablesources/operator.go	`68.42% <ø> (-10.53%)`	⬇️
...al/resolution/variablesources/installed_package.go	`62.06% <62.06%> (ø)`
internal/resolution/util/predicates/predicates.go	`40.00% <66.66%> (-31.43%)`	⬇️
internal/resolution/entities/bundle_entity.go	`56.77% <68.75%> (-38.55%)`	⬇️
...al/resolution/variablesources/bundle_deployment.go	`71.42% <71.42%> (ø)`
internal/controllers/variable_source.go	`100.00% <100.00%> (ø)`
...nternal/resolution/entitysources/catalogdsource.go	`59.70% <100.00%> (+1.23%)`	⬆️
internal/resolution/util/sort/sort.go	`54.28% <100.00%> (-40.00%)`	⬇️
internal/resolution/variables/installed_package.go	`100.00% <100.00%> (ø)`
...lution/variablesources/bundles_and_dependencies.go	`50.00% <100.00%> (-36.67%)`	⬇️

... and 6 files with indirect coverage changes

internal/resolution/entities/bundle_entity.go

perdasilva · 2023-07-17T11:51:48Z

internal/resolution/entities/bundle_entity_test.go

@@ -269,6 +265,27 @@ var _ = Describe("BundleEntity", func() {
 		})
 	})

+	Describe("Replaces", func() {


perdasilva · 2023-07-17T11:52:13Z

internal/resolution/entitysources/catalogdsource.go

@@ -106,6 +110,9 @@ func getEntities(ctx context.Context, client client.Client) (input.EntityList, e
 				if catalogScopedEntryName == bundle.Name {
 					channelValue, _ := json.Marshal(property.Channel{ChannelName: ch.Name, Priority: 0})
 					props[property.TypeChannel] = string(channelValue)
+					// TODO(jmprusi): Add the proper PropertyType for this


should we address this todo?

I know the following suggestion is a bigger refactor, and I'm not suggesting we do it now, but I think we should refactor things such that there are:

variable source for bundles

variable source for channels

Bundle entities would no longer carry upgrade edge or channel membership information. Rather, channel variables would incorporate bundle membership and upgrade edges.

IMO, something like my suggestion should happen almost immediately after this PR lands so that we set ourselves up for a more extensible future where different channel implementations can exist and we don't have to keep adding synthetic properties to bundles (which might have undesirable effects if/when we support more arbitrary contraint definitions in the future)

perdasilva · 2023-07-17T11:53:48Z

internal/resolution/variables/installed_package.go

+	return r.bundleEntities
+}
+
+func NewInstalledPackageVariable(bundleImage string, bundleEntities []*olmentity.BundleEntity) *InstalledPackageVariable {


should we add a unit test for this function?

Seems like a unit test was added here.

internal/resolution/variablesources/bundle_deployment.go

perdasilva · 2023-07-17T11:55:29Z

internal/resolution/variablesources/bundle_deployment.go

+	}
+}
+
+func (o *BundleDeploymentVariableSource) GetVariables(ctx context.Context, entitySource input.EntitySource) ([]deppy.Variable, error) {


we should probably have good test coverage of this function, wdyt?

done, CI failing due to codecov issues..

perdasilva · 2023-07-17T11:56:58Z

internal/resolution/variablesources/installed_package.go

+	}
+
+	// sort by channel and version
+	// TODO: this is a bit of a hack and it assumes a well formed catalog.


could you please create issue/s for these todos and link them in the comments here?

perdasilva

Looks really good ^^ Thank you for this! I know it was tough. I think we just need to address a couple of the todos, add issue/s for the others, and maybe a little bit more unit test coverage. Great work!

Signed-off-by: Joaquim Moreno Prusi <joaquim@redhat.com> Signed-off-by: Per Goncalves da Silva <pegoncal@redhat.com> Co-authored-by: Per Goncalves da Silva <pegoncal@redhat.com>

awgreene

/approve

awgreene · 2023-07-27T13:48:06Z

internal/resolution/entities/bundle_entity_test.go

@@ -296,4 +313,46 @@ var _ = Describe("BundleEntity", func() {
 			Expect(err.Error()).To(Equal("error determining bundle mediatype for entity 'operatorhub/prometheus/0.14.0': property 'olm.bundle.mediatype' ('badtype') could not be parsed: invalid character 'b' looking for beginning of value"))
 		})
 	})
+
+	// Increase test coverage


Nit: I would remove this or add a TODO if this is meant to be done in the future, non-blocking though.

awgreene · 2023-07-27T13:56:16Z

internal/resolution/variables/installed_package.go

+	return r.bundleEntities
+}
+
+func NewInstalledPackageVariable(bundleImage string, bundleEntities []*olmentity.BundleEntity) *InstalledPackageVariable {


Seems like a unit test was added here.

perdasilva · 2023-08-03T13:53:05Z

internal/resolution/entitysources/catalogdsource.go

@@ -106,6 +110,9 @@ func getEntities(ctx context.Context, client client.Client) (input.EntityList, e
 				if catalogScopedEntryName == bundle.Name {
 					channelValue, _ := json.Marshal(property.Channel{ChannelName: ch.Name, Priority: 0})
 					props[property.TypeChannel] = string(channelValue)
+					// TODO(jmprusi): Add the proper PropertyType for this
+					replacesValue, _ := json.Marshal(replacesProperty{Replaces: b.Replaces})
+					props["olm.replaces"] = string(replacesValue)


should we move this to a constant? (olm.replaces)

perdasilva

lgtm - small non-blocking issue around a magic string. But we can tackle that on in the next PR ^^

m1kola · 2023-08-04T10:41:47Z

internal/resolution/variables/installed_package.go

+	id := deppy.IdentifierFromString(fmt.Sprintf("installed package %s", bundleImage))
+	entityIDs := make([]deppy.Identifier, 0, len(bundleEntities))
+	for _, bundle := range bundleEntities {
+		entityIDs = append(entityIDs, bundle.ID)
+	}
+	return &InstalledPackageVariable{
+		SimpleVariable: input.NewSimpleVariable(id, constraint.Mandatory(), constraint.Dependency(entityIDs...)),
+		bundleEntities: bundleEntities,
+	}
+}


@jmprusi what is the difference between this and RequiredPackageVariable?

They are semantically different concepts that get translated to the same sort of structure (variable with mandatory and dependency constraints). I guess we could separate that out as a concept if we want to simplify the code here a bit. wdyt?

They are semantically different concepts that get translated to the same sort of structure (variable with mandatory and dependency constraints).

@perdasilva Is it just semantic difference? Or are they/will they be processed differently? As far as I understnad - they are exactly the same today.

I guess we could separate that out as a concept if we want to simplify the code here a bit. wdyt?

For now I'm just trying to understand how it works. But I don't follow - what do you mean by separating? They are already separate structures.

For now it is. It could be that these two evolve on different paths. I meant separate out the pattern of a Variable with a Mandatory and Dependency constraints. I think it could be a common pattern. Then we can just typedef the required and installed package variables as that.

Thanks for the clarification.

I meant separate out the pattern of a Variable with a Mandatory and Dependency constraints. I think it could be a common pattern. Then we can just typedef the required and installed package variables as that.

Sounds like a good idea.

openshift-ci bot added the do-not-merge/work-in-progress Indicates that a PR should not merge because it is a work in progress. label Jun 27, 2023

openshift-merge-robot added the needs-rebase Indicates a PR cannot be merged because it has merge conflicts with HEAD. label Jun 27, 2023

jmprusi force-pushed the jmprusi/upgrade-edges branch from 04efb43 to 9892759 Compare June 27, 2023 14:48

openshift-merge-robot removed the needs-rebase Indicates a PR cannot be merged because it has merge conflicts with HEAD. label Jun 27, 2023

jmprusi force-pushed the jmprusi/upgrade-edges branch from 9892759 to 92db9ba Compare June 27, 2023 14:49

jmprusi changed the title ~~WIP - Implement upgrade edges support~~ Implement upgrade edges support Jun 27, 2023

openshift-merge-robot added the needs-rebase Indicates a PR cannot be merged because it has merge conflicts with HEAD. label Jun 29, 2023

jmprusi force-pushed the jmprusi/upgrade-edges branch from 92db9ba to 26e844e Compare July 3, 2023 15:03

openshift-merge-robot removed the needs-rebase Indicates a PR cannot be merged because it has merge conflicts with HEAD. label Jul 3, 2023

jmprusi force-pushed the jmprusi/upgrade-edges branch 6 times, most recently from fff05ac to a6205a7 Compare July 6, 2023 12:15

jmprusi force-pushed the jmprusi/upgrade-edges branch 6 times, most recently from fd5be1f to 2da8d70 Compare July 12, 2023 11:36

jmprusi marked this pull request as ready for review July 12, 2023 11:36

openshift-ci bot removed the do-not-merge/work-in-progress Indicates that a PR should not merge because it is a work in progress. label Jul 12, 2023

perdasilva reviewed Jul 17, 2023

View reviewed changes

internal/resolution/entities/bundle_entity.go Outdated Show resolved Hide resolved

perdasilva reviewed Jul 17, 2023

View reviewed changes

internal/resolution/variablesources/bundle_deployment.go Outdated Show resolved Hide resolved

perdasilva reviewed Jul 17, 2023

View reviewed changes

perdasilva requested changes Jul 17, 2023

View reviewed changes

jmprusi force-pushed the jmprusi/upgrade-edges branch from 2da8d70 to 3be9c40 Compare July 18, 2023 13:03

jmprusi requested a review from a team as a code owner July 18, 2023 13:03

jmprusi force-pushed the jmprusi/upgrade-edges branch 9 times, most recently from e87975d to d6084e3 Compare July 25, 2023 10:50

Implement support for upgrade Edges

a9c0905

Signed-off-by: Joaquim Moreno Prusi <joaquim@redhat.com> Signed-off-by: Per Goncalves da Silva <pegoncal@redhat.com> Co-authored-by: Per Goncalves da Silva <pegoncal@redhat.com>

jmprusi force-pushed the jmprusi/upgrade-edges branch from d6084e3 to a9c0905 Compare July 25, 2023 13:16

Merge branch 'operator-framework:main' into jmprusi/upgrade-edges

552f129

awgreene approved these changes Jul 31, 2023

View reviewed changes

perdasilva reviewed Aug 3, 2023

View reviewed changes

perdasilva approved these changes Aug 3, 2023

View reviewed changes

awgreene added this pull request to the merge queue Aug 3, 2023

Merged via the queue into operator-framework:main with commit c145c46 Aug 3, 2023
7 of 9 checks passed

m1kola reviewed Aug 4, 2023

View reviewed changes

m1kola mentioned this pull request Aug 4, 2023

entity for package "project-quay" not found in solution when trying to upgrade to the next version using replaces #321

Closed

joelanford mentioned this pull request Aug 31, 2023

[Epic] Initial upgrade support #377

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Implement upgrade edges support #275

Implement upgrade edges support #275

jmprusi commented Jun 27, 2023 •

edited

Loading

codecov bot commented Jul 6, 2023 •

edited

Loading

perdasilva Jul 17, 2023

perdasilva Jul 17, 2023

joelanford Jul 18, 2023

joelanford Jul 18, 2023

perdasilva Jul 17, 2023

awgreene Jul 27, 2023

perdasilva Jul 17, 2023 •

edited

Loading

jmprusi Jul 25, 2023

perdasilva Jul 17, 2023

perdasilva left a comment

awgreene left a comment

awgreene Jul 27, 2023

awgreene Jul 27, 2023

perdasilva Aug 3, 2023

perdasilva left a comment

m1kola Aug 4, 2023

perdasilva Aug 4, 2023

m1kola Aug 4, 2023

perdasilva Aug 4, 2023

m1kola Aug 4, 2023

Implement upgrade edges support #275

Implement upgrade edges support #275

Conversation

jmprusi commented Jun 27, 2023 • edited Loading

Description

Reviewer Checklist

codecov bot commented Jul 6, 2023 • edited Loading

Codecov Report

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

perdasilva Jul 17, 2023 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

perdasilva left a comment

Choose a reason for hiding this comment

awgreene left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

perdasilva left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

jmprusi commented Jun 27, 2023 •

edited

Loading

codecov bot commented Jul 6, 2023 •

edited

Loading

perdasilva Jul 17, 2023 •

edited

Loading