Consolidate code of core component into one repo

[qiujian16]

We currently have different component repos:

• registration
• work
• placement
• registration-operator
• clusteradm
• addon-framework

It is hard to do e2e tests with many components and hard to do dependency check. I propose,we consolidate the code in these repos to one single repo:

• code in the work/registration/placement/operator/clusteradm component is merged into https://github.com/open-cluster-management-io/OCM
• split addon-framework to pure library code and addon-manager code. The addon-manager code is also put into https://github.com/open-cluster-management-io/OCM
• archive the the independent repo

After that we will have

• https://github.com/open-cluster-management-io/api
• https://github.com/open-cluster-management-io/OCM
• https://github.com/open-cluster-management-io/addon-framework (just library, remove unnecessary deps)

With this consolidation, we will have the benefit that

1. we do not need webhook per component, we can have one single webhook server
2. we can build hub and agent as one single binary
3. easier e2e testing
4. better understanding and dev experience for the contributor

note from @deads2k we should validate gates for PR that we do not have cross deps among different components.

[clyang82]

Can we rename https://github.com/open-cluster-management-io/OCM to https://github.com/open-cluster-management-io/ocm? It seems the lower-case repo name is a common practices.

[xuezhaojun]

https://github.com/xuezhaojun/ocm-demo: An example merged repo, keeping the same commit history.

Using commands in this post: https://stackoverflow.com/a/52933095/6371360

[dhaiducek]

I'm on board with consolidating the repos, but I'm not sure having clusteradm in the same repo with the controllers makes sense. It seems like the CLI should have its own repo since it's not a controller/API?

[xuezhaojun]

I'm on board with consolidating the repos, but I'm not sure having clusteradm in the same repo with the controllers makes sense. It seems like the CLI should have its own repo since it's not a controller/API?

But it seems including CLI in the main repo is a common pattern, is there a particular advantage we can gain from not including it?

[dhaiducek]

I wasn't aware including the CLI in the main repo was a common pattern (kubectl was the only one I checked). I was thinking having it separate might make it more discoverable and would keep the ocm repo focused on controllers.

[xuezhaojun]

I checked a few repos like argo-cd, istio and one similar usage repo karmada.

It may not be a common pattern, but if we separate it, we have to maintain 2 e2e tests. If we keep it in ocm, any changes would trigger test cases in the controller and CLI, and we gain confidence in the final user experience.

[mprahl]

I checked a few repos like argo-cd, istio and one similar usage repo karmada.

It may not be a common pattern, but if we separate it, we have to maintain 2 e2e tests. If we keep it in ocm, any changes would trigger test cases in the controller and CLI, and we gain confidence in the final user experience.

Note that you can reuse GitHub workflows from another repo so you could run the same tests in both repos without duplicating them.

[xuezhaojun]

Hello, everyone, since we are going to reach release 0.11.0, now it's a good time to discuss the next executable steps of code consolidation. I made a draft plan, please take a review when you have time, Thanks!

• Release 0.11.0 @qiujian16
• Rename OCM to ocm: Can we rename https://github.com/open-cluster-management-io/OCM to https://github.com/open-cluster-management-io/ocm? It seems the lower-case repo name is a common practices. #139
• Merge the Pull Request: Consolidate code from 0.11.0 #143
  • include pre github actions checks, but doesn't includepost and release.
• Add Post github actions back.
• Add Release github actions back.
• Disable origin repos' GitHub actions on the main branch.
• Announce in channel and mail group.
• Update Readme in the ocm repo and origin repos.
  • Provide a Dev Guidence in the main branch in theocm.
  • Add a note mentioning dev works moved to the ocm repo in origin repos.
    • Add Stop Edit Note and Remove post github action placement#109
• Update website references. Update links point to the repo ocm. open-cluster-management-io.github.io#361

The following task happens on the ocm repo:

• Refactor e2e out of staging.
• Consolating integration test. Refactore integration test #150
• Init Codecoverage. Add: codecov configuration. #166
• Add Owners file.
• Refactor utils pkg(including version, dependencymanagement, etc).

[xuezhaojun]

We have a new PR to relocate code out of staging: #146
The code needs further merging in the future to reduce redundancy.

Change Notes

For all components:

We left the e2e-tests and some helper scripts still under the staging/ folder. If you want to run e2e, for example, placement, try a command like: make -C ./staging/src/open-cluster-management.io/placement test-e2e to specify the Makefile.

For registration-operator:

We have manifests/, deploy, and hack/ of the origin registration-operator repo copied in ocm. The content is unchanged.

For placement, registration, and work:

• The integration-test used to depend on CRDs files under /deploy, now it's depending on CRDs in vendor directly. For example, in placement:
  • Before: https://github.com/open-cluster-management-io/placement/blob/2572e0a55c2d4635b372fbd7dd7382e35983cd56/test/integration/suite_test.go#L39
  • After:

    ocm/test/placement/integration/suite_test.go

    Line 46 in ad38b94

    CRDDirectoryPaths: CRDPaths,

    .
• We have a unified integration-test makefile now, the make test-integration is to run tests of each component one by one:

  ocm/test/integration-test.mk

  Line 49 in ad38b94

  test-integration: ensure-kubebuilder-tools test-registration-operator-integration test-registration-integration test-placement-integration test-work-integration

  .
• Use make build can build binaries for all 4 components, Use make test can run unit-test for all 4 components.

For placement: @haoqing0110

• The benchmark depends on pkg in integration, so we move util pkg up a level.
  • Before: https://github.com/open-cluster-management-io/placement/tree/main/test/integration/util
  • After: https://github.com/open-cluster-management-io/ocm/tree/main/test/placement/util
• The benchmark CRDs is depending on vendor files as well. You may need to run in the root directory of ocm.

Unsolved or affected:

• Release(including OLM release) @zhiweiyin318

[qiujian16]

@xuezhaojun the progress seems good. I think we should have a discussion and start creating some small tasks. e.g.

• refactor e2e out of staging
• further consolidating the integration
• refactor cmd/dependencymagnet/version code

[xuezhaojun]

/close

[openshift-ci]

@xuezhaojun: Closing this issue.

In response to this:

/close

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.