🎉 [Google Summer of Code 2023] - Leverage Phase 2 Plugins API to allow creation and usage of external plugins

[everettraven]

🔈 Kubebuilder is part of the Google Summer Code Program 2023. This project is applied under the CNFC organization.

This discussion is meant to help talk about Phase 2 Plugins for the Google Summer of Code.

Contributor Proposal Submission period: March 20 - April 4 (1800 UTC)

Starting Monday, March 20 at 1800 UTC, contributors can submit their proposals via the program website.

To attend the program, you must be an open-source beginner or a student. Check out the GSoC FAQ entry for eligibility requirements. Also, you can check out some videos to know more about.

You must apply as a student (GSoC Contributor) to the program and propose this project. It does not need to be big or verbose. Then, you can be stunned by its timeline to know if your proposal was accepted and when you should begin to work on the project. We recommend you make a few small contributions against contributors to check how that works and see if that fits your purposes to apply.

Leverage Phase 2 Plugins API to allow the creation and usage of external plugins

Kubebuilder introduced Phase 2 Plugins to make itself more extensible as a CLI and as a library. This enables the community to develop, use, maintain and release their own plugins independently. Phase 2 Plugins have been available for a few releases. To further increase and help facilitate the adoption of this functionality, we need to create an extensive sample, documentation and tutorial on how to build and use external plugins. To achieve this goal, we can use Go, Ansible, Python or Helm to create the sample external plugin that will be used as a central reference plugin that serves as an example of an external plugin that leverages Phase 2 Plugins.

Expected outcome:

• Finish the remaining tasks in the meta task (the big task that tracks various subtasks for the feature): Phase 2 Plugins - Meta Issue #2600
  • Ensure the quality of the sample and Phase 2 Plugins API by:
    • Adding e2e tests should validate that the sample can be built and used with the Kubebuilder CLI.
    • Implementing a maintainable solution to ensure the sample is updated always.
      • We might need to so something like: ✨ Update component-config tutorial automatically #3191
      • However, a dependabot for the sample might suffice in this case, to ensure the dependencies.
  • Review and improve the Plugin documentation to ensure that we the community, can easily leverage the Phase 2 Plugins API: https://book.kubebuilder.io/plugins/plugins.html
  • Verify that we are generating go docs for the Phase 2 Plugins API.
  • Create a tutorial with the sample to let the community and users know how to build and use their external plugins.
  • Add an extra sample using any other language other than Golang.
  • Create a video demo to showcase the Phase 2 Plugin working with the sample. This will demonstrate the achievements and will be shared with the community. We also will add the link to the demo in the docs.

Mentors: @rashmigottipati and @everettraven (myself) will be the mentors for this idea and are happy to answer any questions!

FAQ:

💁‍♀️ What does Kubebuilder? How is it used?

Kubebuilder is often used as a CLI tool that generates projects and provides features to help people create "Operator Solutions".
"Operator solutions" are projects which follow the Operator pattern and extend the K8S API and has the purpose of managing solutions on K8s clusters.

TL'DR:

All Operators are made of controllers, but not all sets of controllers are operators. Operators are sets of controllers that encode human operational knowledge for a specific application (e.g. the Prometheus Operator). Controllers are more general -- for instance, the namespace service account controller isn't an operator because it doesn't know a specific application -- it's very general. Kubebuilder helps constructs sets of controllers. Some of those are Operators, and some might not be.

Also, Kubebuilder can be used as a lib to help other projects extend their features and/or create nice plugins which can be used as specific helpers. For example, Operator-SDK uses Kubebuilder and scaffolds the same plugins/projects but also provide additional features and other plugins on top.

See:

• Readme Overview
• Readme Motivatios
• Who is this for

💡 Examples of its usage:

See https://operatorhub.io/ to have a better idea

❓ Why Operators?

By using operators, it’s possible to provide all expected resources and manage them dynamically, programmatically, and at execution time. To illustrate this idea, imagine if someone accidentally changed a configuration or removed a resource by mistake; in this case, the operator could fix it without human intervention.

What is the difference between Kubebuilder and SDK?
See: https://sdk.operatorframework.io/docs/faqs/#what-are-the-the-differences-between-kubebuilder-and-operator-sdk

🚀 How can I begin to become familiarized with the Project?

OpenSource projects usually have a Contribution guide; see CONTRIBUTING and also VERSIONING for further details.

Also, feel free to check the open issues in the project and begin to work and contribute with anything you wish.
Your help and contributions 🥇 are very welcome.

The maintainers actively manage the issues list and try to highlight issues suitable for newcomers (You can filter by good+first+issue also contributions to help on the docs can fit well for the first ones), but you are free to help as please you. The project follows the typical GitHub pull request model. Before you start any work, please either comment on a current issue or file a new one.

✋ How to get help?

Feel free to ping us and join the #kubebuilder channel in the kubernetes.slack.com slack.
You also can open issues in its repo if you see that fits.

💡 How to skill up and get started?

• You can follow the Kubebuilder QuickStart and tutorial: https://book.kubebuilder.io/quick-start.html
• Check out Deploy Mmage Plugin developed in the latest Google Summer Program 2022 to understand how the projects are done. You will see that by using this plugin, you will have a project with the whole code to distribute and manage a solution on a Kubernetes cluster. You can also look at the demonstration video done by the student in the Community meeting.
• Another amazing project built last year as part of this program was the Grafana plugin; feel free to check its demonstration video as well

How are the mentors of this project?

Mentors: @rashmigottipati and @everettraven (myself) will be the mentors for this idea and are happy to answer any questions!
However, if you need to contact us, please create a group and add us to them so we can answer as soon as possible.

Students who reached out so far.

Slack	Github
Eileen Yu	@Eileen-Yu

[Eileen-Yu]

So excited when finding the idea and glad to see it for GSoC! I have great interest in this project, and would love to apply.

Noticed that there is already a detailed proposal, as well as a prototype. It seems the rest task is mainly about testing and docs as listed in this meta issue.
So I’m bit curious what is expected to be achieved? Is there any new ideas or changes that need to be implemented?

[everettraven]

Hi @Eileen-Yu - those are great questions!

Here is a list of the expectations I have:

• Documentation on:
  • Writing a Phase 2 Plugin
  • Installing a Phase 2 Plugin
  • Using a Phase 2 Plugin
• Testing:
  • Robust unit tests (IIRC the testing coverage for Phase 2 Plugins could be improved)
  • End-to-End (E2E) tests that verify a properly developed and installed Phase 2 Plugin works as expected
• Enhancements
  • Phase 2 Plugins are still in an early development stage and there is likely some improvements that could be made in the code base. I think watching for issues created around the Phase 2 Plugins and looking through the implementation in general would allow us to understand currently lacking areas and how the design could be improved.
    • Some issues created that shows we need(ed) to make some improvements:
      • How external plugins can work with PROJECT config #3245
      • External plugins discovery fails when XDG_CONFIG_HOME is set #3224
      • PluginRequest.Universe doesn't get populated before calling an external plugin #3214

I don't have any ideas off the top of my head for potential improvements, but I am happy to discuss any that you or others may find!

@rashmigottipati feel free to add to or update any of these expectations!

[camilamacedo86]

🎉 We're also excited to announce that @Eileen-Yu will be working on this one under the Google Summer Code Program.
Congrats !!! We are looking further to work with you.

[camilamacedo86]

A record has been created to showcase the outcomes of this project. You can view it here: https://youtu.be/ksMUkIgfpXA?si=vPTMx0JP5eAy3Ixe

🎉 We'd like to congratulate the outstanding student behind this achievement, @Eileen-Yu. A big shoutout to @everettraven for the excellent mentoring throughout this project. We also extend special recognition to @Kavinjsir, who co-mentored and provided unwavering support to @Eileen-Yu alongside @rashmigottipati . Great work, team! 🌟

[camilamacedo86]

Closing since the project finished