Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Create grpc deployment when starting operator #332

Open
wants to merge 9 commits into
base: main
Choose a base branch
from

Conversation

troy0820
Copy link
Member

@troy0820 troy0820 commented Jun 20, 2024

What does this change

Creates the GRPC deployment, configmap, service when starting the operator

  • Create go routines which will handle the deployment creation asynchronously
  • Create deployment, configmap, service for grpc server

What issue does it fix

Closes #308

Notes for the reviewer

Put any questions or notes for the reviewer here.

Checklist

  • Did you write tests? Not yet
    • We have to untangle the integration tests because most of this is in main.
  • Did you write documentation?
  • Did you make any API changes? Update the corresponding API documentation.

@troy0820 troy0820 force-pushed the troy0820/create-grpc-deployment branch 2 times, most recently from 36f0901 to 2c48935 Compare June 25, 2024 16:08
main.go Outdated Show resolved Hide resolved
@troy0820 troy0820 marked this pull request as ready for review July 2, 2024 20:18
@troy0820 troy0820 changed the title WIP: Create grpc deployment when starting operator Create grpc deployment when starting operator Jul 2, 2024
Signed-off-by: Troy Connor <troy0820@users.noreply.github.com>
Signed-off-by: Troy Connor <troy0820@users.noreply.github.com>
Signed-off-by: Troy Connor <troy0820@users.noreply.github.com>
Signed-off-by: Troy Connor <troy0820@users.noreply.github.com>
Signed-off-by: Troy Connor <troy0820@users.noreply.github.com>
Signed-off-by: Troy Connor <troy0820@users.noreply.github.com>
…igmap

Signed-off-by: Troy Connor <troy0820@users.noreply.github.com>
Signed-off-by: Troy Connor <troy0820@users.noreply.github.com>
@troy0820 troy0820 force-pushed the troy0820/create-grpc-deployment branch from 89bf6fa to 3ec5f36 Compare July 2, 2024 20:23
Containers: []corev1.Container{
{
Name: "porter-grpc-service",
Image: "ghcr.io/getporter/server:v1.1.0",
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can me make the image version a const? I think it's fine to have the server version compiled into the operator controller to ensure compatibility but it should be easy to upgrade when we want to release a new version

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I can update this.

},
}

var GrpcConfigMap = &corev1.ConfigMap{
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is where I think the complexity is for managing the grpc server as part of the controller. The porter config that this uses HAS to be the porter config that the installations used when running so that means it needs to use the porter config for the namespace. We put the restriction in the only allow a single porter config per namespace instead of a porter config to be defined for every installation. The operator sets up a default config if one doesn't exist. This service should be using the porter config that's applied to the namespace, either the default one or the user provided one. It also has to track if that PorterConfig ever changes and then reload with the new config so that it can hit the backing stores where the installations actually live.

Btw @schristoff we should add support for moving porter storage data from one backend to another. This will "just work" for secrets as long as the secrets are managed externally for the new store like they should be but for storage backend AFAIK there's no way to move from one DB to another if that changes in the porter config and we should definitely support that!

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The porter config that this uses HAS to be the porter config that the installations used when running so that means it needs to use the porter config for the namespace. We put the restriction in the only allow a single porter config per namespace instead of a porter config to be defined for every installation. The operator sets up a default config if one doesn't exist. This service should be using the porter config that's applied to the namespace, either the default one or the user provided one. It also has to track if that PorterConfig ever changes

That means we can't do this implementation because this is trying to install the grpc server before we even do an installation. If the installation needs to happen to success to then create a deployment/configmap/service to get the Porter config in the namespace, (that resource isn't created until something makes it get created) then we will have to do that after the first installation. Relying on the default Porter config at runtime seems a little tricky as that default is making assumption around the installation resource process.

What I can do is move this to be done once at setup during the first installation and dynamically create the configmap that will map to the porter config by the installation we have in the namespace once the installation is complete.

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is really interesting because of how the Porter config gets resolved for an AgentAction:

func (r *AgentActionReconciler) resolvePorterConfig(ctx context.Context, log logr.Logger, action *porterv1.AgentAction) (porterv1.PorterConfigSpec, error) {

We probably need to create a "default" grpc server in the operator namespace then have namespace specific servers IF a PorterConfig is specified for that namespace BUT not for the system..... This just feels gross.... But the resolvePorterConfig should be able to handle checking if a grpc server exists that will be able to handle the AgentAction based on the PorterConfig that is selected to run that AgentAction

@troy0820 troy0820 added the hold - 🛑 Don't merge PR until discussion is resolved. label Jul 8, 2024
)

const (
PorterNamespace = "porter-operator-system"
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is already defined in controllers/installation_controller.go

Signed-off-by: Troy Connor <troy0820@users.noreply.github.com>
@troy0820
Copy link
Member Author

After discussion, we agreed that this should be created on a per PorterConfig basis. If none exists, fall back to the default one that is created in the porter-operator-system namespace.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
hold - 🛑 Don't merge PR until discussion is resolved.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Bundle grpc server with controller-manager in the operator pod spec
2 participants