-
Notifications
You must be signed in to change notification settings - Fork 487
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Add logging support to operator #748
Conversation
// +kubebuilder:resource:path="pod-logs" | ||
// +kubebuilder:resource:singular="pod-logs" |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
The hyphens really aren't how k8s expects you to refer to multi-word names (daemonsets, not daemon-sets). I only changed this for podlogs but I suspect we'll want to do the same for grafana-agent, prometheus-instance and logs-instance in a later PR.
@@ -124,6 +124,43 @@ func CreateOrUpdateStatefulSet(ctx context.Context, c client.Client, ss *apps_v1 | |||
return nil | |||
} | |||
|
|||
// CreateOrUpdateDaemonSet applies the given DaemonSet against the client. | |||
func CreateOrUpdateDaemonSet(ctx context.Context, c client.Client, ss *apps_v1.DaemonSet) error { |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I've said it before and I'll say it again: generics can't come fast enough :)
Owns(applyGVK(&apps_v1.DaemonSet{})). | ||
Owns(applyGVK(&core_v1.Secret{})). | ||
Owns(applyGVK(&core_v1.Service{})). | ||
Watches(watchType(&core_v1.Secret{}), events[resourceSecret]). | ||
Watches(watchType(&grafana_v1alpha1.LogsInstance{}), events[resourceLogsInstance]). | ||
Watches(watchType(&grafana_v1alpha1.PodLogs{}), events[resourcePodLogs]). |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
The diff here is kind of gnarly because I sorted everything.
if len(d.Prometheis) == 0 { | ||
key := types.NamespacedName{Namespace: svc.Namespace, Name: svc.Name} | ||
|
||
var service core_v1.Service | ||
err := r.Client.Get(ctx, key, &service) | ||
if k8s_errors.IsNotFound(err) { | ||
return nil | ||
} else if err != nil { | ||
return fmt.Errorf("failed to find stale Service %s: %w", key, err) | ||
} | ||
|
||
err = r.Client.Delete(ctx, &service) | ||
if err != nil { | ||
return fmt.Errorf("failed to delete stale Service %s: %w", key, err) | ||
} | ||
return nil | ||
} |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This is the only new logic in this function.
// Don't generate anything if there weren't any instances. | ||
if len(d.Prometheis) == 0 { | ||
continue | ||
} | ||
|
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This is the only new logic in this function.
// | ||
// Note that stages is a string and because SIG API Machinery does not support | ||
// recursive types. Be careful not to mistype anything. | ||
Stages string `json:"stages,omitempty"` |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I tried a few different ways to avoid this, but this really seems to be the only sane way to do it without creating more CRDs.
@@ -143,53 +145,6 @@ func (r *reconciler) fillStore(ctx context.Context, refs []config.AssetReference | |||
return nil | |||
} | |||
|
|||
// createConfigurationSecret creates the Grafana Agent configuration and stores | |||
// it into a secret. | |||
func (r *reconciler) createConfigurationSecret( |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Moved to reconciler_metrics.go.
|
||
// createGoverningService creates the service that governs the (eventual) | ||
// StatefulSet. It must be created before the StatefulSet. | ||
func (r *reconciler) createGoverningService( |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Moved to reconciler_metrics.go.
} | ||
|
||
// createStatefulSets creates a set of Grafana Agent StatefulSets, one per shard. | ||
func (r *reconciler) createStatefulSets( |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Moved to reconciler_metrics.go.
9b72261
to
d637120
Compare
This is ready for review now as an initial implementation. There are some known issues but given the size of this PR I want to do the fixes separately. Ditto with docs. |
(Because I reverted the logs fix in 57ecfb3 this will work but there will be a lot of errors about not being able to save the positions file) |
d637120
to
8192345
Compare
8192345
to
285eaba
Compare
@@ -60,6 +64,66 @@ func trimSlice(s []interface{}) []interface{} { | |||
return res | |||
} | |||
|
|||
// intoStages converts the a yaml slice of stages into a Jsonnet array. | |||
func intoStages(i []interface{}) (interface{}, error) { | |||
text, ok := i[0].(string) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Do we need to protect against 0 length arrays?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Nah, jsonnet will prevent that function from being called if you don't have an argument matching for the list of params passed to vm.NativeFunction
.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Looks good, mostly nits/opinions, changing the version and the loki naming as the primary items.
42d92d6
to
45364f4
Compare
* deploy daemonsets for logs * changelog entry * working example e2e * undo docs changes until a future PR * defer logs fix for a future PR * remove duplicated intoStages code * undo base rebase change * undo accidental commit * review feedback
PR Description
Completes logging support for the operator.
Which issue(s) this PR fixes
Closes #628.
Notes to the Reviewer
No docs changes, I'll do those in a follow up PR.
I found a few issues. Once this is merged I'll open issues and separate PRs for each.
PR Checklist