-
Notifications
You must be signed in to change notification settings - Fork 152
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Segregate System Logs From Datapath Logs #1497
Conversation
Thanks for submitting this pull request 🎉. The team will review it soon and get back to you. If you haven't already, please take a moment to review our project contributing guideline and Code of Conduct document. |
docs/tasks/logs.rst
Outdated
events like interactions with the Kubernetes APIs, CRUD operations on | ||
blueprints and actionsets etc. | ||
|
||
Datapath logs, on the other hand, are logs emitted by data mover task pods |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I am not sure if we can call them data mover task pods. In a lot of scenarios, the pods actually doesn't move data but just runs some command. Example is KubeExec
function.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Also, do we have some other options than data path logs? for example functionPodLogs
or kanFunPodLogs
etc.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I don't want to leak implementation concepts like function
, kanFunc
etc. to the logs. Fundamentally, I want to convey high-level concepts like "system vs. user", "control plane vs data plane" etc. Other than operations like scale up and down, even most KubeExec
and KubeOps
calls made by the users involve some kind of data operations, right?
But yes, maybe we can replace "data mover task pods" with just "task pods"?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I am more inclined towards changing the value for const LogKindDatapath
and the reason is, when users see the logs with this extra field they should be able to figure out what are these logs.
I think, dataPath
is not intuitive enough to tell that these logs are from the commands that are there in the kanister functions. What do you think?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
What kind of information would you like to convey to the user in the logs? Will using the ActionSet
and LogKind
fields together provide the level of granularity you are referring to? Adding more fields to the logs may not necessary make things more intuitive as they tend to clutter the log outputs.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I agree, I was not requesting to add more fields. I was asking to change the value datapath
to something else, if possible. Because, according to me, datapath
is not able to convey which logs are these exactly.
If possible, we should be able to convey that, these logs are output of the commands that you (users) have mentioned in the blueprint.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Signed-off-by: Ivan Sim <ivan.sim@kasten.io>
Signed-off-by: Ivan Sim <ivan.sim@kasten.io>
Signed-off-by: Ivan Sim <ivan.sim@kasten.io>
Signed-off-by: Ivan Sim <ivan.sim@kasten.io>
Signed-off-by: Ivan Sim <ivan.sim@kasten.io>
Signed-off-by: Ivan Sim <ivan.sim@kasten.io>
Signed-off-by: Ivan Sim <ivan.sim@kasten.io>
Signed-off-by: Ivan Sim <ivan.sim@kasten.io>
Signed-off-by: Ivan Sim <ivan.sim@kasten.io>
Signed-off-by: Ivan Sim <ivan.sim@kasten.io>
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
LGTM 🚀
Change Overview
This PR adds a new
LogKind
field to log lines generated by theKubeExec
andKubeTask
functions. This field will be assigned the valuedatapath
to distinguish datapath logs from the controller's system logs.This PR also update the documentation to include a reference architecture with instructions on setting up Loki and Grafana to visualize and segregate Kanister logs.
Pull request type
Please check the type of change your PR introduces:
Issues
Test Plan
Deploy this blueprint that uses
flog
to generate Apache log lines:Deploy this actionset to invoke the log streaming actions:
The Kanister logs should show log lines with the
LogKind
fields: