Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Enable log parsing from the UI #1779

Closed
2 tasks
ankitnayan opened this issue Nov 29, 2022 · 4 comments · Fixed by #2457
Closed
2 tasks

Enable log parsing from the UI #1779

ankitnayan opened this issue Nov 29, 2022 · 4 comments · Fixed by #2457
Assignees
@raj-k-singh @nityanandagohain
Labels
enhancement New feature or request logs

Comments

@ankitnayan
Copy link
Collaborator

ankitnayan commented Nov 29, 2022 •
edited by raj-k-singh
Loading

The currently suggested approach is to enable parsers and use regex at otel-collectors. Writing regex becomes a nightmare when it is just more than basics.

  • We should enable UI-based parsing rules to easily extract fields from ingested logs data to be able to convert them to columns and add an index for better filtering and query performance

  • We can also enable a user to choose a parsing template from the UI (and enable customisation) for standard components like Nginx and famous application loggers.
@ankitnayan ankitnayan added enhancement New feature or request logs labels Nov 29, 2022
@ankitnayan
Copy link
Collaborator Author

ankitnayan commented Nov 29, 2022 •
edited
Loading

https://github.com/SigNoz/engineering-pod/issues/837
https://github.com/SigNoz/engineering-pod/issues/790
https://github.com/SigNoz/engineering-pod/issues/860

@pranay01 pranay01 added this to the v0.13.0 milestone Dec 16, 2022
@nityanandagohain
Copy link
Member

SigNoz/signoz-otel-collector#55

@pranay01 pranay01 modified the milestones: v0.13.0, v0.14.0 Dec 30, 2022
@ankitnayan ankitnayan mentioned this issue Jan 4, 2023
Open
@makeavish makeavish modified the milestones: v0.14.0, v0.15.0 Jan 17, 2023
@makeavish makeavish modified the milestones: v0.15.0, v0.16.0 Jan 27, 2023
@makeavish makeavish removed this from the v0.16.0 milestone Feb 13, 2023
This was referenced Mar 15, 2023
Merged
Merged
@ankitnayan
Copy link
Collaborator Author

Issues to handle

  • Multi-line error logs
  • parsing from the JSON body from the UI (working with array keys in the json is an issue)

@raj-k-singh raj-k-singh reopened this Aug 1, 2023
This was referenced Aug 22, 2023
Closed
Merged
@raj-k-singh raj-k-singh mentioned this issue Sep 6, 2023
Merged
This was referenced Oct 14, 2023
Merged
Merged
Merged
Merged
Merged
This was referenced Oct 19, 2023
Merged
Closed
@raj-k-singh
Copy link
Collaborator

Now it is possible to parse logs from the UI

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@raj-k-singh raj-k-singh

@nityanandagohain nityanandagohain

Labels
enhancement New feature or request logs
Projects
None yet
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

feat: integrate pipelines API SigNoz/signoz
5 participants
@pranay01 @raj-k-singh @makeavish @ankitnayan @nityanandagohain