Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

tools: Update offline_log_viewer for serde-encoded controller logs & test it #5293

Closed
jcsp opened this issue Jun 30, 2022 · 6 comments
Closed

Comments

@jcsp
Copy link
Contributor

jcsp commented Jun 30, 2022

The log decoder in tools/offline_log_viewer is a very useful diagnostic. It has some hand-crafted decoding logic for decoding some controller log messages into JSON.

In 22.2, when the serde_raft_0 feature flag activates after upgrade, we'll start writing controller messages in a new serde encoding (instead of old ADL).

The metadata_viewer tool needs to understand both encodings, because upgraded systems will have a mixture of encodings in their controller logs.

This is a good time to add an automated ducktape test for the tool, that runs a cluster through some administrative operations and then checks the tool can decode the resulting controller log.

@jcsp jcsp changed the title Update metadata_viewer for serde-encoded controller logs & test tools: Update metadata_viewer for serde-encoded controller logs & test Jun 30, 2022
@jcsp jcsp added this to the v22.2.1 milestone Jun 30, 2022
@jcsp jcsp changed the title tools: Update metadata_viewer for serde-encoded controller logs & test tools: Update metadata_viewer for serde-encoded controller logs & test it Jun 30, 2022
@dotnwat
Copy link
Member

dotnwat commented Jul 2, 2022

visibility for @felixguendling

@jcsp jcsp changed the title tools: Update metadata_viewer for serde-encoded controller logs & test it tools: Update offline_log_viewer for serde-encoded controller logs & test it Jul 4, 2022
@jcsp
Copy link
Contributor Author

jcsp commented Jul 4, 2022

Related: a small start on hooking this tool into automated tests is in #5315

@jcsp
Copy link
Contributor Author

jcsp commented Feb 14, 2023

A bunch of offline_log_viewer stuff merged since this ticket was last updated - is there still work to do?

@piyushredpanda
Copy link
Contributor

@andijcr ^?

@andijcr
Copy link
Contributor

andijcr commented Feb 15, 2023

Tool is updated and hooked in random_node_operation_tes.py. it should be hooked in the next upgrade test pr #8253 , but that work belongs to another ticket #7310.

Let me check the changelog for serde::version, to check that nothing is missing

@andijcr
Copy link
Contributor

andijcr commented Feb 15, 2023

cmd git diff 617d042edd9795ad7c63e9c60cbfb6de02f464b1 | awk 'BEGIN {RS="\+\+\+"; ORS="\n\n=======++++++\n+++"} /serde::version<[^0]/ (a commit from october)

removing entries from test files i get this file serde_versions_mods.txt that should contains the changes where serde::version is incremented. i'll run down this later to check if something is missing

@andijcr andijcr closed this as completed Oct 3, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging a pull request may close this issue.

5 participants