[O11y][IBM MQ] Lens migration for visualizations to Kibana version 8.5.0 #5495

rajvi-patel-22 · 2023-03-09T14:35:11Z

Enhancement

What does this PR do?

Migrate visualizations to lens for IBM MQ integration package
Statistics for IBM MQ Lens migration:

IBM MQ dashboard	Before Migration		After Migration
	Lens	Visualization	Lens	Visualization	New Control Panel
[Metrics IBM MQ] Subscriptions Overview	0	7	7	0	0
[Logs IBM MQ] Overview of error log	0	7	5	1	1
[Metrics IBM MQ] Calls Overview	0	11	11	0	0
[Metrics IBM MQ] Messages Overview	0	12	12	0	0
	0	37	35	1	1

Note: One visualization is migrated to new input controls.

Below visualization is not migrated to the lens

Dashboard	Visualisation Name	Link to the issue
[Logs IBM MQ] Overview of error log	Top 5 Errors [Logs IBM MQ]	#5662

Checklist

I have added an entry to my package's changelog.yml file.
I have verified that panels are populated with data.
I have verified that panels are not distorted after being migrated to lens.
I have updated screenshots of dashboard.
I have verified that data count are matching and panel aggregations are same as before.

Author's Checklist

Migrated panels should be removed from visualization folder.
Migrated visualizations are populating in current Kibana version 8.5.0 itself.

Related issues

Relates Lens Migration of OBS integrations #5221

Issues Identified

Gradient and rainbow color palettes are not available in lens and won't be available in the future versions as well. Hence, the closest color palettes (positive and negative) have been used.
[Lens] Support for Gradient and Rainbow color palette in Lens kibana#152699
The text type field is not supported in table type visualizations of Lens. ~~Hence, we have removed Description field from the table.~~ Created the issue to migrate the Top 5 Errors [Logs IBM MQ] panel in future.
Lens only shows keyword fields, not text fields kibana#96576
Added custom label for multilayer visualizations to identify the individual graphs (for eg., Succeeded and Failed in MQCB calls succeeded/failed [Metrics IBM MQ] panel)

elasticmachine · 2023-03-09T14:54:11Z

💚 Build Succeeded

the below badges are clickable and redirect to their specific view in the CI or DOCS

Expand to view the summary

Build stats

Start Time: 2023-03-23T12:23:05.279+0000
Duration: 17 min 41 sec

Test stats 🧪

Test	Results
Failed	0
Passed	14
Skipped	0
Total	14

🤖 GitHub comments

Expand to view the GitHub comments

To re-run your PR in the CI, just comment with:

/test : Re-trigger the build.

elasticmachine · 2023-03-09T14:54:24Z

🌐 Coverage report

Name	Metrics % (`covered/total`)	Diff
Packages	100.0% (`2/2`)	💚
Files	100.0% (`2/2`)	💚
Classes	100.0% (`2/2`)	💚
Methods	100.0% (`20/20`)	💚
Lines	98.904% (`451/456`)	👍 3.709
Conditionals	100.0% (`0/0`)	💚

kush-elastic

LGTM!

…s_migration-ibmmq-8.5.0

agithomas · 2023-03-21T10:24:53Z

The issue of 'The text type field is not supported in table type visualizations of Lens. Hence, we have removed Description field from the table.' is a blocker to me.

Removing a field may not be the best option. We may then retain the old viz panel that supports text data. Please create a backlog issue to migrate this panel in future.

rajvi-patel-22 · 2023-03-23T12:46:11Z

Removing a field may not be the best option. We may then retain the old viz panel that supports text data. Please create a backlog issue to migrate this panel in future.

@agithomas, We have reverted the changes to the old visualization and created the issue to migrate the Top 5 Errors [Logs IBM MQ] panel in future.

agithomas · 2023-03-27T04:31:00Z

@agithomas, We have reverted the changes to the old visualization and created the issue to migrate the Top 5 Errors [Logs IBM MQ] panel in future.

Please update the summary chart on how many vis are not migrated.

agithomas · 2023-03-27T04:33:14Z

Last bucket empty is a common problem seen across multiple dashboards. @SubhrataK , kindly confirm if this problem is a blocker for vis migration to Lens. Hope you can clarify this today. This clarification is a blocker for multiple integration tasks.

@rajvi-elastic , if this is a blocker, kindly revert the migration for these panels to Lens and update the summary chart.

rajvi-patel-22 · 2023-03-27T05:09:31Z

Please update the summary chart on how many vis are not migrated.

@agithomas, I have updated the summary below migration stat table. Please let me know if anything else needs to be added.

agithomas · 2023-03-27T05:11:41Z

Please update the summary chart on how many vis are not migrated.

@agithomas, I have updated the summary below migration stat table. Please let me know if anything else needs to be added.

Please update in the PR description as well : #5495 (comment)

SubhrataK · 2023-03-27T05:52:40Z

Last bucket empty is a common problem seen across multiple dashboards. @SubhrataK , kindly confirm if this problem is a blocker for vis migration to Lens. Hope you can clarify this today. This clarification is a blocker for multiple integration tasks.

@agithomas and @rajvi-elastic IMO, this should not be a blocker. As a user, it shows a legend for me and I can hover and see the values. Also, this is only in the case where the last bucket values are zero in area and line charts (bar charts are fine).
Infact, the point that it shows the values in the last bucket would have been slightly confusing, as I would have expected it to show an aggregate of the values over the selected time range. There is another discussion thread on what data should be displayed, here: elastic/kibana#101353.
However, there has to be a consistency in the values displayed in the different charts, and that has been raised as a bug.

I suggest we go ahead and migrate the visualizations.

rajvi-patel-22 · 2023-03-28T04:24:28Z

Please update in the PR description as well : #5495 (comment)

@agithomas I have added the table for panels that are not migrated to the lens. Can you please check? #5495 (comment)

agithomas · 2023-03-28T04:32:04Z

Please update in the PR description as well : #5495 (comment)

@agithomas I have added the table for panels that are not migrated to the lens. Can you please check? #5495 (comment)

@rajvi-elastic , please check below. I was referring to this table. If certain visualisation is not migrated to Lens, i expect certain count under Visualisation not equal to 0, after migration. Or, is my understanding incorrect?

With Subrata's clarification, i believe, the last bucket issue is not longer a concern. So, please unlink to referrences to Lens original issue from all the applicable Lens migration issues

rajvi-patel-22 · 2023-03-28T04:50:22Z

@rajvi-elastic , please check below. I was referring to this table. If certain visualisation is not migrated to Lens, i expect certain count under Visualisation not equal to 0, after migration. Or, is my understanding incorrect?

@agithomas My bad! Updated the table and removed the last bucket issue link from the PR and meta issue as well.

agithomas · 2023-03-30T03:17:15Z

@rajvi-elastic ,i see that there is a steep jump in the kibana version from 8.2.0 to 8.5.0. With some of the changes made , does the change to 8.5.0 still needed. If yes, why?

rajvi-patel-22 · 2023-03-30T05:18:12Z

@rajvi-elastic ,i see that there is a steep jump in the kibana version from 8.2.0 to 8.5.0. With some of the changes made , does the change to 8.5.0 still needed. If yes, why?

Yes @agithomas. Please find the below analysis for [Metrics IBM MQ] Calls Overview dashboard which shows the minimum version and the functionality that has been used for the panels.

Visualization Name	Minimum kibana version	Functionality used
MQCB calls succeeded/failed [Metrics IBM MQ]	8.5.0	Adds the ability to show the metric name if there are multiple layers with breakdowns in Lens
MQCLOSE calls succeeded/failed [Metrics IBM MQ]	8.5.0	Adds the ability to show the metric name if there are multiple layers with breakdowns in Lens
MQCONN/MQCONNX calls succeeded/failed [Metrics IBM MQ]	8.5.0	Adds the ability to show the metric name if there are multiple layers with breakdowns in Lens
MQDISC calls succeeded [Metrics IBM MQ]MQDISC calls succeeded [Metrics IBM MQ]	8.2.0	include empty rows
MQCTL calls succeeded [Metrics IBM MQ]	8.2.0	include empty rows
MQSTAT calls succeeded [Metrics IBM MQ]	8.2.0	include empty rows
MQCTL calls succeeded [Metrics IBM MQ]	8.2.0	include empty rows
MQOPEN calls succeeded/failed [Metrics IBM MQ]	8.5.0	Adds the ability to show the metric name if there are multiple layers with breakdowns in Lens
MQINQ calls succeeded/failed [Metrics IBM MQ]	8.5.0	Adds the ability to show the metric name if there are multiple layers with breakdowns in Lens
MQSET calls succeeded/failed [Metrics IBM MQ]	8.5.0	Adds the ability to show the metric name if there are multiple layers with breakdowns in Lens
MQSUBRQ calls succeeded/failed [Metrics IBM MQ]	8.5.0	Adds the ability to show the metric name if there are multiple layers with breakdowns in Lens

agithomas · 2023-03-31T05:19:07Z

@rajvi-elastic ,i see that there is a steep jump in the kibana version from 8.2.0 to 8.5.0. With some of the changes made , does the change to 8.5.0 still needed. If yes, why?

Yes @agithomas. Please find the below analysis for [Metrics IBM MQ] Calls Overview dashboard which shows the minimum version and the functionality that has been used for the panels.

Visualization Name Minimum kibana version Functionality used
MQCB calls succeeded/failed [Metrics IBM MQ] 8.5.0 Adds the ability to show the metric name if there are multiple layers with breakdowns in Lens
MQCLOSE calls succeeded/failed [Metrics IBM MQ] 8.5.0 Adds the ability to show the metric name if there are multiple layers with breakdowns in Lens
MQCONN/MQCONNX calls succeeded/failed [Metrics IBM MQ] 8.5.0 Adds the ability to show the metric name if there are multiple layers with breakdowns in Lens
MQDISC calls succeeded [Metrics IBM MQ]MQDISC calls succeeded [Metrics IBM MQ] 8.2.0 include empty rows
MQCTL calls succeeded [Metrics IBM MQ] 8.2.0 include empty rows
MQSTAT calls succeeded [Metrics IBM MQ] 8.2.0 include empty rows
MQCTL calls succeeded [Metrics IBM MQ] 8.2.0 include empty rows
MQOPEN calls succeeded/failed [Metrics IBM MQ] 8.5.0 Adds the ability to show the metric name if there are multiple layers with breakdowns in Lens
MQINQ calls succeeded/failed [Metrics IBM MQ] 8.5.0 Adds the ability to show the metric name if there are multiple layers with breakdowns in Lens
MQSET calls succeeded/failed [Metrics IBM MQ] 8.5.0 Adds the ability to show the metric name if there are multiple layers with breakdowns in Lens
MQSUBRQ calls succeeded/failed [Metrics IBM MQ] 8.5.0 Adds the ability to show the metric name if there are multiple layers with breakdowns in Lens

@lalit-satapathy / @rameshelastic please advice here. Is it Ok to migrate to 8.5.0 .

agithomas

LGTM!

elasticmachine · 2023-04-06T06:34:21Z

Package ibmmq - 0.4.0 containing this change is available at https://epr.elastic.co/search?package=ibmmq

lens migration of ibmmq to 8.5.0

fe35e17

rajvi-patel-22 requested a review from a team as a code owner March 9, 2023 14:35

Update changelog

633c53f

rajvi-patel-22 changed the title ~~lens migration of ibmmq to 8.5.0~~ [O11y][IBM MQ] Lens migration for visualizations to Kibana version 8.5.0 Mar 9, 2023

rajvi-patel-22 mentioned this pull request Mar 9, 2023

[O11y][IBM MQ] Lens Migration #5494

Closed

6 tasks

kush-elastic assigned rajvi-patel-22 Mar 10, 2023

kush-elastic requested review from lalit-satapathy, kush-elastic and agithomas March 10, 2023 09:34

kush-elastic added enhancement New feature or request v8.5.0 Team:Service-Integrations Label for the Service Integrations team labels Mar 10, 2023

kush-elastic approved these changes Mar 13, 2023

View reviewed changes

Merge branch 'main' of github.com:rajvi-elastic/integrations into len…

1b080c9

…s_migration-ibmmq-8.5.0

rajvi-patel-22 and others added 3 commits March 23, 2023 17:37

Merge branch 'elastic:main' into lens_migration-ibmmq-8.5.0

8a02de3

Revert Top 5 errors panel to support text field types

d178b3a

Revert changelog

1eccf0b

kush-elastic mentioned this pull request Mar 29, 2023

Lens Migration of OBS integrations #5221

Closed

36 tasks

agithomas approved these changes Apr 6, 2023

View reviewed changes

kush-elastic merged commit bde6789 into elastic:main Apr 6, 2023

milan-elastic mentioned this pull request Jun 8, 2023

[Meta][Lens migration] Phases and Issues tracker #6489

Closed

5 tasks

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[O11y][IBM MQ] Lens migration for visualizations to Kibana version 8.5.0 #5495

[O11y][IBM MQ] Lens migration for visualizations to Kibana version 8.5.0 #5495

rajvi-patel-22 commented Mar 9, 2023 •

edited

Loading

elasticmachine commented Mar 9, 2023 •

edited

Loading

Build stats

Test stats 🧪

elasticmachine commented Mar 9, 2023 •

edited

Loading

kush-elastic left a comment

agithomas commented Mar 21, 2023

rajvi-patel-22 commented Mar 23, 2023

agithomas commented Mar 27, 2023

agithomas commented Mar 27, 2023

rajvi-patel-22 commented Mar 27, 2023

agithomas commented Mar 27, 2023

SubhrataK commented Mar 27, 2023

rajvi-patel-22 commented Mar 28, 2023

agithomas commented Mar 28, 2023

rajvi-patel-22 commented Mar 28, 2023 •

edited

Loading

agithomas commented Mar 30, 2023

rajvi-patel-22 commented Mar 30, 2023

agithomas commented Mar 31, 2023

agithomas left a comment

elasticmachine commented Apr 6, 2023

[O11y][IBM MQ] Lens migration for visualizations to Kibana version 8.5.0 #5495

[O11y][IBM MQ] Lens migration for visualizations to Kibana version 8.5.0 #5495

Conversation

rajvi-patel-22 commented Mar 9, 2023 • edited Loading

What does this PR do?

Checklist

Author's Checklist

Related issues

Issues Identified

elasticmachine commented Mar 9, 2023 • edited Loading

💚 Build Succeeded

Build stats

Test stats 🧪

🤖 GitHub comments

elasticmachine commented Mar 9, 2023 • edited Loading

🌐 Coverage report

kush-elastic left a comment

Choose a reason for hiding this comment

agithomas commented Mar 21, 2023

rajvi-patel-22 commented Mar 23, 2023

agithomas commented Mar 27, 2023

agithomas commented Mar 27, 2023

rajvi-patel-22 commented Mar 27, 2023

agithomas commented Mar 27, 2023

SubhrataK commented Mar 27, 2023

rajvi-patel-22 commented Mar 28, 2023

agithomas commented Mar 28, 2023

rajvi-patel-22 commented Mar 28, 2023 • edited Loading

agithomas commented Mar 30, 2023

rajvi-patel-22 commented Mar 30, 2023

agithomas commented Mar 31, 2023

agithomas left a comment

Choose a reason for hiding this comment

elasticmachine commented Apr 6, 2023

rajvi-patel-22 commented Mar 9, 2023 •

edited

Loading

elasticmachine commented Mar 9, 2023 •

edited

Loading

elasticmachine commented Mar 9, 2023 •

edited

Loading

rajvi-patel-22 commented Mar 28, 2023 •

edited

Loading