-
Notifications
You must be signed in to change notification settings - Fork 8.3k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
[Logs UI] Add category anomalies to anomalies page #70982
[Logs UI] Add category anomalies to anomalies page #70982
Conversation
- Adds an anomalies API endpoint - Adds server side sorting / pagination to anomalies endpoint - Renders category examples in the anomalies table - Renames log entry rate examples to log entry examples
Pinging @elastic/logs-metrics-ui (Team:logs-metrics-ui) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
👀 I'll review this over the next days with the intention to minimize the conflicts between this and the modules list I'm working on (#64757).
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I'm submitting this in several parts to shorten the feedback cycle. I hope it won't be too inconvenient that I'm jumping around between various aspects and comment on them as I notice them.
I'm seeing a weird effect that seems to be related to the pagination. This is how I can reproduce it:
- load the page
- disable the auto-refresh
- paginate to the second page
- click "refresh" besides the datepicker
This causes the "no data" screen to be displayed for me. Another click on "refresh" causes data to be shown again, but the pagination state is lost (i.e. I'm on page one again).
x-pack/plugins/infra/common/http_api/log_analysis/results/log_entry_anomalies.ts
Outdated
Show resolved
Hide resolved
x-pack/plugins/infra/common/http_api/log_analysis/results/log_entry_anomalies.ts
Outdated
Show resolved
Hide resolved
x-pack/plugins/infra/common/http_api/log_analysis/results/log_entry_anomalies.ts
Outdated
Show resolved
Hide resolved
x-pack/plugins/infra/common/http_api/log_analysis/results/log_entry_anomalies.ts
Outdated
Show resolved
Hide resolved
x-pack/plugins/infra/common/http_api/log_analysis/results/log_entry_anomalies.ts
Outdated
Show resolved
Hide resolved
x-pack/plugins/infra/common/http_api/log_analysis/results/log_entry_anomalies.ts
Outdated
Show resolved
Hide resolved
x-pack/plugins/infra/common/http_api/log_analysis/results/log_entry_anomalies.ts
Outdated
Show resolved
Hide resolved
x-pack/plugins/infra/public/pages/logs/log_entry_rate/sections/anomalies/table.tsx
Outdated
Show resolved
Hide resolved
Hmm, I'm struggling to replicate this. Following the same steps I see the following: So I'm not thrown incorrectly into the "no data" state. I am returned to page 1, which would be expected as clicking "refresh" would change the time range we're looking at, and as soon as that changes pagination would need to be reset. I've tried a few different date expressions etc 🤔 |
…entry_anomalies.ts Co-authored-by: Felix Stürmer <weltenwort@users.noreply.github.com>
…entry_anomalies.ts Co-authored-by: Felix Stürmer <weltenwort@users.noreply.github.com>
…entry_anomalies.ts Co-authored-by: Felix Stürmer <weltenwort@users.noreply.github.com>
…entry_anomalies.ts Co-authored-by: Felix Stürmer <weltenwort@users.noreply.github.com>
…entry_anomalies.ts Co-authored-by: Felix Stürmer <weltenwort@users.noreply.github.com>
…entry_anomalies.ts Co-authored-by: Felix Stürmer <weltenwort@users.noreply.github.com>
…entry_anomalies.ts Co-authored-by: Felix Stürmer <weltenwort@users.noreply.github.com>
Ok, it seems I didn't control for all the variables 🙈 I can reproduce it if I expand a row on the second page and then click "refresh". |
Still no luck reproducing with the expanded row, interesting 🤔 |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
While testing the page and reading the code I came a cross a few things that we might have to handle:
-
As mentioned below, the examples don't seem to be filtered by the category pattern.
-
The page doesn't work with only the categories job deployed. This is somewhat intertwined with the ml job list flyout, so it might make sense if I tackled this in [Logs UI] Show log analysis ML jobs in a list #71132 or even a follow-up PR. What would the page look like without log rate data?
-
When deploying the log rate job anew, there was quite a long period where the anomalies API reported results but the page showed the empty state prompt:
This is probably also a relic of the previous focus of the page on the log entry rate results. The "hasResults" checks and related conditions might need refining.
x-pack/plugins/infra/public/pages/logs/log_entry_rate/sections/anomalies/expanded_row.tsx
Show resolved
Hide resolved
x-pack/plugins/infra/public/pages/logs/log_entry_rate/page_results_content.tsx
Outdated
Show resolved
Hide resolved
Hmm. Yeah, I missed this to be honest. Just applied the "examples around" like log rate anomalies. I'll get this working, probably with the outlook to merge what needs to be merged in a followup. I've merged certain things along the way, and left others separate.
That would be great if you don't mind. I guess we'd want to show a callout in place of the chart without log rate data, telling the user to create the job.
Ah, yes, need to tweak those has data checks (like you said). |
…es.ts Co-authored-by: Felix Stürmer <weltenwort@users.noreply.github.com>
…b.com:Kerry350/kibana into 64755-expand-anomalies-page-to-add-categories
…es.ts Co-authored-by: Felix Stürmer <weltenwort@users.noreply.github.com>
…b.com:Kerry350/kibana into 64755-expand-anomalies-page-to-add-categories
…/anomalies/table.tsx Co-authored-by: Felix Stürmer <weltenwort@users.noreply.github.com>
…b.com:Kerry350/kibana into 64755-expand-anomalies-page-to-add-categories
…alies-page-to-add-categories
@weltenwort I think everything is addressed and working now. Main things:
Definitely. I think there's a few things we can improve once this all comes together and we have a chance to breathe and reassess. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Well done regarding the reducer, feels pretty consistent and much easier to read now.
Great job overall, this was definitely a challenging enhancement to pivot the tab's focus like that 👏
sourceConfiguration: { sourceId }, | ||
} = useLogEntryRateModuleContext(); | ||
}> = ({ anomaly, timeRange }) => { | ||
const { sourceId } = useLogSourceContext(); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Yes, this reduces the scope of the dependency 👍
tooltip={tooltipProps} | ||
baseTheme={isDarkMode ? DARK_THEME : LIGHT_THEME} | ||
xDomain={{ min: timeRange.startTime, max: timeRange.endTime }} | ||
<> |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Oh, ok then... I would have expected something like
return !isLoading && !series.length ? (
<EuiEmptyPrompt ...>
) : (
<LoadingOverlayWrapper ...>
);
to work as well.
Forgot to mention, the empty category example retrieval now works for me 👍 And I think putting the |
Another late thought, sorry 🙈 As the failing i18n check points out we lost a bit of accessibility of the loading spinner. Do we want to resurrect that |
@weltenwort Thanks for the reviews 🙏 Pushed up those last little tweaks, hopefully this can go green in the background 🤞 |
💚 Build SucceededBuild metrics@kbn/optimizer bundle module count
History
To update your PR or re-run it, just comment with: |
* Add category anomalies to anomalies page Co-authored-by: Felix Stürmer <weltenwort@users.noreply.github.com> Co-authored-by: Elastic Machine <elasticmachine@users.noreply.github.com>
* Add category anomalies to anomalies page Co-authored-by: Felix Stürmer <weltenwort@users.noreply.github.com> Co-authored-by: Elastic Machine <elasticmachine@users.noreply.github.com> Co-authored-by: Felix Stürmer <weltenwort@users.noreply.github.com> Co-authored-by: Elastic Machine <elasticmachine@users.noreply.github.com>
Summary
Implements #64755 and #65046.
Overview
search_after
is used to implement paging backwards.LogEntryRateExample
resources have been renamedLogEntryExample
as they're in no way specific to log rate.Notes / caveats
I was unsure on the best route to take regarding the document tiebreaker for the sorting. Usually we would use the source configurations'
tiebreaker
property, but the anomalies are ML documents and not Filebeat documents. I'd considered the_id
, but the docs advise strongly against this unless the_id
is duplicated in another field that hasdoc value
enabled, given we don't control that that wasn't possible. I've used_doc
, which is our default. I know thedoc
properties have their own caveats, but wasn't sure what else could be used.The auto refresh of the page really doesn't work very well with the pagination. Given the pagination and sorting is now determined server side, if the time range changes, then the pagination needs to be reset. If the time range shifts, the information we have is incorrect, and we need to re-evaluate. Two possible solutions:
If somebody is really looking to stream their anomalies, rather than page through them and look at historical data, it's just a case of setting the sort (on start time) and staying on page 1.
Pagination options (page size) are technically all hooked up server and client side, but not exposed in the UI. I didn't know if we wanted this, or just to just stick with a sensible number (I've used 25).
On the design
dataset
andcategory
are in bold in the rows. Given recommendations seem to be to usei18n.translate
overFormattedMessage
now I omitted the bold. If we think this is really important, it can be changed.Next steps
We should add dataset filtering. This is technically in the design for the adding categories work, but wasn't part of the outlined work. I'd like to open up this PR now due to FF, and an already large changeset, but that would be the next logical step around this functionality.(PR open)There is definitely some further cleanup that can be done, e.g. we can remove some of the alternate formattings of the log rate results data now.
Testing