Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Ingest pipelines] add support for fingerprint processor #100541

Merged
merged 9 commits into from
May 26, 2021

Conversation

sabarasaba
Copy link
Member

Fixes: #90708

Release Note

The Ingest Node Pipelines UI added support to configure a fingerprint processor. This processor computes a hash based on certain fields of the document’s content.

Default description

Screenshot 2021-05-25 at 13 41 35

New processor fields

Screenshot 2021-05-25 at 13 42 19

@sabarasaba sabarasaba added v8.0.0 Team:Kibana Management Dev Tools, Index Management, Upgrade Assistant, ILM, Ingest Node Pipelines, and more Feature:Ingest Node Pipelines Ingest node pipelines management v7.14.0 release_note:feature Makes this part of the condensed release notes labels May 25, 2021
@sabarasaba sabarasaba self-assigned this May 25, 2021
@sabarasaba sabarasaba requested a review from a team as a code owner May 25, 2021 11:46
@elasticmachine
Copy link
Contributor

Pinging @elastic/es-ui (Team:Elasticsearch UI)

@jrodewig jrodewig self-requested a review May 25, 2021 11:52
Copy link
Contributor

@jrodewig jrodewig left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for adding this @sabarasaba.

The help text for Target field and Ignore missing require some changes. I also left some minor suggestions for a few other fields. After those are addressed, I'll take another look.

Copy link
Contributor

@jrodewig jrodewig left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM. Thanks again @sabarasaba.

Copy link
Contributor

@alisonelizabeth alisonelizabeth left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for adding support for the fingerprint processor! Looks great. I left a couple minor comments in the code. Let me know what you think.

@sabarasaba
Copy link
Member Author

@elasticmachine merge upstream

Copy link
Contributor

@alisonelizabeth alisonelizabeth left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for making the changes @sabarasaba! I left another comment around the implementation for the method default value. Let me know what you think. I'm going to go ahead and approve to unblock.

method: {
type: FIELD_TYPES.TEXT,
defaultValue: 'SHA-1',
serializer: from.emptyStringToUndefined,
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think the serializer will need to be updated as well. If you take a look at the serializer for sort processor/order parameter linked to above, you'll see we're stripping the default value from the request. I think we should follow the same approach here to be consistent.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

uhh, thanks for having a look again @alisonelizabeth! 🙏🏼🙏🏼🙏🏼

@sabarasaba
Copy link
Member Author

@elasticmachine merge upstream

@kibanamachine
Copy link
Contributor

💚 Build Succeeded

Metrics [docs]

Module Count

Fewer modules leads to a faster build time

id before after diff
ingestPipelines 493 494 +1

Async chunks

Total size of all lazy-loaded chunks that will be downloaded as the user navigates the app

id before after diff
ingestPipelines 682.5KB 686.4KB +3.9KB
Unknown metric groups

References to deprecated APIs

id before after diff
canvas 29 25 -4
crossClusterReplication 8 6 -2
fleet 22 20 -2
globalSearch 4 2 -2
indexManagement 12 7 -5
infra 261 149 -112
lens 67 45 -22
licensing 18 15 -3
lists 239 236 -3
maps 286 208 -78
ml 121 115 -6
monitoring 109 56 -53
securitySolution 386 342 -44
stackAlerts 101 95 -6
total -342

History

To update your PR or re-run it, just comment with:
@elasticmachine merge upstream

cc @sabarasaba

@sabarasaba sabarasaba merged commit ae63fb1 into elastic:master May 26, 2021
gmmorris added a commit to gmmorris/kibana that referenced this pull request May 26, 2021
* master: (60 commits)
  [Docs] Index patterns REST API docs (elastic#100549)
  [Ingest pipelines] add support for fingerprint processor (elastic#100541)
  ping Core team when renovate bot bumps es client version (elastic#100662)
  [Maps] Add draw wizard (elastic#100278)
  Use documentation link service for index pattern field editor (elastic#100609)
  [Maps] filter dashboard by map extent (elastic#99860)
  [ftr] migrate screenshots and snapshots services to FtrService class (elastic#100514)
  fix anomaly functional test (elastic#100504)
  update breaking changes template to incorporate ES deprecations (elastic#100621)
  improve default time ranges (elastic#100536)
  [Gauge] Fixes wrong translations on ranges less than symbol (elastic#100535)
  [ftr] migrate "globalNav" service to FtrService class (elastic#100604)
  [ftr] migrate "testSubjects" to FtrService class (elastic#100512)
  Fix spaces test flakyness (elastic#100605)
  [Ingest pipelines] add support for ip type in convert processor (elastic#100531)
  [ftr] migrate "browser" to FtrService class (elastic#100507)
  [ftr] migrate "find" service to FtrService class (elastic#100509)
  [telemetry] report config deprecations (elastic#99887)
  [ftr] migrate "docTable" service to FtrService class (elastic#100595)
  [ftr] migrate "listingTable" service to FtrService class (elastic#100606)
  ...
sabarasaba added a commit to sabarasaba/kibana that referenced this pull request May 26, 2021
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Feature:Ingest Node Pipelines Ingest node pipelines management release_note:feature Makes this part of the condensed release notes Team:Kibana Management Dev Tools, Index Management, Upgrade Assistant, ILM, Ingest Node Pipelines, and more v7.14.0 v8.0.0
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Add support for fingerprint ingest processor
5 participants