-
Notifications
You must be signed in to change notification settings - Fork 24.6k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Ingest node (time) duration processor #39857
Labels
:Data Management/Ingest Node
Execution or management of Ingest Pipelines including GeoIP
>enhancement
Team:Data Management
Meta label for data/management team
Comments
jakelandis
added
>enhancement
:Data Management/Ingest Node
Execution or management of Ingest Pipelines including GeoIP
labels
Mar 8, 2019
Pinging @elastic/es-core-features |
Could we also imagine something like: {
"duration": {
"start_field": "begin",
"end_field": "end",
"to_period": "seconds",
"target_field": "time.duration_in_seconds"
}
} So it will also help replacing custom painless script to compute a duration between 2 dates. |
9 tasks
Am I correct in assuming this is entirely impossible in the current version (7.13)? So I have a |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Labels
:Data Management/Ingest Node
Execution or management of Ingest Pipelines including GeoIP
>enhancement
Team:Data Management
Meta label for data/management team
Time duration's are often represented as Strings. Elasticsearch uses a simple way to define duration's. This is much simpler then the ISO durations.
This enhancement request is to create a
duration
processor that can read Elasticsearch's definition of duration's in String form and convert the that to along
type in the time period chosen. This processor would re-use Elasticsearch's time parsing and would be limited in scope to it's support. For example, since Elasticsearch does not currently support fractional duration's, neither would this processor.An initial use case for this is for use of ingesting Elasticsearch's own logs which can log with duration's defined in this manner. For example, beats currently uses a custom painless script to handle this: https://github.com/elastic/beats/pull/9603/files#diff-0e8149f64b561ec89a4f9a01077b6f25R38. The introduction of this processor would allow for them to remove that custom code.
Future versions may be able to read ISO duration's too, but is out of scope for this request.
For example:
Could convert "1d" -> 86400
Related: #31737
Related: #31244
The text was updated successfully, but these errors were encountered: