Total over time (integrals)

[passkey1510]

I'm wondering if there is a way to have a chart of total hits over time. Here is an use case:

• Each time a new user is registered, a log message is sent, hence there is 1 hit
• I would like to see the chart of user growth during a period of time, I'm thinking of a histogram with total hits in that period.

But I don't know how to do this in histogram configuration, any help please?

[kryachkov]

Same question and the same purpose 👍

[kryachkov]

Did anyone do this before?

[jorgelbg]

Why can't you use the provided histogram? In the log message you have the timestamp, and therefor each time you have a document in ES is a new event, so you could use the count option for the chart value; With this configuration (the same you see in kibana in the events over time graph, for each interval of time ES will count how many events (i.e registrations) occurred and plot that value in the histogram).

For instance, I've a graph that looks like this:

In this case in my log file every time a query hits the search server (which I'm monitoring) a log line with a lot of information is stored in ES, and in kibana (in this graph) I show how many events (i.e queries or registrations) are registered.

[kryachkov]

Standard histogram is not enough becase it shows number of events that came in specified moment of time (for example your histogram shows that at 00:00 10-28 200 events were logged), we need to see total growth of events count overtime. Thus chart will be always growing.

[passkey1510]

@kryachkov 👍. Yes, that is what we would like to have. A simple use case is when you want to see how your total user growth during a month.

[kryachkov]

For better understanding, this how it should behave:

This is simple histogram: http://goo.gl/bI6mwP
Tick "Cumulative" in "Graph options" on the right. This is what we are seeking for

[rashidkpc]

Right now there is no elasticsearch aggregation that makes this possible. It is possible you could fake it by doing a histogram facet for your current time range, and then a query facet for everything before it. Then add the before number to the first bucket and totaling the buckets sequentially, adding the result of the previous bucket to the next.

[Francis88]

I think I got the idea but struggle with the "add the before number to the first bucket and totaling the buckets" part. I have one histogram facet and one query facet like (time<"at"). How do I access the result of the query as a "field" I can then add to the histogram

[dblado]

+1

[oliviervaussy]

+1

[Francis88]

+1

[galarragas]

+1

[RickHoving]

Any updates on this functionality?

[skymeyer]

+1

[rashidkpc]

This would require a method for reducing/combining buckets in Elasticsearch. I believe this is being worked on and we might be able to make it work in the future.

[tcucchietti]

+1

[sbehrens]

+1

[TobiasOtt]

+1

[atulsm]

+1

[ghost]

+1

[Lloydshove]

+1
Is this achievable already in Kibana 4?

[the-fine]

+1

[tcoupland]

+1

[pemontto]

👍

[m-breen]

+1

[Greybird]

+1

[ebertti]

+1

[patrickkusebauch]

For those interested for just cumulative sum graphs. I have made a port of a kibana plugin, that can do that. https://github.com/patrickkusebauch/kibana-plugin-line-sg

[leeway23]

+1

[Kudryavets]

+1

[liucheng98]

+1

[JulieZY]

+1

[andremilk]

I was wondering if there is a way to make the cumulative "global".
I managed to get a date histogram of cardinality over time and use cumulative_sum over that, but the problem is that the cardinality is only true to that particular interval of my date histogram.

[T3rm1]

Wow, is this still not possible although elasticsearch has support for over a year now?
EDIT: Just tried Kibana 5 alpha 5 and even there is no support for pipeline aggregations. Can someone explain to me why such an important and basic feature is not being worked on?

[jasonmhead]

+1

[pwiatrowski]

+1

[epotocko]

+1

[brooksandrew]

+1

[olivierlambert]

+1

[czimamori-daichi]

+1

[MohMaz]

+1

[Dom-nik]

+1

[ghost]

Im looking something similar, calculate the time over threshold, in a day histogram,
for example: how many hours was the value over 15. ten minutes, 90minutes..
is it posible ?

[suyogdilipkale]

In Kibana 5.5.2, using Cumulative Sum aggregation matrix I am able to plot hourly hits vs total hits count on area chart: