Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Akka.Streams: add monitoring stages to measure flow throughput / graph execution time #4852

Closed
Aaronontheweb opened this issue Mar 15, 2021 · 1 comment

Comments

@Aaronontheweb
Copy link
Member

Without getting overly prescriptive on the specs for this:

  1. Want one flow stage which measures the rate at which stream items are flowing through it and periodically emits that value to an external Sink of some kind. That Sink can be a Sink.ActorRef or whatever else, but the idea being that this stream can observe the quantity of messages / second passing through it and periodically emit that counter value out to the Sink. This can be used explicitly for monitoring / observability purposes or it can be used for flow-control, i.e. observing how close a system is performing compared to some internal or external resource limit and pre-emptively throttling.
  2. This one is more complicated - need a way of observing how long it takes a specific "tracer round" or a user-defined message to pass from one point of the stream to another. This is designed to measure per-message latency as a gauge or histogram - so in the event that a stream begins rapidly backpressuring, that value can be observed in an external monitoring system or perhaps an alert can be fired.

We are planning on building some Phobos-specific implementations of these, but given the value they can bring outside of observability (flow control, adaptive backoff, etc) we thought they merited inclusion into the Akka.Streams core library itself.

@Aaronontheweb Aaronontheweb added this to the 1.4.18 milestone Mar 15, 2021
@Aaronontheweb Aaronontheweb modified the milestones: 1.4.18, 1.4.19 Mar 23, 2021
@Aaronontheweb Aaronontheweb modified the milestones: 1.4.19, 1.4.20 Apr 28, 2021
@Aaronontheweb Aaronontheweb modified the milestones: 1.4.20, 1.4.21 May 12, 2021
@Aaronontheweb Aaronontheweb modified the milestones: 1.4.21, 1.4.22 Jun 16, 2021
@Aaronontheweb Aaronontheweb modified the milestones: 1.4.22, 1.4.23, 1.4.24 Aug 5, 2021
@Aaronontheweb Aaronontheweb modified the milestones: 1.4.24, 1.4.25 Aug 18, 2021
@Aaronontheweb Aaronontheweb modified the milestones: 1.4.25, 1.4.26 Sep 8, 2021
@Aaronontheweb Aaronontheweb modified the milestones: 1.4.26, 1.4.27 Sep 28, 2021
@Aaronontheweb Aaronontheweb modified the milestones: 1.4.27, 1.4.28 Oct 11, 2021
@Aaronontheweb
Copy link
Member Author

closed via #5357 - that should help quite a bit.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

1 participant