capture/analyze application performance #384
Labels
Comments
|
Putting this here for now. Since it appeared that the latency for a v3 single device read increased from a v2 single device read, I decided to look into that first, using cProfile and graphviz, I came up with the following data Synse v2Profile outputCall graph
Synse v3Profile outputCall graph
|
|
Initial analysis suggests that |
|
Confirmed that
The abridged results are as follows:
The question around this is whether this is acceptable or not. Obviously lower latency is better, but I feel like the value structlog brings in being able to associate request IDs or other bits of data for the lifecycle of the request is pretty valuable moving forward. We could look around and see if there are any other structured logging libraries and how they perform if this is an issue. I haven't seen many others with the same capabilities as structlog. Another option could be to write our own, but that feels like overkill. At least for the read request, there is not a lot of room for micro-optimizations otherwise, as the majority of the remaining execution time is around the grpc network call. |
|
I did some brief analysis on some of the other logging solutions out there to see how they compare to structlog, regarding performance, results are below: these results reflect the time (in seconds) it takes to execute 1000000 calls to The comparisons here aren't necessarily fair, because of the 5 packages listed above, only structlog and loguru have structured logging capabilities built-in, so its not as simple as just replacing structlog with one of the better performing libraries at this moment. |
|
I think that the data collection and results of this are sufficient to close this issue. To summarize, it appears that extra latency is being added in fulfilling requests for synse v3 because of the overhead brought on by At this time, it feels like the value that structlog affords outweighs the overhead that it brings. This may change in the future and we may eventually want to swap out the logging backend, but that doesn't feel necessary in the near future. |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
After doing some quick comparisons of throughput/latency tests between synse v2 and synse v3, it appears that v3 has a bit higher latency. this could be due to a number of things, one of which being the added instrumentation for capturing application metrics, but that is just my current hypothesis and is likely not the sole cause.
This issue is to dive deeper into measuring application performance, making note of any bottlenecks/areas which may be optimized, etc. Even if the output of this is just confirming that the additional latency is solely from application metrics overhead, that is valuable information.
The text was updated successfully, but these errors were encountered: