Question - metrics sampling

[NguyHa03]

I see that we could override sampling for request (docs), do we have something like that for custom metrics ?
I want to keep all metrics at 100% and other telematry at sampling (like 10%)

I found that Standard Metric would be suitable for my scenario (docs), but java codeless is currently not support. Could we have any other solution ?

[heyams]

@NguyHa03 can you try metrics filtering. It was introduced in telemetry processor since 3.1.1.

[NguyHa03]

@heyams Thanks for reply
metrics filtering means that I have to set sampling rate of application insight to 100% and filter out metrics that I don't want. It's not suitable for my case.
I need to keep sampling for all telematry like (traces, perfomances..) at 10% to reduce cost and keep all metrics at 100% for business accuracy.

[heyams]

@NguyHa03 can you try increasing the interval metirc-interval. by default, perf counters and custom metrics are captured every 60 seconds.. when you use a longer interval, it will generate less telemetries. this doesn't cover traces though.

alternatively, please also try sampling. It applies to all telemetries except metrics. I think this might work. please let me know how it goes. thanks.

[NguyHa03]

@heyams I tried sampling and set enviroment variable like that APPLICATIONINSIGHTS_SAMPLING_PERCENTAGE=0, but I see that my metricss are not sent to insight metric, when I set APPLICATIONINSIGHTS_SAMPLING_PERCENTAGE=100, the metrics are sent normally.

I think metric is still impact by sampling.

[heyams]

are you using 2.x sdk to send custom metrics? sampling percentage affects metrics coming from 2.x SDK currently.

[heyams]

if you're using "trackMetrics" from the 2.x SDK, please use metric-interval for now. it doesn't affects calls to "trackMetrics", i.e. custom metrics, but will impact metrics that are auto-collected.

[NguyHa03]

I used 3.x agent along with micrometer-core 1.7.3 to send metric like this
static final Counter counter = Metrics.counter("test_counter"); counter.increment();

[trask]

thanks for the explanations @NguyHa03! agree that it is unexpected for sampling percentage to apply to metrics, will fix in #1942