System.Diagnostics (Observability) work planned for .NET 7

[tarekgh]

This issue captures the planned work for .NET 7. This list is expected to change throughout the release cycle according to ongoing planning and discussions, with possible additions and subtractions to the scope.

Planned for .NET 7

• Investigate what metrics need to be added to the runtime #63649
• Support Metrics UpDownCounter instrument #63648
  • Support UpDownCounter open-telemetry/opentelemetry-dotnet#2362
• [API Proposal]: System.Diagnostics.Activity: Enumeration API [Iteration 2] #67207
• [API Proposal]: Api handle Activity.Current value changes #67276
• Samplers should be allowed to modify tracestate #63652
• ActivityContext.TryParse should support IsRemote #42575
• [API Proposal]: Activity.IsStopped (functionality required to conform to OpenTelemetry specification) #63353
• [API Proposal]: Activity.HasRemoteParent (and problematic docs for Activity.Parent) #65660
• Libraries like HttpClient use legacy activity creation, requiring hacks in OpenTelemetry SDK #63868
• [API Proposal]: System.Diagnostics.ActivityLink/ActivityEvent: Enumeration API #68056

Planned for .NET 8+

• Add "hints" in Metric API to provide things like histogram bounds #63650
• [Feature Request] Allow libraries that don't use DI to make use of ILogger #50777
• Explore using EventSource in front of ILogger #12750
• EventSource improvements for text logging #12730
• ActivityTraceFlags is not reflected in Activity Id when updated after activity start() #61857
  • Currently there is an applied workaround for that
• Add schema URL support #63651
  • Spec exists already
• Strong type support in Microsoft.Extensions.Logging #61947
  • @reyang is following up with creating a prototype using Source Generators
• Activity should have AddExceptionMethod #53641
  • OTel specs dependency
• Attribute Extractor Support
  • OTel Specs dependency
  • @noahfalk following up with this item
• Add Timer instrument
  • OTel specs dependency
• Support Bound instruments
  • OTel specs dependency
• OTel Baggage signal is independent of tracing, where .NET Baggage is associated with Activity
  • Cijo going to send the link for this item
  • Need to figure out who is asking for that and need to clarify the ask
  • P3 till we get specific ask
• OTel propagators support OTel Baggage, .NET propagators does not
  • Cijo going to send the link for this item
  • Need to figure out who is asking for that and need to clarify the ask
  • P3 till we get specific ask

[tarekgh]

CC @noahfalk @shirhatti @cijothomas @reyang @maryamariyan

[ghost]

Tagging subscribers to this area: @tommcdon
See info in area-owners.md if you want to be subscribed.

Issue Details

---

This issue is listing the candidate features/work for .NET 7.0. This list is expected to change according to the discussions and planning. We expect this list will be filtered and possibly other items can be added.

• [Feature Request] Allow libraries that don't use DI to make use of ILogger #50777.
• Explore using EventSource in front of ILogger #12750.
• EventSource improvements for text logging #12730.
• Investigate what metrics need to be added to the runtime and/or adapted to use the newer Metrics API in addition to EventCounters.
• ActivityTraceFlags is not reflected in Activity Id when updated after activity start() #61857.
• Strong type support in Microsoft.Extensions.Logging #61947..
• Activity should have AddExceptionMethod #53641.
• Attribute Extractor Support.
• ActivityContext.TryParse should support IsRemote #42575.
• Add UpDownCounter, ObservableUpDownCounter to the API. (Spec exists already).
• Add "hints" in Metric API to provide things like histogram bounds, recommended tags etc. (OTel spec dependency).
• Support "schema_url" in Meter (Spec exists already).
• Add Timer instrument. (OTel spec dependency).
• Support Bound instruments - to offer the ability to specify some tag keys along with its values, at the time of instrument creation. (OTel spec dependency).
• Explore additions to API to provide Tag keys at instrument creation time, and values only at record time. (Spec allows this already).
• Explore if .NET runtime can expose base classes for Aggregator. (OTel spec dependency).
• OTel Baggage signal is independent of tracing, where .NET Baggage is associated with Activity.
• OTel propagators support OTel Baggage, .NET propagators does not.
• Libraries like HttpClient use legacy activity creation, requiring hacks in OTel SDK.

Author:	tarekgh
Assignees:	-
Labels:	Epic, area-System.Diagnostics
Milestone:	.NET 7.0

[ejsmith]

@tarekgh this OpenTelemetry spec feature is really important for a lot of scenarios:

https://github.com/open-telemetry/opentelemetry-specification/blob/main/specification/metrics/api.md#multiple-instrument-callbacks

I am attempting to populate queue stats like queue count, working item count and deadletter counts as 3 different observable gauges. Getting the counts requires an expensive call and, when I make that, it returns all 3 counts at the same time. The problem is since gauges are required to be observable, I have to give different callbacks for each gauge. I think there are a lot of similar use cases that would greatly benefit from this feature being implemented.

[tarekgh]

@ejsmith thanks for the suggestion. For now, you may solve this by using the following simple workaround:

    public class InstrumentsValues<T1, T2, T3>
                    where T1 : struct
                    where T2 : struct
                    where T3 : struct
    {
        private T1? _value1;
        private T2? _value2;
        private T3? _value3;
        private Func<(T1, T2, T3)> UpdateValues { get; set; }

        public InstrumentsValues(Func<(T1, T2, T3)> readValues)
        {
            _value1 = null;
            _value2 = null;
            _value3 = null;

            UpdateValues = readValues;
        }

        public T1 GetValue1()
        {
            if (!_value1.HasValue)
            {
                (_value1, _value2, _value3) = UpdateValues();
            }

            T1 value = _value1.Value;
            _value1 = null;
            return value;
        }

        public T2 GetValue2()
        {
            if (!_value2.HasValue)
            {
                (_value1, _value2, _value3) = UpdateValues();
            }

            T2 value = _value2.Value;
            _value2 = null;
            return value;
        }

        public T3 GetValue3()
        {
            if (!_value3.HasValue)
            {
                (_value1, _value2, _value3) = UpdateValues();
            }

            T3 value = _value3.Value;
            _value3 = null;
            return value;
        }
    }

and here is an example how to use it:

            Random random = new Random();
            using Meter meter = new Meter("My Meter");
            InstrumentsValues<int, long, double> values = new InstrumentsValues<int, long, double>(() => (random.Next(), random.NextInt64(), random.NextDouble()));

            ObservableCounter<int> observableCounter1 = meter.CreateObservableCounter<int>("Counter 1", () => new Measurement<int>(values.GetValue1(), new KeyValuePair<string, object?>("Counter 1", null)));
            ObservableGauge<long> observableGauge = meter.CreateObservableGauge<long>("Gauge 1", () => new Measurement<long>(values.GetValue2(), new KeyValuePair<string, object?>("Gauge 1", null)));
            ObservableCounter<double> observableCounter2 = meter.CreateObservableCounter<double>("Counter 2", () => new Measurement<double>(values.GetValue3(), new KeyValuePair<string, object?>("Counter 2", null)));

I assume your gauges are going to be observed at the same time. But if not, it is easy to tweak this implementation to invalidate the values periodically.

CC @noahfalk @reyang @CodeBlanch

[ejsmith]

@tarekgh I'll give this method a try. I tried something similar, but it didn't work.

[ejsmith]

@tarekgh that did work. Thank you!

[noahfalk]

@tarekgh - how do you feel about closing this issue and we can re-open a new one once we've made plans with the OpenTelemetry folks for .NET 8? We can always refer back to this issue if we want to see the list of items we were considering - the list won't be lost.