MDCAmbit to assist in creating and closing MDC entries

[ceki]

MDCClosable was supposed to help users remove entries at the end of a try/catch block. Unfortunately, as discussed in SLF4J-557 closable instances, including MDCCloseable instances are closed before the catch clause. This is very counter-intuitive and most probably goes against the wishes of the developer.

To alleviate this problem, MDCAmbit is being proposed under the branch_2.1.x branch.

Usage examples can be seen in the javadocs of MDCAmbit and well as in MDCAmbitTest

Given that this MDCAmbit is slated to be added to the org.slf4j package, your feedback is requested.

For example, assuming MDCAmbit is added to the org.slf4j package, what should be the SLF4J release version? Should it be version 2.0.10, 2.1 or 3.0?

[philsttr]

The concept seems ok, but the name MDCHelper does not convey the purpose of the class. Perhaps MDCScope would be better, since it will hold MDC keys scoped to a particular section of code.

Also the name removeSet() is not quite clear. Set seems like an internal concept within the class that should not be part of the public interface. Perhaps if the class is named MDCScope, then a method name like end() could be used to denote that the scope is ending. Or maybe use clear() (to be similar to MDC.clear()) or just remove().

The downside of the term "scope" is that it is also used by the upcoming Scoped Values feature. So if there is ever a future where the MDC (or something like it) is a ScopedValue instead of a ThreadLocal (which I think is definitely worth considering at some point), using the term "scope" could cause confusion.

Naming is hard.

[ceki]

Thank you for your comments. Naming is hard indeed.

Preview features may be great for experimenting with new APIs but as far as libraries are concerned, preview features are unusable given the great complexity it adds. We have no choice but to wait until Scoped Values are no longer a preview feature. However, you are right to observe that Scoped Values is something we will need to take into consideration in the future.

How about MDCAmbit instead of MDCHelper?

I also agree with your comments regarding removeSet() .

The API has been updated in response to your comments.

[mourjo]

Could we use something like a Runnable instead of a helper?

public static void withMDC(Map<String, String> ctx, Runnable lambda) {
    var previousCtx = MDC.getCopyOfContextMap();
    try {
      MDC.clear();
      MDC.setContextMap(ctx);
      lambda.run();
    } finally {
      MDC.clear();
      MDC.setContextMap(previousCtx);
    }
  }

public static void withMDC(Runnable lambda) {
    with(Map.of(), lambda);
  }

withMDC(() -> {
   MDC.put("some", "thing");
});

[ceki]

Changes to the MDC state should be additive. The idea is to set some keys and to clear them at the end withtout touching any previously set keys. Moreover, in the statement with(Map.of(), lambda); which method does with refer to?

[mourjo]

Thanks for responding! Sorry for the confusion, let me clarify what I meant.

which method does with refer to?

with refers to the mapping context the runnable starts with. Something akin to of in Map.of. So it would be to initialize the mapping context with some preset keys. Like:

withMDC(Map.of("another", "thing"), () -> {
   MDC.put("some", "thing"); // this has both "some" and "another" in the context
});

I see, but to the outside world, it would be additive - because clear and restore happens inside the call to withMDC.

Changes to the MDC state should be additive.

Combining both I guess we could use something like this:

public static void withBoundedCtx(Runnable lambda) {
    var previousCtx = MDC.getCopyOfContextMap();
    try {
      lambda.run();
    } finally {
      if (previousCtx != null ) {
         MDC.setContextMap(previousCtx);
       } else {
         MDC.clear();
       }
    }
  }

Usage:

MDC.put("another", "thing"); 
 // MDC has ony "another" as keys

withBoundedCtx(() -> {
   MDC.put("some", "thing");      // MDC has "another" and "some" as keys
});

 // MDC has ony "another" as keys

[hkupty]

I think a typical use-case for this would be a layer that always adds the same key and delegates that to a block:

public void handleRequest(Request req) {
    MDCAmbit
      .of("requestId", getRequestId(req))
      .withMdc(controller::handleRequest);
}

This design resonates well with the ScopedValue (proposed) implementation and, as I said in my comment below, if SLF4J backends can implement their own versions, I'd prefer to use ScopedValues altogether instead of ThreadLocal and this would be a nice way to abstract that.

[hkupty]

I think this is a great addition. One thing I would consider is making it extendable/implementable by the libraries implementing SLF4J. It could potentially replace MDC itself and allow ScopedValues in newer JVM instead of a ThreadLocal, which can be problematic with virtual threads.

Additionally, I'd second having a method that takes a runnable, but maybe, to preserve the additive property, prepare the state outside and only apply before running the code block, cleaning up right after...

[drub0y]

Love this API. I have two requests from a devx perspective:

First, it would be great if MDCAmbit implemented Closeable so we could use succinct try blocks like so that instead of this:

MDCAmbit mdcAmbit = new MDCAmbit();
// populate here

try {
  // logic here
} finally {
  mdcAmbit.clear();
}

we could just do this:

try(MDCAmbit mdcAmbit = new MDCAmbit()) {
   // populate here

   // logic here
}

Second, it would be awesome if the API provided a builder API so that we can do this:

MDCAmbit mdcAmbit = MDCAmbit.builder()
   .put("k0", value0)
   .put("k1", value1)
   .build();

If you were to combine the Closeable and the builder together you could have the following clean setup to every scope:

try(MDCAmbit mdcAmbit = MDCAmbit.builder()
   .put("k0", value0)
   .put("k1", value1)
   .build()) {
  // logic here
}

[ceki]

Hi @drub0y,

Thank you for your input. As mentioned at the top of this page and as discussed in SLF4J-557 Closable instances instances are closed before the catch clause is reached. This is very counter-intuitive and usually goes against the wishes of the user.

Thus, Closable is unfortunately not a good idea in this particular case.

It follows that the builder idea is not that useful either because you need to call clear within a finally block.

[zacthompson]

This MDCAmbit feature seems purely additive to the API, and as such is perfectly fine to release as part of any 2.x version in my opinion, even 2.0.10. But if you're wanting to ever mark MDCCloseable/putCloseable as deprecated for any reason, that should at minimum be a new "minor version" (2.1.x or something). No need for a 3.x line if there are no breaking changes. (e.g., if you were to ever remove MDCCloseable completely, that would probably warrant 3.x.)

(A small question: Is it accurate to say that MDCAmbit, like MDCClosable before it, is not meant for use with "stack/queue" interfaces like push/popByKey? That's probably for the best: I don't really know of any significant use of those push/pop methods "in the wild", and they seem to be largely focused on supporting older use cases.)

[huxi]

The ticket #404 is also relevant to MDCAmbit.

[pedrolamarao]

I think it critical for MDCAmbit to be designed towards implementation as a ScopedValue.
Java is moving forward to embrace structured concurrency supporting "millions" of concurrent virtual threads.
In this world, one would set up an MDC and execute a structured concurrency construct under its scope.
ScopedValue was designed explicitly to keep such programs performant.
I suppose all this means that clear has to go.

[pedrolamarao]

And perhaps, as suggested above, this would then be a "Scoped MDC".

[hkupty]

I'd like to add that, being based on ScopedValue would likely mean it more compatible with other languages on top of the JVM, such as kotlin, where ScopedValue would likely have to be supported by coroutines, for example.

[henryju]

Maybe that's because I am not a native English speaker, but what does Ambit mean? This is the first time I have seen it. It doesn't sound like a common word, so it will not help users understand what it is about.

[hkupty]

There's a similar word in portuguese (Âmbito) which roughly translates to context/surroundings so I can only guess this word comes from a latin root and likely will be foreign to many germanic-based languages and somewhat easier to understand for those with a latin-based language background. This is, of course, very speculative.
edit: I just saw you're from France so my theory is not that solid 😅

[henryju]

I am French, so mostly Latin-based language, and I can't think of any related word ;)

My point is that this class is not doing something extraordinary from a computer science PoV. If I understood well, it is capturing a context, then restoring it at the end. We have many well-know words in CS that most developers in the world would understand, because they already saw them before. Here are a few examples from the top of my head:

• context/capture/restore/revert
• snapshot/rollback (database-like)
• stack/push/pop

For example:

  MDCContext mdc = MDCContext.capture();
  try {
    mdc.put("k0", "v0");
    doSomething();
  } catch (RuntimeException e) {
    // here MDC.get("k0") would return "v0"
  } finally {
    // MDC remove "k0"
    mdc.restore();
  }

or

  MDCSnapshot mdc = MDCSnapshot.take();
  try {
    mdc.put("k0", "v0");
    doSomething();
  } catch (RuntimeException e) {
    // here MDC.get("k0") would return "v0"
  } finally {
    // MDC remove "k0"
    mdc.rollback();
  }

or

  MDCStack mdc = new MDCStack();
  try {
    mdc.push("k0", "v0");
    doSomething();
  } catch (RuntimeException e) {
    // here MDC.get("k0") would return "v0"
  } finally {
    // MDC remove "k0"
    mdc.pop();
  }

[hkupty]

I agree.. Personally, I think MDCContext might be a bit more fitting but the other two examples are much more upfront with their intended usage than MDCAmbit.

[zacthompson]

Discussion of the name already happened in the first comment in this discussion, because it was originally named MDCHelper, which didn't really convey any meaning at all. MDCAmbit is much better.

I'm just one voice, but as an English speaker, I think "ambit" is exactly the right name, despite it being a pretty uncommon word, even in English. You probably know related words such as "ambient" or "ambiance", essentially meaning "surroundings". Even the word "ambition" comes from the idea of "going around" (to get votes). You can think of the ambit of something as being its range / scope / area / domain / circuit / realm. Yes, it is kind of like "context" (though, "ambit" has a bit more of a sense of something with a boundary, which is appropriate here).

You might think Ambit is an awkward term at first, but it's better than all the other words available. You want a way of saying, "all of these things go together, in a little kingdom of their own."

• "Collection"/"Set" have very specific meanings in Java, as do "Stack"/"Circuit" for CS in general, and they don't correspond well with the Ambit concept nor its proposed usage. (MDCAmbit is not like taking/restoring a snapshot in general: the proposed design clearly includes adding pre-existing elements into an Ambit and then clearing them later.)
• "Context" would be okay, except that MDC already means "Mapped Diagnostic Context". MDCContext would mean Mapped Diagnostic Context Context. People already make a lot of "AbstractAdapterFactoryBuilder" jokes about Java, and I don't think we should give them more fuel.
• "Scope" is pretty good, except that (as mentioned above) people would be almost certain to confuse it with ScopedValues, which has some similar ideas but is still a different thing.
• "Domain" or "Range" aren't bad, but aren't very good either, and have specific meanings in math (functions) that might lead to confusion. "Realm" is an even worse match.

Ambit is a good name, and I can't think of a better one, given the limits above.

[mikehearn]

Ambit has the problem that even native English speakers are unlikely to have ever encountered it. I've never seen this word used in a computer science context before and so the class name might as well be a random number for all the help it gives.

What's wrong with MDC.Scope? Everyone knows what a scope is, and there's no confusion with ScopedValue IMHO because they're conceptually the same thing but just in different libraries (and ScopedValue might well end up being an implementation strategy if lambdas are the only way to open an MDC scope).

[mikehearn]

There's a typo in the javadoc for this class: DCAmbit mdca = new MDCAmbit(); (the declaration is missing an M).

I'd suggest:

1. Renaming to MDC.Scope, which reads well and communicates what it is clearly. "Ambit" may make sense in some languages but not really in English, IMHO.
2. Eliminating .clear() (or rather making it private), so the only interface possible is the lambdas. This is sadly painful because unlike Kotlin, Java doesn't auto-box (preferring to expose the "effectively final" concept). But it does enable an optionally much faster ScopedValue implementation in future. Unfortunately ScopedValue can only be set using a lambda, not by hand, so that forces the API design here.

[henryju]

Regarding the typo, I already opened a PR: #443

[hkupty]

As an author of a backend for SLF4J (I'm not here for advertising it so I won't link, but it's quite easy to find) I think it makes sense to use something like ScopedValue for the implementation. ThreadLocal unfortunately forces you to create a value even when MDC is empty so at least ScopedValue's cost is associated with its usage.

[ppkarwasz]

2. Eliminating .clear() (or rather making it private), so the only interface possible is the lambdas. This is sadly painful because unlike Kotlin, Java doesn't auto-box (preferring to expose the "effectively final" concept). But it does enable an optionally much faster ScopedValue implementation in future. Unfortunately ScopedValue can only be set using a lambda, not by hand, so that forces the API design here.

This change will make MDCAmbit virtually identical to the proposed ScopedContext interface in Log4j API. The ScopedContext API will be released as a separate artifact, so that both Logback and Log4j Core can implement it. The baseline of the artifact can also be increased to the first Java version featuring a non-preview version of ScopedValue.

[pedrolamarao]

Multi-release JARs would allow variant implementations of the new mechanism, with and without ScopedValue.