ErrorMessages are mangled by observed message channels

[SkylerArnold-Corista]

In what version(s) of Spring Integration are you seeing this issue?

6.3.0

Describe the bug

When micrometer observation is enabled for a MessageChannel that's being used for error handling, the ErrorMessages that are sent through the channel are silently converted into MutableMessages, and the originalMessage is lost.

From the example repo linked below:

    @Bean
    public IntegrationFlow integrationFlow() {
        return IntegrationFlow.fromSupplier(() -> new GenericMessage<>(UUID.randomUUID().toString()), c -> c.poller(p -> p
                        .fixedDelay(3000)
                        .errorChannel("errorChannel")  // THE PROBLEMATIC BIT
                        .taskExecutor(exampleTaskExecutor())
                ))
                .handle(m -> {
                    throw new RuntimeException();
                })
                .get();
    }

If I set up a service activator to handle the ErrorMessages published to errorChannel, it can't actually receive ErrorMessages and instead it only gets MutableMessage<Throwable>s.

This is an issue in my real project because my error handling logic needs access to the original message (from ErrorMessage::getOriginalMessage), which isn't possible if the message gets converted into a MutableMessage.

I've done a bit of digging and it seems this is happening in AbstractMessageChannel::sendWithObservation and MutableMessage::of. I assume this is an issue any time you send a non-mutable message type through an observed channel & expect it to come out the other side as the same type, not just for my usecase.

Workaround

This is similar to this issue that was raised in the Sleuth project a while ago. Unfortunately, the recommended workaround for that situation doesn't work here, because the interceptors are applied to the message after it's already been converted to a MutableMessage. I also tried to use a custom ErrorMessageStrategy to wrap the ErrorMessage in a MutableMessage before it gets sent from the MessagePublishingErrorHandler, but sadly the ErrorMessageStrategy can only return ErrorMessage types, so that didn't work.

Instead, I'm currently just disabling observation for my error handling channel all together, which has the unfortunate effect of causing the traceId of the logs for the error handling code not to line up with the traceId of the rest of the IntegrationFlow.

Sample

Here's a small example that reproduces the problem. If you run the project you'll see a lot of ugly stack traces, instead of the expected Error from [...] logs that should be happening. If you uncomment the !errorChannel observation pattern, the problem goes away.

[artembilan]

Yeah... I see the problem: didn't think about tracing on error channels.
Will fix it shortly.
As a workaround I can suggest only an extension of the ErrorMessage where you would put originalMessage into headers.
This way, even if it is swwapped to the MutableMessage in the tracing channel, that info is still going to be available in the respective custom header.