Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Ping Message were received on all consumers #178

Open
davimorao opened this issue Jan 12, 2023 · 4 comments
Open

Ping Message were received on all consumers #178

davimorao opened this issue Jan 12, 2023 · 4 comments
Labels
bug Something isn't working

Comments

@davimorao
Copy link

davimorao commented Jan 12, 2023

Hi,

I'm using the AddOutboundEndpointsCheck method and noticed that ping messages are being received from all consumers...

The expected result should be that no consumer receives the ping message, how can we do that?

Code:
services.AddHealthChecks() .AddConsumersCheck() .AddOutboundEndpointsCheck() .AddOutboxCheck();

There are two situations:

1 - Using .AddInbound<T>, the ping message is received in the consumer without errors in all consumers, because it tries to serialize the ping message as if it were from the consumer

2 - Using .AddInbound(...), an error is generated:

Processing inbound message. | endpointName: order-events, messageType: Silverback.Messaging.HealthChecks.PingMessage, Silverback.Integration, Version=3.8.0.0, Culture=neutral, PublicKeyToken=null, messageId: 03f4917a-8a4c-44dc-a0d9-43068d9736ea, offset: [0]@57, kafkaKey: 03f4917a-8a4c-44dc-a0d9-43068d9736ea
fail: Silverback.Messaging.Inbound.Transaction.TransactionHandlerConsumerBehavior[1002]
Error occurred processing the inbound message. | endpointName: order-events, messageType: Silverback.Messaging.HealthChecks.PingMessage, Silverback.Integration, Version=3.8.0.0, Culture=neutral, PublicKeyToken=null, messageId: 03f4917a-8a4c-44dc-a0d9-43068d9736ea, offset: [0]@57, kafkaKey: 03f4917a-8a4c-44dc-a0d9-43068d9736ea
Silverback.Messaging.Publishing.UnhandledMessageException: No subscriber could be found to handle the message of type Silverback.Messaging.Messages.InboundEnvelope`1[[Silverback.Messaging.HealthChecks.PingMessage, Silverback.Integration, Version=3.8.0.0, Culture=neutral, PublicKeyToken=null]].
at Silverback.Messaging.Publishing.Publisher.PublishCoreAsync(Object message, Boolean throwIfUnhandled, Boolean executeAsync)
at Silverback.Messaging.Outbound.Routing.OutboundRouterBehavior.HandleAsync(Object message, MessageHandler next)
at Silverback.Messaging.Outbound.Routing.ProduceBehavior.HandleAsync(Object message, MessageHandler next)
at Silverback.Messaging.Inbound.PublisherConsumerBehavior.PublishEnvelopeAsync(ConsumerPipelineContext context, Boolean throwIfUnhandled)
at Silverback.Messaging.Inbound.PublisherConsumerBehavior.HandleAsync(ConsumerPipelineContext context, ConsumerBehaviorHandler next)
at Silverback.Messaging.Sequences.SequencerConsumerBehaviorBase.HandleAsync(ConsumerPipelineContext context, ConsumerBehaviorHandler next)
at Silverback.Messaging.Sequences.SequencerConsumerBehavior.HandleAsync(ConsumerPipelineContext context, ConsumerBehaviorHandler next)
at Silverback.Messaging.Headers.HeadersReaderConsumerBehavior.HandleAsync(ConsumerPipelineContext context, ConsumerBehaviorHandler next)
at Silverback.Messaging.Validation.ValidatorConsumerBehavior.HandleAsync(ConsumerPipelineContext context, ConsumerBehaviorHandler next)
at Silverback.Messaging.Serialization.DeserializerConsumerBehavior.HandleAsync(ConsumerPipelineContext context, ConsumerBehaviorHandler next)
at Silverback.Messaging.BinaryFiles.BinaryFileHandlerConsumerBehavior.HandleAsync(ConsumerPipelineContext context, ConsumerBehaviorHandler next)
at Silverback.Messaging.Encryption.DecryptorConsumerBehavior.HandleAsync(ConsumerPipelineContext context, ConsumerBehaviorHandler next)
at Silverback.Messaging.Inbound.ExactlyOnce.ExactlyOnceGuardConsumerBehavior.HandleAsync(ConsumerPipelineContext context, ConsumerBehaviorHandler next)
at Silverback.Messaging.Sequences.SequencerConsumerBehaviorBase.HandleAsync(ConsumerPipelineContext context, ConsumerBehaviorHandler next)
at Silverback.Messaging.Inbound.Transaction.TransactionHandlerConsumerBehavior.HandleAsync(ConsumerPipelineContext context, ConsumerBehaviorHandler next)

@BEagle1984
Copy link
Owner

Sorry for the delay and thank you for notifying me about the error.

The error message is strange but it's very possible that there is a bug there. I'll have a look.

@BEagle1984 BEagle1984 added the bug Something isn't working label Jan 18, 2023
@davimorao
Copy link
Author

No problem, thanks for checking. Let me know if you need more information.

@davimorao
Copy link
Author

Taking advantage of this theme, is it possible to customize the sending of the Ping message for just one topic?
This would prevent services that aren't in .NET and don't know about ping messages from having to have specific treatments to ignore them.

@BEagle1984
Copy link
Owner

No, this is not possible at the moment.

Sorry but the ping health check isn't very sophisticated, since I don't use it productively.
In the end, we monitor our services via logs and metrics. If a service fails to produce a message, this should be reported in the logs (in our case collected in Splunk and we get alarmed). For the consumer, we monitor the reported lag on the consumer group (a metric exposed by the broker) and we alarm if it exceeds a certain threshold. This approach might not work very well if it's a topic that is written only from time to time, and from here came the idea to send a ping message at a defined interval, but it is detrimental to interoperability as you correctly pointed out.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working
Projects
None yet
Development

No branches or pull requests

2 participants