Publish trace decisions through pub/sub

[VinozzZ]

The decider node will make a decision and publish it to other nodes via a pub/sub system. Decisions are published once, and all nodes receive them through their subscriptions.

Publishing

• Create a pub/sub channel for keep decisions and another for drop decisions.
• Keep decision messages should contain all metadata that need to be added to every span within the trace.
• Drop decisions should be batched, either when a specified number of decisions is reached or after a timeout, whichever comes first.

When a decision is published:

• Publish the decision when a trace decision has been made
• If a span proxy arrives and the decider node already has a trace decision for that trace, it should re-publish the existing trace decision.( I believe this is the only place that's dependent on the span proxy feature)

Receiving

• When receiving a decision message, send a signal to the collect() goroutine so that it can start process the decisions
• When processing a kept decision, we should make sure all metadata is correctly attached as well as updating the decision cache
• When processing a drop decision, we should make sure the decision cache is correctly updated

POC: https://github.com/honeycombio/refinery/pull/1337/files#diff-6be1bc790ae041da2185cc51c6e6aea13d635fdaa79c8d567c2530923fe3ecfeR1254

[MikeGoldsmith]

I think this needs to wait for the following to completed first:

• Forward only key fields to peers #1318
• Abstract unique sampling fields from config #1325

[VinozzZ]

@MikeGoldsmith I believe this is no longer blocked by the span proxy issue anymore. I have updated the issue to include more details. Let me know if you have any questions