fix(kafka sink): retry messages that result in kafka policy violations

Problem: Some messages were getting dropped by Vector due to Kafka
throwing `PolicyViolation` errors. These should be retried as a policy
can be as simple as a more aggressive rate limit.

----------

Solution: Retry any messages that had the `RDKafkaErrorCode::PolicyViolation` error.

----------

Note: A dynamic back off may be better, as there may be a rate limit out
there that still needs more than 100ms to back off on requests.

----------

See the original issue at #22026

Closes #22026

changelog.d/22026-retry-kafka-policy-violations.fix.md

@@ -0,0 +1,3 @@
+Retry Kafka messages that error with `RDKafkaErrorCode::PolicyViolation` so messages are not lost.
+
+authors: PriceHiller

src/sinks/kafka/service.rs

@@ -159,9 +159,12 @@ impl Service<KafkaRequest> for KafkaService {
                             })
                             .map_err(|(err, _)| err);
                     }
-                    // Producer queue is full.
+                    // Producer queue is full or a policy has been violated and the request should
+                    // be retried
                     Err((
-                        KafkaError::MessageProduction(RDKafkaErrorCode::QueueFull),
+                        KafkaError::MessageProduction(
+                            RDKafkaErrorCode::QueueFull | RDKafkaErrorCode::PolicyViolation,
+                        ),
                         original_record,
                     )) => {
                         if blocked_state.is_none() {