Error handling

Error handling and propagation

Typically librdkafka hides any non-permanent errors from the application and handles them gracefully internally by retrying.

Permanent errors are handled in different ways:

• global errors, e.g., all brokers down, are passed to the error_cb and served through rd_kafka_poll() (e.g. RD_KAFKA_RESP_ERR__ALL_BROKERS_DOWN).
• Producer: message delivery failures (and success) are passed to the dr_cb/dr_msg_cb callback and served through rd_kafka_poll()
• Consumer: Consumer errors, such as mild notifications (RD_KAFKA_RESP_ERR__PARTITION_EOF) or more serious errors are passed as rkmessage->err for the .._consume*() functions that return or pass an rd_kafka_message_t.

This means that a producer application should be calling rd_kafka_poll() every now and then, and a consumer application should both consume messages (duh..) and call rd_kafka_poll() every now and then.