KafkaProducer: The Producer thread is getting stuck when the sender hits an exception.

[sgrandhi76]

apps-dataplane-cfproducer-76b498c7b4-kxsw2-last10000.log

Tried debugging. In sender::run_once when the batches are getting expired should the future of the batch be set or should we call _complete_batch?

This is causing the producer who is waiting on this batch to be stuck.

Do you think this is an issue?

Have attached the logs

[flaneur2020]

we've got the same issue: #1577

in my opinion the the sender thread is taken advantage to decouple the producer thread from the blocking operations on kafka. there is a 'max_block_time' option to restrict the producer from blocking on send() from buffer is full or metadata unavailable, but the producer could also hangs at the write_w pipe which have no socket timeout. it would be better to raise an exception in the producer thread other than hangs without any error.

[jeffwidman]

There have been a lot of changes to the producer connection / sending code since this was posted, so I'm going to close this. If you can still replicate with the 2.0.0 release please post here and we'll re-open.