Quit gracefully on interrupt

[HannesP]

I have a service which reads from a Kafka topic using KafkaConsumer. When the service receives a SIGTERM, I want to shut it down gracefully. However, I can't find a way to stop the consumer without causing an error. This is my approach:

import signal

def sigterm_handler(signo, frame):
	consumer.close()
signal.signal(signal.SIGTERM, sigterm_handler)

consumer = KafkaConsumer('my-topic')
for msg in consumer:
	process_message(msg)

I've also tried this approach, but since the topic may be written to very rarely, it isn't useful for shutting the service down:

import signal

has_stopped = False
def sigterm_handler(signo, frame):
	has_stopped = True
signal.signal(signal.SIGTERM, sigterm_handler)

consumer = KafkaConsumer('my-topic')
for msg in consumer:
	if has_stopped:
		break
	process_message(msg)

[tvoinarovskyi]

Are you running the code on python2. 7?

[HannesP]

@tvoinarovskyi no, python 3.6.3

[tvoinarovskyi]

Hmm, on python3 the upper example should work. Probably close should wake up the poll. For now you can use the 'poll' API with a relatively small timeout

[dpkp]

I think #1978 should help here