Skip to content

Support FetchRequest v4 / Message Format v2 #1119

Closed
@dpkp

Description

@dpkp

In Kafka 0.11, the structure of the 'MessageSet' and 'Message' were significantly
changed. Not only were new fields added to support new features like exactly once
semantics and record headers, but the recursive nature of the previous versions of
the message format was eliminated in favor of a flat structure. A 'MessageSet' is
now called a 'RecordBatch', which contains one or more 'Records' (and not 'Messages').
When compression is enabled, the RecordBatch header remains uncompressed,
but the Records are compressed together. Further, multiple fields in the 'Record'
are varint encoded, which leads to significant space savings for larger batches.

See https://cwiki.apache.org/confluence/display/KAFKA/A+Guide+To+The+Kafka+Protocol#AGuideToTheKafkaProtocol-Messagesets

Metadata

Metadata

Assignees

Labels

No labels
No labels

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions