Apache Kafka interview questions 13 questions in 5 chapters

The partition resumes from the last committed offset, so the record is processed again. That is at-least-once delivery. I make the handler idempotent with a dedupe table or an upsert keyed by the event id, in the same transaction as the side effect. For a Kafka-to-Kafka pipeline I can use transactions and a read_committed consumer. An external database update or payment call is still at-least-once, so it needs its own idempotency.

Duplicates are normal under at-least-once delivery. I give every event a stable id and record it in a processed-events table with a unique constraint, in the same database transaction as the business write. A replay then hits the constraint and is skipped. If the write is naturally idempotent, an upsert or a version check is enough. I do not rely on Kafka exactly-once for an external payment or email, because that effect lives outside Kafka.

At-most-once means I commit the offset before processing, so a crash can lose the record. At-least-once means I process first and then commit, so a crash can deliver it again; that is Kafka's default, and the consumer must be idempotent. Exactly-once applies to a read-process-write pipeline inside Kafka, using a transactional producer and a read_committed consumer. It does not, by itself, make a database update or a payment API happen only once.

I never write to the database and to Kafka in two separate steps, because one can succeed after the other fails. I write the business row and an outbox row in the same database transaction, and a relay, either polling or change data capture, publishes the outbox to Kafka. Delivery is still at-least-once, so consumers deduplicate by event id. The idempotent producer stops a retried send from creating a second log entry.

Kafka only preserves order within a single partition. I use the entity id, such as the order id, as the message key so every event for that entity lands on the same partition and is consumed in offset order. I keep one consumer in the group on that partition, and I leave producer idempotence on so retries cannot reorder in-flight batches. I do not add partitions later without a plan, because that changes the key-to-partition map.

Lag means the consumer is behind the log end offset. If every partition is behind, producers are faster than the group, so I speed up processing, add partitions and then consumers, or pause while a downstream cools down. Adding consumers beyond the partition count does nothing, because one partition is assigned to only one consumer in a group. If only one partition is behind, I look for a hot key or a stuck record, not more consumers.

On the classic protocol with the default Range assignor, rebalancing is eager: every member revokes all partitions, in-flight work must be committed in onPartitionsRevoked, and processing stops until the new assignment. Cooperative sticky revokes only the partitions that must move. I set group.instance.id so a short restart does not rebalance the group. Since Kafka 4.0 I can opt into the KIP-848 consumer protocol with group.protocol=consumer, which is incremental and broker-coordinated.

A hot partition happens when a skewed key, such as a popular product id, sends most records to one partition. That one consumer falls behind while the others sit idle, because a group assigns each partition to only one member. I change the key, salt the hot keys and re-aggregate, or split the work into a local pre-aggregation and a global merge. More consumers alone will not help until the traffic is spread across more partitions.

A healthy pod only proves the process is up, not that the consumer is reading. I check lag and records-consumed-rate: if lag climbs and the rate is zero, the member was probably evicted after the poll loop blocked past max.poll.interval.ms. I keep processing short, shrink max.poll.records, and make liveness depend on consumer progress so Kubernetes restarts a stuck member. I also alert on lag, not only on pod health.

A poison record fails every time, so a blocking retry loop stalls the whole partition and lag grows behind it. I treat deserialization and validation errors as non-retryable and publish them to a dead-letter topic, then commit and continue. Spring Kafka's ErrorHandlingDeserializer catches corrupt payloads before the listener, and DefaultErrorHandler with DeadLetterPublishingRecoverer can park the record. I alert on dead-letter volume so the parked events are actually fixed.

I retry immediately only for a clearly transient, very short failure, and even then I cap the attempts. For anything that may stay down, I use exponential backoff with jitter, and I keep the work inside max.poll.interval.ms so the member is not evicted. After the limit, or for a deterministic error such as a bad schema, I send the record to a dead-letter topic. Indefinite retries on a poison message just block the partition.

I put the events through a schema registry and pick a compatibility mode. If older consumers are still running, backward compatibility is not enough, because a new producer field would break them; I need forward or full compatibility, and I only add optional fields with defaults. I never rename or reuse a field. If I cannot control when consumers upgrade, I use FULL or FULL_TRANSITIVE and I roll the producer out gradually.

I choose Kafka when I need a durable, replayable log, high-throughput event streams, and several independent consumer groups reading the same data at their own pace. I choose RabbitMQ when I need flexible routing with exchanges, competing consumers on a work queue, or per-message acknowledgement and retry. RabbitMQ also offers streams if I want a log-like structure there, but queues remain its core model. The two solve different shapes of problem, not the same problem at different speeds.