How do you publish events reliably when you update the database and send to Kafka

This is the dual-write problem. A database commit and a Kafka send() are not one atomic operation. If you write the order row and then the process crashes before send() completes, the event never leaves. If you send first and then the database rolls back, consumers act on an order that does not exist.

The transactional outbox solves it by making the event part of the same database transaction as the business write. The outbox table holds an id, the aggregate id, a type, a payload, and a published flag or timestamp. A relay then publishes unpublished rows to Kafka. Two common relays:

• A worker that SELECT ... FOR UPDATE SKIP LOCKED a batch, produces the records, and marks them published.
• Change data capture on the outbox table, for example Debezium's outbox event router, so the publication is driven from the write-ahead log rather than from polling.

Delivery from the relay is still at-least-once: a crash after a successful produce and before marking the row published will send the event again. Consumers deduplicate by event id. The idempotent producer (enable.idempotence=true) stops the relay's own retries from appending a second copy of the same produce request.

Kafka transactions do not replace the outbox when the source of truth is the database. They atomicise writes to Kafka, including the consumer offset, not a write to Postgres plus a write to Kafka.

The same pattern, aimed at a microservices audience, is also answered in How do you publish events reliably when you update the DB and send to Kafka?.