Most engineers who use Kafka can produce to a topic and consume from a consumer group, and would struggle to say what a partition is on disk, why a slow broker is worse for a cluster than a dead one, or what exactly-once actually writes into the log. Those gaps are not academic. They are the reason a partition count gets chosen badly and cannot be undone, the reason a cluster runs out of network before it runs out of disk, and the reason a failover reprocesses a week of history.

This course takes Kafka apart. It starts with the log as an abstraction and works downward into the record batch format, the segment and index files, and the page cache and sendfile path that make a broker fast. From there it covers cluster coordination in the KRaft era, the in-sync replica set and what leader election costs, and the unclean election that trades durability for availability. The producer and consumer are treated as the complicated pieces of software they are: the accumulator and its batching, the partitioner and the key skew it produces, sequence numbers and idempotence, group assignment and the rebalance protocols, offset commits and the delivery semantics that follow from them.

The second half covers transactions and the coordinator, read_committed and the last stable offset, and what an exactly-once pipeline is composed of end to end. Then the ecosystem, Connect and Streams and the Schema Registry, each from the angle of its failure modes rather than its quickstart. The course closes with operations: sizing a cluster from throughput and retention, the metrics that are always wrong when something is wrong, consumer lag measured in time rather than in records, reading an incident from the four commands that matter, and multi-cluster replication with an honest account of what a disaster recovery plan for a log can promise.

Every section names the configuration settings it discusses together with their default values, works through the arithmetic with real numbers, and prefers the failure case to the happy path. The test applied throughout is whether a section could have been written without opening the broker's source or its documentation. If it could, it is not deep enough.

By the end you will be able to size a cluster and defend the numbers, choose a partition count knowing what it forecloses, read a broker's metrics and logs during an incident, explain which guarantees survive a replication link and which do not, and run the failures deliberately before they run themselves.