General Training Session: Kafka Tutorial
General Training Session: Kafka Tutorial


The speakers offer an introduction to Apache Kafka, covering basic concepts, the data lifecycle, features, and APIs. The speaker then walks you through at a high level using Kafka’s data pipeline, stream processing, database replication, and CDC, explains how to evaluate if a Kafka cluster is healthy and how to analyze the performance of a Kafka cluster, and shares LinkedIn’s operation tools for Kafka.

LinkedIn has been using Kafka and Samza to build a stream processing platform that processes 1 trillion messages per day. With the above introduction to Kafka, we will first share in-depth knowledge on how we use Kafka to support realtime data ingestion, database change capture, and data re-processing. We will also share how we use Samza to process the data from Kafka to produce the realtime analytical results. Finally, we will talk about how the results are sent to our online serving stores.


Dong Lin is a Sr. Software Engineer at Linkedin where he works on all things Kafka, the Apache distributed stream processing platform. As part of his work he helps support Kafka's operation at LinkedIn, develop new features in Apache Kafka to better serve its users, and participate in the open source community discussion on the evolution of Kafka's design and implementation. Dong completed his PhD from U. Penn in 2015 and has been at Linkedin since then. Dong is interested in distributed systems and various aspects of stream processing and actively contributes to the open source community and forums.

Privacy Settings
We use cookies to enhance your experience while using our website. If you are using our Services via a browser you can restrict, block or remove cookies through your web browser settings. We also use content and scripts from third parties that may use tracking technologies. You can selectively provide your consent below to allow such third party embeds. For complete information about the cookies we use, data we collect and how we process them, please check our Privacy Policy
Consent to display content from - Youtube
Consent to display content from - Vimeo
Google Maps
Consent to display content from - Google