General Training Session: Kafka Tutorial
General Training Session: Kafka Tutorial


The speakers offer an introduction to Apache Kafka, covering basic concepts, the data lifecycle, features, and APIs. The speaker then walks you through at a high level using Kafka’s data pipeline, stream processing, database replication, and CDC, explains how to evaluate if a Kafka cluster is healthy and how to analyze the performance of a Kafka cluster, and shares LinkedIn’s operation tools for Kafka.

LinkedIn has been using Kafka and Samza to build a stream processing platform that processes 1 trillion messages per day. With the above introduction to Kafka, we will first share in-depth knowledge on how we use Kafka to support realtime data ingestion, database change capture, and data re-processing. We will also share how we use Samza to process the data from Kafka to produce the realtime analytical results. Finally, we will talk about how the results are sent to our online serving stores.


Prateek Maheshwari is an Apache Samza committer and a Staff Software Engineer at LinkedIn. He's passionate about building scalable, performant and easy to use systems that empower developers to create awesome products. He has recently been working on Samza's new High Level programming API that offers simple and powerful abstractions for creating complex real-time data processing pipelines.

Prateek joined LinkedIn in 2013 with a Master's degree in Computer Science from UT Austin and in Physics from BITS Pilani. Before joining the streams infrastructure team, he worked on data analytics, frontend and backend services for new member sign-ups, and service infrastructure at LinkedIn.

Privacy Settings
We use cookies to enhance your experience while using our website. If you are using our Services via a browser you can restrict, block or remove cookies through your web browser settings. We also use content and scripts from third parties that may use tracking technologies. You can selectively provide your consent below to allow such third party embeds. For complete information about the cookies we use, data we collect and how we process them, please check our Privacy Policy
Consent to display content from - Youtube
Consent to display content from - Vimeo
Google Maps
Consent to display content from - Google