English | MP4 | AVC 1280×720 | AAC 44KHz 2ch | 120 lectures (8h 23m) | 3.11 GB
START HERE: Learn Apache Kafka 3.0 Ecosystem, Core Concepts, Real World Java Producers/Consumers & Big Data Architecture
If you look at the documentation, you can see that Apache Kafka is not easy to learn…
Thanks to my several years of experience in Kafka and Big Data, I wanted to make learning Kafka accessible to everyone.
We’ll take a step-by-step approach to learn all the fundamentals of Apache Kafka.
At the end of this course, you’ll be productive and you’ll know the following:
- The Apache Kafka Ecosystem Architecture
- The Kafka Core Concepts: Topics, Partitions, Brokers, Replicas, Producers, Consumers, and more!
- Launch your own Kafka cluster in no time using native Kafka binaries – Windows / MacOS X / Linux
- Learn and Practice using the Kafka Command Line Interface (CLI)
- Code Producer and Consumers using the Java API
- Real world project using Wikimedia as a source of data for a producer and OpenSearch as a sink for our consumer
- Overview of Advanced APIs (Kafka Connect, Kafka Streams)
- Real World Case Studies and Big Use cases
- Overview of Advanced Kafka for Administrators
- Advanced Topic Configurations
Note: The hands-on section is based on Java, which is the native Kafka programming language. But, good news! Your learning in Java will be completely applicable to other programming languages, such as Python, C#, Node.js or Scala, and Big Data frameworks such as Spark, NiFi or Akka
What you’ll learn
- Understand Apache Kafka Ecosystem, Architecture, Core Concepts and Operations
- Master Concepts such as Topics, Partitions, Brokers, Producers, Consumers
- Start a personal Kafka development environment
- Learn major CLIs: kafka-topics, kafka-console-producer, kafka-console-consumer, kafka-consumer-groups, kafka-configs
- Create your Producers and Consumers in Java to interact with Kafka
- Program a Real World Twitter Producer & ElasticSearch Consumer
- Extended APIs Overview (Kafka Connect, Kafka Streams), Case Studies and Big Data Architecture
- Practice and Understand Log Compaction
Table of Contents
Kafka Introduction
1 Course Introduction
2 Apache Kafka in 5 minutes
3 Course Objectives
4 Welcome About your instructor
Code Download
5 Code Download
Kafka Fundamentals
6 Kafka Fundamentals
Kafka Theory
7 Topics Partitions and Offsets
8 Producers and Message Keys
9 Consumers & Deserialization
10 Consumer Groups & Consumer Offsets
11 Brokers and Topics
12 Topic Replication
13 Producer Acknowledgements & Topic Durability
14 Zookeeper
15 Kafka KRaft Removing Zookeeper
16 Theory Roundup
Starting Kafka
17 Important Starting Kafka & Lectures Order
18 FAQ for Setup Problems
19 Starting Kafka with Conduktor Multi Platform
20 Mac OS X Download and Setup Kafka in PATH
21 Mac OS X Start Zookeeper and Kafka
22 Mac OS X Using brew
23 Linux Download and Setup Kafka in PATH
24 Linux Start Zookeeper and Kafka
25 Windows WSL2 Download Kafka and PATH Setup
26 Windows WSL2 Start Zookeeper & Kafka
27 Windows WSL2 How to Fix Problems
28 Windows WSL2 Extra Instructions
29 Windows nonWSL2 Start Zookeeper and Kafka
Starting Kafka without Zookeeper
30 Note not ready for production yet
31 Mac OS X Start Kafka in KRaft mode
32 Linux Start Kafka in KRaft mode
33 Windows WSL2 Start Kafka in KRaft mode
CLI Command Line Interface 101
34 CLI Introduction
35 WINDOWS WARNING PLEASE READ
36 Kafka Topics CLI
37 Kafka Console Producer CLI
38 Kafka Console Consumer CLI
39 Kafka Consumers in Group
40 Kafka Consumer Groups CLI
41 Resetting Offsets
Kafka UI
42 Conduktor Demo
Kafka Java Programming 101
43 Kafka SDK List
44 Creating Kafka Project
45 Java Producer
46 Java Producer Callbacks
47 Java Producer with Keys
48 Java Consumer
49 Java Consumer Graceful Shutdown
50 Java Consumer inside Consumer Group
51 Java Consumer Incremental Cooperative Rebalance & Static Group Membership
52 Java Consumer Incremental Cooperative Rebalance Practice
53 Java Consumer Auto Offset Commit Behavior
54 Programming Advanced Tutorials
Kafka Real World Project
55 Real World Project Overview
56 Real World Exercise Solution
Kafka Wikimedia Producer & Advanced Producer Configurations
57 Wikimedia Producer Project Setup
58 Wikimedia Producer Implementation
59 Wikimedia Producer Run
60 Wikimedia Producer Producer Config Intros
61 Producer Acknowledgements Deep Dive
62 Producer Retries
63 Idempotent Producer
64 Safe Kafka Producer Settings
65 Wikimedia Producer Safe Producer Implementation
66 Kafka Message Compression
67 lingerms and batchsize Producer settings
68 Wikimedia Producer High Throughput Implementation
69 Producer Default Partitioner & Sticky Partitioner
70 Advanced maxblockms and buffermemory
OpenSearch Consumer & Advanced Consumer Configurations
71 OpenSearch Consumer Project Overview
72 OpenSearch Consumer Project Setup
73 Setting up OpenSearch on Docker
74 Setting up OpenSearch on the Cloud
75 OpenSearch 101
76 OpenSearch Consumer Implementation Part 1
77 OpenSearch Consumer Implementation Part 2
78 Consumer Delivery Semantics
79 OpenSearch Consumer Implementation Part 3 Idempotence
80 Consumer Offsets Commit Strategies
81 OpenSearch Consumer Implementation Part 3 Delivery Semantics
82 OpenSearch Consumer Implementation Part 5 Batching Data
83 Consumer Offset Reset Behavior
84 OpenSearch Consumer Implementation Part 6 Replaying Data
85 Consumer Internal Threads
86 Consumer Replica Fetching Rack Awareness
Kafka Extended APIs for Developers
87 Kafka Extended APIs Overview
88 Kafka Connect Introduction
89 Kafka Connect Hands On Warning
90 Kafka Connect Wikimedia & ElasticSearch Hands On
91 Kafka Streams Introduction
92 Kafka Streams HandsOn
93 Kafka Schema Registry Introduction
94 Kafka Schema Registry Hands On
95 Which Kafka API should I use
Real World Insights and Case Studies Big Data Fast Data
96 Case Study MySocialMedia
97 Case Study MyBank
98 Case Study Big Data Ingestion
99 Case Study Logging and Metrics Aggregation
100 Choosing Partition Count & Replication Factor
101 Kafka Topics Naming Convention
102 Case Study MovieFlix
103 Case Study GetTaxi
Kafka in the Enterprise for Admins
104 Kafka Cluster Setup High Level Architecture Overview
105 Kafka Monitoring & Operations
106 Kafka Security
107 Kafka Multi Cluster & MirrorMaker
108 Advertised Listeners Kafka Client & Server Communication Protocol
Advanced Kafka
109 Advanced Kafka
Advanced Topics Configurations
110 Changing a Topic Configuration
111 Segment and Indexes
112 Log Cleanup Policies
113 Log Cleanup Delete
114 Log Compaction Theory
115 Log Compaction Practice
116 Unclean Leader Election
117 Large Messages in Kafka
Next Steps
118 Whats Next
119 THANK YOU
120 Bonus Lecture
Resolve the captcha to access the links!