The Problem

Teams adopting Apache Kafka face a steep learning curve: core concepts like topics, partitions, consumer groups, and message delivery semantics are abstract until practiced. Most training material is either slideware or a single monolithic demo that doesn't map to a structured curriculum. This repo gives instructors and self-learners a lab-by-lab path from setup to advanced security (LDAP), with scripts to run a local Kafka instance.

What This Does

kafka-training is a courseware repository: each labs/LabNN_*/ folder contains a README.md (objectives), steps.md (guided exercises), and config/ files (server.properties, log4j.properties) to run Kafka for that exercise. Labs progress from setup (Lab01_Setup) through tools (Lab02_Tools), messages (Lab03_Messages), topics (Lab04_Topics), performance (Lab05_Performance), and LDAP auth (Lab08_LDAP).

Two root-level shell scripts, labs/kafka-server.sh and labs/ldap-server.sh, manage the local Kafka broker and an LDAP server for the security lab. Assessments live in assessments/questions.md; root files include a syllabus and materials list.

How It Is Wired

Execution starts at labs/kafka-server.sh. Run without args, it prints usage. The script wraps Kafka's bin/kafka-server-start.sh and bin/kafka-server-stop.sh with a local config/server.properties. The LDAP variant, labs/ldap-server.sh, starts an Apache Directory Server instance using labs/Lab08_LDAP/config/auth.ldif and pirates.ldif for test users.

The import graph is minimal: 0 internal modules, 0 import edges across the 3 code files (1 Java, 2 Shell). The only Java file, labs/Lab08_LDAP/src/TestLDAP.java, is a standalone client that connects to the LDAP server and Kafka broker; it has no internal dependencies. Control flow is script-driven, not code-driven: the shell scripts are the entry points, and the Java test is run manually per Lab08_LDAP/steps.md. The wiring for automated orchestration has not been mapped because none exists — there is no CI, no build system, and no test runner.

File-by-file responsibility:

  • labs/kafka-server.sh — starts/stops the local Kafka broker.
  • labs/ldap-server.sh — starts/stops the LDAP server for Lab08.
  • labs/Lab08_LDAP/src/TestLDAP.java — validates LDAP auth against the broker.
  • Each labs/LabNN_*/steps.md — the actual lab instructions.

How To Use It

Setup: Requires Java JDK 15+, Bash, and git on PATH. No build step — clone and go.

git clone https://github.com/moses-y/kafka-training
cd kafka-training/labs

Running: Start the broker with ./kafka-server.sh (no args prints usage). Follow labs/Lab01_Setup/steps_local.md or steps_virtualization.md for environment-specific first-run steps. For Lab08, run ./ldap-server.sh then compile and run TestLDAP.java per labs/Lab08_LDAP/steps.md.

Configuration: Each lab's config/server.properties sets broker ports and log dirs. Lab08_LDAP/config/client.properties holds Kafka client auth settings. No environment variables are required.

Real-World Use

A team onboarding new Kafka developers runs a weekly session: Lab01–Lab02 in week one (setup, CLI tools), Lab03–Lab04 in week two (producers/consumers, topics), Lab05 for performance tuning, and Lab08 for security. Each student gets a local broker via kafka-server.sh, works through steps.md, and the instructor uses assessments/questions.md for a written check. The LDAP lab doubles as a production rehearsal for teams moving to Kerberos or LDAP-backed ACLs.

Code Health & Issues

Static analysis (measured, not opinion) found 1 high-severity issue:

  • High – Duplicated code blocks – labs/kafka-server.sh and labs/ldap-server.sh share 29 repeated 6-line blocks across the 2 files. The shutdown/cleanup logic is copy-pasted; extracting a shared helper would reduce drift risk.

SDLC observations from the file structure:

  • Med – No CI/CD pipeline – no .github/ or CI config; nothing gates changes.
  • Low – Sparse test coverage – 1 test file, and it is a manual LDAP probe, not an automated suite.
  • Low – No lockfile – expected for shell/Java courseware, but means no dependency pinning.

The Bottom Line

Solid, well-organized courseware for hands-on Kafka training — the lab progression and local scripts are practical. The duplicated shell logic is a maintenance smell but not a blocker. Use it if you need a structured, self-contained Kafka curriculum; skip it if you want production-grade automation or test coverage.