Learn Prompt Hacking

The Problem

Teams building generative‑AI applications often lack a single, reproducible source that teaches both offensive prompt‑hacking techniques and defensive counter‑measures. Without structured, runnable examples, learners waste time recreating experiments and miss best‑practice safeguards.

What This Does

The repository bundles a curriculum of 60 files that walk a user through prompt fundamentals, attack scenarios, red‑team tooling, and blue‑team defenses. Core learning material lives in the notebook files:

1_Basics/*.ipynb – introduces AI concepts and prompt‑engineering basics.
3_Prompting_Hacking/*.ipynb – demonstrates jailbreaks, leakage attacks, and bypasses.
4_RedTeam_LLM/*.ipynb – shows large‑scale red‑team workflows and evaluation scripts (e.g., Use_DSPy_to_evaluate_LLMs.ipynb).
5_BlueTeam_Defence/*.ipynb – implements input‑filtering and Llama‑Guard wrappers.
7_Evaluation_and_BenchmarksTesting/*.ipynb – provides benchmark notebooks that call external tools such as Inspect, Mindgard, and garak.

Supplementary markdown files in Hack_Resource_Collections/, Paper_Collections/, and Conference_Slides/ give background reading and slide decks.

How It Is Wired

The repo contains no executable script, package, or import graph. Each notebook is a self‑contained entry point; they import only standard libraries (e.g., openai, pandas) and external tools that must be installed manually. No functions are shared across notebooks, so there is no internal call graph to trace. Consequently:

File / Folder	Primary Responsibility
`1_Basics/*.ipynb`	Teach prompt‑engineering fundamentals.
`3_Prompting_Hacking/*.ipynb`	Run offensive attack demos; each notebook runs its own code cells.
`4_RedTeam_LLM/*.ipynb`	Execute red‑team pipelines, call evaluation libraries (`DSPy`).
`5_BlueTeam_Defence/*.ipynb`	Apply defensive filters (Llama‑Guard, Prompt‑Guard).
`7_Evaluation_and_BenchmarksTesting/*.ipynb`	Invoke third‑party benchmark suites (`inspect`, `garak`).
Markdown resources	Provide theory, references, and slide content.

Because there is no shared module, changing a technique in one notebook does not affect the others, but also means there is no single “run‑all” orchestrator.

How To Use It

# Clone the repository (exact URL required)
git clone https://github.com/moses-y/Learn-Prompt-Hacking
cd Learn-Prompt-Hacking

# Install a Python environment manually (no manifest is provided)
python -m venv venv
source venv/bin/activate
pip install openai pandas  # plus any library referenced inside notebooks

No requirements file exists; you must inspect each notebook for additional imports.

Open a notebook in JupyterLab or VS Code:

jupyter lab 1_Basics/01.😄Introduction_to_AI.ipynb

Run cells sequentially; the notebooks contain explanatory markdown and code blocks. For evaluation notebooks, install the referenced tools (e.g., pip install inspect-ai garak) before execution.

Real‑World Use

A security‑team lead could adopt the 5_BlueTeam_Defence notebooks to prototype an input‑filter pipeline for an internal LLM API. By copying the code from Apply_Input_processing_with_Prompt_Guard_(meta_llama).ipynb into a microservice, the team gains a ready‑made guard that can be benchmarked with the 7_Evaluation_and_BenchmarksTesting suite to verify mitigation strength against known jailbreak prompts.

Code Health & Issues

HIGH – Pin the environment – 17 notebooks lack a dependency manifest; reproducibility is impossible. Fix: add requirements.txt or environment.yml.
MEDIUM – Strip notebook outputs – 3_Prompting_Hacking/04_Prompt_Leakage_Attack.ipynb (and two others) contain >1 MB of stored output. Fix: run nbstripout to clean outputs.
Tests present – 4 test files exist, but they are not wired to any CI pipeline.
CI – No .github/ workflows or other CI configuration.
Dockerfile – None provided.
License – LICENSE is present.
Lockfile – No requirements.txt, Pipfile.lock, or similar.
Committed secrets – None detected.

The Bottom Line

The repo supplies a rich, notebook‑driven curriculum covering prompt hacking and defense, valuable for training and quick prototyping. However, the lack of a dependency manifest, CI, and shared code modules limits reproducibility and integration into production pipelines. It is best suited for educational or research environments where users are comfortable installing libraries manually and running notebooks in isolation.