How to Start Hacking AI

"Hacking AI" is ambiguous, so worth clarifying upfront: this roadmap is about attacking AI systems — prompt injection, jailbreaking, and the other vulnerability classes specific to LLM and agentic applications — not about using AI tools to help with traditional hacking. The first is a genuinely new, fast-growing discipline with its own OWASP Top 10 and real bug bounty payouts. That's what this covers.

This assumes the fundamentals from the general roadmap — AI red teaming builds on normal web application security, it doesn't replace it. Most LLM applications are still web apps with a model bolted on, and they still have all the normal web vulnerabilities on top of the AI-specific ones.

You don't need to be a machine learning engineer

This is the biggest misconception keeping people out of this space. Applied LLM security is much closer to web application security than it is to ML research — you're testing how an application handles untrusted input and constrains a model's behavior, not designing neural network architectures. If you're already comfortable with Burp Suite and basic web app testing, you already have most of the transferable skill you need.

The vulnerability classes: OWASP Top 10 for LLM Applications

This is the standard reference framework, and worth knowing by name — it's what bug bounty programs and security teams are actually scoring against:

Prompt Injection — getting a model to ignore its instructions or follow attacker-supplied instructions embedded in input. The most actively exploited category by a wide margin.
Sensitive Information Disclosure — getting a model to leak training data, system prompts, or other information it shouldn't reveal.
Supply Chain Vulnerabilities — compromised models, datasets, or plugins pulled into an application.
Data and Model Poisoning — corrupting training or fine-tuning data to change model behavior.
Improper Output Handling — treating model output as trusted when it feeds into another system (a classic injection vulnerability, just with an LLM as the untrusted input source now).
Excessive Agency — an LLM-powered agent given more permissions or autonomy than the task actually requires.
System Prompt Leakage — extracting the hidden instructions that shape a model's behavior.
Vector and Embedding Weaknesses — attacks against RAG (retrieval-augmented generation) pipelines specifically.
Misinformation — a model confidently producing false information that a downstream system or user trusts.
Unbounded Consumption — resource exhaustion / denial-of-wallet attacks against pay-per-token model APIs.

Excessive Agency and Improper Output Handling are worth extra attention right now — as applications move from simple chatbots to agents with real tool access (browsing, code execution, sending emails), those two categories are where the actual damage happens. A prompt injection is a curiosity in a chatbot; the same injection against an agent that can execute code or send money is a real incident.

Where to practice

Gandalf (Lakera) — free, 8 escalating levels, the standard first stop. Pure prompt injection, no setup required.
HackAPrompt — more structured: a guided "AI Red Teamer" learning path, a separate Offensive AI Security CTF, and a "Certified AI Red Teamer" credential if you want something to put on a resume. A reasonable next step after Gandalf.
Crucible (Dreadnode) — 70+ challenges spanning both LLM and classical ML security (prompt injection through model inversion and evasion). More advanced, and broader than just LLMs — used as the platform for official competitions at Black Hat and GovTech Singapore. Save this for once the basics are comfortable.

Where the money actually is

Every major AI lab now pays for AI-specific vulnerabilities. As of 2026:

OpenAI runs a dedicated AI-safety bug bounty (launched March 2026) specifically scoped to prompt injection, agent data exfiltration, and harmful autonomous actions — up to $7,500 for a reproducible high-severity report, $100K max for exceptional findings.
Microsoft accepts Copilot/AI/LLM submissions through the standard MSRC portal (the "Microsoft Copilot Bounty").
Google, Anthropic, and xAI all pay for AI-specific findings too, with entry-level payouts starting around $200.

One important, current caveat: Google stopped accepting AI-generated vulnerability reports in March 2026 because the volume of low-quality, AI-written submissions became unmanageable. That's a preview of where this whole space is heading — as report volume grows, low-effort submissions get filtered out faster than in traditional bug bounty. A clear, manually-verified, well-written report matters even more here than in normal web bug bounty. See how to start bug bounty hunting for report-writing fundamentals that apply directly.

Common mistakes

Assuming you need ML research skills to start. You don't — application-layer LLM security is the accessible entry point, and it's also where most of the real-world vulnerabilities actually are.
Collecting jailbreak prompts without understanding why they work. A prompt that works today often stops working after the next model update. Understanding the underlying vulnerability class (why prompt injection works at all) transfers; memorized prompts don't.
Skipping web fundamentals because "it's just prompts." Improper output handling, for example, is a classic injection vulnerability — you need to recognize it as one, which means you need the web security fundamentals first.
Submitting low-effort or AI-generated reports. Given where Google and others are already drawing the line, this is actively counterproductive now, not just bad practice.

Next step

Get comfortable with Burp Suite if you aren't already — you'll use it to inspect and manipulate the actual HTTP traffic between an application and its model, which is where a lot of practical LLM security testing happens. Then start with Gandalf; it's free and takes an afternoon.