Adversarial testing · Instruction attacks

Prompt Injection & Jailbreak Testing

Instruction attacks must be traced from untrusted text to prompts, retrieval, tools, permissions, data, and downstream actions under approved rules of engagement.

An instruction may arrive in a user message or sit inside a document the system retrieves. We follow both routes through the application, prompts, policies, tools, permissions, protected data, and downstream actions. Each verified bypass comes with a controlled trace and a retest that includes ordinary valid requests.

Each defense is recorded next to the valid requests it still has to serve, alongside the authorized attack corpus, controlled bypass traces, and impact findings.

Test the instruction boundaryView the security service
Illustration of Prompt Injection & Jailbreak Testing: a team probing an AI system for security weaknesses

Some of the 500+ brands we've worked with

See all references
  • GE
  • Defacto
  • Eureko Sigorta
  • GS Store

Steps, gates, and who decides

How we work

Attack-case design and defense acceptance remain separate. After remediation, the affected attacks and their corresponding valid requests run again so the fix and its regressions can be judged together.

  1. Authorize the corpus and handling rules

    Together we set the system boundary, protected assets, known attacks, test environment, prompts, policies, retrieval sources, tools, permissions, logs, stop conditions, and evidence-handling rules. No case runs until the charter permits it.

    AI assist
    Known injection patterns seed candidate corpus entries. The security owner chooses what enters the approved set.
    Human gate
    Are the approved corpus and handling rules complete for every authorized route, safety limit, and stop condition? Your security owner authorizes the attack scope, safety limits, and stop conditions.

  2. Follow each instruction to its effect

    We exercise reviewed malicious and benign cases through direct input, indirect content, and tool-impact routes. The retained trace shows which instruction reached the model, how the control responded, and what behavior or side effect followed.

    AI assist
    Inside the authorized environment, an agent may run approved cases and record the instruction path.
    Human gate
    Can a second authorized tester reproduce the bypass or blocked outcome from the retained trace? A Zeo security specialist verifies a bypass before it is recorded as a finding.

  3. Judge the bypass in context

    Each verified bypass is reviewed against the affected asset, impact, detection, severity, unauthorized tool or data exposure, and false-positive cost. Consequential paths are reported separately so an aggregate result cannot obscure them.

    AI assist
    Bypass evidence is sorted by affected asset, impact, and detection so a specialist can judge it.
    Human gate
    Which verified findings must close before your security lead will accept the tested system state? Which verified findings require remediation before acceptance is a call for your security lead.

  4. Retest the defense

    After a change, we repeat the affected attacks and the related benign cases. The linked results show whether the bypass closed and whether the new defense rejected legitimate behavior.

    AI assist
    An agent may rerun the approved attacks and benign cases, then attach the results to the original record.
    Human gate
    Do corpus coverage, critical paths, and valid-request checks meet the defense-regression threshold you approved? You decide whether the defense-regression threshold has been met.

Named artifacts you keep

What you get

Your engineers receive controlled evidence that lets them reproduce a verified bypass, inspect the relevant control, and compare the remediation result without exposing the record beyond approved access.

  • Dataset

    Approved test charter and attack corpus

    The authorized boundary plus the malicious and benign cases selected for direct, indirect, and tool-impact routes.

  • Test evidence

    Controlled bypass reproduction trace file

    Controlled evidence tying the test input to the instruction that reached the model, its response, any tool or data effect, and the control outcome.

  • Risk register

    Reviewed impact and control findings

    Verified bypasses organized by affected asset, impact, detection behavior, severity, false positives, and the control expected to respond.

  • Report

    Remediation notes and retest record

    Defensive recommendations linked to each finding, with results from repeating the affected attacks and valid requests.

Scope and honest limits

When to bring us in

This work fits applications where untrusted text can influence a tool, protected data, retrieved material, or an action with consequences.

A good fit when

  • Untrusted text enters through user messages or retrieved documents, but nobody has traced which instructions reach the model and its tools.
  • A successful override can call a tool or expose protected data, so one bypass may trigger a downstream action with consequences.
  • The defense blocks an attack case, yet the team lacks paired valid-request evidence to show that ordinary behavior still works.
  • Instructions move through prompts, policies, retrieval sources, tools, and permissions, but the full route to protected assets remains untraced.
  • Direct and indirect attacks are known, but the benign and malicious cases lack one reviewed corpus.
  • A bypass is reproduced, but its impact, detection, severity, false positives, remediation, and retest result are not reviewed together.
  • The approved input has a controlled trace, while the instruction path, control response, and observed tool or data effect remain disconnected.

Better handled as other work when

  • You want a verified bypass to settle a regulatory or legal question. Audit opinions and certification remain the work of your qualified authority.
  • You need assurance for every future prompt, model version, tool, or indirect-injection route, although the corpus covers only reviewed cases.
  • You need the remediation deployed to production, because this engagement ends with defensive notes and linked retest evidence.

If one of these is closer to your situation, start here instead: View the security service

We operate the systems we test

It's hard to test a system well if you've never had to keep one running. We operate production AI ourselves, so our evaluation, security testing, and LLMOps work starts from what actually breaks. The people on it are senior engineers, and Zeo has been doing client work since 2011.

Content we've produced on this topic

Articleby Burak PehlivanWhat is WebMCP? Agentic Web and Websites Interacting with AI AgentsArticleby Yiğit KonurMCP Server Architecture: State Management, Security & Tool Orchestration

Tools we use

Tools behind this work

  • Promptfoo

    the plugin suite covering both direct jailbreaks and document-borne indirect injection

  • garak

    the maintained probe library covering publicly documented jailbreak families

  • Lakera Guard

    the runtime layer the defense-regression retest is actually run against

  • Giskard

    the scan run that scores false positives alongside true findings in one pass

Next step

Follow the untrusted instruction all the way through

Bring the application boundary, retrieval sources, tool and data permissions, and the security owner who can approve the cases and stop conditions. Each authorized instruction path is traced to its result.
Test injection paths

Before you decide

Questions we hear a lot

We usually need the deployed or test application, architecture, prompts, policies, retrieval sources, tools, permissions, protected assets, logs, known attacks, and approved rules of engagement. Before any sensitive input is handled, we agree its purpose, access limit, and retention conditions.