Skip to main content
PPenTesting.org

AI and LLM penetration testing: what a test covers

An AI penetration test is a security test of a system built around an AI model. The model is one part. The test also covers the application around it, the instructions it is given, the data it retrieves and the tools it can use.

What is in scope

CREST’s Security Testing of AI standard says testing also needs to consider applications, prompts and system instructions, retrieval mechanisms, data sources, memory, tools, plugins, APIs, orchestration layers and downstream systems influenced by AI outputs. That is a useful checklist for any AI test, whoever does it.

The parts of an AI system a penetration test can touchSix parts in the order data flows through them: inputs the model reads; the application and system prompt; the model and its supply chain; the retrieval store; tools, agents and connected services; spend and capacity.1Inputs the model readsChat, files, web pages, emailLLM012The applicationSystem prompt, output handlingLLM05 · 073The modelHosted, fine-tuned or openLLM03 · 044Retrieval storeDocuments, vector indexLLM085Tools, agents, servicesWhat it can read or changeLLM066Spend and capacityRequests, tokens, costLLM10
Tags are items in the OWASP Top 10 for LLM Applications 2025. Which part touches which item is our mapping; the item names are OWASP’s.

What a test is mapped to

OWASP publishes a Top 10 for LLM Applications 2025 (the list on OWASP’s site). The table is our mapping of each part of the system to the items it relates to, as a way to write down scope.

Part of the systemWhat a tester looks atOWASP LLM 2025 items
Inputs the model readsChat, uploads, web pages, emails and records that reach the model as textLLM01 Prompt Injection
The application and system promptHidden instructions, how model output is used or displayed, access between usersLLM07 System Prompt Leakage; LLM05 Improper Output Handling; LLM02 Sensitive Information Disclosure
The model and its supply chainHosted API, fine-tuned or open model, plug-ins, training and fine-tuning dataLLM03 Supply Chain; LLM04 Data and Model Poisoning
Retrieval storeDocuments and vector index the model searches, and who can write to themLLM08 Vector and Embedding Weaknesses; LLM02 Sensitive Information Disclosure
Tools, agents and connected servicesWhat the model can read, send or change, and with whose permissionsLLM06 Excessive Agency
Spend and capacityLimits on requests, tokens and cost per userLLM10 Unbounded Consumption

For systems where the model plans and acts, OWASP also publishes the Top 10 for Agentic Applications for 2026 (OWASP Top 10 for Agentic Applications for 2026, dated 9 December 2025).

What a test can and cannot show

The NCSC says “prompt injection attacks may never be totally mitigated in the way that SQL injection attacks can be”. A clean report is therefore not a promise that no instruction can ever steer the model.

On how to respond, the NCSC says to “approach it by seeking to reduce the risk and the impact; if the system’s security cannot tolerate the remaining risk, it may not be a good use case for LLMs”. So a good test spends its time on the second half: when an instruction does get through, what could it reach? That depends on the permissions, data and tools around the model, which is why prompt injection testing reports impact, not only success.

What an AI penetration test doesFour steps: map the system, try instructions through every input, follow what they reach, report what to change.1MapInputs, tools,data, accounts2TryInstructionsvia each input3FollowWhat data andtools it reaches4ReportRepeat steps,what to change
A generic outline of the work. Ask each firm for its own method.

How it differs from a web application test

  • The target takes instructions in plain language. A web app test checks that data cannot become code. With a model, the NCSC’s point is that instructions and data are not cleanly separated, so tests look at what each input could make the system do.
  • The surface is the connections. In our view, much of the exposure sits in what the model is allowed to call or read, such as a mailbox, a database or a payment API, so a test lists those first.
  • Findings are about impact. The same injected text can be harmless in one system and serious in another that gives the model more access.
  • The usual web tests still apply. The application, API and cloud setup around the model are tested the normal way. See web application penetration testing and API testing.

CREST’s two AI accreditations

CREST has launched two AI accreditations in 2026. They answer different questions.

AI-Enabled Penetration Testing

The question: how does this firm use AI in its own testing?

  • CREST AI-Enabled Penetration Testing assesses responsible, secure and appropriate use of AI within penetration testing services.
  • CREST AI-Enabled Penetration Testing is Annex B of its Penetration Testing Accreditation Standard.
  • CREST opened applications for its AI-enabled services accreditation on 28 July 2026.

Security Testing of AI

The question: can this firm test our AI system?

  • CREST launched its Security Testing of AI standard and accreditation on 20 August 2026.
  • CREST Security Testing of AI builds on CREST’s Penetration Testing Accreditation, which organisations must hold or apply for alongside it.
  • CREST’s Security Testing of AI standard says testing also needs to consider applications, prompts and system instructions, retrieval mechanisms, data sources, memory, tools, plugins, APIs, orchestration layers and downstream systems influenced by AI outputs.

CREST AI-Enabled Penetration Testing is about how a provider uses AI in its own services; Security Testing of AI assures a provider can test AI systems. Which firms hold either changes as CREST accredits more, so we do not list holders here; confirm a firm’s status with CREST directly.

What drives the cost

No price figures are published here; there is no sourced price for AI tests yet. What moves a quote:

What moves the price of an AI penetration testSix drivers: tools and permissions; inputs; retrieval sources; user roles; environment; depth of access.Cost driversYour AI systemTools and permissionsHow many, and read or writeInput channelsChat, files, email, webRetrieval sourcesHow many, and who can editUser roles and tenantsAccess between accountsEnvironmentTest copy or liveDepth of accessWith or without the prompt

See what a penetration test costs for the general picture.

Which firms list AI and LLM testing

5 firms in the directory list AI or LLM testing on their own websites (3 serve the UK). That is what the firms say about themselves; it is not an accreditation and it says nothing about depth. See the list and each firm’s source.

What to ask any firm that tests AI systemsFive questions: what its method covers; a sample report; who does the work; whether it uses AI in its own testing; whether a retest is included.Before you chooseAsk each firmMethodApp, prompts, retrieval, toolsA sample reportWith the client removedWho does the workNamed people and theirexperienceIts own use of AIAnd any AI-EnabledaccreditationRetestIncluded or priced

Next steps

Sources

Frequently asked

Is an AI penetration test just testing the model?

No. CREST’s Security Testing of AI standard says testing needs to consider more than the model: applications, prompts and system instructions, retrieval mechanisms, data sources, memory, tools, plugins, APIs, orchestration layers and downstream systems influenced by AI outputs.

Can a test prove my AI system is safe from prompt injection?

No. The NCSC says prompt injection may never be totally mitigated the way SQL injection can be, and that it remains a residual risk to manage through design, build and operation. A test shows what an injected instruction could reach in your system today and whether your safeguards held.

Do I need a firm with a CREST AI accreditation?

Treat it as one signal among several. CREST’s Security Testing of AI accreditation assesses a firm’s ability to test AI systems, and requires the firm to hold or apply for CREST Penetration Testing Accreditation alongside it. Ask any firm for its evidence of AI testing work, whether or not it holds the accreditation.

Find firms that fit your test

Answer a few questions about what needs testing and why, and see matched firms with a link to each one’s contact page.

Match with firms