Testing AI Features: A QA Playbook for Non-Deterministic Systems

Traditional test automation rests on one assumption: the same input produces the same output. Large language models break that assumption by design. Ask a support assistant the same question twice and you'll get two differently worded answers, and both may be correct.

That doesn't make AI features untestable. It means the test strategy has to shift from checking exact outputs to checking properties, boundaries and behaviour at scale.

Separate the deterministic from the probabilistic

Most "AI features" are mostly ordinary software. The prompt template, retrieval logic, tool calls, input validation, rate limiting and UI are all deterministic, and they should be tested exactly as you would any other code.

Draw a clear line around the model call itself. Everything outside that line gets conventional unit, integration and end-to-end tests. Everything inside it gets the techniques below.

Assert on properties, not strings

Instead of expect(answer).toBe("..."), assert on qualities every good answer must share:

• Structure: the response parses as the JSON schema the UI expects.
• Grounding: every fact cited appears in the retrieved source documents.
• Boundaries: the answer stays on topic and declines out-of-scope requests.
• Safety: no personal data, credentials or internal URLs leak into the output.
• Format: length limits, language and tone match the product brief.

Many of these can be checked with plain code. The rest can be scored by a second model acting as a judge, as long as the judge's rubric is explicit and you spot-check its verdicts by hand.

Build a golden evaluation set

Collect a few dozen to a few hundred representative inputs: common questions, awkward edge cases, adversarial prompts and real queries from production logs (anonymised). For each one, record what a good answer must and must not contain.

Run the whole set on every prompt change, model upgrade or retrieval tweak, and track the pass rate over time rather than treating one failure as a broken build. A drop from 96% to 88% after a "small" prompt edit is exactly the regression signal you want.

Test the failure modes on purpose

The riskiest behaviour rarely shows up in happy-path testing. Make room in every test cycle for:

1. Prompt injection: inputs that try to override the system instructions.
2. Empty or irrelevant retrieval: what happens when the knowledge base has no answer?
3. Provider outages and timeouts: does the UI degrade gracefully, or hang?
4. Hand-off to a human: does escalation trigger when it should?

Keep humans in the loop

Automated evaluation catches regressions; it doesn't define quality. Schedule regular reviews where product owners and domain experts read a sample of real conversations. Their findings become new cases in the golden set, so the suite gets sharper with every release.

The takeaway

QA for AI isn't about forcing determinism onto a probabilistic system. It's about agreeing what "good" looks like, measuring it consistently and catching drift early. That's the same discipline good test management has always been about, applied to a new kind of component.