An AI agent can give a good demo and still fail the customer's real workflow. The demo may use a clean record, a cooperative user and a successful tool response. Acceptance testing should answer a harder question: can this release perform the agreed task, inside the customer's rules, with evidence that an operator can inspect?
Start with the task and the customer owner. Do not begin with a target score borrowed from another product. A useful acceptance plan names the supported work, the cases that must stop, the evidence needed to pass and the person who can accept the result.
Define the unit of work before writing tests
Choose one bounded workflow. Preparing a candidate summary, drafting a customer reply and updating a CRM record are different units of work. Each needs its own input, expected result, allowed actions and review rule. A broad requirement such as handle customer onboarding is too vague to test.
For an illustrative recruiting workflow, define the task as preparing a review note for one application. Require each factual statement to match supplied evidence. Allow the agent to save a draft, but not reject the candidate or send a message. This gives reviewers a concrete result to judge and a clear boundary to test.
Record the starting system state too. A correct result may depend on the application being active, the record belonging to the right customer and a reviewer having approved the content. Include those conditions in the test fixture rather than leaving them in someone's memory.
Turn acceptance criteria into observable checks
Write each requirement so a reviewer can say what passed and what failed. Use direct checks for exact identifiers, required fields and persisted changes. Use a written human rubric for judgments such as whether a summary preserves the important evidence without adding unsupported claims.
OpenAI's evaluation guidance recommends task-specific tests, representative data and human feedback to calibrate automated scoring. Apply those principles to the customer workflow. Do not assume that a high score on answer quality also proves correct tool use or safe writes.
| Requirement | Evidence to inspect | Example failure |
|---|---|---|
| Correct record | Input ID, tool arguments and saved result | Right content on the wrong account |
| Supported claims | Source material and output | Invented experience in a candidate note |
| Approval required | Approval state and action log | Message sent before review |
| Safe repetition | Repeated run and destination state | Another copy of an existing task |
| Clear stop | Error result and operator queue | Success reported after a failed write |
Build a customer-specific test set
Collect approved examples that represent the actual inputs. Include ordinary cases, incomplete records, unusual formats and conflicting information. Keep sensitive data access narrow. Where production data is inappropriate, use synthetic fixtures that preserve the relevant structure and have a customer expert review them.
Keep a held-out group of cases that the builder does not use for prompt tuning. Otherwise the team can improve familiar examples without improving the workflow. Record why each case exists, which requirement it checks and what outcome the customer expects.
Include negative cases deliberately. A missing required identifier should stop the write. An ambiguous instruction should reach review. A record outside the permitted customer scope should remain untouched. Treat these as expected successful stops, not as failures to complete more work.
Inspect tool actions and final system state
Read the generated answer, but also inspect what the agent actually did. OpenAI describes traces as records of model calls, tool calls, guardrails and handoffs. These records can help locate a wrong tool choice or a missed handoff. They are useful evidence, not a substitute for checking the destination.
After a write test, read the target record independently. Confirm its identifier, changed fields and associations. Check that unrelated records did not change. An agent saying saved successfully is not proof that the application accepted the write or that it accepted the intended values.
For a draft-only release, confirm that the output stayed a draft. Inspect the send queue or destination system rather than trusting the final sentence. Keep technical assertions separate from model grading so a fluent explanation cannot override a failed state check.
Agree how human and automated grading work
Give human reviewers examples of acceptable and unacceptable output. Ask them to explain disagreements before using the rubric at scale. If two reviewers interpret a requirement differently, clarify the requirement rather than hiding the difference in an aggregate result.
A model judge can help review defined text criteria. Calibrate its decisions against human labels and inspect disagreements. OpenAI's guidance notes response-order and verbosity biases in model grading. Keep the rubric specific, and do not reward a longer answer when the requirement is accurate extraction.
For repeated runs, preserve every outcome. Generated results can vary for the same input. Agree which critical cases must be repeated and how failures affect the release decision before running them. Do not keep rerunning a failed case until one favorable output appears.
Test failure paths without live consequences
Use a controlled environment or test doubles for actions that could contact people or change business records. Simulate a timeout, a denied permission, a partial tool response and an interrupted run. Check whether the agent stops, queues review or safely resumes according to the written requirement.
Test a repeated request after a successful action. Then test a retry after the destination accepted a write but the caller lost the response. Define the expected result in advance. The test should show whether the workflow can distinguish unfinished work from work already completed.
Package the release decision
- Freeze the prompt, model configuration, tool definitions and integration version being tested.
- Run the agreed cases and preserve outputs, tool records and destination checks.
- List failed requirements and unresolved reviewer disagreements.
- Block release when a critical condition fails.
- Record the customer owner's decision and the exact supported scope.
- Rerun the relevant suite after a material change.
Keep exceptions visible. If a workflow is approved only for drafts or only for one record type, make that limit part of configuration and operator instructions. A meeting note alone cannot prevent the next release from exceeding the accepted scope.
Mistakes to avoid in acceptance testing
- Using only clean examples chosen by the builder.
- Grading the answer without checking tool arguments or saved state.
- Changing pass criteria after seeing the results.
- Combining serious action failures with minor wording issues in one score.
- Testing a new prompt while reporting results from an older version.
- Calling a successful test run proof that future runs cannot fail.
How Remqa supports customer rollout testing
Remqa provides forward deployed engineering capacity for customer integrations and AI rollouts. Bring one workflow, sample inputs and the person who owns business acceptance. Define the release checks before expanding the scope.
Use the pilot-to-production guide to frame the wider rollout. After acceptance, the agent runbook checklist helps turn the accepted behavior into an operating handoff. Keep the test evidence with the release so the next change can be judged against the same requirements.