Define the exact range of inputs your agent is expected to handle and test systematically beyond those boundaries. What happens when a user submits an oversized payload? A malformed JSON structure? An input in an unexpected language?

Output boundaries matter equally. Verify that responses stay within acceptable format constraints, token limits, and content policies — not just under ideal conditions, but under load and with adversarial inputs.

Trusys tool: TruEval runs structured boundary tests across input/output dimensions as part of your evaluation suite.

Prompt injection remains one of the most exploitable vulnerabilities in production LLM agents. Attackers embed instructions inside user inputs — or within documents and data sources the agent retrieves — to hijack its behaviour, bypass guardrails, or exfiltrate sensitive information.

Before go-live, run a dedicated prompt injection test suite covering direct injection (user input), indirect injection (retrieved content), and multi-turn injection attempts where instructions are spread across conversation turns.

Trusys tool: TruScan scans for injection vulnerabilities and known LLM security risks before deployment.

Every LLM agent will hallucinate at some rate. The question is whether your team knows that rate — and whether it is within acceptable tolerance for your use case. A hallucination rate of 2% might be acceptable in a content summarisation tool. It is not acceptable in a medical information system or a financial compliance chatbot.

Test hallucination rate under a range of conditions: normal load, high query volume, low-context inputs, and adversarial prompts designed to induce confabulation. Establish a baseline and set a hard threshold before deployment.

Guardrails are not a passive safety net. They need to be actively verified to confirm they intercept the right outputs, do not over-block legitimate responses, and respond consistently across varied phrasing of the same request.

Test each guardrail policy against a representative sample of expected violations, near-miss cases, and clearly acceptable content. Document pass/fail rates and resolve conflicts between policies before they reach production.

Trusys tool: TruGuard's policy engine can be run in simulation mode against test cases to validate guardrail coverage before promoting policies to production.

Context window limits create a failure mode most teams underestimate: when an agent’s conversation history or retrieved documents approach the token limit, model behaviour can degrade significantly — producing truncated reasoning, ignoring earlier instructions, or silently dropping important context.

Test how your agent behaves at 70%, 90%, and 100% of its context limit. Verify that truncation is handled gracefully, that critical instructions persist, and that the agent communicates limitations to users rather than silently degrading.

Agentic AI systems — particularly those executing multi-step workflows, tool calls, or autonomous decision chains — must produce consistent results across repeated executions of the same task. Non-deterministic behaviour in a customer-facing workflow is a reliability failure, not a feature.

Run the same task sequence multiple times under identical conditions and measure variance in outputs, intermediate steps, and tool call sequences. Any significant divergence needs to be understood and bounded before deployment.

Trusys tool: TruEval supports automated eval generation for multi-step LLM agent workflows.

For agentic systems with tool-calling capabilities, unauthorized or escalated tool use is one of the most common real-world failure modes — and one of the hardest to catch in a one-time review. An agent that behaves correctly in isolated tests can still chain permitted actions in unintended ways once it operates autonomously.

Before go-live, verify that the agent cannot invoke tools outside its authorized scope, cannot chain tool calls to escalate privileges or access systems beyond its intended boundary, and that tool outputs are validated before the agent acts on them. Test with adversarial prompts specifically designed to induce out-of-scope tool calls, and confirm that failed or rejected tool calls are logged and surfaced rather than silently retried.

Trusys tool: TruScan and TruGuard together can be used to test tool-call boundaries and enforce authorization policies on agent actions before deployment.

AI agents that have access to enterprise data — through RAG pipelines, tool integrations, or system prompts containing sensitive context — can inadvertently surface that data in user-facing outputs. This is a data leakage risk with direct GDPR and HIPAA implications.

Before launch, test whether the agent can be prompted into surfacing system prompt contents, revealing details from retrieved documents outside their intended scope, or outputting PII that was included in training or retrieval context. Document what data the agent has access to and confirm that boundary holds under adversarial pressure.

Trusys tool: TruScan identifies data leakage vulnerabilities in LLM systems before they reach production.

Compliance is not a post-deployment concern. Before launch, verify that your agent’s behaviour, data handling, and documentation satisfy the requirements of the regulatory frameworks that apply to your deployment context. For EU-based or EU-serving enterprises, this means EU AI Act Articles 12 and 22 (transparency and human oversight). For ISO/IEC 42001, it means demonstrating documented risk assessment and governance controls.

Map each compliance requirement to a specific control in your deployment — and where gaps exist, document them with a remediation plan before go-live. Regulators will ask for this evidence.

Trusys tool: TruScout maps agent behaviour to compliance requirements across EU AI Act, ISO 42001, and NIST AI RMF.

Distinct from broader compliance mapping, human oversight — as required under EU AI Act Article 14 — is one of the most commonly overlooked items in pre-launch reviews, because it depends on workflows that only activate in edge cases. An agent can pass every functional test and still lack a working path for a human to intervene when it matters.

Before go-live, verify that escalation paths are defined for low-confidence outputs, high-stakes decisions, and policy violations; that override and pause mechanisms actually halt agent action when triggered, not just log the event; and that the humans responsible for oversight have the context and access needed to act, not just a notification. Test these triggers under realistic conditions, not just in isolation.

Trusys tool: TruScout and TruGuard can be used together to map oversight requirements to specific escalation and override controls, and verify those controls function as configured.

Model drift — where an AI system’s performance degrades silently over time as input distributions shift — is one of the most common causes of post-deployment failures. The time to set up drift detection is before launch, not after you notice a problem.

Before go-live, verify that you have established baseline performance metrics, defined drift thresholds that will trigger alerts, and confirmed that your monitoring infrastructure is active and receiving production traces. If you cannot answer ‘how will I know when this agent starts degrading?’ — you are not ready to ship.

Trusys tool: TruPulse provides continuous production monitoring and drift detection with configurable alert thresholds.

When something goes wrong with a production AI agent — and eventually something will — your ability to investigate, remediate, and demonstrate compliance depends entirely on what you logged. Incomplete audit trails do not just create operational blind spots; they are a direct regulatory liability under frameworks like the EU AI Act, which mandates traceability for high-risk AI systems.

Before go-live, confirm that every agent action — including tool calls, intermediate reasoning steps, retrieved documents, and output decisions — is logged with sufficient context for post-incident analysis. Verify retention periods, access controls on log data, and that your logging does not itself create a PII exposure risk.

​

Trusys tool: TruPulse captures structured production traces that form the foundation of an audit trail for regulator review.

Consider a financial services chatbot that passed every pre-launch evaluation with strong scores on accuracy and tone. Three months into production, a seasonal spike in query volume pushed conversation histories closer to the model’s context limit than they had been in testing. At 90% of the context window, the agent began dropping earlier compliance disclaimers from its responses — a failure mode that had never been tested, because the original evaluation only ran at typical, low-volume conditions.

Nothing in the agent’s code changed. The failure was entirely a function of a test gap — one that Section 5 of this checklist exists to close. This is the pattern behind most post-deployment AI incidents: not a single catastrophic flaw, but an untested edge case that only surfaces under real-world conditions.

The most common reason teams skip items on this checklist is not negligence — it is the false confidence that comes from a successful pre-launch evaluation. A one-time test tells you how your agent performs on the day you tested it. It says nothing about how it performs next month, after a model update, after your data distribution shifts, or after a user discovers an edge case your test suite never considered.

Genuine AI assurance requires continuous governance — not a launch gate. The 12 items above are your pre-deployment baseline. The real work is maintaining them in production.

Trusys helps enterprise teams implement this checklist systematically — covering pre-deployment evaluation, security scanning, guardrail verification, compliance mapping, and continuous production monitoring from a single platform.

Book a demo to see how TruEval, TruScan, TruGuard, TruPulse, and TruScout work together to make your AI agents governable — before and after they go live.