A practical framework for validating AI interactions across Utterances, Topics, Actions, and Responses.

“How do you validate software that doesn't follow a strict, predictable script?”

Why AI Changes the QA Equation

AI in the Salesforce ecosystem is no longer just predicting churn or recommending the next best action; in 2026, it is actively executing complex tasks. Salesforce’s Agentforce framework empowers organizations to deploy autonomous AI agents that can reason, converse, and update CRM data in real-time.

For QA professionals, this introduces a major challenge: how do you validate software that doesn't follow a strict, predictable script?

Standard regression scripts and Apex code coverage checks are no longer enough. QA must validate the entire lifecycle of an AI interaction.

“The real challenge is validating behavior, not just code paths.”

The Anatomy of an Agentforce Test Case

Traditional Salesforce QA is simple: Input A → Expected Output B. With Agentforce, the agent must understand, decide, act, and communicate. The Agentforce Testing Center breaks automated AI testing into four critical evaluation pillars.

‍

1. The Utterance — The Input

The utterance is the raw text prompt provided by the user. Because users are unpredictable, the test suite must include many variations of utterances for the same goal.

• Case escalation: “Escalate Case 00112233.”
• “I need to speak to a manager about my ticket.”
• “Why is my support issue still unresolved?”

2. Expected Topic — The Classification

Before the agent acts, it must classify what the user wants. Agentforce uses Topics to group capabilities. QA must validate that the utterance triggered the correct Topic; otherwise, the agent may load the wrong instructions.

Example: “Push the Acme Corp deal to next quarter” should route to Opportunity_Management rather than Account_Support.

3. Expected Action — The Execution

Once the topic is identified, the agent chooses a tool or action to execute, such as an Autolaunched Flow, Apex class, or prompt template. This is the most critical technical validation step.

• Verify that Update_Opp_Close_Date_Flow fired with the right parameters.
• Validate extraction of “Acme Corp” and calculation of “next quarter.”
• Prevent junk or unintended data from being committed during the test.

4. Expected Response — The Generation

Because LLMs generate text dynamically, strict string matching is not appropriate. QA should evaluate whether the response is concise, accurate, and semantically confirms the expected business result.

“Test the meaning of the response—not the exact sentence.”

Instead of asserting an exact response string, use LLM-as-a-judge evaluation to assess whether the response accurately confirms the stage or date change based on the Opportunity record retrieved in the Action step.

4 Rules for the New QA Approach

1. Test for Intent, Not Exact Words

Abandon strict string matching. Rely on AI-driven scoring or semantic similarity checks; the exact phrasing can change while the business value remains constant.

2. Test Multi-Turn Conversations

Simulate complex chats. Test whether the agent correctly remembers an Account ID mentioned several messages earlier.

3. Stress-Test the Topic Routing

Inject vague utterances and verify that the agent asks a clarifying question rather than guessing the wrong Topic.

4. Guard Against Unauthorized Actions

Use prompt-injection scenarios to test whether the agent can bypass policy or Salesforce security.

The Catch: Current Limitations of Agentforce QA

QA teams must orchestrate their tests around several native Salesforce limits and AI-specific constraints.

‍

• Testing Center Execution Limits: Jobs are capped at a maximum of 10 per hour, with a hard limit of 500 test cases per job.
• Performance Bottlenecks: Salesforce recommends smaller batches of around 20–30 test cases per execution to optimize performance and avoid rate limits.
• Sandbox and Test-Data Dependency: Realistic regression testing depends heavily on representative sandbox data, relationships, permissions, and configurations.
• Non-Deterministic Evaluation: LLM-based evaluation can vary as underlying models and testing services evolve.
• Limited Explainability: Session Tracing can provide visibility into conversation, reasoning steps, selected subagents, actions, prompts, and execution details, but may not provide a deterministic explanation.
• Regression Result Stability: Results can potentially change as Salesforce’s underlying testing and AI services evolve.

‍

Final Thoughts

Deploying an AI agent without rigorous testing risks reputational damage and data leaks. By structuring QA around the rigorous evaluation of Utterances, Topics, Actions, and Responses, teams can turn unpredictable AI into a reliable enterprise asset.

“Turn unpredictable AI into a reliable enterprise asset.”

Practical QA Checklist

• Cover multiple utterance variations for each business intent.
• Verify Topic routing before validating execution.
• Validate action selection, parameters, permissions, and side effects.
• Evaluate responses semantically rather than through exact-string assertions.
• Include multi-turn conversations and memory/context checks.
• Stress-test ambiguous prompts and routing boundaries.
• Test prompt-injection and unauthorized-action scenarios.
• Manage execution limits with controlled batches.
• Maintain representative sandbox data and permissions.
• Use Session Tracing to investigate unexpected behavior.