Test your Agent in Playground
Run realistic, repeatable conversations in the isolated Playground and diagnose failures at the correct source.
Playground is the interactive test surface for the customer-facing Agent. It lets authorized teammates try realistic conversations without relying on the public storefront. The B2B workspace opens Playground through a secure, short-lived handoff, so use the in-product action rather than saving an old test URL.
This guide shows how to design useful scenarios, inspect the whole response, classify failures, and turn important cases into repeatable batch tests. It also explains what Playground cannot prove about live installation.
How this fits into Humind
AI Agent is the configuration space for the customer-facing assistant. Knowledge and Catalog provide facts; Guidance shapes response behavior; Tools add actions; Escalation defines human support; Test surfaces let you review the result before deployment.
Changes can affect many conversations, so test representative buying and support scenarios after each meaningful update. A visually correct chat is not enough: verify the answer, product context, available action, and handoff behavior together.
Before you start
Access: Playground requires Agent configuration read access. Correcting the source may also require Knowledge, Catalog, or Agent configuration write access.
- Confirm the active company, catalog market, Agent configuration, and language.
- Prepare expected outcomes for product, policy, support, and edge-case questions.
- Use non-sensitive test data and clearly separate test conversations from real customer work.
Step-by-step workflow
Open a fresh Playground session
Open AI Agent and choose Playground. Humind opens the dedicated surface for the active company. Start a new conversation when validating a source change so earlier messages do not influence the result.
Record the configuration or content change under test and the exact question. A vague statement such as 'the Agent is better' cannot be repeated or reviewed by another teammate.
Run a balanced scenario set
Test common product discovery, a specific product detail, a policy stored in Knowledge, a support request, an unavailable or no-result case, a follow-up question, and an input that should be moderated or declined according to configuration.
Vary phrasing and include realistic ambiguity. The goal is not to force one memorized sentence, but to verify correct facts, honest uncertainty, relevant products, appropriate tools, and the expected handoff path.
Inspect the whole response
Review text, cited or linked content, recommended products, variants, prices, tool output, follow-up suggestions, and escalation behavior. A fluent answer can still be wrong if it selects the wrong market, ignores availability, or uses a stale policy.
When the result fails, identify the source area before editing: Knowledge, Catalog, Guidance, Tools, Escalation, interface, or deployment. Fixing the source usually produces a more durable result than adding another broad instruction.
Create a repeatable regression check
For important scenarios, open Batch testing. Create or select a dataset, add questions manually, import a CSV, or generate questions from the catalog where appropriate. Run the set, review answer status and ratings, and open individual conversations for detail.
Rate or document the outcome using a consistent standard. Export the CSV report when a reviewed artifact is needed, and rerun selected questions after a targeted change.
Finish with a live smoke test
Playground validates Agent behavior, not the storefront installation. After deploying through Sales channels, test a smaller set on the actual allowed domain in a private browser on desktop and mobile.
Verify that the entry point, consent, customer identity collection, product context, and Inbox handoff behave correctly in the live environment.
Permissions and important caveats
- A Playground pass does not prove the widget is installed or styled correctly on the storefront.
- Conversation context affects follow-up answers; use a fresh session for source-level regression checks.
- Generated batch questions are starting material and still require merchant review.
- Do not use real customer personal data in test prompts.
Verify the result
Use this checklist before considering the work complete:
- Each scenario has an explicit expected fact, product behavior, action, or escalation outcome.
- Failures are assigned to a source area before changes are made.
- Critical cases exist in a repeatable Batch testing dataset.
- A post-deployment storefront smoke test passes on desktop and mobile.
Troubleshooting
Playground does not open
Return to the B2B workspace, confirm the active company and Agent configuration access, then use the current Playground action again. Do not rely on a previously saved handoff URL.
A corrected answer still looks old
Confirm the source is saved or published in the active company, remove conflicts, and start a new Playground conversation. If catalog data changed, wait for synchronization and any filter materialization.
Batch and interactive results differ
Compare company, language, product context, dataset question, conversation history, and the time each run occurred. Rerun the exact selected question after confirming the same current configuration.