Test an LLM

LLM Chat tests your agent’s text LLM directly through the provider’s native chat API. No browser, no phone, no audio: the AI tester sends messages as text and reads the replies as text, turn by turn. It is the fastest and cheapest connection type, which makes it the right tool for a high-frequency sanity check — run it on every deploy to catch regressions in prompt logic, tool calls, and response content before you spend credits on phone or browser tests.

Audio quality, TTS voice, and widget UI are out of scope here — for those, use phone, web voice, or web chat.

Before you start

• A Testzilla account (sign up).
• A Retell agent with chat enabled in the Retell dashboard. (Vapi text chat is not supported in v1; voice-only agents will be rejected.)
• The Retell chat agent ID.

Set it up

1. Create a project, or let Tessie do it from a plain-language prompt.

2. Add an LLM Chat channel

   Click + Channel, pick the LLM Chat adapter (the platform is forced to Retell), and supply the Retell Chat Agent ID.

3. Write a test with the scenario and pass/fail criteria — the conversational logic, prompt instructions, dynamic variables, or tool responses you want to verify.

4. Run it. Because it is text-only, runs are quick and cheap.

5. Read the result — transcript, verdict, score, and analysis.

Note

LLM Chat costs 1 credit per round-trip. A 10-turn conversation costs 10 credits.

Chat must be enabled

If you see “Cannot start a chat session with selected agent”, the agent’s chat mode is not enabled. Enable it in the Retell dashboard for each agent you want to test via LLM Chat.

Next steps

Run it on every deploy

Wire LLM Chat into CI as a fast regression gate.

Testing Retell

Retell across all four channels.

Run a suite

Benchmark the same suite across LLM, voice, and chat.