Amazon Nova Sonic voice agent evaluation tool

AWS has introduced the Nova Sonic Test Harness, an open-source framework designed to streamline the evaluation of Amazon Nova Sonic voice agents. This tool facilitates rapid iteration for tuning system prompts and tool configurations by allowing users to run conversations, review results, and make adjustments quickly. It also serves as a comprehensive framework for validating voice agent quality at scale, automatically running multi-turn conversations and employing LLM-as-judge techniques. Notably, it can detect discrepancies between a model's text and audio output, eliminating the need for a physical microphone.