AI Agent
For audio from the LLM, make sure to add custom parameters for modalities and audio. Example:
Select array and enter:
key: modalities
value: audio
Select object and enter:
key: audio
value: { "voice": "alloy", "format": "pcm16" }
Enabling image input will:
The AI Avatar feature requires Agora tokens to be enabled in your Agora project. You cannot use an AppID that does not have tokens turned on.
To enable tokens:
At least one output modality must be selected
You have two options for live subtitles:
Requires RTM to be enabled on your Agora AppID. Supports both viewing transcripts and sending messages.
Uses RTC data stream. No RTM required. View transcripts only, cannot send messages.
If signaling is not properly configured:
Both the client and agent need to be configured with a single token that works for both RTC and RTM for live subtitles to work properly. The transcription system relies on RTM to deliver real-time speech recognition data.