Issue Description
When integrating the Agora Conversational AI Plugin within the Dify AI platform for a voice-to-voice streaming use case, users experienced the following issues:
The session successfully initialised and played the greeting message initially.
Afterwards, the agent stopped responding to further speech inputs or returned an error message:
“Something went wrong, please try again.”
This affected the continuity of real-time conversational flow, preventing full testing of the voice interaction feature.
Platform/SDK
Platform: Web
Product: Agora Conversational AI Engine
Integration: Dify AI Plugin
Agora Web SDK Version: 4.24.0
Error Message
-
Text-to-Speech (TTS) Error (from 11Labs):
[tts] send_tts_failure_event: get error code 1008, message: "Unusual activity detected. Free Tier usage disabled."Cause: The free-tier 11Labs TTS service was temporarily disabled due to usage restrictions.
-
ASR/Language Parameter Error:
Error code: 1007 Error details: Invalid 'language' query parameter or unknown custom deployment.Cause: The
"language"field in the configuration payload was invalid or missing.
Step-by-Step Solution
-
Verify Text-to-Speech (TTS) Configuration
The Conversational AI engine uses TTS to generate audio responses.
-
If using 11Labs, ensure:
Your API key belongs to a paid plan (the free tier may disable requests after threshold breaches).
-
The TTS credentials are updated in both:
Dify Agora Conversational AI Plugin settings
Agora App Builder configuration
If you encounter the 1008 error, upgrade to a paid plan or generate a new API key.
-
Validate Language Parameter in the Payload
The
languageparameter must match supported ASR/LLM formats.-
For English, use the following key-value pair:
"language": "en-US" Incorrect formats (e.g.,
"language": "en") may trigger theError code: 1007.
-
Ensure Backend Agent Configuration
-
The Dify Agent should handle all backend logic, including:
Speech-to-Text (STT)
Text-to-Speech (TTS)
Real-time response orchestration
The plugin facilitates a 1:1 voice interaction where Dify manages the agent’s conversation logic automatically.
-
-
Re-run the Session
-
After applying the above corrections:
Regenerate the agent configuration with the corrected language and API credentials.
Relaunch the plugin interface URL (e.g.,
https://<your-plugin-id>.ai-plugin.io/convoai-web/index.html).Verify that the greeting message and voice responses function consistently.
-
-
Confirm Successful Integration
-
A successful setup will allow:
Continuous two-way voice interaction.
No disconnections after the initial greeting.
Proper audio response from the agent.
-
Root Cause
The issue occurred due to:
A disabled 11Labs Free Tier TTS account, which prevented voice response generation.
An invalid
languageparameter in the API request ("en"instead of"en-US"), which stopped the ASR service from processing input correctly.