Skip to main content

Agora Conversational AI Engine v2.4 Release

×

March 2026 Agora Monthly Digest Meet our new AI agents

×

Resolving Android Application Deadlocks Caused by Custom Beauty SDK Integration and Video Frame Callbacks

Updated

Issue Description

When integrating custom beauty filter SDKs with the Agora Video SDK on Android, the application may occasionally freeze, become unresponsive, or trigger an Application Not Responding (ANR) error. This issue occurs during video capture when both the beauty processing filter and local video rendering pipeline are running concurrently.

 

Platform and SDK Context

  • Operating System: Android (arm64-v8a)

  • SDK Component: Agora Video SDK (v4.x)

  • Target Issue: Thread deadlock / ANR during video frame capture processing

Root Cause Analysis

This crash/freeze is caused by a thread synchronization deadlock triggered when SDK state-control APIs are invoked inside real-time video frame callbacks.

Under the Hood (Technical Mechanism):

  1. Thread Execution Conflict: The Agora SDK executes onCaptureVideoFrame() on the dedicated video capture/processing thread (VideoCaptureThread). Concurrently, third-party beauty SDKs perform texture and frame format transformations (such as toI420 or EGL context binding) on their own rendering threads.

  2. Lock Acquisition Order: Calling synchronous SDK configuration methods — such as setLocalRenderMode() or mirror mode toggles—inside onCaptureVideoFrame() forces the Agora engine to acquire internal display locks while waiting on the capture thread. Simultaneously, the beauty SDK holds a native frame buffer lock and waits for the Agora rendering thread to complete its cycle.

  3. Deadlock State: Both threads end up waiting on each other's locks, causing a complete pipeline freeze and hanging the application.

     

Solution and Resolution Steps

  1. Decouple Render Mode API Calls from Frame Callbacks

    Never invoke setLocalRenderMode() or other engine state modifiers directly within the onCaptureVideoFrame() callback. Configure the local render mode once outside the frame observer pipeline. 

     

  2. Standardize Mirroring Logic in the Beauty Pipeline

    Avoid toggling mirror modes inside the raw video processing pipeline, as this introduces thread blocking:

    • Delegate mirror transformation handling entirely to Agora’s rendering engine by using VIDEO_MIRROR_MODE_AUTO.

    • Ensure custom beauty SDK configurations do not override the default mirror flags (e.g., set local beauty mirror processing to false and keep getMirrorApplied() returning false).

       

  3. Optimize Heavy Pixel Transformations in Custom Filters

    If your beauty pipeline requires CPU-intensive operations (such as converting RGBA to I420 format via toI420):

    • Offload pixel conversion tasks to GPU shaders or asynchronous worker threads.

    • Ensure the beauty SDK returns the processed frame within the 33 ms threshold (for 30 fps) to avoid blocking the video pipeline.

       

  4. Verification

    After separating render settings from the video callback, rebuild the application and conduct stress testing with repeated camera flips, beauty effect toggles, and view rotations to confirm zero ANR occurrences.

 

Prevention and Best Practice

  • Keep Callbacks Lightweight: Treat onCaptureVideoFrame() purely as a data-transformation hook. Do not update UI components, query layout states, or re-configure SDK options inside this callback.

  • Asynchronous Offloading: If UI or state changes must occur in response to a frame event, dispatch the action asynchronously to the Main Thread via a Handler or Coroutine rather than blocking the capture thread.

Related to