The custom video collection refers to the function where the developer provides custom video input source data to the NERTC SDK, which then encodes and pushes the data.

Overview

Generally, the app uses the default device to collect video data, which is usually the camera module of the device. However, you may need to use a custom video source in some scenarios. For example:

• The third-party Face++ SDK is responsible for video data collection and pre-processing while the NERTC is responsible for video data encoding and cloud pushing.
• Some video collection devices are exclusively owned by the system. In order to avoid conflicts with other services, flexible device management strategies are required. For example, a short video needs to be recorded during a live streaming event.
• It is required to use external video sources, that is, data collected by devices other than the camera, for example, playing local video files, screen sharing, or game live streaming.

Based on the above scenarios, the NERTC SDK enables developers to use custom video sources to implement the related requirements in these business scenarios. This article describes how to implement the custom video collection function.

Considerations

In a custom video collection scenario, you need to manage the collection and processing of video data by yourself.

Method

The NERTC SDK currently provides the Push method to implement custom video sources. In Push mode, the SDK crops or zooms the data source according to the set video profile to achieve the target resolution, and the frame rate of the video is controlled by the data source itself.

You can take the following steps to implement the custom video source function through the Push method in your project:

1. You can specify the external video collection device by calling setExternalVideoSource before calling enableLocalVideo.

2. After that, the developer manages the video data collection and processing by himself.

3. After the video data is processed, it is sent to the SDK by calling pushExternalVideoFrame for subsequent operations.

   To meet actual needs, you can modify the video data before sending it back to the SDK through NERtcVideoFrame. For example, you can set rotation as 180 to make the video frame rotate clockwise by 180°.

Sample code

javaNERtcEx.getInstance().setExternalVideoSource(true); //Set to using custom data source
NERtcEx.getInstance().enableLocalVideo(true)//Start the video
NERtcEx.getInstance().pushExternalVideoFrame(videoFrame); //Push the custom data

private void changeExternalVideo(boolean external) {
    if (external) {
        NERtcEx.getInstance().enableLocalVideo(false); //Stop the original video first
        NERtcEx.getInstance().setExternalVideoSource(true); //Set it as external video
        if (mExternalVideoType == 2) {
            mExternalVideoSource = new ExternalTextureVideoSource(mExternalVideoPath, this);
        } else {
            mExternalVideoSource = new ExternalYuvVideoSource(mExternalVideoType, mExternalVideoPath,
                    mExternalVideoWidth, mExternalVideoHeight, mExternalVideoFps, mExternalVideoRotation);
        }
        NERtcEx.getInstance().enableLocalVideo(true); //Start the video push
        mExternalVideoSource.start(); //Send external data to SDK
    } else {
        if (mExternalVideoSource != null) { //Stop sending external data to SDK
            mExternalVideoSource.stop();
            mExternalVideoSource = null;
        }
        NERtcEx.getInstance().enableLocalVideo(false); //End the video push
        NERtcEx.getInstance().setExternalVideoSource(false); //Set it as internal camera
        NERtcEx.getInstance().enableLocalVideo(true); //Start the video push
    }
}