In audio and video calls or live streaming, by default, only members in the channel can speak. To create an atmosphere and foster an environment where various voices are allowed, users can add background music, accompaniment, or scene effects such as applause and whistle to the human voice through the mixing feature. Then the system can play the synthesized voice to others members in the channel.

Overview

Mixing means that SDK gets audio data from the App. The SDK integrates audio data provided by the App with SDK-captured audio data. In most cases, mixing is used in scenarios such as music live streaming, online KTV, co-hosting PK, and game live streaming.

NERTC SDK provides mixing-related methods to play short-time sound effects and background music.

• Play short-time sound effects:

  Short-time audio files are played during calls or live streaming. In most cases, you can use the feature to create the room atmosphere. We offer short-time sound effects such as game sound effects, applause, whistles, cheers, laughter, and support overplaying multiple sound effects.

• Play background music:

  Mixing allows you to play local or online music files in the room as background sound during calls or live streaming. Other members in the room can hear the music. You can use NERTC mixing method to play longer background sounds such as accompaniment, ambient white noise, and background music.

The NERTC SDK mixing feature supports the following settings:

• Mixing: The SDK mixes (overlies) and encodes the audio stream of music file with the audio stream captured by a microphone, then sends the mixing to the other member.
• Loop: You can set whether to loop the mixing file, and the times of loop.
• Volume adjustment: You can control the mixing volume played locally or sent by encodes separately.
• Position: You can start playing the mixing task from anywhere in the background music file.

Considerations

• If the returned value is lower than 0 when you use accompaniment-related method, you fail to call the method.
• Start the mixing task after you successfully join the room.
• Local mix files supported formats: MP3, M4A, AAC, 3GP, WMA, and WAV, local files, and online URLs.
• You can receive information about music file playback state changes through onAudioMixingStateChanged callbacks. If playback error occurs, you can troubleshoot through corresponding error codes.

Method

To implement mixing in your project, see following steps for details.

1. After users join the room, you can call startAudioMixing to create a mixing task.

2. Mixing audio task management.

   After the SDK starts a mix, you can take other ways to implement more features of mixing task management.

   • Process management: You can pause or resume mixing through pauseAudioMixing and resumeAudioMixing, and set playback process of mixing files through setAudioMixingPosition.
   • Volume management: You can set playback volume of mixing and sending volume of accompaniment through such ways as setAudioMixingPlaybackVolume andsetAudioMixingSendVolume .
   • Get information related to mixing: For example, you can get playback process of mixing files through such APIs as getAudioMixingCurrentPosition.

3. Calls stopAudioMixing to stop mixing before a user leaves a room.

Sample code

    // Set parameters for audio mixing first.
    NERtcCreateAudioMixingOption option = new NERtcCreateAudioMixingOption();
    option.path = "/sdcard/test.mp3"; //The path of audio mixing file.
    option.playbackEnabled = true;    //Specify whether to play locally.
    option.playbackVolume = 100;      //The local playback volume.
    option.sendEnabled = true;        //Specify whether to send by encoding.
    option.sendVolume = 100;          //The sent volume. 
    option.loopCount = 1;             //The times of loop.
    int ret = NERtcEx.getInstance().startAudioMixing(option);; //Starts audio mixing.
    if(ret == NERtcConstants.ErrorCode.OK) {
        //A mixing audio task is created. 
    }else {
        //A mixing audio task is not created. 
    }
    
    // Pause and stop mixing tasks.
    NERtcEx.getInstance().pauseAudioMixing();
    NERtcEx.getInstance().resumeAudioMixing();
    
    // Get and set the local playback volume of mixing audio. The volume range is 0 to 100.
    int volume = NERtcEx.getInstance().getAudioMixingPlaybackVolume();
    NERtcEx.getInstance().setAudioMixingPlaybackVolume(volume);
    
    //  Get and set the volume of sending encoded mixing audio. The volume range is 0 to 100.
    int volume = NERtcEx.getInstance().getAudioMixingSendVolume();
    NERtcEx.getInstance().setAudioMixingSendVolume(volume);
    
    // Get the duration of a mixing audio file.
    long time = NERtcEx.getInstance().getAudioMixingDuration();
    
    // Get the playback position of a mixing audio file.
    long position = NERtcEx.getInstance().getAudioMixingCurrentPosition();
    
    // Set the playback position of a mixing audio file.
    NERtcEx.getInstance().seekAudioMixingPosition(position);
    
    // Stop a mixing audio task. 
    NERtcEx.getInstance().stopAudioMixing();

API reference