Moderation

Update time: 2024/03/07 11:13:40

CommsEase provides three types of content moderation services: client-side content moderation, content moderator provided by GuardEase, and custom moderation.

Client-side content review

To activate the client-side review, contact the sales for help. Detect text messages in P2P chats, group chats, and chat rooms for unwanted content using local word filters. Two matching methods are supported: keywords and regular expressions.

/// You can manage and configure keyword filters in the CommsEase console, and the SDK can download and manage the filters. Three rules are applied for keyword hits

  • Replacement: The unwanted content in hit messages are replaced with specified text and the messages with replaced text are delivered to the server.
  • Interception: Hit messages are blocked. /// Server-side handling: You can configure the message attributes as hits by server-side moderation, and then sends the hit message to the server. The sender can see the message being sent, but the IM server will not deliver the message and the recipient will not receive the message. For information about how to configure the message properties, see the following.

Retrieve keyword filters

The SDK will automatically retrieve the word filters from the server during login.

Content detection

  • API prototype
dartclass MessageService {
  /// Check local anti-spam lexicon. Text messages in P2P chats, group chats, and chat rooms can be handled for moderation.
  /// <p>[content] text to be moderated.
  /// <p>[replacement] text used to replace the hit text in a message.
  /// <p>[NIMLocalAntiSpamResult] result of moderation
  ///
  Future<NIMResult<NIMLocalAntiSpamResult>> checkLocalAntiSpam(
      String content, String replacement);
}

SDK for Web does not support the local moderation interface.

  • NIMLocalAntiSpamResult
Parameter Description
operator The type of action for hits, 0: no hit; 1: client-side replacement; 2: client-side interception; 3: server-side processing.
content the content with unwanted content replaced.
  • Example
dartString content = "USA"
String replacement = "America"

LocalAntiSpamResult result = await NimCore.instance.messageService.checkLocalAntiSpam(content,replacement);

final operator = result.operator;
if (operator == LocalAntiSpamResult.pass){
    // missing
}else if(operator == LocalAntiSpamResult.clientReplace){
    // replacement
    content = result.content;
}else if(operator == LocalAntiSpamResult.clientIntercept){
    // block
    // the hit message is dropped
}else if(operator == LocalAntiSpamResult.serverIntercept){
    // Delivered to the server
    // Set an attribute for hits
    message.clientAntiSpam = true;
}

Content moderator provided by GuardEase

You can activate the content moderator service provided by GuardEase in the CommsEase console. The content moderator service can review messages in P2P chats, group chats, and chat rooms, avatars and user profiles.

If you want to use content moderator, activate the service in the console. After the service is activated, go to the console to configure the type of services you want to apply the moderator. For example: P2P chats, group chat, text and image messages in chat rooms and more.

To moderate custom messages, you must configure the custom message in the antiSpamOption of NIMMessage for message types, such as text, image and video, used for content moderation.

If your app is served by the content moderator, messages sent from your app clients are forwarded to GuardEase for moderation. if you want to moderate a message, call the antiSpamOption of IMMessage for the message to be sent, passing the NIMAntiSpamOption object as a parameter. You must construct this NIMAntiSpamOption object. The configurable options are as follows:

  • Parameters

NIMAntiSpamOption representation

Parameter Description
enable boolean, whether Content Moderator provided by GuardEase is used to review messages. If Content Moderator is not activated for your app, this setting is invalid. If the value is set to false, Content Moderator is not used
content String, custom content field, valid only for custom messages
antiSpamConfigId String, optional, the ID of a moderator configuration . You can specify a specific configuration for moderating a message.

The content parameter must be a JSON object. The size of content cannot exceed 5,000 bytes in the following format:

json{
  "type": 1, //1: text; 2: image; 3: video
  "data": "" //text, image URL, or video URL
}
  • Example
dart// Testing account is used.
String account = "testAccount";
// P2P chat
NIMSessionType sessionType = NIMSessionType.p2p;
String text = "This message is not moderated";
// Create a text message.
NIMMessage textMessage = await MessageBuilder.createTextMessage(
    sessionId: account,
    sessionType: sessionType,
    text: text,
);

// Construct an Option object for content moderation
NIMAntiSpamOption antiSpamOption = new NIMAntiSpamOption();
antiSpamOption.enable = false;
textMessage.antiSpamOption = antiSpamOption;

// Send messages to the recipient.
final result = await NimCore.instance.messageService.sendMessage(
    message: textMessage,
    resend: false,
);
Was this page helpful?
Yes
No
  • Client-side content review
  • Retrieve keyword filters
  • Content detection
  • Content moderator provided by GuardEase