Moderation
Update time: 2024/03/07 11:13:40
CommsEase provides three types of content moderation services: client-side content moderation, content moderator provided by GuardEase, and custom moderation.
Client-side content review
To activate the client-side review, contact the sales for help. Detect text messages in P2P chats, group chats, and chat rooms for unwanted content using local word filters. Two matching methods are supported: keywords and regular expressions.
/// You can manage and configure keyword filters in the CommsEase console, and the SDK can download and manage the filters. Three rules are applied for keyword hits
- Replacement: The unwanted content in hit messages are replaced with specified text and the messages with replaced text are delivered to the server.
- Interception: Hit messages are blocked. /// Server-side handling: You can configure the message attributes as hits by server-side moderation, and then sends the hit message to the server. The sender can see the message being sent, but the IM server will not deliver the message and the recipient will not receive the message. For information about how to configure the message properties, see the following.
Retrieve keyword filters
The SDK will automatically retrieve the word filters from the server during login.
Content detection
- API prototype
dartclass MessageService {
/// Check local anti-spam lexicon. Text messages in P2P chats, group chats, and chat rooms can be handled for moderation.
/// <p>[content] text to be moderated.
/// <p>[replacement] text used to replace the hit text in a message.
/// <p>[NIMLocalAntiSpamResult] result of moderation
///
Future<NIMResult<NIMLocalAntiSpamResult>> checkLocalAntiSpam(
String content, String replacement);
}
SDK for Web does not support the local moderation interface.
- NIMLocalAntiSpamResult
Parameter | Description |
---|---|
operator | The type of action for hits, 0: no hit; 1: client-side replacement; 2: client-side interception; 3: server-side processing. |
content | the content with unwanted content replaced. |
- Example
dartString content = "USA"
String replacement = "America"
LocalAntiSpamResult result = await NimCore.instance.messageService.checkLocalAntiSpam(content,replacement);
final operator = result.operator;
if (operator == LocalAntiSpamResult.pass){
// missing
}else if(operator == LocalAntiSpamResult.clientReplace){
// replacement
content = result.content;
}else if(operator == LocalAntiSpamResult.clientIntercept){
// block
// the hit message is dropped
}else if(operator == LocalAntiSpamResult.serverIntercept){
// Delivered to the server
// Set an attribute for hits
message.clientAntiSpam = true;
}
Content moderator provided by GuardEase
You can activate the content moderator service provided by GuardEase in the CommsEase console. The content moderator service can review messages in P2P chats, group chats, and chat rooms, avatars and user profiles.
If you want to use content moderator, activate the service in the console. After the service is activated, go to the console to configure the type of services you want to apply the moderator. For example: P2P chats, group chat, text and image messages in chat rooms and more.
To moderate custom messages, you must configure the custom message in the antiSpamOption of NIMMessage for message types, such as text, image and video, used for content moderation.
If your app is served by the content moderator, messages sent from your app clients are forwarded to GuardEase for moderation. if you want to moderate a message, call the antiSpamOption
of IMMessage
for the message to be sent, passing the NIMAntiSpamOption
object as a parameter. You must construct this NIMAntiSpamOption
object. The configurable options are as follows:
- Parameters
NIMAntiSpamOption representation
Parameter | Description |
---|---|
enable | boolean, whether Content Moderator provided by GuardEase is used to review messages. If Content Moderator is not activated for your app, this setting is invalid. If the value is set to false, Content Moderator is not used |
content | String, custom content field, valid only for custom messages |
antiSpamConfigId | String, optional, the ID of a moderator configuration . You can specify a specific configuration for moderating a message. |
The content parameter must be a JSON object. The size of content cannot exceed 5,000 bytes in the following format:
json{
"type": 1, //1: text; 2: image; 3: video
"data": "" //text, image URL, or video URL
}
- Example
dart// Testing account is used.
String account = "testAccount";
// P2P chat
NIMSessionType sessionType = NIMSessionType.p2p;
String text = "This message is not moderated";
// Create a text message.
NIMMessage textMessage = await MessageBuilder.createTextMessage(
sessionId: account,
sessionType: sessionType,
text: text,
);
// Construct an Option object for content moderation
NIMAntiSpamOption antiSpamOption = new NIMAntiSpamOption();
antiSpamOption.enable = false;
textMessage.antiSpamOption = antiSpamOption;
// Send messages to the recipient.
final result = await NimCore.instance.messageService.sendMessage(
message: textMessage,
resend: false,
);