Moderation

Update time: 2023/02/09 15:41:47

CommsEase provides three types of content moderation services: client-side content moderation, content moderator provided by GuardEase, and custom moderation.

Client-side moderation

To activate the client-side review, contact the sales for help. Detect text messages in P2P chats, group chats, and chat rooms for unwanted content using local word filters. Two matching methods are supported: keywords and regular expressions.

You can configure and manage the local word filters in the CommsEase console. Make sure you Publish the configured filters. The SDK downloads and manages the filter database. The following rules are applied for hits

  • Replacement: The unwanted content in hit messages are replaced with specified text and the messages with replaced text are delivered to the server.
  • Interception: Hit messages are blocked.
  • Server-side handling: You can configure the message attributes as hits by server-side moderation, and then sends the hit message to the server. The sender can see the message being sent, but the IM server will not deliver the message and the recipient will not receive the message. For information about how to configure the message properties, see the following.

Retrieve keyword filters

The SDK for iOS will automatically retrieve the word filters from the server during login.

Content detection

Pass the text to be checked and a string for replacing hit content (for example, if you want to replace the hit text with , pass), the SDK will check whether the message hits the word filters, and return the result.

@protocol NIMAntispamManager <NSObject>

/**
 *  Local moderation
 *
 * @param option Option for local moderation. The option.content indicates the text message to be checked, option.replacement indicates the string used for replacement.
 *  @param error         Error message
 * @discussion This extension will not be synced with other clients. tThe upper layer must ensure that NSDictionary can be converted to JSON.
 *  @return              Local moderation results updated synchronously after each login
 * Due to network issues or no logins, the local moderation results will become invalid. The error parameter contains the specific error message
 */
- (NIMLocalAntiSpamCheckResult *)checkLocalAntispam:(NIMLocalAntiSpamCheckOption *)option
                                              error:(NSError **)error;

@end

Properties

Field Type Description
option NIMLocalAntiSpamCheckOption Option used for local moderation. The option allows users to replace the hit text with the specified string.

The returned result is of the NIMLocalAntiSpamCheckResult type. The possible results are:

/**
 * Result types for local moderation
 */
typedef NS_ENUM (NSInteger, NIMLocalAntiSpamOperate){
    /*  Local word filters do not exist. */
    NIMAntiSpamOperateFileNotExists,
    /* Local replacement */
    NIMAntiSpamResultLocalReplace,
    /* Local block */
    NIMAntiSpamResultLocalForbidden,
    /* Server block */
    NIMAntiSpamResultServerForbidden,
    /* Verified without hits */
    NIMAntiSpamResultNotHit,
    /* Parsing failure */
    NIMAntiSpamResultError
};
/**
 * Local moderation option
 * @discussion Option used for local check. The option allows user to replace the hit with the specified string.
 */
@interface NIMLocalAntiSpamCheckResult : NSObject

/**
 * Result type
 */
@property (nonatomic, assign, readonly)            NIMLocalAntiSpamOperate type;

@end

If a message hits the word filters on the server, you must configure the corresponding properties before sending the message. If no properties are configured, the server will not block the message. Set NIMMessage - antiSpamOption - hitClientAntispam to YES .

Content moderator

Content moderator is a value-added service provided by CommsEase and allows you to detect user-generated content for compliance..

Content moderation overview

Content moderator can review content in the messages of specified types. If you activate the content moderation service for your application in the CommsEase console, messages are moderated before sending to recipients. Content Moderator can handle text and image messages from P2P chats, group chats, and chat rooms, profile pictures, profile attributes,, custom messages. For information about how to detect custom messages, see Detect custom messages

Detect custom messages

After Content Moderator is activated, specified types of messages in P2P chats and group chats will be detected based on the pre-defined rules. To moderate custom messages, you must configure the custom message in the antiSpamOption of NIMMessage for message types, such as text, image and video, used for content moderation.

Custom moderation

If your app has access to a third-party content moderation service, you can use the [webhook](https://doc.commsease.com/docs/TM5MzM5Njk/ jI3ODc2ODE?platformId=60353) service to implement third-party moderation service. After a message is sent, the message is delivered to the third-party moderation service and handled based on the result.

Review the result

SDKv8.7.0 or later and Content Moderator is activated

The following fields contain the moderation result:

  • Android: IMMessage#getYidunAntiSpamRes
  • iOS: NIMMessage#yidunAntiSpamRes

These two fields are returned by the corresponding callbacks:

yidunAntiSpamRes will is returned only when messages are intercepted by Content Moderator. Suspicious messages are handled based on the pre-defined rules. If a suspicious message is intercepted, yidunAntiSpamRes is returned, and if the suspicious message is delivered, no yidunAntiSpamRes is returned.

  • Voice and video messages are detected asynchronously for moderation. The detection result is notified to the app server using the data sync service. Although CommsEase will automatically delete the files on the server after the message hits the pre-defined rules, the message may have been delivered to the recipient before the detection result comes out. Therefore, if the hit result is synced by the data sync service, users can unsend a message by calling the corresponding server API for P2P chats and group chats. For chat room messages, call Unsend messages in chat rooms.
  • If a message hits the client's local moderation rules, the SDK returns the result. If server-side interception is configured, the message synced with the data sync service is with the antispam field.

:::

SDK v8.7.0 or earlier

If a message hits moderation rules, the result is transparent to the sender and the SDK returns a success but the recipient cannot receive the message.

If a message hits the Content Moderator rules and the data sync service for IM is activated, the yidunRes field is included in the result and the action field indicates the action taken based on the result. 0: pass, 1: suspicious, 2: hit. Only the yidunBusType filed contains 0 or 2, this field is included in the data for data sync.

The yidunBusType enums:

  • 0: text moderation
  • 1: image moderation
  • 2: profile moderation
  • 3: avatar moderation

The corresponding image returns the labels field, text moderation reference The labels field< /a>, image moderation reference The labels field.

Was this page helpful?
Yes
No
  • Client-side moderation
  • Retrieve keyword filters
  • Content detection
  • Content moderator
  • Content moderation overview
  • Detect custom messages
  • Custom moderation
  • Review the result