CACTER AI Laboratory: Application of large AI models in the field of email security

CACTER AI Lab used the Tsinghua ChatGLM large model to conduct experimental analysis on millions of email data and discovered three highlights of the AI ​​large model in phishing email detection:

The large AI model has indeed shown a strong ability in identifying and capturing malicious emails, and can identify a variety of phishing emails including link types, social engineering types, image QR code types, and attachment document types. However, large AI models may cause misjudgments in actual operations.

The large AI model demonstrates powerful intent extraction and analysis capabilities, which can help identify whether an email has the intention of inducing users to perform certain actions, such as opening attachments or accessing links. This ability can also help us identify the characteristics of whether this email is a phishing email.

During experiments, the CACTER AI laboratory found that even if there were no phishing samples in specific minor languages ​​in the training samples, the large AI model could still identify phishing emails in certain minor languages ​​in the test set, showing its cross-language detection potential. The assistance of large AI models can help CACTER AI Lab identify and capture phishing emails that attempt to use small languages ​​​​to bypass the detection engine.

APT (Advanced Persistent Threat) attacks are usually highly concealed and targeted. The application of large AI models in APT attack detection shows new potential:

02. Exploration of multi-modal large models

Compared with text large models, multi-modal large models can provide richer information data sources, such as text, images, audio data, etc., to support phishing detection. Multimodal large models can also perform comprehensive operations on various types of information and conduct cross-modal correlation analysis.

CACTER AI Lab will use multi-modal large models to detect known phishing emails. The large multi-modal model is not only based on text understanding capabilities, but can also simulate visual analysis and process multimedia content such as images and link landing pages. By analyzing the text content, images in the email, and the appearance of the web page pointed to by the link, the model can make a preliminary judgment and identify the true intention of the email.

Confrontational training for attack and defense

The impact of AI on the field of email security must be two-sided. By simulating the attacker's perspective, defenders can better improve their defensive strategies. Attackers can also use large models to iterate their attacks in multiple ways to try to bypass our defenses.

An attacker may apply large models through three levels:

·Primary application: Use historical attack samples to generate new attack emails in batches to reduce manual operations.

·Mid-level application: Discover the weaknesses of defense products and automatically update attack email templates.

·High-end applications: Use the learning capabilities of large models to gain an in-depth understanding of defense methods, generate attack emails that are more difficult to identify, and iteratively bypass detection engines.

02. Defensive application

As a defender, we can use large AI models to conduct confrontation training and feature mining and other strategies for defense.

·Adversarial training: Use large models for adversarial training to improve the performance of the anti-spam system.

·Feature mining: Analyze new attack samples through large models and quickly update detection logic to adapt to new attacks.

The application of large AI models in email anti-phishing detection is still in the exploratory stage, but its potential is encouraging. With the further development and optimization of technology, AI large models are expected to play a more important role in the field of email security and become an indispensable part of email security protection.