Multimodal Fake News Detection: A Survey of Text and Visual Content Integration Methods

Abstract

The widespread use of social media has significantly facilitated the transmission of misleading information, making fake news a complicated and important global issue. While misinformation has emerged for decades, it has evolved from simple text-based content to more complex formats that include images, audio, and video. This transition requires more advanced detection methods capable of processing and integrating various types of data effectively. Multimodal fake news detection addresses this challenge by integrating information from different modalities, such as text and images, to improve accuracy. Since most social media content today includes both visual and textual elements, using them in combination allows models to detect inconsistencies and patterns that might not be evident when analyzing a single modality. For example, a misleading caption may be associated with a manipulated image, and understanding the relationship between the two is essential for accurate detection. Despite its potential, effective multimodal fake news detection remains in its early stages. While many studies have focused on single-modality detection, combining different data types each with unique structures and dimensions poses technical challenges. The core difficulty involves developing robust fusion strategies that can meaningfully combine information from different modalities. This survey paper focuses specifically on deep learning (DL) methods for multimodal fake news detection on social media. It reviews key works in the field, highlighting the deep learning techniques used, the data types analyzed (with a focus on text and images), and the fusion mechanisms employed. The paper also discusses major limitations in current state-of-the-art approaches.