Multilingual Sentiment Analysis: A Review of Deep Learning Transformer Models and Ensemble Techniques

Abstract

Sentiment analysis, especially in the multilingual sentiment analysis (MSA) context, can be complicated owing to the nuances, sarcasm, diversity in languages, culture, and data sparsity involved in identifying, discerning, and interpreting sentiment. In this paper, we review and summarize the progress and current work being undertaken in Multilingual Sentiment Analysis, with a focus on modern advancement and implementation of state-of-the-art deep learning Transformer models, and ensemble learning methods, in sentiment classification tasks. We review the use of well-known and powerful Transformer architectures such as XLM-R, LaBSE, MPNet, mBERT, DistilBERT, and their variations. These models can perform well to capture cross-lingual contextual meanings in relation to sentiment. We also review the literature in their use of ensemble methods in sentiment classification tasks, for example, stacking/bagging/boosting/voting approaches, as a contributions to gain higher accuracy, robust predictions, and generalizability of those predictions from the literature. The review consequently presents a summary of key findings from previous studies on the applicability of transformer-based and ensemble models in multilingual sentiment classification and their related advantages and disadvantages when used on multilingual datasets. We highlight the advantages of using ensemble techniques, that various a group of models can be optimized to have both collective and individual gain higher performance over a single learner. We also include challenges and potential limitations in the literature, such as cost of computing through complex ensemble, and diversity among base models. The review acts as a synthesis and definitive summary overview of the state of the art, and aims to provide future research directions for employing advanced deep learning techniques and their ensemble techniques to understanding complexities of multilingual sentiment analysis.