BERT-based two-channel neural network model text emotion analysis
DOI:
https://doi.org/10.56294/saludcyt20251444Keywords:
Bidirectional Encoder Representations from Transformers (BERT), Two-Channel Neural Network, text emotion, Augmented Recurrent Neural Networks-mutated Unidirectional Long Short-Term Memory (ARNN-Uni-LSTM)Abstract
Introduction: Text emotion analysis, or sentiment analysis, is a crucial area in natural language processing (NLP) focused on identifying emotions within textual data. Challenges in this field include disappearing gradients, information loss, and the lack of contextual semantics.
Methods: To address these challenges, we propose a BERT-based model utilizing a two-channel neural network for enhanced emotion classification. The model transforms text into word vectors using BERT, which excels in capturing contextual information. The architecture includes Augmented Recurrent Neural Networks-mutated Unidirectional Long Short-Term Memory (ARNN-Uni-LSTM) to extract local semantic features and capture long-range dependencies. Preprocessing involved tokenization and Word2Vec on publicly available text emotion datasets. The first channel employs ARNN for local feature extraction, while the second uses Uni-LSTM for broader context.
Results: Experiments conducted in Python demonstrated that our model outperformed traditional methods, achieving precision of 97.18%, recall of 94.56%, and an F1 score of 96.26%.
Conclusions: The BERT-based model shows significant promise for applications such as customer feedback analysis, social media monitoring, and mental health diagnostics, offering a foundation for advanced emotion recognition systems.
References
1. Wu Y, Jin Z, Shi C, Liang P, Zhan T. Research on the Application of Deep Learning-based BERT Model in Sentiment Analysis. arXiv preprint arXiv:2403.08217. 2024 Mar 13. https://doi.org/10.54254/2755-2721/71/2024MA
2. Bashynska I, Sarafanov M, Manikaeva O. Research and Development of a Modern Deep Learning Model for Emotional Analysis Management of Text Data. Applied Sciences. 2024 Feb 27;14(5):1952. https://doi.org/10.3390/app14051952
3. Rezapour M. Emotion Detection with Transformers: A Comparative Study. arXiv preprint arXiv:2403.15454. 2024 Mar 18.https://doi.org/10.48550/arXiv.2403.15454
4. Murfi H, Gowandi T, Ardaneswari G, Nurrohmah S. BERT-based combination of convolutional and recurrent neural network for indonesian sentiment analysis. Applied Soft Computing. 2024 Jan 1;151:111112.https://doi.org/10.1016/j.asoc.2023.111112
5. Wan B, Wu P, Yeo CK, Li G. Emotion-cognitive reasoning integrated BERT for sentiment analysis of online public opinions on emergencies. Information Processing & Management. 2024 Mar 1;61(2):103609.https://doi.org/10.1016/j.ipm.2023.103609
6. Zhang L, Li Z, Guo T, Huang P. BDCM: BERT-based Dual-Channel Hybrid Model for Aspect Sentiment Analysis. In2024 IEEE 7th Advanced Information Technology, Electronic and Automation Control Conference (IAEAC) 2024 Mar 15 (Vol. 7, pp. 258-261). IEEE. https://doi.org/10.1109/IAEAC59436.2024.10503758
7. Liang S, Jin J, Du W, Qu S. A Multi-Channel Text Sentiment Analysis Model Integrating Pre-training Mechanism. Information Technology and Control. 2023 Jul 15;52(2):263-75. https://doi.org/10.5755/j01.itc.52.2.31803
8. Gan C, Feng Q, Zhang Z. Scalable multi-channel dilated CNN–BiLSTM model with attention mechanism for Chinese textual sentiment analysis. Future Generation Computer Systems. 2021 May 1;118:297-309. https://doi.org/10.1016/j.future.2021.01.024
9. Acheampong FA, Nunoo-Mensah H, Chen W. Transformer models for text-based emotion detection: a review of BERT-based approaches. Artificial Intelligence Review. 2021 Dec;54(8):5789-829.https://doi.org/10.1007/s10462-021-09958-2
10. Chiorrini A, Diamantini C, Mircoli A, Potena D. Emotion and sentiment analysis of tweets using BERT. InEdbt/icdt workshops 2021 Mar 23 (Vol. 3, pp. 1-7).
11. Abas AR, Elhenawy I, Zidan M, Othman M. BERT-CNN: A Deep Learning Model for Detecting Emotions from Text. Computers, Materials & Continua. 2022 May 1;71(2).https://doi.org/10.32604/cmc.2022.021671
12. Kumar P, Raman B. A BERT based dual-channel explainable text emotion recognition system. Neural Networks. 2022 Jun 1;150:392-407.https://doi.org/10.32604/cmc.2022.021671
13. Liu N, Zhao J. A BERT‐Based Aspect‐Level Sentiment Analysis Algorithm for Cross‐Domain Text. Computational Intelligence and Neuroscience. 2022;2022(1):8726621.https://doi.org/10.1155/2022/8726621
14. Kumar A, Jain AK. Emotion detection in psychological texts by fine-tuning BERT using emotion–cause pair extraction. International Journal of Speech Technology. 2022 Sep;25(3):727-43.https://doi.org/10.1007/s10772-022-09982-9
15. Chen N, Sun Y, Yan Y. Sentiment analysis and research based on two‐channel parallel hybrid neural network model with attention mechanism. IET Control Theory & Applications. 2023 Nov;17(17):2259-67.https://doi.org/10.1049/cth2.12463
16. Na L, Cao T, Bai S, Li D. A Parallel Two-Channel Emotion Classification Method for Chinese Text. IEEE Access. 2024 Jan 5.https://doi.org/10.1109/ACCESS.2024.3350190.
Downloads
Published
Issue
Section
License
Copyright (c) 2025 Yingying Mei , Mideth Abisado (Author)
This work is licensed under a Creative Commons Attribution 4.0 International License.
The article is distributed under the Creative Commons Attribution 4.0 License. Unless otherwise stated, associated published material is distributed under the same licence.
