A Hybrid GloVe-BERT Fusion Model with Multi-Level Attention-Based CNN-BiLSTM for Sentiment Analysis
Article Sidebar
Main Article Content
Abstract
Gauging public sentiment toward climate policy from information-rich news headlines remains challenging for conventional text classification approaches. Conventional sentiment analysis tools miss contextual subtleties in brief headlines, whereas deep learning models capture the public perception more accurately. The proposed work presents hybrid Convolutional Neural Network (CNN) and Bidirectional Long Short-Term Memory (BiLSTM) model that uses combination of GloVe and BERT embeddings with an attention layer for sentiment analysis of climate change news headlines. The novelty of this research lies in the use of GloVe and BERT embeddings through a multi-stage fusion strategy and an attention mechanism to enhance text classification performance in a hybrid model. The architecture employs a hierarchical layering approach to fuse static GloVe embeddings with dynamic, contextualized BERT representations through attention modules that enables the network to selectively focus on salient features. To further model complex semantic dependencies, the design incorporates parallel CNN-BiLSTM branches, structured with residual connections and bolstered with additional layers of attention. Evaluated on 1,023 climate-related headlines annotated on a three-point polarity scale, the proposed model achieves an accuracy of 80.47%, outperforming classi- cal baselines (SVM, Naive Bayes, K-NN) and single branch deep networks (CNN:78.63%, BiLSTM:78.36%). The predictive accuracy of the hybrid model is evaluated using a paired t-test to determine whether the difference between models is statistically significant; this is confirmed by rejecting null hypothesis and accepting alternate hypothesis.This study demonstrates that compact, domain-adaptive deep learning models incorporating contextual embeddings and attention mechanisms that can effectively extract sentiment from news headlines, offering scalable, evidence based tools for tracking climate discourse and information policy decisions.
Article Details
This work is licensed under a Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 International License.
References
S. Adhikari, R. Kaushik, A. J. Obaid, S. Jeyalaksshmi, D. Balaganesh and F. H. Hanoon, “YouTube sentimental analysis using a combined approach of KNN and K-means clustering algorithm,” in Proceedings of 3rd International Conference on Mathematical Modeling and Computational Science, Singapore: Springer Nature Singapore, pp. 37–50, 2023.
M. Alruily, “Sentiment analysis for predicting stress among workers and classification utilizing CNN: Unveiling the mechanism,” Alexandria Engineering Journal, vol. 81, pp. 360–370, 2023.
D. Amangeldi, A. Usmanova and P. Shamoi, “Understanding Environmental Posts: Sentiment and Emotion Analysis of Social Media Data,” in IEEE Access, vol. 12, pp. 33504-33523, 2024.
R. J. Brulle, J. Carmichael and J. C. Jenkins, “Shifting public opinion on climate change: An empirical assessment of factors influencing concern over climate change in the U.S., 2002–2010,” Climatic Change, vol. 114, no. 2, pp. 169–188, Sep. 2012.
C. Bucur, B. Tudorica, J. V. Andrei, D. Dusmanescu, D. Paraschiv and C. Teodor, “Sentiment analysis of global news on environmental issues: Insights into public perception and its impact on low-carbon economy transition,” Frontiers in Environmental Science, vol. 12, p. 1360304, Jun. 2024.
D. Chi, T. Huang, Z. Jia and S. Zhang, “Research on sentiment analysis of hotel review text based on BERT-TCN-BiLSTM-attention model,” Array, vol. 25, p. 100378, Mar. 2025.
E. D. Cubuk, B. Zoph, D. Man´e, V. Vasudevan and Q. V. Le, “AutoAugment: Learning Augmentation Strategies From Data,” 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), Long Beach, CA, USA, pp. 113-123, 2019.
M. K. Daradkeh, “A hybrid data analytics framework with sentiment convergence and multi-feature fusion for stock trend prediction,” Electronics, vol. 11, no. 2, p. 11020250, Jan. 2022.
J. Devlin, M.-W. Chang, K. Lee and K. Toutanova, “BERT: Pre-training of deep bidirectional transformers for language understanding,” in Proceedings of NAACL-HLT 2019, pp. 4171–4186, 2019.
S. Divya, S. Kiruthika, A. N. Anton and S. Padmavathi, “Segmentation, tracking and feature extraction for Indian sign language recognition,” International Journal on Computational Science & Applications, vol. 4, no. 2, pp. 57–72, 2014.
D. N. Dwivedi and G. Mahanty, “Mental health in messages: Unravelling emotional patterns through advanced text analysis,” in Using Machine Learning to Detect Emotions and Predict Human Psychology, Hershey, PA, USA: IGI Global, pp. 187–208, 2024.
L. Edwards et al., “Rapid evidence assessment on online misinformation and media literacy,” Final Report, 2021.
M. El Barachi, M. AlKhatib, S. Mathew and F. Oroumchian, “A novel sentiment analysis framework for monitoring the evolving public opinion in real-time: Case study on climate change,” Journal of Cleaner Production, vol. 312, p. 127820, Aug. 2021.
M. Geetha, N. Aloysius, D. A. Somasundaran, A. Raghunath and P. Nedungadi, “Toward RealTime Recognition of Continuous Indian Sign Language: A Multi-Modal Approach Using RGB and Pose,” in IEEE Access, vol. 13, pp. 6027060283, 2025.
T. Gokcimen and B. Das, “Exploring climate change discourse on social media and blogs using a topic modeling analysis,” Heliyon, vol. 10, no. 11, p. e32464, 2024.
J. P. Gujjar and H. P. Kumar, “Sentiment analysis: Text blob for decision making,” International Journal of Scientific Research & Engineering Trends, vol. 7, no. 2, pp. 1097–1099, 2021.
N. S. Harzevili and S. H. Alizadeh, “Mixture of latent multinomial naive Bayes classifier,” Applied Soft Computing, vol. 69, pp. 516–527, 2018.
J. Heide, “Polarizing social figures? Climate activists in German media and popular discourse,” European Societies, vol. 27, no. 5, pp. 959–991, 2025.
S. Hota and S. Pathak, “KNN classifier based approach for multi-class sentiment analysis of Twitter data,” International Journal of Engineering & Technology, vol. 7, no. 3, pp. 1372–1375, 2018.
C. Hutto and E. Gilbert, “VADER: A parsimonious rule-based model for sentiment analysis of social media text,” Proc. Int. Conf. The Eighth International AAAI Conference on Weblogs and Social Media (ICWSM-14), vol. 8, no. 1, pp. 216–225, 2014.
T. Islam et al., “Lexicon and deep learning-based approaches in sentiment analysis on short texts,” Journal of Computer and Communications, vol. 12, no. 1, pp. 11–34, 2024.
T. R. Karl and K. E. Trenberth, “Modern global climate change,” Science, vol. 302, no. 5651, pp. 1719–1723, 2003.
T. A. Khan, R. Sadiq, Z. Shahid, M. M. Alam and M. B. M. Su’ud, “Sentiment analysis using support vector machine and random forest,” Journal of Informatics and Web Engineering, vol. 3, no. 1, pp. 67–75, 2024.
F. S. Khatibi, A. Dedekorkut-Howes, M. Howes and E. Torabi, “Can public awareness, knowledge and engagement improve climate change adaptation policies?,” Discover Sustainability, vol. 2, no. 18, pp. 1–24, 2021.
S. Khruahong, O. Surinta and S. C. Lam, “Sentiment analysis of local tourism in Thailand from YouTube comments using BiLSTM,” in Proc. Int. Conf. Multi-disciplinary Trends in Artificial Intelligence, Switzerland: Springer, pp. 169–177, 2022.
S. O’Neill et al., “Dominant frames in legacy and social media coverage of the IPCC Fifth Assessment Report,” Nature Climate Change, vol. 5, no. 4, pp. 380–385, 2015.
A. Razaque et al., “State-of-art review of information diffusion models and their impact on social network vulnerabilities,” Journal of King Saud University Computer and Information Sciences, vol. 34, no. 1, pp. 1275–1294, 2022.
B. Pang and L. Lee, “Opinion mining and sentiment analysis,” Foundations and Trends in Information Retrieval, vol. 2, no. 1–2, pp. 1–135, 2008.
N. M. Sham and A. Mohamed, “Climate change sentiment analysis using lexicon, machine learning and hybrid approaches,” Sustainability, vol. 14, no. 8, p. su14084723, 2022.
K. L. Tan, C. P. Lee, and K. M. Lim, “A survey of sentiment analysis: Approaches, datasets, and future research,” Applied Sciences, vol. 13, no. 7, p. app13074550, 2023.
J. Xie, B. Chen, X. Gu, F. Liang and X. Xu, “Self-Attention-Based BiLSTM Model for Short Text Fine-Grained Sentiment Classification,” in IEEE Access, vol. 7, pp. 180558-180570, 2019.
