Hierarchical Contextual Attention Network with Dynamic Sentiment Routing for Aspect-Based Sentiment Analysis
Keywords:
Aspect-based sentiment analysis, Hierarchical attention, Capsule network, Dynamic routing, BERT, Cross-aspect interactior, SemEvalAbstract
Aspect-based Sentiment Analysis (ABSA) attempts to detect the sentiment polarity associated with particular aspects and to predict such polarities accurately. Although pre-trained language models like BERT have greatly improved the performance of ABSA, they still suffer from the problems of discriminating relevant context of aspects from noise, capturing dependencies among multiple aspects and aggregating sentiment evidence effectively, especially for the small neutral class. We propose a Hierarchical Contextual Attention Network with Dynamic Sentiment Routing (HCAN-DSR) to solve these problems. Our model leverages a multi-level hierarchical attention module on top of a frozen BERT encoder to extract local and global contextual information with focusing aspect-related features. The cross-aspect graph with syntax guidance can model semantic and syntactic dependencies among multiple aspects for enriching interaction among them. Moreover, our Dynamic Sentiment Routing (DSR) mechanism is used as an alternative to traditional pooling to iteratively aggregate aspect-specific evidence, thus preserving fine-grained sentiment information and improving the neutral sentiment classification. Experiment results on Laptop14, Restaurant14 and Twitter benchmark datasets show that HCAN-DSR performs better than all state-of-the-art graph and attention based ABSA models with accuracy of 81.38% on Laptop14 and 89.14% on Restaurant14.
Downloads
References
[1] Pang, B., & Lee, L. (2008). Opinion mining and sentiment analysis. Now Publishers Inc.
[2] Sun, C., Huang, L., & Qiu, X. (2019, June). Utilizing BERT for aspect-based sentiment analysis via constructing auxiliary sentence. In Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers) (pp. 380-385).
[3] Devlin, J., Chang, M. W., Lee, K., & Toutanova, K. (2019, June). Bert: Pre-training of deep bidirectional transformers for language understanding. In Proceedings of the 2019 conference of the North American chapter of the association for computational linguistics: human language technologies, volume 1 (long and short papers) (pp. 4171-4186).
[4] Liu, Y., Ott, M., Goyal, N., Du, J., Joshi, M., Chen, D., ... & Stoyanov, V. (2019). Roberta: A robustly optimized bert pretraining approach. arXiv preprint arXiv:1907.11692.
[5] Lan, Z., Chen, M., Goodman, S., Gimpel, K., Sharma, P., & Soricut, R. (2019). Albert: A lite bert for self-supervised learning of language representations. arXiv preprint arXiv:1909.11942.
[6] Wang, Y., Huang, M., Zhu, X., & Zhao, L. (2016, November). Attention-based LSTM for aspect-level sentiment classification. In Proceedings of the 2016 conference on empirical methods in natural language processing (pp. 606-615).
[7] Tang, D., Qin, B., Feng, X., & Liu, T. (2016, December). Effective LSTMs for target-dependent sentiment classification. In Proceedings of COLING 2016, the 26th international conference on computational linguistics: technical papers (pp. 3298-3307).
[8] Sun, K., Zhang, R., Mensah, S., Mao, Y., & Liu, X. (2019, November). Aspect-level sentiment analysis via convolution over dependency tree. In Proceedings of the 2019 conference on empirical methods in natural language processing and the 9th international joint conference on natural language processing (EMNLP-IJCNLP) (pp. 5679-5688).
[9] Liang, B., Yin, R., Gui, L., Du, J., & Xu, R. (2020, December). Jointly learning aspect-focused and inter-aspect relations with graph convolutional networks for aspect sentiment analysis. In Proceedings of the 28th international conference on computational linguistics (pp. 150-161).
[10] Liang, B., Yin, R., Gui, L., Du, J., & Xu, R. (2020, December). Jointly learning aspect-focused and inter-aspect relations with graph convolutional networks for aspect sentiment analysis. In Proceedings of the 28th international conference on computational linguistics (pp. 150-161).
[11] He, R., Lee, W. S., Ng, H. T., & Dahlmeier, D. (2018, July). Exploiting document knowledge for aspect-level sentiment classification. In Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Volume 2: Short Papers) (pp. 579-585).
[12] Han, S., Dong, H., Teng, X., Li, X., & Wang, X. (2021). Correlational graph attention-based Long Short-Term Memory network for multivariate time series prediction. Applied Soft Computing, 106, 107377.
[13Tian, Y., Chen, G., & Song, Y. (2021, June). Aspect-based sentiment analysis with type-aware graph convolutional networks and layer ensemble. In Proceedings of the 2021 conference of the North American chapter of the association for computational linguistics: human language technologies (pp. 2910-2922).
[14] Sabour, S., Frosst, N., & Hinton, G. E. (2017). Dynamic routing between capsules. Advances in neural information processing systems, 30.
[15] Zhao, W., Ye, J., Yang, M., Lei, Z., Zhang, S., & Zhao, Z. (2018). Investigating capsule networks with dynamic routing for text classification. In Proceedings of the 2018 conference on empirical methods in natural language processing (pp. 3110-3119).
[16] Yang, C., Liu, Z., Zhao, D., Sun, M., & Chang, E. Y. (2015, July). Network representation learning with rich text information. In Ijcai (Vol. 2015, pp. 2111-2117).
[17] Li, L., Wang, H., Sun, X., Chang, B., Zhao, S., & Sha, L. (2015, September). Multi-label text categorization with joint learning predictions-as-features method. In Proceedings of the 2015 Conference on Empirical Methods in Natural Language Processing (pp. 835-839).
[18] Pontiki, M., Galanis, D., Papageorgiou, H., Androutsopoulos, I., Manandhar, S., Al-Smadi, M., ... & Eryiğit, G. (2016, June). Semeval-2016 task 5: Aspect based sentiment analysis. In Proceedings of the 10th international workshop on semantic evaluation (SemEval-2016) (pp. 19-30).
[19] Pontiki, M., Galanis, D., Papageorgiou, H., Androutsopoulos, I., Manandhar, S., Al-Smadi, M., ... & Eryiğit, G. (2016, June). Semeval-2016 task 5: Aspect based sentiment analysis. In Proceedings of the 10th international workshop on semantic evaluation (SemEval-2016) (pp. 19-30).
[20] Pontiki, M., Galanis, D., Papageorgiou, H., Androutsopoulos, I., Manandhar, S., Mohammad, A.-L., Al-Ayyoub, M., Zhao, Y., Qin, B., & De Clercq, O. (2016). SemEval-2016 Task 5: Aspect based sentiment analysis. In Proceedings of the 10th International Workshop on Semantic Evaluation (SemEval 2016) (pp. 19–30). Association for Computational Linguistics.
[21] Jiang, Q., Chen, L., Xu, R., Ao, X., & Yang, M. (2019, November). A challenge dataset and effective models for aspect-based sentiment analysis. In Proceedings of the 2019 conference on empirical methods in natural language processing and the 9th international joint conference on natural language processing (EMNLP-IJCNLP) (pp. 6280-6285).
[22] Loshchilov, I., & Hutter, F. (2017). Decoupled weight decay regularization. arXiv preprint arXiv:1711.05101.
[23] Sarzynska-Wawer, J., Wawer, A., Pawlak, A., Szymanowska, J., Stefaniak, I., Jarkiewicz, M., & Okruszek, L. (2021). Detecting formal thought disorder by deep contextualized word representations. Psychiatry research, 304, 114135.
[24] Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A. N., ... & Polosukhin, I. (2017). Attention is all you need. Advances in neural information processing systems, 30.
[25] Yang, Z., Yang, D., Dyer, C., He, X., Smola, A., & Hovy, E. (2016, June). Hierarchical attention networks for document classification. In Proceedings of the 2016 conference of the North American chapter of the association for computational linguistics: human language technologies (pp. 1480-1489).
[26] Tang, D., Qin, B., & Liu, T. (2016, November). Aspect level sentiment classification with deep memory network. In Proceedings of the 2016 conference on empirical methods in natural language processing (pp. 214-224).
[27] Ma, D., Li, S., Zhang, X., & Wang, H. (2017). Interactive attention networks for aspect-level sentiment classification. arXiv preprint arXiv:1709.00893.
[28] Chen, P., Sun, Z., Bing, L., & Yang, W. (2017, September). Recurrent attention network on memory for aspect sentiment analysis. In Proceedings of the 2017 conference on empirical methods in natural language processing (pp. 452-461).
[29] Song, Y., Wang, J., Jiang, T., Liu, Z., & Rao, Y. (2019). Attentional encoder network for targeted sentiment classification. arXiv preprint arXiv:1902.09314.
[30] Phan, M. H., & Ogunbona, P. O. (2020, July). Modelling context and syntactical features for aspect-based sentiment analysis. In Proceedings of the 58th annual meeting of the association for computational linguistics (pp. 3211-3220).
[31] Zhang, C., Li, Q., & Song, D. (2019, November). Aspect-based sentiment classification with aspect-specific graph convolutional networks. In Proceedings of the 2019 conference on empirical methods in natural language processing and the 9th international joint conference on natural language processing (EMNLP-IJCNLP) (pp. 4568-4578).
[32] Li, R., Chen, H., Feng, F., Ma, Z., Wang, X., & Hovy, E. (2021, August). Dual graph convolutional networks for aspect-based sentiment analysis. In Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing (Volume 1: Long Papers) (pp. 6319-6329).
[33] Gartner III, T. E., Piaggi, P. M., Car, R., Panagiotopoulos, A. Z., & Debenedetti, P. G. (2022). Liquid-liquid transition in water from first principles. Physical review letters, 129(25), 255702.
[34] Liang, B., Yin, R., Gui, L., Du, J., & Xu, R. (2020, December). Jointly learning aspect-focused and inter-aspect relations with graph convolutional networks for aspect sentiment analysis. In Proceedings of the 28th international conference on computational linguistics (pp. 150-161).
[35] Majumder, N., Poria, S., Gelbukh, A., & Cambria, E. (2017). Deep learning-based document modeling for personality detection from text. IEEE intelligent systems, 32(2), 74-79.
[36] Poria, S., Cambria, E., Bajpai, R., & Hussain, A. (2017). A review of affective computing: From unimodal analysis to multimodal fusion. Information fusion, 37, 98-125.
[37] Hinton, G. E., Krizhevsky, A., & Wang, S. D. (2011, June). Transforming auto-encoders. In International conference on artificial neural networks (pp. 44-51). Berlin, Heidelberg: Springer Berlin Heidelberg.
[38] Li, X., Bing, L., Lam, W., & Shi, B. (2018, July). Transformation networks for target-oriented sentiment classification. In Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) (pp. 946-956).
[39] Tang, R., Lu, Y., Liu, L., Mou, L., Vechtomova, O., & Lin, J. (2019). Distilling task-specific knowledge from bert into simple neural networks. arXiv preprint arXiv:1903.12136.
[40] Zhang, C., Li, Q., & Song, D. (2019, July). Syntax-aware aspect-level sentiment classification with proximity-weighted convolution network. In Proceedings of the 42nd international ACM SIGIR conference on research and development in information retrieval (pp. 1145-1148).
Downloads
Published
Issue
Section
License
Copyright (c) 2026 Otmane Houdaif , Meryem Tamir , Elarbi Lamfarrad , Amine Bouobida (Author)
This work is licensed under a Creative Commons Attribution 4.0 International License.
