Chen, Qian, Ragusa, Edoardo, Chaturvedi, Iti, Cambria, Erik, and Zunino, Rodolfo (2023) Text-Image Sentiment Analysis. In: Lecture Notes in Computer Science (13397) pp. 169-180. From: CICLing 2018: 19th International Conference on Computational Linguistics and Intelligent Text Processing, 18–24 March 2018, Hanoi, Vietnam.

PDF (Published Version) - Published Version Restricted to Repository staff only

Abstract

Expressiveness varies from one person to another. Most images posted on Twitter lack good labels and the accompanying tweets have a lot of noise. Hence, in this paper we identify the contents and sentiments in images through the fusion of both image and text features. We leverage on the fact that AlexNet is a pre-trained model with great performance in image classification and the corresponding set of images are extracted from the web. In particular, we present a novel method to extract features from Twitter images and the corresponding labels or tweets using deep convolutional neural networks trained on Twitter data. We consider fine tuning AlexNet pre-trained CNNs to initialize the model and AffectiveSpace of English concepts as text features. Lastly, to combine the image and text predictions we propose a novel sentiment score. Our model is evaluated on Twitter dataset of images and corresponding labels and tweets. We show that accuracy by merging scores from text and image models is higher than using any one system alone.

Item ID:	78260
Item Type:	Conference Item (Research - E1)
ISBN:	978-3-031-23804-8
ISSN:	1611-3349
Related URLs:	Publisher
Copyright Information:	© Springer Nature Switzerland AG 2023
Date Deposited:	15 May 2023 23:59
FoR Codes:	46 INFORMATION AND COMPUTING SCIENCES > 4602 Artificial intelligence > 460208 Natural language processing @ 30% 46 INFORMATION AND COMPUTING SCIENCES > 4603 Computer vision and multimedia computation > 460306 Image processing @ 40% 46 INFORMATION AND COMPUTING SCIENCES > 4602 Artificial intelligence > 460206 Knowledge representation and reasoning @ 30%
SEO Codes:	22 INFORMATION AND COMMUNICATION SERVICES > 2204 Information systems, technologies and services > 220403 Artificial intelligence @ 50% 22 INFORMATION AND COMMUNICATION SERVICES > 2204 Information systems, technologies and services > 220408 Information systems @ 25% 28 EXPANDING KNOWLEDGE > 2801 Expanding knowledge > 280115 Expanding knowledge in the information and computing sciences @ 25%
Downloads:	Total: 3
	More Statistics