Convolutional MKL based multimodal emotion recognition and sentiment analysis

Poria, Soujanya, Chaturvedi, Iti, Cambria, Erik, and Hussian, Amir (2017) Convolutional MKL based multimodal emotion recognition and sentiment analysis. In: Proceedings of the 16th International Conference on Data Mining. pp. 439-448. From: ICDM 2016: 16th International Conference on Data Mining, 12-15 December 2016, Barcelona, Spain.

[img] PDF (Published Version) - Published Version
Restricted to Repository staff only

View at Publisher Website:


Technology has enabled anyone with an Internet connection to easily create and share their ideas, opinions and content with millions of other people around the world. Much of the content being posted and consumed online is multimodal. With billions of phones, tablets and PCs shipping today with built-in cameras and a host of new video-equipped wearables like Google Glass on the horizon, the amount of video on the Internet will only continue to increase. It has become increasingly difficult for researchers to keep up with this deluge of multimodal content, let alone organize or make sense of it. Mining useful knowledge from video is a critical need that will grow exponentially, in pace with the global growth of content. This is particularly important in sentiment analysis, as both service and product reviews are gradually shifting from unimodal to multimodal. We present a novel method to extract features from visual and textual modalities using deep convolutional neural networks. By feeding such features to a multiple kernel learning classifier, we significantly outperform the state of the art of multimodal emotion recognition and sentiment analysis on different datasets.

Item ID: 63233
Item Type: Conference Item (Research - E1)
ISBN: 978-1-5090-5473-2
Keywords: Multimodal sentiment analysis, Deep learning, Convolutional neural networks, Multiple kernel learning
Copyright Information: © IEEE
Date Deposited: 03 Sep 2020 01:31
FoR Codes: 47 LANGUAGE, COMMUNICATION AND CULTURE > 4704 Linguistics > 470403 Computational linguistics @ 30%
46 INFORMATION AND COMPUTING SCIENCES > 4611 Machine learning > 461103 Deep learning @ 70%
SEO Codes: 97 EXPANDING KNOWLEDGE > 970108 Expanding Knowledge in the Information and Computing Sciences @ 100%
Downloads: Total: 3
More Statistics

Actions (Repository Staff Only)

Item Control Page Item Control Page