Understanding emotional impact of images using Bayesian multiple kernel learning

He Zhang, Mehmet Gönen, Zhirong Yang, Erkki Oja

Research output: Contribution to journalArticle

9 Scopus citations

Abstract

Affective classification and retrieval of multimedia such as audio, image, and video have become emerging research areas in recent years. The previous research focused on designing features and developing feature extraction methods. Generally, a multimedia content can be represented with different feature representations (i.e., views). However, the most suitable feature representation related to people's emotions is usually not known a priori. We propose here a novel Bayesian multiple kernel learning algorithm for affective classification and retrieval tasks. The proposed method can make use of different representations simultaneously (i.e., multiview learning) to obtain a better prediction performance than using a single feature representation (i.e., single-view learning) or a subset of features, with the advantage of automatic feature selections. In particular, our algorithm has been implemented within a multilabel setup to capture the correlation between emotions, and the Bayesian formulation enables our method to produce probabilistic outputs for measuring a set of emotions triggered by a single image. As a case study, we perform classification and retrieval experiments with our algorithm for predicting people's emotional states evoked by images, using generic low-level image features. The empirical results with our approach on the widely-used International Affective Picture System (IAPS) data set outperform several existing methods in terms of classification performance and results interpretability.

Original languageEnglish (US)
Pages (from-to)3-13
Number of pages11
JournalNeurocomputing
Volume165
DOIs
StatePublished - Oct 1 2015

Keywords

  • Image emotions
  • Low-level image features
  • Multiple kernel learning
  • Multiview learning
  • Variational approximation

ASJC Scopus subject areas

  • Computer Science Applications
  • Cognitive Neuroscience
  • Artificial Intelligence

Fingerprint Dive into the research topics of 'Understanding emotional impact of images using Bayesian multiple kernel learning'. Together they form a unique fingerprint.

  • Cite this