Bayesian supervised dimensionality reduction

Research output: Contribution to journalArticle

11 Scopus citations

Abstract

Dimensionality reduction is commonly used as a preprocessing step before training a supervised learner. However, coupled training of dimensionality reduction and supervised learning steps may improve the prediction performance. In this paper, we introduce a simple and novel Bayesian supervised dimensionality reduction method that combines linear dimensionality reduction and linear supervised learning in a principled way. We present both Gibbs sampling and variational approximation approaches to learn the proposed probabilistic model for multiclass classification. We also extend our formulation toward model selection using automatic relevance determination in order to find the intrinsic dimensionality. Classification experiments on three benchmark data sets show that the new model significantly outperforms seven baseline linear dimensionality reduction algorithms on very low dimensions in terms of generalization performance on test data. The proposed model also obtains the best results on an image recognition task in terms of classification and retrieval performances.

Original languageEnglish (US)
Article number6475174
Pages (from-to)2179-2189
Number of pages11
JournalIEEE Transactions on Cybernetics
Volume43
Issue number6
DOIs
StatePublished - Dec 1 2013

Keywords

  • Dimensionality reduction
  • Gibbs sampling
  • Handwritten digit recognition
  • Image recognition
  • Image retrieval
  • Multiclass classification
  • Subspace learning
  • Variational approximation

ASJC Scopus subject areas

  • Software
  • Control and Systems Engineering
  • Information Systems
  • Human-Computer Interaction
  • Computer Science Applications
  • Electrical and Electronic Engineering

Fingerprint Dive into the research topics of 'Bayesian supervised dimensionality reduction'. Together they form a unique fingerprint.

  • Cite this