Bayesian supervised dimensionality reduction

Mehmet Gonen

Research output: Contribution to journalArticlepeer-review

16 Scopus citations


Dimensionality reduction is commonly used as a preprocessing step before training a supervised learner. However, coupled training of dimensionality reduction and supervised learning steps may improve the prediction performance. In this paper, we introduce a simple and novel Bayesian supervised dimensionality reduction method that combines linear dimensionality reduction and linear supervised learning in a principled way. We present both Gibbs sampling and variational approximation approaches to learn the proposed probabilistic model for multiclass classification. We also extend our formulation toward model selection using automatic relevance determination in order to find the intrinsic dimensionality. Classification experiments on three benchmark data sets show that the new model significantly outperforms seven baseline linear dimensionality reduction algorithms on very low dimensions in terms of generalization performance on test data. The proposed model also obtains the best results on an image recognition task in terms of classification and retrieval performances.

Original languageEnglish (US)
Article number6475174
Pages (from-to)2179-2189
Number of pages11
JournalIEEE Transactions on Cybernetics
Issue number6
StatePublished - Dec 1 2013


  • Dimensionality reduction
  • Gibbs sampling
  • Handwritten digit recognition
  • Image recognition
  • Image retrieval
  • Multiclass classification
  • Subspace learning
  • Variational approximation

ASJC Scopus subject areas

  • Software
  • Control and Systems Engineering
  • Information Systems
  • Human-Computer Interaction
  • Computer Science Applications
  • Electrical and Electronic Engineering


Dive into the research topics of 'Bayesian supervised dimensionality reduction'. Together they form a unique fingerprint.

Cite this