Factor analysis of correlation matrices when the number of random variables exceeds the sample size

Research output: Contribution to journalArticle

Abstract

Factor analysis which studies correlation matrices is an effective means of data reduction whose inference on the correlation matrix typically requires the number of random variables, p, to be relatively small and the sample size, n, to be approaching infinity. In contemporary data collection for biomedical studies, disease surveillance and genetics, p > n limits the use of existing factor analysis methods to study the correlation matrix. The motivation for the research here comes from studying the correlation matrix of log annual cancer mortality rate change for p = 59 cancer types from 1969 to 2008 (n = 39) in the U.S.A. We formalise a test statistic to perform inference on the structure of the correlation matrix when p > n. We develop an approach based on group sequential theory to estimate the number of relevant factors to be extracted. To facilitate interpretation of the extracted factors, we propose a BIC (Bayesian Information Criterion)-type criterion to produce a sparse factor loading representation. The proposed methodology outperforms competing ad hoc methodologies in simulation analyses, and identifies three significant underlying factors responsible for the observed correlation between cancer mortality rate changes.

Original languageEnglish (US)
Pages (from-to)246-256
Number of pages11
JournalStatistical Theory and Related Fields
Volume1
Issue number2
DOIs
StatePublished - Jul 3 2017

Fingerprint

Correlation Matrix
Factor analysis
Factor Analysis
Random variables
Exceed
Sample Size
Random variable
Cancer
Mortality Rate
Group Sequential
Bayesian Information Criterion
Methodology
Data Reduction
Surveillance
Annual
Test Statistic
Data reduction
Infinity
Correlation matrix
Sample size

Keywords

  • Alpha spending function
  • BIC
  • cancer surveillance
  • eigenvalues
  • sparse factor loadings

ASJC Scopus subject areas

  • Statistics and Probability
  • Analysis
  • Applied Mathematics
  • Statistics, Probability and Uncertainty
  • Computational Theory and Mathematics

Cite this

Factor analysis of correlation matrices when the number of random variables exceeds the sample size. / Marino, Miguel; Li, Yi.

In: Statistical Theory and Related Fields, Vol. 1, No. 2, 03.07.2017, p. 246-256.

Research output: Contribution to journalArticle

@article{fbf12a107c1d44089288cc7ba22e4670,
title = "Factor analysis of correlation matrices when the number of random variables exceeds the sample size",
abstract = "Factor analysis which studies correlation matrices is an effective means of data reduction whose inference on the correlation matrix typically requires the number of random variables, p, to be relatively small and the sample size, n, to be approaching infinity. In contemporary data collection for biomedical studies, disease surveillance and genetics, p > n limits the use of existing factor analysis methods to study the correlation matrix. The motivation for the research here comes from studying the correlation matrix of log annual cancer mortality rate change for p = 59 cancer types from 1969 to 2008 (n = 39) in the U.S.A. We formalise a test statistic to perform inference on the structure of the correlation matrix when p > n. We develop an approach based on group sequential theory to estimate the number of relevant factors to be extracted. To facilitate interpretation of the extracted factors, we propose a BIC (Bayesian Information Criterion)-type criterion to produce a sparse factor loading representation. The proposed methodology outperforms competing ad hoc methodologies in simulation analyses, and identifies three significant underlying factors responsible for the observed correlation between cancer mortality rate changes.",
keywords = "Alpha spending function, BIC, cancer surveillance, eigenvalues, sparse factor loadings",
author = "Miguel Marino and Yi Li",
year = "2017",
month = "7",
day = "3",
doi = "10.1080/24754269.2017.1399740",
language = "English (US)",
volume = "1",
pages = "246--256",
journal = "Statistical Theory and Related Fields",
issn = "2475-4269",
number = "2",

}

TY - JOUR

T1 - Factor analysis of correlation matrices when the number of random variables exceeds the sample size

AU - Marino, Miguel

AU - Li, Yi

PY - 2017/7/3

Y1 - 2017/7/3

N2 - Factor analysis which studies correlation matrices is an effective means of data reduction whose inference on the correlation matrix typically requires the number of random variables, p, to be relatively small and the sample size, n, to be approaching infinity. In contemporary data collection for biomedical studies, disease surveillance and genetics, p > n limits the use of existing factor analysis methods to study the correlation matrix. The motivation for the research here comes from studying the correlation matrix of log annual cancer mortality rate change for p = 59 cancer types from 1969 to 2008 (n = 39) in the U.S.A. We formalise a test statistic to perform inference on the structure of the correlation matrix when p > n. We develop an approach based on group sequential theory to estimate the number of relevant factors to be extracted. To facilitate interpretation of the extracted factors, we propose a BIC (Bayesian Information Criterion)-type criterion to produce a sparse factor loading representation. The proposed methodology outperforms competing ad hoc methodologies in simulation analyses, and identifies three significant underlying factors responsible for the observed correlation between cancer mortality rate changes.

AB - Factor analysis which studies correlation matrices is an effective means of data reduction whose inference on the correlation matrix typically requires the number of random variables, p, to be relatively small and the sample size, n, to be approaching infinity. In contemporary data collection for biomedical studies, disease surveillance and genetics, p > n limits the use of existing factor analysis methods to study the correlation matrix. The motivation for the research here comes from studying the correlation matrix of log annual cancer mortality rate change for p = 59 cancer types from 1969 to 2008 (n = 39) in the U.S.A. We formalise a test statistic to perform inference on the structure of the correlation matrix when p > n. We develop an approach based on group sequential theory to estimate the number of relevant factors to be extracted. To facilitate interpretation of the extracted factors, we propose a BIC (Bayesian Information Criterion)-type criterion to produce a sparse factor loading representation. The proposed methodology outperforms competing ad hoc methodologies in simulation analyses, and identifies three significant underlying factors responsible for the observed correlation between cancer mortality rate changes.

KW - Alpha spending function

KW - BIC

KW - cancer surveillance

KW - eigenvalues

KW - sparse factor loadings

UR - http://www.scopus.com/inward/record.url?scp=85070475648&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=85070475648&partnerID=8YFLogxK

U2 - 10.1080/24754269.2017.1399740

DO - 10.1080/24754269.2017.1399740

M3 - Article

VL - 1

SP - 246

EP - 256

JO - Statistical Theory and Related Fields

JF - Statistical Theory and Related Fields

SN - 2475-4269

IS - 2

ER -