TY - JOUR
T1 - How many parameters can a model have and still be testable?
AU - Bamber, Donald
AU - van Santen, Jan P.H.
N1 - Funding Information:
The first author’s work on this paper was supported by Veterans Administration Medical Research funds. The second author was supported by United States Air Force, Life Sciences Directorate, Grant AFOSR 80-0279. This paper has benefitted from extensive comments by Consulting Editor William Batchelder. Conversations with Richard Schweickert and James Townsend have helped to clarify some of the ideas presented in this paper. Also, careful reading by James Harper, and comments by James Neely and Peter Schoenemann, are appreciated. Morris W. Hirsch and Ralph Carr provided us with proof outlines for Appendix B and Example 4, respectively. Portions of this paper were presented at the Sixteenth Annual Mathematical Psychology Meeting, Boulder, Colorado, August 11, 1983.
PY - 1985/12
Y1 - 1985/12
N2 - A standard rule of thumb states that a model has too many parameters to be testable if and only if it has at least as many parameters as empirically observable quantities. We argue that when one asks whether a model has too many parameters to be testable, one implicitly refers to a particular type of testability, which we call quantitative testability. A model is defined to be quantitatively testable if the model's predictions have zero probability of being correct by chance. Next, we propose a new rule of thumb, based on the rank of the Jacobian matrix of a model (i.e., the matrix of partial derivatives of the function that maps the model's parameter values onto predicted experimental outcomes). According to this rule, a model is quantitatively testable if and only if the rank of the Jacobian matrix is less than the number of observables. (The rank of his matrix can be found with standard computer algorithms.) Using Sard's theorem, we prove that the proposed new rule of thumb is correct provided that certain "smoothness" conditions are satisfied. We also discuss the relation between quantitative testability and reparameterization, identifiability, and goodness-of-fit testing.
AB - A standard rule of thumb states that a model has too many parameters to be testable if and only if it has at least as many parameters as empirically observable quantities. We argue that when one asks whether a model has too many parameters to be testable, one implicitly refers to a particular type of testability, which we call quantitative testability. A model is defined to be quantitatively testable if the model's predictions have zero probability of being correct by chance. Next, we propose a new rule of thumb, based on the rank of the Jacobian matrix of a model (i.e., the matrix of partial derivatives of the function that maps the model's parameter values onto predicted experimental outcomes). According to this rule, a model is quantitatively testable if and only if the rank of the Jacobian matrix is less than the number of observables. (The rank of his matrix can be found with standard computer algorithms.) Using Sard's theorem, we prove that the proposed new rule of thumb is correct provided that certain "smoothness" conditions are satisfied. We also discuss the relation between quantitative testability and reparameterization, identifiability, and goodness-of-fit testing.
UR - http://www.scopus.com/inward/record.url?scp=0039029896&partnerID=8YFLogxK
UR - http://www.scopus.com/inward/citedby.url?scp=0039029896&partnerID=8YFLogxK
U2 - 10.1016/0022-2496(85)90005-7
DO - 10.1016/0022-2496(85)90005-7
M3 - Article
AN - SCOPUS:0039029896
SN - 0022-2496
VL - 29
SP - 443
EP - 473
JO - Journal of Mathematical Psychology
JF - Journal of Mathematical Psychology
IS - 4
ER -