Analyses of factors related to positive test bias in software testing

Laura Marie Leventhal, Barbee Eve Teasley, Diane Rohlman

Research output: Contribution to journalArticle

14 Citations (Scopus)

Abstract

In earlier work, we have shown that software testers exhibit positive test bias. Positive test bias is the pervasive behavioral phenomenon in which hypothesis testers tend to test a hypothesis with data which confirms the hypothesis. However, in software testing this behavior may be counter-productive, since it may be more effective to test with data which are designed to disconfirm the hypothesis. The first study considered how positive test bias is influenced by the expertise level of the subjects, the completeness of the software specifications and whether or not the programs contained errors. The results demonstrated strong evidence of positive test bias regardless of condition. The effects appear to be partially mitigated by increasingly higher levels of expertise and by increasingly more complete specifications. In some cases, the effect is also increased by the presence of errors. A second study used talk-aloud protocols to explore the kinds of hypotheses testers generate during testing. The results further emphasize that subjects test their programs in a biased way and support the notion that the program specification drives testers' hypotheses. We conclude that positive test bias is a critical concern in software testing and may have a seriously detrimental effect on the quality of testing. The results further emphasize the importance of complete and thorough program specifications in order to enhance effective testing.

Original languageEnglish (US)
Pages (from-to)717-749
Number of pages33
JournalInternational Journal of Human Computer Studies
Volume41
Issue number5
DOIs
StatePublished - Nov 1994

Fingerprint

Software testing
Statistical Factor Analysis
Software
Specifications
trend
Testing
expertise
test subject
Network protocols
software
evidence

ASJC Scopus subject areas

  • Artificial Intelligence
  • Human-Computer Interaction
  • Experimental and Cognitive Psychology

Cite this

Analyses of factors related to positive test bias in software testing. / Leventhal, Laura Marie; Teasley, Barbee Eve; Rohlman, Diane.

In: International Journal of Human Computer Studies, Vol. 41, No. 5, 11.1994, p. 717-749.

Research output: Contribution to journalArticle

Leventhal, Laura Marie ; Teasley, Barbee Eve ; Rohlman, Diane. / Analyses of factors related to positive test bias in software testing. In: International Journal of Human Computer Studies. 1994 ; Vol. 41, No. 5. pp. 717-749.
@article{c73aec26e36a4309a16726ac08054f25,
title = "Analyses of factors related to positive test bias in software testing",
abstract = "In earlier work, we have shown that software testers exhibit positive test bias. Positive test bias is the pervasive behavioral phenomenon in which hypothesis testers tend to test a hypothesis with data which confirms the hypothesis. However, in software testing this behavior may be counter-productive, since it may be more effective to test with data which are designed to disconfirm the hypothesis. The first study considered how positive test bias is influenced by the expertise level of the subjects, the completeness of the software specifications and whether or not the programs contained errors. The results demonstrated strong evidence of positive test bias regardless of condition. The effects appear to be partially mitigated by increasingly higher levels of expertise and by increasingly more complete specifications. In some cases, the effect is also increased by the presence of errors. A second study used talk-aloud protocols to explore the kinds of hypotheses testers generate during testing. The results further emphasize that subjects test their programs in a biased way and support the notion that the program specification drives testers' hypotheses. We conclude that positive test bias is a critical concern in software testing and may have a seriously detrimental effect on the quality of testing. The results further emphasize the importance of complete and thorough program specifications in order to enhance effective testing.",
author = "Leventhal, {Laura Marie} and Teasley, {Barbee Eve} and Diane Rohlman",
year = "1994",
month = "11",
doi = "10.1006/ijhc.1994.1079",
language = "English (US)",
volume = "41",
pages = "717--749",
journal = "International Journal of Human Computer Studies",
issn = "1071-5819",
publisher = "Academic Press Inc.",
number = "5",

}

TY - JOUR

T1 - Analyses of factors related to positive test bias in software testing

AU - Leventhal, Laura Marie

AU - Teasley, Barbee Eve

AU - Rohlman, Diane

PY - 1994/11

Y1 - 1994/11

N2 - In earlier work, we have shown that software testers exhibit positive test bias. Positive test bias is the pervasive behavioral phenomenon in which hypothesis testers tend to test a hypothesis with data which confirms the hypothesis. However, in software testing this behavior may be counter-productive, since it may be more effective to test with data which are designed to disconfirm the hypothesis. The first study considered how positive test bias is influenced by the expertise level of the subjects, the completeness of the software specifications and whether or not the programs contained errors. The results demonstrated strong evidence of positive test bias regardless of condition. The effects appear to be partially mitigated by increasingly higher levels of expertise and by increasingly more complete specifications. In some cases, the effect is also increased by the presence of errors. A second study used talk-aloud protocols to explore the kinds of hypotheses testers generate during testing. The results further emphasize that subjects test their programs in a biased way and support the notion that the program specification drives testers' hypotheses. We conclude that positive test bias is a critical concern in software testing and may have a seriously detrimental effect on the quality of testing. The results further emphasize the importance of complete and thorough program specifications in order to enhance effective testing.

AB - In earlier work, we have shown that software testers exhibit positive test bias. Positive test bias is the pervasive behavioral phenomenon in which hypothesis testers tend to test a hypothesis with data which confirms the hypothesis. However, in software testing this behavior may be counter-productive, since it may be more effective to test with data which are designed to disconfirm the hypothesis. The first study considered how positive test bias is influenced by the expertise level of the subjects, the completeness of the software specifications and whether or not the programs contained errors. The results demonstrated strong evidence of positive test bias regardless of condition. The effects appear to be partially mitigated by increasingly higher levels of expertise and by increasingly more complete specifications. In some cases, the effect is also increased by the presence of errors. A second study used talk-aloud protocols to explore the kinds of hypotheses testers generate during testing. The results further emphasize that subjects test their programs in a biased way and support the notion that the program specification drives testers' hypotheses. We conclude that positive test bias is a critical concern in software testing and may have a seriously detrimental effect on the quality of testing. The results further emphasize the importance of complete and thorough program specifications in order to enhance effective testing.

UR - http://www.scopus.com/inward/record.url?scp=0028548323&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=0028548323&partnerID=8YFLogxK

U2 - 10.1006/ijhc.1994.1079

DO - 10.1006/ijhc.1994.1079

M3 - Article

AN - SCOPUS:0028548323

VL - 41

SP - 717

EP - 749

JO - International Journal of Human Computer Studies

JF - International Journal of Human Computer Studies

SN - 1071-5819

IS - 5

ER -