Analyses of factors related to positive test bias in software testing

Laura Marie Leventhal, Barbee Eve Teasley, Diane Schertler Rohlman

Research output: Contribution to journalArticle

14 Scopus citations

Abstract

In earlier work, we have shown that software testers exhibit positive test bias. Positive test bias is the pervasive behavioral phenomenon in which hypothesis testers tend to test a hypothesis with data which confirms the hypothesis. However, in software testing this behavior may be counter-productive, since it may be more effective to test with data which are designed to disconfirm the hypothesis. The first study considered how positive test bias is influenced by the expertise level of the subjects, the completeness of the software specifications and whether or not the programs contained errors. The results demonstrated strong evidence of positive test bias regardless of condition. The effects appear to be partially mitigated by increasingly higher levels of expertise and by increasingly more complete specifications. In some cases, the effect is also increased by the presence of errors. A second study used talk-aloud protocols to explore the kinds of hypotheses testers generate during testing. The results further emphasize that subjects test their programs in a biased way and support the notion that the program specification drives testers’ hypotheses. We conclude that positive test bias is a critical concern in software testing and may have a seriously detrimental effect on the quality of testing. The results further emphasize the importance of complete and thorough program specifications in order to enhance effective testing.

Original languageEnglish (US)
Pages (from-to)717-749
Number of pages33
JournalInternational Journal of Human Computer Studies
Volume41
Issue number5
DOIs
StatePublished - Nov 1994

ASJC Scopus subject areas

  • Software
  • Human Factors and Ergonomics
  • Education
  • Engineering(all)
  • Human-Computer Interaction
  • Hardware and Architecture

Fingerprint Dive into the research topics of 'Analyses of factors related to positive test bias in software testing'. Together they form a unique fingerprint.

Cite this