Software Testing
A JOURNAL OF LTEX CLASS FILES, VOL. 6, NO. 1, JANUARY 2007
1
Enhancing Performance of Random Testing Through Markov Chain Monte Carlo Methods
Bo Zhou, Hiroyuki Okamura, Member, IEEE, and Tadashi Dohi, Member, IEEE
Abstract—In this paper, we propose a probabilistic approach to finding failure-causing inputs based on Bayesian estimation. According to our probabilistic insights of software testing, the test case generation algorithms are developed by Markov chain Mote Carlo (MCMC) methods. Dissimilar to existing random testing schemes such as adaptive random testing, our approach can also utilize the prior knowledge on software testing. In experiments, we compare effectiveness of our MCMC-based random testing with both ordinary random testing and adaptive random testing in real program sources. These results indicate that MCMC-based random testing can drastically improve the effectiveness of software testing. Index Terms—Software testing, Random testing, Adaptive random testing, Bayes statistics, Markov chain Monte Carlo.
!
1
I NTRODUCTION
Software testing, defined as the systematic execution of the software system with the aim of revealing failures, is an important phase to validate the correctness of software system. The activities of software testing consists of (i) making test cases and (ii) validating the behavior of software system by executing the test cases. In general, since we cannot validate all of the execution paths, software testing is performed with limited test cases. Thus the quality of test cases directly affects the quality of software products, and one of the quality attributes of test cases is the ability of detecting as-yet-undiscovered failures. A number of different test case generation methods were discussed in the software engineering research field. Random testing (RT) is one of the most classical software testing strategies. Although the RT is quite simple for implementation [1], it is often argued that RT is inefficient,
1
Enhancing Performance of Random Testing Through Markov Chain Monte Carlo Methods
Bo Zhou, Hiroyuki Okamura, Member, IEEE, and Tadashi Dohi, Member, IEEE
Abstract—In this paper, we propose a probabilistic approach to finding failure-causing inputs based on Bayesian estimation. According to our probabilistic insights of software testing, the test case generation algorithms are developed by Markov chain Mote Carlo (MCMC) methods. Dissimilar to existing random testing schemes such as adaptive random testing, our approach can also utilize the prior knowledge on software testing. In experiments, we compare effectiveness of our MCMC-based random testing with both ordinary random testing and adaptive random testing in real program sources. These results indicate that MCMC-based random testing can drastically improve the effectiveness of software testing. Index Terms—Software testing, Random testing, Adaptive random testing, Bayes statistics, Markov chain Monte Carlo.
!
1
I NTRODUCTION
Software testing, defined as the systematic execution of the software system with the aim of revealing failures, is an important phase to validate the correctness of software system. The activities of software testing consists of (i) making test cases and (ii) validating the behavior of software system by executing the test cases. In general, since we cannot validate all of the execution paths, software testing is performed with limited test cases. Thus the quality of test cases directly affects the quality of software products, and one of the quality attributes of test cases is the ability of detecting as-yet-undiscovered failures. A number of different test case generation methods were discussed in the software engineering research field. Random testing (RT) is one of the most classical software testing strategies. Although the RT is quite simple for implementation [1], it is often argued that RT is inefficient,