Copyright © 2003 SRI International
Early studies of effectiveness
By 1991, this author’s research team at Michigan had carried out meta-analyses of findings from 121 controlled studies of teaching in colleges and universities (J. Kulik et al., 1980; C. Kulik & Kulik, 1986; C. Kulik & Kulik, 1991). The studies contained results from a variety of computer applications in a number of different disciplines. Among the 121 studies were a substantial number on computer tutorials and computer simulations in science. The results from these early studies provide a good standard for gauging recent contributions of tutorials and simulations to science teaching, and are therefore reviewed here. Listed in the 1986 and 1991 reviews were achievement effect sizes from 37 studies of computer tutoring in college courses. Results of these studies favored the computer-tutored students by a small amount. In 26 of the 37 studies, the tutorial group outperformed the control group; in the remaining 11 studies, the control group scores were higher. The effect sizes in the 37 studies were between –1.20 and 1.25. The median effect size was 0.15. This effect is not large enough to be considered educationally meaningful. It suggests that computer-tutored students would perform at the 56th percentile on relevant achievement tests, whereas conventionally taught students would performat the 50th percentile. Results of computer tutoring in science courses were similar to results in nonscience areas. These reviews also contained findings from 13 studies of computer simulations in science. Results of these studies were favorable to the groups that worked with the computer simulations. In 11 of the 13 studies, the simulation group outperformed the control group, but in the remaining studies, the control group outscored the simulation group. The effect sizes in the 13 studies were between –0.14 and 1.27. The median effect size was 0.25. Effect sizes of 0.25 and over are usually considered to be educationally meaningful. By this standard, the effects of computer simulations are just large enough to be judged as educationally meaningful. An effect size of 0.25 suggests that students who worked with simulations would perform at the 60percentile on relevant achievement tests, whereas conventionally taught students would perform at the 50th percentile. Computer tutoring
Seven studies of computer tutorials from the 1990s were identified for this literature review. The studies examined two kinds of instructional outcomes: student achievement and student attitudes (Table 7). Effects of computer tutorials on both outcomes were mixed. Large or moderate positive effects. The effects of computer tutoring were large and positive in two studies (Kitz & Thorpe, 1995; Vitale & Romance, 1992). Both of these studies examined effectiveness of videodisc software from Systems Impact Corporation. Conclusion
It is clear that computers can contribute substantially to the improvement of college teaching. Evaluation studies of the past decade usually found that college courses taught with computer help were more effective than similar courses taught without such help. These recent studies produced far more favorable results than did studies of the 1960s, 1970s, and 1980s. In 119 studies carried out between 1967 and 1986, the median effect of instructional technology was to raise scores on examinations by 0.30 standard deviations (C. Kulik & Kulik, 1986, 1991). In the 46 more recent studies reviewed in this report, the average effect of instructional technology was to raise student scores by 0.46 standard deviations. Both gains are...