Meta Analysis

Topics: Effect size, Statistical significance, Meta-analysis Pages: 22 (7579 words) Published: August 5, 2013
Chapter 22 Meta-Analysis

‘Fett’s Law: Never replicate a successful experiment’

Content list What is meta-analysis? Examples of meta-analytic studies Conducting a meta-analysis Replication and meta-analysis Comparing studies by effect size Combining studies by effect size Comparing studies by significance levels Combining studies by significance levels Comparing and combining more than two effect sizes and significance levels Some issues in meta-analysis 533 535 536 539 540 542 544 545 547 547

By the end of this chapter you will understand:
1 What meta-analysis is. 2 How it helps in confirming research findings. 3 How to undertake a meta-analysis.

Introduction Meta-analysis has become an important research strategy as it enables researchers to combine the results of many pieces of research on a topic to determine whether the finding holds generally. This is better than trying to assume that the findings of a single study have global meaning.

What is meta-analysis?
Each strand of a rope contributes to the strength of the rope. But the rope is stronger than any individual strand. Similarly, when a particular finding is obtained repeatedly,

534

EXTENSION CHAPTERS ON ADVANCED TECHNIQUES

under a variety of conditions, we are strongly confident that there exists a general principle. The results of small localized individual studies, no matter how well conducted, are often insufficient to provide us with confident answers to questions of general importance. Meta-analysis allows us to compare or combine results across a set of similar studies. In the individual study, the units of analysis are the individual observations. In meta-analysis the units of analysis are the results of individual studies. The term meta-analysis means ‘an analysis of analysis’. A particular topic may have been replicated in various ways, using, for example, differently sized samples, and conducted in different countries under different environmental, social and economic conditions. Sometimes results appear to be reasonably consistent; others less so. Meta-analysis enables a rigorous comparison to be made rather than a subjective ‘eyeballing’. However, the technique relies on all relevant information being available for each of the examined studies. If some crucial factors like sample size and methodology are missing then comparison is not feasible.

Meta-analysis. An objective and quantitative methodology for synthesizing previous studies and research on a particular topic into an overall finding.

If you are asked to write a report for your manager on the economics of recycling waste paper, or on the relationship between air travel fares and ticket sales, or on the relationship between money supply and mortgage rates, you will search for information from a variety of sources, including in-house documents, the Internet, the national and local libraries, etc. The strategy is to read studies relevant to the topic you wish to investigate, summarize the findings, and then integrate the existing knowledge. From this you may conclude that a particular variable is of crucial importance, or that the relationships between particular variables are worthy of note. This is the standard literature survey where you draw essentially subjective conclusions based on your critical evaluation of the literature. You often use a ‘voting method’ as a crude index of where the balance of results lies. This method is flawed and inexact because: • • • you are unable to deal with the large number of studies on a topic, and so focus on a small subset of studies, often without describing how the subset was selected; you often cite the conclusions of previous reviews without examining those reviews critically; you are interested in a particular issue so you might not be inclined to give full weight to evidence that is contrary to your own desired outcome.

As a result, your subjective conclusion may not accurately reflect the actual strength of the relationship. You...

References: Cooper, H. & Rosenthal, R. 1982. Statistical versus traditional methods for summarising research findings. Psychological Bulletin, 87, 442–449. De Dreu, C.K. & Weingart, L.R. 2003. Task versus relationship conflict, team performance and team member satisfaction: a meta analysis. Journal of Applied Psychology, 88 (4), 741–749. Gully, S., Incalcaterra, K., Joshi, A. & Beaubien, J. 2002. A meta analysis of team efficiency, potency and performance. Journal of Applied Psychology, 87 (5), 819–832. Hosada, M., Stone-Romero, E. & Coats, G. 2003. The effects of physical attractiveness on job related outcomes. Personnel Psychology, 56 (2), 431–448. Iaffaldano, M. & Muchinsky, P.M. 1985. Job satisfaction and job performance: a meta analysis. Psychological Bulletin, 97, 251–273. Jenkins, J. 1986. Financial incentives. In Generalising from Laboratory to Field Settings. Locke, E. (ed). Lexington: Lexington Books. Judge, T.A., Colberet, A. & Illies, R. 2004. Intelligence and leadership. Journal of Applied Psychology, 89 (3), 542–552. Mullen, B. & Copper, C. 1994. The relation between group cohesion and performance. Psychological Bulletin, 115 (2), 210–227. Mullen, B. & Rosenthal, R. 1985. Basic Meta-analysis. Hillsdale: Lawrence Erlbaum. Rosenthal, R. 1979. The file drawer problem. Psychological Bulletin, 86, 638–641. Rosenthal, R. 1984. Meta analytic procedures for social research, in Applied Social Science Research Methods, Vol 6. Beverly Hills: Sage. Rosenthal, R. 1994. Interpersonal expectancy effects. A 30 year perspective. Current Directions in Psychological Science, 3, 176–179. Rosenthal, R. 1991. Meta-analytic Procedures for Social Research. Newbury Park: Sage. Smith, M. & Glass, G. 1977. Meta-analysis of psychotherapy outcome studies. American Psychologist. 32, 752–760. Thorsteinson, T.J. 2003. Job attitudes of part-time vs. full-time workers. Journal of Occupational and Organisational Psychology, 76 (2), 151–177.
You will never have to undertake meta-analysis by hand. There are computer programs for this. For example: 1 Meta-Stat – A Tool for the Meta-Analysis of Research Studies. Produced by Lawrence M. Rudner, Gene V. Glass, David L. Evartt and Patrick J. Emery. Meta-Stat is a DOSbased computer program that automates the many complex tasks that are required to
META-ANALYSIS 551
2
perform a meta-analysis. The data can easily be output in a format ready for use by SPSS. Meta-Stat is free for non-commercial, educational use. Meta-Stat is available through the auspices of the ERIC Clearinghouse on Assessment and Evaluation, Department of Measurement, Statistics and Evaluation, University of Maryland, College Park. Comprehensive Meta-Analysis Program (CMA) produced by Biostat. A free trial download is available at www.power.analysis.com/about.biostat.htm
Now turn to the website page for this chapter and undertake the activities there.
Continue Reading

Please join StudyMode to read the full document

You May Also Find These Documents Helpful

  • meta-analysis critique cochrane review Essay
  • Leader-member exchange meta-analysis Essay
  • Critical Review of Meta Analysis Essay
  • Cohesion and Performance in Groups: A Meta-Analytic Clarification of Construct Relations Essay
  • effectiveness applied behaviour analysi Research Paper
  • Nlp Meta-Modeling Questions Essay
  • Pest Analysis Essay
  • Tescos Pest Analysis Essay

Become a StudyMode Member

Sign Up - It's Free