External Validity

Only available on StudyMode
  • Download(s) : 227
  • Published : April 24, 2011
Open Document
Text Preview
Methodologies and Research Design 18:

Research Design IV: External Validity and


• to unpack different types of external validity

• to describe crucial issues in sampling
- precision vs. representativeness vs. cost
- probability vs. non-probability / 'judgement'

• to describe stages in the process of sampling, and the possible intrusion of ‘bias’

• to describe methods of probability sampling
and methods of non-probability sampling

• to be able to estimate your desired sample size,
- using rules of thumb
- or charts (e.g. in deVaus)

• to describe the problem of non-response, and how to minimise it

• to consider methods used for ‘sampling’ and generalisability in qualitative / ‘flexible’ designs External Validity, or Generalisibility

Population Validity - generalisibility from selected sample of cases

to population of interest

Ecological Validity - generalisibility / applicable
from particular
• time
• place / setting, incl. conditions of the research
• event(s)

to other settings,

to other time-periods

to other “events”, OR

to other forms of the “treatment” (independent variable),

to other measure of the outcome (dependent variable)

(1) Random Sampling for Representativeness

Basic trade-off: Validityvs. Cost

esp. External

esp. Population

Representativeness: avoiding (sampling) bias

Precision: source – sampling variation
measure – standard error
aims: to estimate
to minimise

Cost (per interview)
• contact costs

• fieldwork costs (incl. travel, postage)

• coding and analysis costs
Stages in the Social Process of Sampling

1. Define the General or Target Population

* 2. Specify the Working Population: “list”

3. Gain Access: to list,
to members of population

* 4. Select the Sample: probability or non-prob. ?

* 5. Contact members of the Sample

* 6. Gain their Cooperation

7. Generalise: statistical inference - Estimation

OR Hypothesis Test

e.g. internet surveys (de Vaus, 2002, 77-80)

* Possible intrusion of bias

SAMPLING - Methods of Selection

Methods of probability sampling

• simple random sampling (SRS) : every member of the population has an equal probability of being selected

• “systematic random sampling”

• stratified random sampling: every member of the population has an non-zero (specified) probability of being selected

• multi-stage / cluster sampling

Methods of non-probability sampling
• quota sampling [“looks like” stratified random sampling]

• critical case: e.g. Luton in The Affluent Worker

• snowball

• volunteer

• convenience


• non-contact
• non-cooperation

Methods of improving response rates

Introduction / communication with respondent
e.g. invitation / request, covering letter

• reasons why the study – BUT NOT hypotheses

• why important to you

• why might be important to respondent

• safeguards: confidentiality vs. anonymity

• connections, solidarity with respondent

• interest, importance: ‘vital information’; sense of having an effect

• promise feedback, copy of report

• money (e.g. FES)

• a chance of winning prize
Estimating Size of Sample needed

A Rules of Thumb:
NB Focus on the smallest subgroup.

• Questionnaire survey

• Interviews

• Semi-structured

B Depending on precision of estimation desired

• Using tables

• Exact calculation (STX4300, using ideas of confidence intervals)

Qualitative / ‘flexible’ designs:
Methods of ‘sampling’ and ensuring generalisability

• (2) Deliberate Sampling for Heterogeneity (Cook & Campbell, 1979) …. OR
‘Theoretical sampling’
[excerpt from (Atkinson, 1979)]

• Replication: not simply repetition [possible?]

In other settings, at other times

[Robson, 2002]

• (‘Thick’) Description and...
tracking img