# External Validity

Topics: Sampling, Sample, Sample size Pages: 8 (910 words) Published: April 24, 2011
Methodologies and Research Design 18:

Research Design IV: External Validity and

Sampling

Objectives
• to unpack different types of external validity

• to describe crucial issues in sampling
- precision vs. representativeness vs. cost
- probability vs. non-probability / 'judgement'

• to describe stages in the process of sampling, and the possible intrusion of ‘bias’

• to describe methods of probability sampling
and methods of non-probability sampling

• to be able to estimate your desired sample size,
- using rules of thumb
- or charts (e.g. in deVaus)

• to describe the problem of non-response, and how to minimise it

• to consider methods used for ‘sampling’ and generalisability in qualitative / ‘flexible’ designs External Validity, or Generalisibility

Population Validity - generalisibility from selected sample of cases

to population of interest

Ecological Validity - generalisibility / applicable
from particular
• time
• place / setting, incl. conditions of the research
• event(s)

to other settings,

to other time-periods

to other “events”, OR

to other forms of the “treatment” (independent variable),

to other measure of the outcome (dependent variable)

(1) Random Sampling for Representativeness

esp. External

esp. Population

Representativeness: avoiding (sampling) bias

Precision: source – sampling variation
measure – standard error
aims: to estimate
to minimise

Cost (per interview)
• contact costs

• fieldwork costs (incl. travel, postage)

• coding and analysis costs
Stages in the Social Process of Sampling

1. Define the General or Target Population

* 2. Specify the Working Population: “list”

3. Gain Access: to list,
to members of population

* 4. Select the Sample: probability or non-prob. ?

* 5. Contact members of the Sample

* 6. Gain their Cooperation

7. Generalise: statistical inference - Estimation

OR Hypothesis Test

e.g. internet surveys (de Vaus, 2002, 77-80)

* Possible intrusion of bias

SAMPLING - Methods of Selection

Methods of probability sampling

• simple random sampling (SRS) : every member of the population has an equal probability of being selected

• “systematic random sampling”

• stratified random sampling: every member of the population has an non-zero (specified) probability of being selected

• multi-stage / cluster sampling

Methods of non-probability sampling
• quota sampling [“looks like” stratified random sampling]

• critical case: e.g. Luton in The Affluent Worker

• snowball

• volunteer

• convenience

Non-Response

• non-contact
• non-cooperation

Methods of improving response rates

Introduction / communication with respondent
e.g. invitation / request, covering letter

• reasons why the study – BUT NOT hypotheses

• why important to you

• why might be important to respondent

• safeguards: confidentiality vs. anonymity

Incentives
• connections, solidarity with respondent

• interest, importance: ‘vital information’; sense of having an effect

• promise feedback, copy of report

• money (e.g. FES)

• a chance of winning prize
Estimating Size of Sample needed

A Rules of Thumb:
NB Focus on the smallest subgroup.

• Questionnaire survey

• Interviews

• Semi-structured

B Depending on precision of estimation desired

• Using tables

• Exact calculation (STX4300, using ideas of confidence intervals)

Qualitative / ‘flexible’ designs:
Methods of ‘sampling’ and ensuring generalisability

• (2) Deliberate Sampling for Heterogeneity (Cook & Campbell, 1979) …. OR
‘Theoretical sampling’
[excerpt from (Atkinson, 1979)]

• Replication: not simply repetition [possible?]

In other settings, at other times

[Robson, 2002]

• (‘Thick’) Description and...