External Validity

Methodologies and Research Design 18:

Research Design IV: External Validity and

Sampling

Objectives • to unpack different types of external validity

• to describe crucial issues in sampling - precision vs. representativeness vs. cost - probability vs. non-probability / 'judgement '

• to describe stages in the process of sampling, and the possible intrusion of ‘bias’

• to describe methods of probability sampling and methods of non-probability sampling

• to be able to estimate your desired sample size, - using rules of thumb - or charts (e.g. in deVaus)

• to describe the problem of non-response, and how to minimise it

• to consider methods used for ‘sampling’ and generalisability in qualitative / ‘flexible’ designs External Validity, or Generalisibility

Population Validity - generalisibility from selected sample of cases

to population of interest

Ecological Validity - generalisibility / applicable from particular • time • place / setting, incl. conditions of the research • event(s)

to other settings,

to other time-periods

to other “events”, OR

to other forms of the “treatment” (independent variable),

to other measure of the outcome (dependent variable)

(1) Random Sampling for Representativeness

Basic trade-off: Validity vs. Cost

esp. External

esp. Population

Representativeness: avoiding (sampling) bias

Precision: source – sampling variation measure – standard error aims: to estimate to minimise

Cost (per interview) • contact costs

• fieldwork costs (incl. travel, postage)

• coding and analysis costs
Stages in the Social Process of Sampling

1. Define the General or Target Population

* 2. Specify the Working Population: “list”

3. Gain Access: to list, to members of population

* 4. Select the Sample: probability or non-prob. ?

* 5. Contact members of the Sample

* 6. Gain their

References: Williams, M. (2000), ‘Interpretivism and Generalisability’, Sociology, 34, 2, 209-224. Williams (2000) considers: Proposition 1: Interpretivists do generalise and this is inevitable – though they may deny the possibility of generalisation, or ignore the issue. Geertz (1979), ‘The Balinese Cockfight’ – generalises from a particular ritual in Balinese life, to Balinese culture in general Fisher (1993) generalises from a typology of young ‘fruit machine’ gamblers to ‘what it is to gamble’ Williams (2000): examples of : • Moderatum generalisations: aspects of research situation are seen to be instances of a broader recognisable set of features... Roper I. and Cunningham I. (2003). ‘Working beyond Contract: Employer Views on Family Friendly Employment Relations’. MUBS / HRM. ‘… where approval is given, … this may be fragile: … may be given in principle, yet in terms of ‘practicality’, these same people may object.’ Scase and Goffee (1989) [ Bryman & Bell, 2003, p52]