The Logic of Experimental Design
THE LOGIC OF EXPERIMENTAL DESIGN
The randomized comparative experiment is one of the most important ideas in statistics. It is designed to allow us to draw cause-and-effect conclusions. Be sure you understand the logic:
• Randomization produces groups of subjects that should be similar in all respects before we apply the treatments.
• Comparative design ensures that influences other than the experimental treatments operate equally on all groups.
• Therefore, differences in the response variable must be due to the effects of the treatments.
We use chance to choose the groups in order to eliminate any systematic bias in assigning the subjects to groups. For example, Sickle-cell anemia is an inherited disorder of the red blood cells that in the
United States affects mostly blacks. It can cause severe pain and many complications. For example, a doctor might subconsciously assign the most seriously ill patients to the hydroxyurea group, hoping that the untested drug will help them. That would bias the experiment against hydroxyurea. Choosing a Simple Random Sample (SRS) of the subjects in one Group gives everyone the same chance to be in either group. We expect the two groups to be similar in all respects age, seriousness of illness, smoker or not, and so on. Chance tends to assign equal numbers of smokers to both groups, for example, even if we don’t know which subjects are smokers.
It should not surprise you to learn that medical researchers adopted randomized comparative experiments only slowly—many doctors think they can tell “just by watching” whether a new therapy helps their patients. Not so. There are many examples of medical treatments that became popular on the basis of one-track experiments and were shown to be worth no more than a placebo when some skeptic tried a randomized comparative experiment. One search of the medical literature looked for therapies studied both by proper comparative trials and by trials with “historical controls.” A
The randomized comparative experiment is one of the most important ideas in statistics. It is designed to allow us to draw cause-and-effect conclusions. Be sure you understand the logic:
• Randomization produces groups of subjects that should be similar in all respects before we apply the treatments.
• Comparative design ensures that influences other than the experimental treatments operate equally on all groups.
• Therefore, differences in the response variable must be due to the effects of the treatments.
We use chance to choose the groups in order to eliminate any systematic bias in assigning the subjects to groups. For example, Sickle-cell anemia is an inherited disorder of the red blood cells that in the
United States affects mostly blacks. It can cause severe pain and many complications. For example, a doctor might subconsciously assign the most seriously ill patients to the hydroxyurea group, hoping that the untested drug will help them. That would bias the experiment against hydroxyurea. Choosing a Simple Random Sample (SRS) of the subjects in one Group gives everyone the same chance to be in either group. We expect the two groups to be similar in all respects age, seriousness of illness, smoker or not, and so on. Chance tends to assign equal numbers of smokers to both groups, for example, even if we don’t know which subjects are smokers.
It should not surprise you to learn that medical researchers adopted randomized comparative experiments only slowly—many doctors think they can tell “just by watching” whether a new therapy helps their patients. Not so. There are many examples of medical treatments that became popular on the basis of one-track experiments and were shown to be worth no more than a placebo when some skeptic tried a randomized comparative experiment. One search of the medical literature looked for therapies studied both by proper comparative trials and by trials with “historical controls.” A