Advantages and Disadvantages of Some Research Designs

Design Advantages Disadvantages
Some pre-experimental designs
Correlation
O		 Simple and easy. Shows if variables are related. Causation not shown.
Instrument reactivity.
Placebo effects.
Pretest-Posttest
O1XO2		 Simple structure.
Simple analysis.		 Cause may be other than Tx.
Instrument reactivity.
Placebo effects.
Longitudinal designs
O1O2O 3--On		 Can use lagging. History not controlled.
Interrupted time series
O1--Oj XOj+1--O n		 Show trends over time. No control for history.
Instrument reactivity (+ or -).
Placebo effects (+or-).
Some quasi-experimental designs
Two-Group Posttest
X1O1
X2O1		 Includes control group. Possible bias in assignment.
Nonrandom subject loss
Multi-Group Posttest
X1O1
X2O1
X3O1
...
XnO1		 More levels of IV.
More control groups.		 Pre-treatment dependent variables not controlled.
Multi-Group
Pretest-Posttest
O1X1O 2
O1X2O 2
...
O1XnO 2		 Get level of DV pre-treatment. Pretest not substitute for control.
Instrument reactivity.
Difference scores controversial.
Ex Post Facto
X1O1 M
X2O1 M		 Better control than if not matched. Not random assignment.
Sample size reduced.
Multi-Group
Time Series
O1--O jX1O j+1--On
O1--OjX 2Oj+1--O n
...
O1--OjX nOj+1--O n		 Shows trend of DV.
Shows duration of treatment.		  
Crossover
O1X1O 2X2O 3
O1X2O 2X1O 3		 Both treatments on each group. Must have washout.
Problem must recur.

These designs, advantages, and disadvantages are discussed in more detail in the references cited in the syllabus.


Another view of the design and timing

O

 Type of Design
Timing of Data Pre-Experimental Quasi-Experimental True Experimental
"One-Shot" O O
O		 R O
R O
Pretest only OX OX1
OX2		 R OX1
R OX2
Posttest only

 XO X1O
X2O		 R X1O
R X2O
Pretest-Posttest

 O1XO 2 O1X 1O2
O1X2O 2		 R O1X 1O2
R O 1X2O 2
Time Series O1O 2O3--O n
O1--Oj XOj-1--O n		 O1--O jX1O j-1--On
O1--OjX 2Oj-1--O n		 R O1--O jX1O j-1--On
R O1--OjX 2Oj-1 --On