What is a Social Experiment?
The summaries in this Digest focus on field studies of social programs in which individuals, households, or (in rare instances) firms or organizations were randomly assigned to two or more alternative treatments. The primary research objective of the experiments was to measure the effects, which are often called "impacts," of the alternative treatments on market behavior (such as the receipt of earnings) and corresponding government fiscal outcomes (such as the receipt of transfer benefits). Thus, a social experiment has at least the following features:
- Random assignment: Creation of at least two groups of human subjects who differ from one another by chance alone.
- Policy intervention: A set of actions ensuring that different incentives, opportunities, or constraints confront the members of each of the randomly assigned groups in their daily lives.
- Follow-up data collection: Measurement of market and fiscal outcomes for members of each group.
- Evaluation: Application of statistical inference and informed professional judgment about the degree to which the policy interventions have caused differences in outcomes between the groups.
Random assignment involves neither choice nor discretion. Whereas human subjects may or may not have the right to choose to participate in the experiment, they do not have the right to decide which group within the experiment they will join. Similarly, those administering the policy intervention may restrict eligibility for participation in the experiment, but once a person is admitted, program staff cannot determine the group in which that subject is enrolled, except by using randomization.
Social experiments test policy interventions: They are attempts to influence the endowments of, and the incentives and disincentives facing, human subjects. Thus, in most social experiments, one of the randomly assigned groups, the control group, represents the status quo and is only eligible for benefits and services under the existing policy regime. The remaining group or groups, the treatment group(s), are subjected to the policy innovation or innovations being tested. Comparisons of the control group with the treatment group(s) indicate the impacts of the tested innovations.
Social experiments are designed to determine whether (or how much) the policies being tested would affect the market behavior of individuals (e.g., their employment and earnings; consumption of food, energy, housing, and health care services; receipt of government benefits). Taken together, the second and third features of our definition exclude random-assignment experiments in medicine, psychology, economics, criminology, and education that are not designed to measure changes in subjects' transactions in their daily environment in response to policy innovations. For example, we exclude randomized clinical trials of prescription drugs intended to affect health status; we also exclude randomization of students to competing school curricula intended to improve scores on standardized tests.
Although outcome data may be collected by a variety of means, readers of this volume's summaries will notice that over time social experimenters have relied increasingly on administrative data rather than surveys. Once collected, outcomes among the groups of randomly assigned individuals are compared. Outcome differences between the groups provide estimates of the impacts of the tested policy interventions. However, the data do not speak for themselves. Analysts must decide what data transformations are appropriate; what, if any, nonexperimental factors should be considered; what statistical techniques to use; and what results do and do not make sense.
Comments