As RCT Becomes Dominant, Randomistas Could Change Social Science Paradigm
Development economics has changed a lot over the past two decades or so, mainly due to the heavy use of “randomized controlled trials” (RCTs). Randomistas are proponents of RCTs for assessing long-term economic productivity and living standards in poor countries. Three randomists, Abhijit Banerjee, Esther Duflo and Michael Kremer, received the 2019 Nobel Prize in Economics for their RCT-based studies of poverty around the world.
The concept of RCT is quite old; cases of RCT can be traced back to the 16th century. However, the statistical basis for RCT was developed by the British statistician Sir Ronald Fisher about 100 years ago, mainly in the context of the design of experiments.
In my experience, I have seen the proportion of events with the same treatment vary between 10% and 35% in different clinical trials. Is it due to an unknown distribution of treatment effects and / or other external effects such as hospital care, hospital location, etc. ? Thus, for an unbiased evaluation of the treatment, its performance should be compared to a “control”, which may be “no treatment” or an “existing treatment” other than the treatment under study.
The next task is to divide the patients between two treatments / interventions at hand. Patients may prefer one treatment over another. Prior knowledge of the treatments to be applied to them could induce a “selection bias” due to unequal proportions of patients unsubscribing from the study. “Randomization” is a procedure used to prevent this by assigning patients using a random mechanism – neither the patient nor the doctor would know the assignment.
“Control” and “randomization” together constitute an RCT. In 1995, statisticians Marvin Zelen and Lee-Jen Wei illustrated a clinical trial to evaluate the hypothesis that AZT antiretroviral therapy reduces the risk of mother-to-child transmission of HIV. A standard randomization scheme was used, resulting in 238 pregnant women receiving AZT and 238 receiving standard treatment (placebo). It is observed that 60 newborns were seropositive in the placebo group and 20 newborns were seropositive in the AZT group. Thus, the placebo failure rate was 60/238, while that of AZT was only 20/238, indicating that AZT was much more effective than placebo. Drawing such an inference, despite the heterogeneity among patients, was only possible through randomization. Randomization makes different treatment groups comparable and also helps to estimate the error associated with inference.
The first applications of RCTs mainly concerned the agricultural field. Sir Ronald Fisher himself was very reluctant to apply statistics to the social sciences, due to their “non-experimental” nature. RCT has grown in importance in clinical trials since the 1960s, so much so that all clinical trials today without RCT were considered almost useless.
Mark a change
Social scientists have slowly discovered that RCT is interesting, feasible and effective. But, in the process, the nature of the social sciences has slowly changed from “non-experimental” to “experimental”. Many interesting applications of RCTs took place in social policy making during the 1960s-90s, and the “randomists” took control of development economics since the mid-1990s. About 1000 RCTs were conducted by Professor Kremer, Professor Banerjee and Professor Duflo and their colleagues in 83 countries such as India, Kenya and Indonesia. These were to examine various dimensions of poverty, including microfinance, access to credit, behavior, health care, immunization programs and gender inequality. While Prof. Banerjee believes that RCTs “are the easiest and best way to assess the impact of a program”, Prof. Duflo refers to RCTs as “the tool of choice”.
Finland’s Basic Income Experiment (2017-18) garnered considerable international attention, where 2,000 unemployed Finns aged 25 to 58 were randomly selected across the country and received € 560 per months instead of basic unemployment benefits. The results of the first year data did not have a significant effect on the employment of the subjects, compared to the control group comprising individuals who were not selected for the experimental group. Essentially, it was also an RCT.
Critics of RCTs in Economic Experiments believe that to conduct RCTs the larger problem is cut into smaller pieces, and any dilution of the scientific method leaves the conclusions questionable. Economists such as Martin Ravallion, Dani Rodrik, William Easterly, and Angus Deaton are very critical of the use of RCTs in economic experiments.
Randomization in clinical trials has an added boost – it ensures that the assignment to a particular treatment remains unknown to the patient and the doctor. This type of “blindness” is central to the philosophy of clinical trials and helps reduce certain types of bias in the trial. It is believed that the “outcome” or “response to treatment” could be influenced if the patient and / or physician is aware of the treatment being given to the patient. However, this type of “blindness” is almost impossible to implement in economic experiments, as the participants would certainly know whether they are receiving financial aid or training. So the randomization should have much less impact. Often, economists miss such an important point.
However, unless randomization is performed, most standard statistical analyzes and inference procedures become meaningless. Previous social experiments lacked randomization and this could be one of the reasons statisticians like Sir Ronald Fisher were unwilling to use statistics in social experiments. Thus, “RCT or no RCT” may not simply be a political decision for the economy; it is the question of the paradigm shift. The “tool” comes with a lot of implied baggage. As randomization dominates development economics, implicitly, economic experiments are becoming more and more statistical. This is a philosophical aspect that economists must deal with.
Apparently, for now, many would agree with Harvard economist Lant Pritchett who criticizes RCTs on a number of points but still agrees that it “is superior to other valuation methods.” The debate would continue, as randomistas continue to gain momentum at this time.
Atanu Biswas is Professor of Statistics, Indian Statistical Institute, Kolkata