“Dangerous Liaisons”: On the ambiguous relationship between development and development economics
By Lucia Rizzica*
Development economics is a very peculiar field of science in that it is scientific research but it also entails an intrinsic altruistic motivation: the goal is to help the poorest, the tools are those of economic science. But the combination of these two characteristics is not an easy one.
A clear example of this ambiguity is the debate about the employment of Randomized Control Trials (RCTs) in developing countries. The underlying intuition is very simple and thus appealing: you take a homogenous group of people, split it into two random subgroups, then treat one and only one of the subgroups in a certain way and afterwards look at the average difference between the two subgroups and interpret that as the average effect of the treatment.
But the use of RCTs does not meet unanimous consensus and their recent impressive success is looked at with scepticism by many economists and development advocates; why is it that the use of an economic technique generates so much discussion?
Last June I attended the presentation of the new book by Esther Duflo and Abhijit Banerjee, “Poor Economics: a radical rethinking of the way to fight poverty” at the London School of Economics1; the crowd that welcomed them was impressive, but why were all those students, academics, policy makers, NGO workers, World Bank representatives and simply curious people so eager to listen to this presentation? In the end it is just another economics book…
The answer to these questions is just rooted in the two-faced character of development economics I referred to above: RCTs is not just an economic technique, and the book by Banerjee and Duflo is not just another economics book, the reasons being that the stakes are the lives of millions of people, that how to save them is of interest not just to economists but to everyone, and that if economics can indicate the way out of poverty, well, then everyone wants to know it.
The two MIT researchers are today undisputed leaders in the field of development economics, founders of the Abdul Latif Jameel Poverty Action Lab (J-PAL), a network of economists around the world “who are united by their use of Randomized Control Trials (RCTs) to answer questions critical to poverty alleviation”2. Their book is a broad discussion of the main themes in development economics supported by evidence provided by various researchers through the means of RCTs.
The whole point of their book, of the presentation at LSE, and of the years Banerjee and Duflo have spent conducting research in poverty alleviation is to promote the use of RCTs as the “gold standard” of development economics; it is only through RCTs, according to Duflo, that economists and policy makers can learn what works and what does not work for development and consequently target their actions.
With such an ambitious objective and a simple recipe for it, their success and the debate about RCTs come as no surprise.
Certainly the enthusiasm about this new paradigm finds its roots in the widespread frustration about aid effectiveness3 and about the role of the World Bank, often criticized for not being able to learn from its projects and thus to design a coherent and comprehensive strategy for development. And it is exactly on this weakness that the supporters of RCTs insist to promote the use of the new methodology, an editorial of the Lancet of 2004 about the adoption of RCTs for the evaluation of World Bank projects was indeed titled “The World Bank is finally embracing science”. And during the presentation of the book Duflo insisted on the idea that the introduction of RCTs in social sciences will prompt a revolution of the discipline as much as did their introduction in medical sciences in the 20th century.
On the other hand many economists and non-economists remain doubtful about the effectiveness of RCTs, for both technical reasons and broader policy concerns.
A competent and truly comprehensive discussion of the technical limits of RCTs can be found in Deaton (2009) or Heckman (1991)4. A key point that is stressed in such articles is that RCTs are far from being universally informative about what works and what does not work in development; at best, assuming all the stringent assumptions required are satisfied, they might tell us what worked in a certain time and geographical context; whether the same would happen in another place or time is not inferable and neither is what would happen in the same context when the experiment is scaled up to cover a larger population. So how can we base policy actions on the results of these small, even if well designed and carefully implemented, RCTs?5
The difficulty of applying RCTs to human beings rather than to cells, for which they were originally designed, brings us back to the distinction between social and natural sciences. While nature typically follows some universal rules, human beings react in ways that are much more difficult to predict and that change depending on the environment in which the individual acts. Therefore researchers struggle to be able to isolate the “pure” effect of the intervention from those generated by the context in which this takes place. The great advantage of RCTs is that the post-intervention difference between the two groups can easily be attributed to the treatment because individuals do not differ from each other in any dimension other than their assignment to the treatment or control group. While this is certainly an advantage of RCTs one might wonder what is the interest of knowing the effect of a treatment as if the treated individuals were closed in a laboratory? They are not, and the real effect actually depends on the context they are living in.
This is not to say that RCTs are uninformative nor useless, but that they should probably be systematically backed up by some solid economic theory which would allow the researcher to predict how the treatment effect would manifest in different contexts; otherwise predicating the application of “RCTs’ truths” to alleviate poverty is just as dangerous as a completely blind distribution of aid funds to developing countries. Unfortunately only few papers present this type of structure and most of them just focus on the results of some well designed, original experiment which will teach us something unexpected.
Let me give you an example of where RCTs economics and theory should meet: there exists solid economic research that focuses on the role of cultural identity and transmission mechanisms; this has provided comprehensive theory and robust empirical evidence that culture and the way this is transmitted over generations play a crucial role in determining the main traits of individual preferences such as discounting, risk aversion and altruism6. This implies that the effects of any particular policy intervention will be conveyed by such mechanisms and its final impact will hinge on the particular cultural traits of the individual treated. Integrating such theories in the design and scaling up of RCTs could provide a more solid basis for policy making, allowing economists to make predictions about the “broader” effects of policy interventions.
A further concern I have is that there are just “too many” experiments being carried out in developing countries. Duflo and Banerjee report in their book that between 2003, year of the foundation of J-PAL, and 2010 the researchers affiliated to this network (just them) “had completed or were engaged in over 240 experiments in 40 countries around the world”. Have we really learnt 240 golden rules for development? Duflo claims that RCTs provide small true facts that can help us slowly recover the recipe for development, but how many ingredients are there in this recipe?
Let’s assume now that some results coming from RCTs truly gave us a clear indication of what to do: are these rules ever implemented world wide or even at a just larger scale? And if they are, would it not be interesting to know what their actual impact eventually was? The J-Pal website currently reports that only four projects have been scaled up7.
Finally I believe that what should really be kept in mind is that randomizations come with a very high ethical cost: some people are randomly chosen not to receive a treatment which is supposed to be beneficial to them. This, in simple words, means children not being immunized, women not being given contraceptives to prevent HIV contagion, men not being given loans to start or sustain an economic activity. If such experiments are not followed by a scaling up of the most successful treatment then they are not only useless but harmful.
* Lucia Rizzica is a PhD candidate at the University College London, UK. Her current research focuses on Economics of Migration and Brain Drain.
1 Banerjee and Duflo’s talk at LSE can be found at: http://www2.lse.ac.uk/newsAndMedia/videoAndAudio/channels/publicLecturesAndEvents/player.aspx?id=1028
2 Quote from http://www.povertyactionlab.org/about-j-pal
3 See on this the first post in this blog by Diego Angemi and Luciano Ciravegna
4Angus S. Deaton, 2009.”Instruments of development: Randomization in the tropics, and the search for the elusive keys to economic development,” NBER Working Papers 14690, National Bureau of Economic Research, Inc.
James J. Heckman, 1991. “Randomization and Social Policy Evaluation,” NBER Technical Working Papers 0107, National Bureau of Economic Research, Inc.
5 Other threats to the validity of the results of RCTs then come from issues of heterogeneity, non compliance and small sample size. See Deaton (2009) for a review.
6 For a complete review of this literature see Alberto Bisin & Thierry Verdier, 2010. “The Economics of Cultural Transmission and Socialization,” NBER Working Papers 16512, National Bureau of Economic Research, Inc.
7 Some projects have been implemented in more than one location for a total of eight scale-ups
Comments
One Comment on “Dangerous Liaisons”: On the ambiguous relationship between development and development economics
-
mariapia on
Sat, 14th Jan 2012 18:24
I really appreciated this article, especially as it comes from a Ph.D student. I’ve seen quite a few students of development economics leaving for a poor country eager to implement a field experiment, presenting thereafter the use of that technique as the main contribution of their research, i.e. as a substitute for substance and thought (those in bad faith would say “they were not good students”, but it is not so easy..). That is pretty depressing because it has to do with the scientific progress on the one hand, and with the lives of many people on the other. It is hard to say which method is better to ultimately fight poverty (“there is no gold standard”), but I just wish all young researchers (and good scientists) had the same dose of critical thinking as Lucia.
You must be logged in to post a comment.