‘More Than Good Intentions’: a review

Two new books are just out from some of the key figures of “the new development economics” (Tyler Cowen’s label, not mine) – More Than Good Intentions: How a New Economics Is Helping to Solve Global Poverty by Dean Karlan and Jacob Appel, and Poor Economics: A Radical Rethinking of the Way to Fight Global Poverty by Esther Duflo and Abhijit Banerjee. I’m not a development economist, but I’m keen to discuss what this movement can contribute to the aid and development debates in general, and the field of water, sanitation and hygiene in particular.

First onto my Kindle: More Than Good Intentions. Karlan and Appel use insights from behavioural economics to show how poor people can be given a helping hand (or a ‘nudge’, following Thaler and Sunstein) in many aspects of their lives, especially microfinance, and to demonstrate how the impact of these interventions can be rigorously evaluated using randomised controlled trials (RCTs). The book is a showcase – or perhaps manifesto – for the work of Innovations for Poverty Action (IPA), the organisation founded by Karlan to put into practice and test out these ideas, in collaboration with a range of partners around the world. Some good examples, all starting from programmes in Kenya, include encouraging farmers to buy fertiliser ready for the next year’s harvest by offering them pre-paid vouchers just after they sell their current crop; working out what interventions are most cost-effective for improving school attendance (deworming is cheaper than giving free school uniforms to the poorest); and – obviously of great interest to me – increasing the number of families using chlorine to treat drinking water by placing simple chlorine dispensers next to water points.

The positives: More Than Good Intentions is a very readable introduction to the field of behavioural economics and RCTs in development, and the authors do a great job of conveying the excitement and potential of development research in general. The format of describing results from investigations by a variety of Karlan’s IPA collaborators interspersed with anecdotes from people Appel meets in his research works well (although the language has some overly cutesy lapses, such as “a great waterfall of pork rinds frozen in mid-cascade” in a small shop in the Philippines). Karlan is particularly strong on interrogating the details of the microfinance movement, and examining the existing ways in which the poor manage their finances (a strength shared with Portfolios of the Poor). I also liked the comparisons with ‘nudge’ ideas which originated to help Western consumers save better or keep their commitments, such as Save More Tomorrow and Stickk.com.

Where I found the book unsatisfying was in its lack of discussion about how to “bridge the gap”, as Karlan and Appel put it, between particular RCTs and wider applicable recommendations. As in any development intervention, context (see an excellent series of posts by Dave Algoso on Find What Works) is crucial; the claim of More Than Good Intentions is that enough RCTs plus an understanding of the underlying behavioural theory can show us what ideas can be successful across different contexts. The concluding chapter lists “seven ideas that work”; some of these are considered to have ‘proven impact’ by IPA’s Proven Impact Initiative while others are at the ‘promising’ stage. But I would have liked more detailed discussion from the authors on how we can draw these wider conclusions and what evidence base they think is needed.

More Than Good Intentions also shows some tantalising glimpses of the politics involved in implementing policy based on evidence from RCTs, but neglects the juicy details and what we could learn from these. For example, Karlan and Appel state that the Progresa programme in Mexico, which provided small cash grants to the poor dependent on their school attendance and health clinic visits, was originally controversial but later accepted, partly thanks to the results shown when it was subjected to an RCT. I would have liked to have seen this example used as the basis for more discussion in the book on the process of using RCT evidence to promote policy change.

So where do we go from here? Happily, More Than Good Intentions finishes with a good list of references, so I’m going to look in more detail at the work so far on chlorine dispensers and explore in a later blog post the wider possible relevance of both ‘nudges’ and RCTs for the water, sanitation and hygiene sector.

‘Children of the kitchen’: defining the household in Mali

Dean Karlan and Jacob Appel’s new book More Than Good Intentions – I’ll post a review soon – has a great example of how apparently straightforward survey questions can be anything but simple. Appel describes testing the wording of a questionnaire with a phone-card seller in Ghana, asking him how many people make up his household, or “share a single living space and take meals together”. As the conversation progresses and Appel probes deeper, the response changes from one person, to five, then eight, then seven, eventually settling at an extended family of (about) nine members.

Households in Mali are also often large, fluid and difficult to define. As in the Ghana example, definitions tend to be around those who share the provision of food. The Bambara word for ‘household members’ (guaden) even translates literally as ‘children of the kitchen’. However, this can still have different understandings for different people in different places, sometimes being taken to mean an extended family with a compound of many different buildings, sometimes interpreted as smaller sub-units of this. Instead of surveying simply by reading out exactly the same question to all respondents, my research assistant and I have decided that it is better to take the time to discuss and reach a common understanding at the start of each interview.

This matters not only for consistency within my own research sample, but also for making comparisons to other datasets. For example, I have used the questions from the Mali Poverty Scorecard to estimate the level of poverty of different households and villages – this is a simple set of ten indicators which correlate well with poverty levels (in terms of expenditure) based on data from the Mali Poverty Evaluation Survey (EMEP) of 2001. So to use this tool to help my research, the definition of the household that I use needs to match the definition from the EMEP.

Some researchers in Mali have gone further and investigated how changing the definition in the question changes the number of household members stated by the respondent. Beaman and Dillon used a randomised survey experiment to investigate the effect of using different household definitions in the Segou region of Mali. They found that adding more requirements to the definition given in the question – for example, requiring defined ‘household members’ to “work together on at least one agricultural plot or in one revenue generating activity” as well as living in the same lodging and recognising the same household head – actually increased the number of people listed as belonging to the household. This is counterintuitive because making the definition more strict should logically only result in a reduced number or the same number of defined household members. However, Beaman and Dillon suggest that perhaps the extra keywords in the stricter definitions prompted respondents to recall particular people that they might not otherwise have thought to include as household members.

This is important because these different definitions of household size can then in turn affect estimates of the poverty levels of households and subsequently wider regions, a good example of how key details of data collection can potentially have an impact on national statistics. For example, Delarue et al discuss in greater depth how data and methodology issues – including the definitions of ‘households’ used by surveyors in different regions – may be responsible for the “Sikasso paradox”, the phenomenon where official statistics suggest that cotton producers in the Sikasso region of Mali are poorer than other farmers, despite cotton-growing households and areas traditionally being thought of as richer.

As Chris Blattman puts it, getting inside the “sausage factory” of data collection is a humbling experience – and as these examples show, a very important one for understanding the possible effects of apparently small details.

The Venn diagrams of life and aid

Back to blogging after further community visits and interviews, and plenty more back-of-the-motorbike thinking time. Following my last post on why I am doing a PhD, I’ve been reflecting more widely on what I might want to do more long-term: some combination of what I enjoy on a day-to-day basis, what I’m good at / qualified for, and what I think is important for contributing to poverty alleviation. Then I found that the excellent la vidaid loca blog had not only got their first, but has articulated these sorts of personal ideas through various colourful Venn diagrams and then extrapolated to the whole field of international development. It’s brilliant, read the whole thing and see all the pretty pictures here.

Circles of life

I’m only slightly biased by the fact that the first two things la vidaid loca proposes that fit in the overlap of ‘what the West is good at’ and ‘what really needs doing to end poverty’ – ‘rights and economic freedom for women’ and ‘plumbing’ – are broadly what my girlfriend and I, respectively, mostly work on.

The circles of aid