By: Kelsey Piper. —
In the last decade, organizations working to improve the lives of people in poor countries have increasingly embraced randomized trials to determine whether the interventions work. When it comes to helping the world’spoor, plenty of interventions — such as work training programs and distributions of free food or medication — have been found to have more impact compared with doing nothing at all.
But is “better than doing nothing at all” the right place to set the bar? “Off the back of RCTs [randomized controlled trials] getting very popular,” Joe Huston, managing director at GiveDirectly, told me, “we’ve realized that pure treatment versus control wasn’t always the right question.”
For the last several years, more and more experts have argued that we should be checking something else: whether a given intervention is more effective than just taking the money you would have spent on that intervention, dividing it up evenly, and giving it to the intended beneficiaries as cash. In other words, is the intervention you’re implementing actually better than just giving people money?
This is called “cash benchmarking.” The idea is that people often know what’s best for them, and by giving them money, they can spend it whatever way best meets their family’s needs. We should introduce other aid programs only when we can demonstrate that they do more good than cash itself. Sometimes they do; often they don’t.
On September 3, researchers Craig McIntosh at UC San Diego and Andrew Zeitlin at Georgetown University published a study using this approach to evaluate a US Agency for International Development (USAID) development program in Rwanda. McIntosh and Zeitlin have looked at this before: In 2018, they found a nutrition program in Rwanda didn’t deliver better results than just giving people an equivalently priced cash transfer.
This time, they collaborated with nonprofit Innovations for Poverty Action to study the Huguka Dukore/Akazi Kanoze program (which means “Get Trained and Let’s Work/Work Well Done”). Beginning in 2017, the Huguka Dukore/Akazi Kanoze program provided 40,000 young people in Rwanda with employment training, including 10 weeks of “workforce readiness preparation,” 10 weeks focused on “individual youth entrepreneurship,” and 10 weeks of technical training for a trade.
The question was:
Does doing all of that improve employment outcomes more than just giving people money?
Eighteen months after the program concluded (there’ll be another study at the three-year mark), the answer appears to be not really. The study found that participants in the work training program were not any likelier to be employed, nor did they have higher incomes or consumption. They did work slightly longer hours and have more savings, and they were happier than the control group and performed better on a test of their business knowledge.
What about the effects ofcash? The money was distributed by GiveDirectly, a nonprofit that has participated in tons of research on cash as a development intervention. One group got about as much money as the cost of the training program, a second group got significantly more, and a third group received both cash and training.
The researchers found that “the cost-equivalent cash grant performed significantly better than Huguka Dukore at increasing monthly income, productive assets, subjective well-being, beneficiary consumption, and household livestock wealth.” Increased business knowledge was the only outcome in which the job training program performed better than cash — however, it appears those higher test scores haven’t translated to more business success, because giving people cash did more for income and household wealth.
What about combining cash and training? It didn’t seem to have led to any synergistic benefits — if anything, the results seem smaller than what would be expected from just adding the results of cash and the results of training.
A courageous approach to global development
Two years ago, when USAID first released the results of benchmarking a study against cash, it made a stir. “The initiative has operated in stealth mode,” wrote the New York Times’s Marc Gunther, who noted that comparing USAID’s programs to cash “poses a threat to hundreds of for-profit companies and nonprofit groups that secure USAID contracts, often with scant evidence of impact.”
My colleague Dylan Matthews fretted, “Government agencies aren’t generally in the business of cooperating with studies that make their programs look bad,” so coverage of the nutrition program — and how it failed to beat cash — might actually create headwinds against such comparisons in the future.
But USAID has continued its work, at least with the programs that were already underway at the time, including Huguka Dukore/Akazi Kanoze. That’s the thing that makes this study so noteworthy, Huston said.
“This is one study out of literally hundreds on cash transfers,” Huston told me. “What’s exciting for us isnot so much that this is a study that shows you give people money and they buy things they need,” a fact already well established in the literature on cash transfers — but that “the biggest aid program in the world is starting to compare their programs to cash.”
Keep in mind that a cash benchmarking study that finds cash works better on most metrics isn’t a failure. It doesn’t mean USAID is wasting its money, as the training program did genuinely improve lives. It just means that poor people are pretty good at improving their own lives, and we should default there while we strive to find programs that do even better.
We should also keep in mind that we don’t have all the answers. At the 18-month mark, it looks like cash is better than business training on almost all metrics, but the program will be reevaluated again at the three-year-mark for a reason: Some interventions take time to pay dividends, especially educational interventions.
Trying to guess which development programs will work best is incredibly difficult. That’s why it’s so important to do a rigorous, systematic review of the evidence and act on what we learn. USAID is an agency with the potential to improve millions of lives around the world. Learning more about how it can do that effectively is good for USAID and good for the world, no matter what answers we find.
