5.3.3 Consider time and cost investments before beginning
Impact evaluations require a significant budget line. In general, the more rigorous the evaluation, the more it will cost. Cost-cutting may sacrifice the quality of the evaluation (though technology use may increase efficiency). For that reason, organizations should understand budgetary implications prior to planning a RCT. Experts report that RCT budgets can run from US$30,000 - $400,000. Budget is influenced by sample size, numbers of waves of data collection, logistical costs in-country, methods of data collection (electronic versus paper-based tools, length of tool, qualitative versus qualitative), and staffing. Data collection tends to be the most expensive part of the process, so an organization that already has data or a streamlined system can achieve significant cost benefits.
In addition to budget, RCTs also take time. The treatment group must receive the full intervention before end-line collection. And data collection, correction or “cleaning,” and analysis can take months depending on the staff structure and collection methods. An RCT involving cash transfers for youth vocational training in Uganda, highlighted in the following box, confirmed that youth were using funds for intended purposes. The initial phase of the study, from baseline survey to mid-term survey, took over two and a half years (although the process of soliciting applications and selecting groups took much longer) and will continue for months more. Not all organizations may be able to dedicate this amount to time to an RCT.
The Youth Opportunity Program of the Northern Uganda Social Action Fund (NUSAF) is a World Bank-funded project that aims to empower communities in Northern Uganda by enhancing their capacity to identify, prioritize, and plan for their needs and implement sustainable development initiatives that improve socio-economic services and opportunities. Through a partnership with the Ugandan government, it sought to provide vocational tools and training to eligible youth groups through cash transfers. The grants totaled approximately US$7,000 for each group, or US$377 per person. The average age of participants was 25 years old and the average education was up to the 8th grade.
In 2007, after a first round of funding, the government had sufficient funds remaining for 265 groups in ten districts. They decided to randomize at the group level; the government selected, screened, and approved 535 groups but only transferred funds to the 265 treatment groups. In 2008, they conducted a baseline survey with five people per group. Roughly two years after initial transfer, a mid-term survey found an effective attrition rate of 8 percent. Short-term outcomes were positive, reflecting improved employment, incomes, and high returns on investment. Funds were used for their intended purpose and there was no evidence of widespread leakage. For more information, see http://web.worldbank.org/external/projects/main?pagePK=104231&theSitePK=....