9.1 Utilizing a Mixed Methods Approach will Result in a more Comprehensive and Accurate Evaluation of a Program

Too often, monitoring, evaluation and impact evaluation draw from either quantitative or qualitative data gathered through a limited number of tools. While both types of data are important, they might only tell part of a program’s story. A single method of data collection might miss other factors that influence results. A mixed methods approach to M&E relies on multiple data collection methods to provide a more comprehensive analysis of program results. For example, the approach can capture both economic and social change, such as whether a young entrepreneur’s business is growing and also whether the young entrepreneur has gained new skills or confidence. YouthSave, a consortium funded by The MasterCard Foundation and led by Save the Children with the Center for Social Development at Washington University, the New America Foundation, and the Consultative Group to Assist the Poor, investigates savings accounts as a tool for youth development and financial inclusion in Ghana, Colombia, Nepal and Kenya. The consortium also utilizes a mixed method learning agenda to better understand how financial capability projects change behavior and improve financial outcomes. See Box 9.3.1 for more information about YouthSave.

9.1.1 Practical Tips: MEDA Describes Mixed Methods

The Mennonite Economic Development Associates (MEDA), an association of compassionate business women and men who invest in the lives of families living in poverty around the world, utilizes the following tools to employ a mixed methods approach to monitoring and evaluation.

  • Select SMART indicators (Box 9.2.1).
  • Start with a baseline. A registration baseline would include population-level demographic data. A sample baseline would mirror follow-up survey.
  • Return with an interim or mid-term evaluation is the bare minimum and allows for course corrections mid-way through the program. The MasterCard Foundation mentioned that they see this of critical importance in a partner, the ability to improve the program based on data.
  • Annual surveys ask youth how the program has impacted their life in terms of income, assets, food security, financial products and services, or other crosscutting elements.
  • Qualitative data follows-up with youth to determine, in their own words, the answer to the following question: “What is the most significant change you have experienced because of project interventions?
  • Meta-analysis of quantitative and qualitative impact.

For more information, see www.meda.org/web.