Are Data Producers and Users Like Mars and Venus in the Office?

Save the Children
M&E has been at the core of Save the Children’s youth employment programs. We have staffed up M&E to produce great data. But are we using it to learn and improve programming? Sometimes it feels like data producers are like Mars and the intended data users are like Venus – Mars doesn’t communicate data enough and Venus doesn’t take action based on the data. The result is lots of missed opportunities to learn and continuously improve programs. 
As data availability increases, translating data into action becomes an important effort at all levels of our programs. Save the Children is thus investing in its frontline operations staff to better use data for smart decision-making and to create a culture of iterative adaptation.
At its Annual Youth Employment Learning Meeting, Save the Children introduced the Structured Experiential Learning (SEL) process, a simple and friendly learning framework to its country program staff working on youth employment. Well received by participants, we are now disseminating and refining the SEL process to drive more data-informed decisions.
Join us at our Making Cents webinar, “What’s all the talk about data-driven decision making? A practical guide to bridge data users and producers,” on Sept 22 at 8:30am EDT, where we will share our SEL resources and case studies and invite feedback from you!
 
Save the Children’s youth employment programs
Save the Children works with at-risk and deprived youth who lack market-relevant skills and job linkages they need to get decent jobs and build businesses. Our youth employment programs create pathways to employment for young people, by building a combination of a) market-relevant skills for the workplace; b) knowledge about the labor market, job search strategies, and their rights as workers; and c) networks and opportunities to strengthen and practice skills through mentorships, apprenticeships and peer to peer learning. Leveraging partnerships with the private and public sectors, the programs offer employability, entrepreneurship, and vocational training, along with job linkage opportunities to disadvantaged youth. Programs currently run in thirteen countries, including Bangladesh, Burkina Faso, China, Egypt, Ethiopia, Indonesia, Iraq, Malawi, Nepal, Philippines, South Africa, Uganda, and Vietnam.   
 
The Challenge: data collection, data quality and data use
At Save the Children, we aim to undertake rigorous research to continuously improve our youth employment programs. In 2014, we successfully completed a randomized controlled trial in Indonesia, which evaluated the training and employment effects of asking youth to cost-share a portion of their vocational training. 
 
We realized from this experience that program learning cannot solely depend on one-time, complex experimental studies that may take too long to design, implement and analyze. Therefore, while we will continue to pursue rigorous research, we will also rely on Structured Experiential Learning (SEL) -- data-driven feedback loops that build on routine monitoring data to rapidly identify problems and correct course if needed. We also learned that Program Managers and M&E staff need to work side-by-side to ensure their youth employment programs remain relevant and on target, despite unexpected hurdles encountered during implementation. 
 
Our efforts to date have mostly relied on strengthening data collection and data quality, and our next step is to develop systematic approaches to use data for smart decision-making.
 
The Goal: Using the Structured Experiential Learning framework for data-driven decisions
We aim to employ the SEL process to systematically use data for strategic decision-making. SEL relies on frequent data collection and reporting, testing, and short feedback loops to enable data-driven adaptations to program design and implementation. 
 
SEL was developed by Lant Pritchett, Salimah Samji, and Jeffrey Hammer at the Center for Global Development. It is also well aligned with the World Bank’s Adaptive Design framework, the problem-driven adaptation (PDIA) approach promoted by the Center for International Development at Harvard University, and the Center for Global Development’s principles of behavioral design. 
 
Our 4 step approach to using SEL: 
Step What it means Key Tasks
DEFINE: Identify problem Narrowly define a problem that is actionable and has programmatic relevance
DIAGNOSE: Gather data to identify root causes of the problem

- Gather internal and external data

- Transform data into information using visual graphics

- Brainstorm root causes with diverse stakeholders and program alumni

DESIGN: Craft potential solutions and pilot most promising

- Brainstorm potential solutions (ideally one for each root cause)

- Select 3 most relevant solutions and test them in time and area-bound pilots

TEST: Monitor key indicators and select best solution

- Identify key indicators to gauge effectiveness

- Track key indicators for each pilot and analyze results

- Select most effective solution

- Test against other potential solutions in the pipeline

- Select best solution

Can we put SEL to work on the ground? A first attempt
We introduced the SEL process to program staff from 6 countries at our Annual Learning Meeting held in Beijing last March. 
As a thought exercise, we presented to participants four conceivable challenges to their youth employment programs on a flipchart:
1. “A high number of youth are not completing the full employability skills training”
2. “Too few girls are enrolled in the vocational training sectors identified as priority in the labor market assessment” 
3. “NGO partners have few connections with employers” 
4. “Employers report low level of satisfaction with the performance of graduates of our program”
Then participants worked in small groups to figure out solutions using the SEL process. In the coming weeks, we will share blogs that describe in detail how these problems can be solved using the SEL process.
 
Our upcoming webinar, “What’s all the talk about data-driven decision making? A practical guide to bridge data users and producers,” on September 22 from 8:30am – 9:30 am EDT, will walk participants through the SEL methodology. Join us for step-by-step examples you can use to set up a culture of iterative adaptation in your own work! Register here.
Regions: 
Tags: