Art of the Possible #1: M&E Technology and Design Thinking

Chris Wood Associates

This is the first entry from Chris Wood in the "Art of the Possible: Technology and Tools for Improved Monitoring and Evaluation" blog series, which covers design, data collection, databases, analysis, and using your results. All articles cover the use of technology, the options available and how to decide on the best approach for your projects.

Art of the Possible #1: M&E Technology and Design Thinking

To design the right M&E system, you have to think backwards! Let me explain by using a simple project as an example. An example project I will use throughout these articles can be described as

  • A 5-year program providing soft-skills training to young people who come from different cities
  • A series of 6 classes cover a range of different employability skills
  • Five hundred young people enrolled during the summer months

The results chain looks like this:

 

Your M&E system needs to answer:

  • Does the program reach the desired number of students?
  • Does the program change attitudes towards work in the disadvantaged students?
  • Do any changes in attitudes towards work last at least 6 months after training ends?
  • Is there a difference depending on the city in which the students live?

We need to work “backwards” by considering the specific questions we want to answer, detailing the data we will need in order to answer these questions and then defining how we will collect that data. By adding a little more detail on to each question, we can generate a table of requirements as in Table 1 below:

Question

Information

Data to collect

The number of girls who attended all training sessions in March?

 

  • Students: gender, name
  • Training: Attendees, Dates, Course name
  • Student information
  • Training class information
  • Attendance register

The increase in motivation of 18-24 year old students from Al Wafrah to seek work

 

  • Students: Age, City, Student ID
  • Survey: Answers, Date, Student ID

Employment Motivation survey

  • At start
  • At end
  • 6 months after end

 

Note that we do NOT need to collect the student nationality or academic status, the duration of the training, the student preferred job or where the survey was completed. This information does not contribute towards answering our questions - therefore do not spend time and effort collecting it.

The flow of design thinking is the opposite to the direction of flow of the data when you implement the system - hence thinking “backwards”. In operation, an M&E system will collect data, store and manage that data, and then produce reports through the analysis of that data (collect, manage, analyse). However to design a system, you need to consider the analysis, and then the data management and finally the collection.

 

You also need to be careful about not trying to answer questions about things that are not under the influence of your project. For instance, our example project delivers training sessions and aims to raise the motivation of our students. We should not attempt to prove the impact of our project by counting the number of students that get a job. That is not what our project does. It might be a side effect, but the results chain gets very tenuous here and will be difficult to “prove”.

If we can collect, manage and analyze the data which we have defined using this process, we should be able to answer the questions and provide some additional comparison analyses.

Question

Analysis

Result

The number of girls who attended all training sessions in March?

 

Select all students

  • Filter by gender
  • Group by month

The increase in motivation of 18-24 year old students from Al Wafrah to seek work

 

Show the difference in motivation ‘scores’

  • Filter by age
  • Filter by city 

Summary

We have explored how to design an M&E system by “working backwards”. Don’t start at the data collection “end” by defining surveys as this will lead to poor design and not being able to answer the key stakeholder questions.

  • The first step must always be to determine the questions you need to answer in your reports.
  • Then you can determine….
  • The information you need to store and manage to answer the questions, and
  • The data you need to collect

Further steps need to consider:

  • When during the delivery process can you collect the data
  • How to collect, store and manage the data
  • Who will be responsible for collecting and managing the data

For more information, this video covers the topics discussed in this article. 

goProve (www.goprove.org) is a cloud-based end-to-end solution providing data collection, data storage, and management as well as built-in analysis and reporting tools. 

Regions: 
Tags: