Art of the Possible #3: Web-based Surveys
This is the third entry from Chris Wood in the "Art of the Possible: Technology and Tools for Improved Monitoring and Evaluation" blog series, which covers design, data collection, databases, analysis, and using your results. All articles cover the use of technology, the options available and how to decide on the best approach for your projects. This article explores how web-based survey tools can be used in your M&E system.
An easy way to collect data is to use one of the many web-based survey tools. These include SurveyMonkey and SurveyGizmo, amongst others. They can be free to use, allow you to send invitations to your beneficiaries and report on the responses.
Let’s put web-based surveys into the youth development context. In the Design Thinking article, I introduced an example project which requires us to measure the change in motivation of our students. To do this, we will:
- Invite all students to complete the employment motivation survey online
- Run the survey 3 times to measure the change in motivation
The basic steps we take for web-based surveying are:
- Create a database of all students we wish to survey - including their email addresses
- Upload this database to our web-based survey account
- Develop the survey questions
- Compose the invitation email and send the invitations
- Once we have some responses, we can then export the data collected by the survey.
This is great - and very easy to do. However.....
We then want to analyse the exported data and disaggregate by several other factors we have already collected about the student such as gender and education level. To be able to do this analysis, we need to reintegrate the survey data with our existing database - linking the records using a unique value such as the email address. This is not a simple task and may take your Excel expert some time to match all of the records together.
For our example project, we need to ask the students to complete the survey several times: at the start of the project (before), immediately after the project (after) and then three months later to analyse the change in their motivation. A single survey will just give us a snapshot in time. To do this using the web-survey solution appears to be simple once again. Upload the student data with their emails, invite them to complete and then export the data set. Again, your Excel expert will need to allocate some time to match the responses to the original student database before you can do any of the analyses you need. It can be done, but you do need to be aware of the technical challenges you will face once you have collected data.
- Generally faster to implement, simpler to operate and cheaper than other survey methods
- Many free providers, although you may need to pay to get the features you need
- Make sure your students have internet access. If not, they cannot respond.
- Longitudinal surveys difficult to link and manage the exported data. This can be done but is a manual task and it takes the time of an Excel expert.
- Typically low response rates. May need to consider incentives for completing the survey
Keep in mind the following when considering web-based data collection:
- Do a cost-benefit analysis to make sure that this approach is appropriate for your situation. If you have less than 100 students to survey, is it worth it?
- Keep your surveys short and precise. Students will leave if the survey is too long and questions difficult to understand.
- For more thorough answers and willingness to complete the follow up survey, explain the purpose of the survey upfront and thank the participants at the end
- Test the survey on colleagues and others who may be unaware of the context. This will help identify any misleading or vague questions.
- Make sure that all test data collected on the survey is deleted before inviting students.
- Add your logo to all of the survey pages and use a colour theme which matches your web site or other materials to give an identity which will be recognised by the student.
goProve (www.goprove.org )is a M&E solution which integrates web-based surveys and automatically links responses to beneficiary records.