Insights from the Kano State Primary Health Care Monitoring and Evaluation systems assessment
Are you planning a data collection exercise? If yes, you will be best served with some of our field experience implementing various data collection activities. eHealth Africa has more than a decade of experience collecting large-scale data, including qualitative and quantitative data. These span geographic information system data, vaccination and vaccinator tracking data, implementation of health systems improvement, and reproductive health services surveys. At every point in a calendar year, eHealth Africa team members are in a remote community interacting with locals and trying to understand the reason for some challenges preventing effective public healthcare service delivery.
We recently concluded data collection efforts in Kano state to assess the challenges to data use in decision making within the monitoring and evaluation (M&E) framework of the Kano State Primary Healthcare Management Board (KSPHCMB). Leveraging support from Technical Advice Connect (TAConnect), eHA designed a mixed study to help us identify the quality of data, their collection process, and how best to encourage empirical decision making and improve the quality of healthcare services delivery. From a sampling population of over 1000 persons within the state primary healthcare (PHC) M&E system, including data generators and data users, we sampled 596 respondents for our quantitative questionnaire and 21 respondents for our qualitative tool. Their responses are now guiding our analysis and findings. Here are five big lessons we learned while delivering this effort.
1. Stakeholder engagement is the key to success , and no stakeholder is more important than the other. Any person’s response could be the insight that unlocks the issues you are trying to solve. But they can make or break your ability to reach all your respondents and access all the communities from where you require information. Our approach was first to map out all the stakeholders and their interests in the project. Next, we agreed on a means of communication and what information was important to them before we reached out. Adequate and open communication is the key to successful stakeholder engagement. We ensure we address all their concerns, make them a part of the project, and, where permissible, include them in helping you to get access to the communities you need to study. Ensure to share your collection tools with stakeholders for their input where necessary. Overall, mainstream stakeholder engagement throughout your collection phases if you want to be successful in data collection.
2. Failing to plan is planning to fail. A field plan helps you understand how much time you need to start and end every data collection effort. Because we have a lot of experience implementing data collection, we can estimate the time required to conclude an exercise accurately. To do this, we establish certain parameters such as the number of data collectors available, how many questionnaires are to be administered, the coverage area, and how long it will take to administer a questionnaire to one respondent. With these figures, draw up a field collection plan to estimate the quantity of data one enumerator can collect in a day. This information is vital if you plan to pay data collectors based on performance or measure their effectiveness. Ensure to include a couple more days for mop-up and recollection. This will help address unforeseen delays and disruptions. eHA has designed a tool, Planfeld, that automates planning for field logistics in public health. Planfeld improves efficiency, reduces your turnaround time, and saves valuable resources. It ensures you do not miss any planned collection location since it allows you to input your planned coverage areas. Planfeld uses the data portal, published by eHA, with over 350,000 points of interest and more than 451,000 settlements across Nigeria and it is interoperable with any geodatabase
3. Test your tools. Our best practice at eHA is finding an equivalent to the sample population outside the study area and administering the proposed instrument. In this study, we leveraged the Jigawa State primary healthcare management officials to pilot our tools. The essence of this exercise is to give us real-time information on the issues we could encounter in the field and plan for them. Field testing will also highlight any problem with your survey tools and allow you to correct such problems before you begin data collection. For example, in the Kano State M&E assessment, we discovered challenges regarding the page-to-page transition. We spent the next couple of days reviewing the open data kit forms. We resolved this issue before commencing data collection in the field. Pilot testing is also the platform to test to see if your collection estimates and timelines are realistic. It is best practice to use pilot testing to simulate if your collection plan is workable.
4. Establish and implement quality checks. For example, collection teams must record the geo-coordinates of the collection locations. It is essential to check the time to complete a single form. These are some ideas that could signal the quality of data collection—for instance, spending five minutes on a form that should take 20 minutes to complete signals that an enumerator is doing something wrong. In a GIS collection project, an enumerator collected several points from one location. Our quality checking standards flagged this, and we immediately rectified it. Quality checks ensure you do not return to the field to implement recollection when you have finished data collection because of quality issues.
5. Engage and train experienced data collectors. Over the years, eHA has built up a cadre of enumerators who understand the job and our quality standards. This lessens the time we spend training them. It has also helped us to reduce field errors and ensure the correct information is collected. Pre-collection training is still important, though, and it is an opportunity to introduce new tools, collection modalities, and quality standards to your enumerators. Training also allows you to address respondents' psychography, social and cultural norms. For instance, do not send male enumerators to interview female respondents in a conservative society. If this must happen, it must be in public and under the supervision of another adult.
An assessment is only as good as the data supporting it. If you collect poor-quality information, the analysis will be flawed. Thus, it is vital to align some of your collection approaches, like the outline above.