Objectives
- Help Riders for Health reduce the time it takes for data on paper forms to be digitized and aggregated into their digital systems
- Explore using state-of-the-art optical character recognition (OCR) technology to digitize data collected on standard Ministry of Health forms that are filled in by hand
Findings
- Prototyped an OCR-assisted tool that recognizes specific regions on a health form from which handwritten data needs to be extracted and predicts the handwritten entries
- Created an end-to-end toolchain that’s flexible enough to use with existing proprietary tools like Google Vision API, Microsoft’s Azure Form Recognizer, or custom models
- Determined that the OCR tools are best coupled with a manual review step to ensure accurate information and to collect properly labeled data for further modeling
- Demonstrated that by using computer vision techniques and machine learning, the time currently taken to digitize paper-based records can be reduced from nearly 60 days to under a day!
Question
Innovations in digitizing written records are critical to the digital health landscape. An essential but tedious task for most frontline health workers is the collection of field data and recent estimates suggest that only ~20% of community health workers globally generate some form of digital data. Most frontline health organizations rely on paper based record-keeping – and that makes sense: it is a trusted, well understood and low-cost medium. But there are huge disadvantages to this — it can take weeks or even months for data on paper to be digitized and this is the primary method for data aggregation and data use across most health systems. Delays in digitization and aggregation of these data for analysis, monitoring & evaluation or to make rapid, information-driven decisions about services and programs can be challenging.
This is something that Riders for Health (Riders) knows well. Riders is an international nonprofit organization that exists to create access to essential healthcare for millions of people that live in some of the hardest-to-reach regions of Africa. With their fleet of motorcycles and specially trained riders and all-terrain drivers, the organization supports Ministries of Health in service delivery in five African countries and reaches nearly 47 million people.
A critical Riders operation is the transport of medical samples for infectious disease diagnosis (mainly tuberculosis, HIV and more recently, for COVID-19) from rural health centers to laboratories where the samples are processed, tested and diagnosed. To maintain the quality and integrity of samples and results during collection, transportation, storage, and analysis, the Riders’ couriers capture details in handwritten logbooks. This laborious process involves the couriers having to manually copy certain important pieces of information (e.g. patient ids, sample types, dates) about their transported samples/results into their logbooks. In some locations, this process generates 1500+ paper-based records everyday. Riders have identified that this time-consuming process limits the volume and frequency of visits couriers can make to health facilities, leads to the degradation of samples, causes delays in diagnosis and therefore the time it takes for patients to get onto essential treatment. On the operations side, couriers struggle with the hassle of the weight of this precious cargo, and once delivered, data entry teams can take up to 60 days to digitally encode the data to be used for anomaly detection, quality control and ultimately for making data-driven program improvements.
Riders wanted to explore ways to ease their couriers’ burdens while speeding up the process of digitizing the data from handwritten forms – ideally in near-real-time! To address this challenge, DataKind Ⓡ developed “ROCR”: a prototype optical character recognition assisted form-capture tool that extracts and predicts key handwritten fields in the standard sample-transport forms used by Riders’ operations in Nigeria – and records them in a digital logbook.
.png)
Riders Team workflow to collect samples and process forms
Sample transport courier Thamae Thamae using mobile data collection at the National Reference Laboratory in Maseru, Lesotho (Credit: Riders for Health, photography by Tom Oldham)
What Happened
Led by Data Ambassador Anna Dixon, the DataKind team of volunteers included Project Manager Amy Roberts and expert Data Scientists: Alexander Sack, Karry Lu and Alex Fried. This team worked very closely with Riders’ Head of Digital Transformation, Simon Devine and the Nigeria-based operations team – Ache Atabo and Anthony Olagunju to identify and plan when and where an OCR-assisted tool could be suitably deployed for aiding paper-to-digital data transfer. Together, the DataCorps team and Riders’ subject matter experts also mapped what data is collected and how, when and where the couriers collect that data. The team evaluated Riders’ record-keeping needs to identify data that could just as easily or preferably be collected by other means (e.g. GPS coordinates and timestamps from a mobile device or app-based human entry) and the data that would be most-suitable for OCR on the forms used. This approach allowed the team to establish which key fields on which forms would be the ROCR tool’s objectives.
Once the team had a test form and fields to predict, building an OCR-assisted form capture tool was a challenging task! Several design questions had to be answered – for instance:
- How to consistently find the key field despite the inherent messiness and variability of images (e.g. skew, resolution, lighting)?
- What handwriting prediction model works best here?
- Was there a way to use knowledge of the form and data collection process to improve the OCR predictions?
In the end, the team built a flexible tool with a modular design that uses computer vision techniques to align images, prepare them for analysis and perform OCR predictions using existing tools – Google Cloud Vision and Microsoft’s Azure Form Recognizer – automatically converting handwritten fields to their digital representation. ROCR also applies several new post-processing techniques to improve performance. One example of a ROCR post-processing method is applying computational linguistics to measure the difference between the output text to a list of known possible values to improve the handwriting prediction results.
OCR-assisted key-field extraction and prediction pipeline
For Riders, the repeated manual copying of data from forms was prone to error. Given that the objective of ROCR is accelerating the transcription of highly important medical information, there is still a need to ensure near perfect accuracy. As a result, the team concluded that while OCR models and tools have improved greatly in recent years, they are still best coupled with a manual review step. This will ensure the paper-to-digital data transfer is accurate and will help to collect properly labeled data for further modeling. ROCR is designed such that if a custom (open-source) model with the necessary high accuracy could be developed, it would be easy to implement instead of using existing proprietary tools.
By developing the ROCR prototype to automatically convert handwritten fields on forms to their digital representation, the team was also able to demonstrate that the tool can scan and store a file that would have taken days in mere minutes. Furthermore, through a series of process-mapping exercises with the experts on the ground, the team was able to demonstrate that the time currently taken to digitize paper-based records can be reduced from nearly 60 days to under a day! This means couriers could spend less time on data entry and more time transporting patient samples to labs, meaning more patients could be diagnosed earlier and begin treatments sooner.
What’s Next
The state of the art for handwriting analysis is still developing and Riders and DataKind partnered to increase the accessibility and usability of this technology to frontline health organizations. ROCR in its current form is well-suited for OCR assisted form capture for repetitive, data entry intensive forms. It can already be easily adapted to a range of new forms that are widespread or standardized in international, national, and subnational health systems.
With this project completed, DataKind and Riders anticipate moving to the next phase which is to develop a web interface to make it user-friendly for end-users and create avenues to improve the tool’s performance. ROCR’s modular nature enables developers to easily expand and modify it for better customization and better handwriting prediction accuracy. Presuming a successful second stage, Riders will test this tool in a pilot area to capture performance metrics, protocols for training field teams and to gather evidence needed to scale this solution. Riders’ goal is to deploy ROCR across its operations in Nigeria where its sample transport couriers serve over 20 million people, then scale to the other Riders operating countries of Lesotho, Malawi, and The Gambia afterwards.
.jpg)
Ministry of Health staff at the Mapoteng Laboratory, Berea District, Lesotho, with Riders for Health sample transport couriers (Credit: Riders for Health, photography by Tom Oldham)
