80% of all healthcare data is in an unstructured format, making it hard to access and harder to use. Unstructured data includes physician notes, radiology reports, and other human analysis. Physicians are writing for other physicians–healthcare is filled with abbreviations, jargon, and clinical terms that require medical knowledge and training to understand. These notes and related medical imagery hold all the nuance of a patient’s longitudinal journey and they require considerable pre-processing for use with analysis tools.
Extracting text from healthcare documents is the foundation of transforming unstructured documents into structured data. Structured data allows healthcare companies to make patient information actionable at scale.
The way the industry tackles this challenge today is through optical character recognition or optical character readers (OCR). OCR is the electronic or mechanical conversion of images of typed, handwritten or printed text into machine-encoded text.
Retina is Mendel’s OCR module, step one of the entire Mendel pipeline. Mendel’s modules are built specifically for healthcare, with powerful Artificial Intelligence that minimizes information loss, and traces results to source evidence.
Because Mendel is built by healthcare for healthcare, we understand that medical documents are dense. There are few filler words in medical documents, so an ideal OCR tool will be able to recognize and extract entire sentences–without loss of meaning. Mendel understands that accuracy is incredibly important. The mistakes and lost information that occur in the OCR step risk downstream data processing steps.
Retina is focused on providing the most accurate extracted output in the market. Unlike off-the-shelf OCRs, Retina:
We have built these capabilities in-house because we understand that all parts of the transformation process need to work together. Passing poor quality text riddled with spelling errors to the downstream tasks in the pipeline will confuse those NLP/NLU modules and result in poor quality output.
Let’s see how Retina’s output compares to a popular OCR tool in the market.
Original Document:
How is Mendel’s Retina able to capture more language with greater accuracy? Current OCR systems are based mainly on computer vision. They slice a character into several tiny slices and try to predict which character it is, then they aggregate the predictions into a character. Their understanding of what they’re reading doesn’t exceed a sequence of a few characters at a time; they do not interpret the words, phrases, or context they scan. As a result, if a word is unclear, they cannot recover it.
The Artificial Intelligence team at Mendel takes a different approach to all our modules. First, we ensure that everything we build is healthcare focused. Our models are trained on the largest medical data set in the market and combined with our own proprietary ontology. Medical Ontologies describe the medical terminology as concepts and define their hierarchy and how they relate to each other. Using these ontologies, along with our novel reasoning algorithms we developed, we attempt to imitate the understanding of a clinician. We want our tools to be able to understand, to approach problems with an ability to interpret and a level of contextual understanding. Because we are using multiple systems to balance decision making, Retina is less error prone than off the shelf OCRs.
Mendel tested Retina against the most popular OCR tool in the market. Not ones to shy away from a challenge, we looked at 120,000 pages of healthcare documents across all medical categories such as pathology reports, progress notes, and administrative documents. This was an unsupervised test, meaning we tested OCR output against OCR output with no human intervention.
Our goal for the evaluation was to show the level of accuracy for both OCR tools. To do this, we compared the number of instances Mendel and the competitor OCR were able to recognize five words in a row correctly (a.k.a. 5-gram). Words in a row matched correctly means less information loss. For this experiment, we used medical English vocabulary sourced from many public sources (pubmed, medical notes and medical ontologies) as a reference for correct words.
The Mendel OCR outperforms the competitor OCR at all word matching levels. However, the improvement increases when looking at the number of words matched in a row. For the 5-gram column, Mendel outperforms the competitor by 8.44%. Compared to the competitor OCR, the Mendel OCR is more fluent with medical documents.
The difference is even more stark when looking at the most difficult pages for both OCR tools. These are pages that may have noise, fading, concentrated ink stains, tables, or figures, they may be tilted or upside-down, for example. To identify those pages, we excluded pages where Retina and the competitor OCR performed similarly. The result is an evaluation set of the hardest 60,000 pages (50%) of the original set.
On the most difficult pages, Mendel OCR outperformed the competitor significantly. Mendel was able to recognize 5 words in a row with 13.48% more accurate recall than the competitor OCR.
Mendel’s Retina is more fluent with medical terminology and provides accurate results, even with the most difficult of documents.
Mendel’ Retina is part of an end-to-end solution that uses the power of a machine and the nuanced understanding of a clinician to structure unstructured patient data at scale.
Want to learn about Mendel’s process and modules? Contact hello@mendel.ai.
