Original Article Source: Author Brian Benoit
The life sciences industry is famous for capital-intensive research and medical devices which have advanced the practices of medical imaging, sample testing, and drug manufacturing. These devices have machine vision capabilities integrated into their design.
Yet for certain lab automation applications, machine vision can’t sufficiently match the flexibility of the human mind to make judgement-based decisions. Computers are famously confused by busy backgrounds and image quality issues, such as specular glare. This makes it incredibly difficult for traditional machine vision algorithms to locate an object or region of interest with precision, especially to identify abnormalities amidst an unstructured scene. It can be time consuming and difficult, if not impossible, for automated systems to successfully identify regions of interest while ignoring irrelevant features.
Today, however, breakthroughs in deep learning-based image analysis can automate these applications so that they are performed reliably and repeatedly—in machine vision parlance, “robustly.”
Life Sciences Defect Detection
Clinical and research microscopy applications that previously required human inspection are being reinvented with the application of deep learning-based image analysis. Pathological and histological samples, for example, require accurate defect detection and segmentation despite defects’ variable and unpredictable patterns.
When you consider the challenge of detecting cell abnormalities and cell damage on a histologic (cell tissue) slide, the potential visual appearances are mind-boggling.
A cancerous cell could appear in a number of sizes and shapes, and its various forms are, in most cases, more different than they are similar. It’s effectively impossible to teach an inspection system to identify all possible anomalies without extensive programming, and even then, the possibility of false identification or rejection is high. In a situation like this, deep learning-based image analysis in unsupervised mode offers a highly accurate and efficient mode of inspection.
In our cell abnormality detection application, a training engineer uses sample images of possible cell abnormalities, like cancer, to teach the software to conceptualize and generalize the normal appearance of a cell or cell clusters. These slides are labeled as “good” examples of healthy cells and take into consideration normal healthy cell variants, like mitosis. Then, during runtime, any variations are flagged as anomalous and likely exemplifying cell damage. This application requires one further step.
Once a cell or cell cluster is flagged, the particular region of interest needs to be dynamically segmented in real-time for further review. The cell exhibits potential damage, after all, because its appearance strays from the norm, but it is not necessarily cancerous. These deviations could be caused by artifacts on the slide.
Normally, a human inspector—likely a pathologist—would have to review this subset of samples to make a firm diagnosis. But again, Cognex’s deep learning-based software can re-run its algorithm over the subset target zones—this time with retraining in supervised mode–to parse between “good” (tolerable, non-damaged) and “bad” (pathological, damaged) cells.
Life Sciences Optical Character Recognition
Many medical suppliers rely on automatic identification for traceability and to meet safety regulations. Human-readable alphanumeric characters can easily present as deformed to the camera of an automated inspection system if it is present on stretchable, moldable material like an IV bag. Specular glare and reflection can also confuse the system, obscuring and changing the code’s natural appearance.
Even without these visual variations, it can still be immensely time-intensive to teach a vision system to recognize different fonts, such as in the case of optical character verification (OCV), when the inspection system can’t anticipate what font style it will encounter. This is where a pre-trained, omni-font library can come in handy. A deep learning-based tool that it pretrained to recognize various fonts essentially works out-of-the-box; there is no upfront image-based training required, and the minimal training that does occur only happens on missed characters to refine the model’s logic.
Fast, easy implementation and limited application adjustments make deep learning-based OCR an obvious choice for applications involving deformed, skewed, and poorly etched characters or in verification applications when the camera is sure to encounter a wide range of unknown fonts.
Life Sciences Assembly Verification
Lab automation devices such as clinical analyzers and in-vitro diagnostic devices rely on machine vision to ensure that samples are perfectly inserted and aligned for optimal testing conditions. Diagnostic device manufacturers’ success relies on the accuracy of their machines’ measurements and results. Perhaps most importantly, they rely on accurate test set-ups and deck assemblage, which provide the device with precise data so that the tests are performed correctly and uniformly.
The correct assemblage of testing samples—blood, urine, or tissue—in what’s known as a pre-assembly verification is essential to reduce any potential errors which could threaten contamination, mix up or mislabel diagnoses, or slow down or break expensive equipment. During these inspections, the automated system must verify that there are no misaligned or absent test tubes, caps that haven’t been removed, or extraneous vessels loaded into the analyzer’s rack. Verifying that the equipment’s rack has been populated completely and correctly involves managing several factors: sample and reagent tubes and vessels vary by manufacturer in shape, size, and dimension, and it can be impossible for the machine to predict the position of samples on the deck.
With these unpredictable variations in test set-ups, it makes sense to use deep learning to perform assembly verification. Cognex deep learning-based software can learn the varying appearance of different samples and reagents, as well as their unpredictable and varying locations, based on a set of training images.
The tool generalizes the distinguishing features of the samples and reagents based on their size, shape, and surface features and learns their normal appearance, as well as their general location on the deck’s racks or microplates. In this way, deep learning is able to automate and solve a previously hard-to-program application in a quick, highly accurate, and easy-to-deploy manner.
Life Sciences Classification
Ascertaining the quality of a blood sample still requires a significant amount of human judgment. This is because a properly prepared sample which has been centrifuged and indexed needs to receive individual scores for turbidity and plasma color. Based on how the samples are loaded into the analyzer machine, their appearances can vary and blood can appear relatively more or less separated. This affects indexing.
For example, a sample with more clearly stratified plasma, buffy coat, and red blood cells would be rated more highly than one with less distinct phases. But in a highly automated lab environment which relies on good workflows, this approach is not ideal. Thankfully, deep learning-based image analysis can mimic human intelligence and assess the quality of a centrifuged sample’s separation. But the quality management process involves one further step: classification.
Only those samples with a passing grade will be allowed for testing. This makes it imperative for the inspection system to be able to generalize and conceptualize the appearance of “good” (i.e, well separated) red blood cell phases. It does this based on factors like plasma color, turbidity, and buffy coat volume, which are all criteria used in sample processing.
Deep learning is the only automation tool able to intelligently classify, sort, and grade multiple objects within a single image. In this case, Cognex Deep Learning is able to sort multiple classes within a single vial of blood to identify and pass only those samples which meet testing criteria.
As the latest automation solution for complex life sciences applications, Cognex’s deep learning-based tools are conveniently available as both off-the-shelf and OEM systems to be designed directly into lab automation devices. With highly reliable results and low demand on additional infrastructures like CPUs or embedded PCs, Cognex’s deep learning-based software is a natural addition to the life science industry’s arsenal of machine vision inspection tools.