How Machine Vision and Deep Learning are changing the field of Automated Inspection in Manufacturing.

Artificial Intelligence is turning out to be a game changer, with countless applications in nearly every domain. It is now making its way into the area of Production and Manufacturing, allowing it to harness the power of deep learning and in doing so, providing automation that is faster, cheaper and more superior. This article aims to give a brief understanding of automated visual assessment and how a deep learning approach can save significant time and effort.

What is Visual Inspection ?

It involves the analysis of products on the production line for the purpose of quality control. Visual inspection can also be used for internal and external assessment of the various equipment in a production facility such as storage tanks, pressure vessels, piping, and other equipment.

It is a process that takes place at regular intervals of time, such as day-to-day. It has been shown repeatedly that visual inspection results in the discovery of most hidden defects during production.

When and where is Visual Inspection needed ?

While Visual inspection is used in manufacturing for quality or defect assessment, in non-production environments, it can be used to determine whether the features indicative of a “target” are present and prevent potential negative impacts.

Among the many industries where visual inspection is required, there are several where visual inspection is considered to be of very high consequence and is high priority activity due to the potentially high cost of any errors that may arise via inspection such as injury, fatality, loss of expensive equipment, scrapped items, rework, or a loss of customers. Such fields where visual inspection is prioritised include nuclear weapons, nuclear power, airport baggage screening, aircraft maintenance, food industry, medicine and pharmaceuticals.

Why not just stick to manual inspection ?

While old might be gold, one could argue that there are several limitations to using the old fashioned way of inspection.

Manual inspection requires the presence of a person, an inspector who performs assessment of the entity under question and passes judgement on it according to some training or previous knowledge. No equipment is required except the naked eye of the trained inspector.

According to research, visual inspection errors typically range from 20% to 30% (Drury & Fox, 1975). Some imperfections can be attributed to human error, while others are due to limitations of space. Certain errors can be reduced through training and practice, but cannot be completely removed.

Visual inspection errors in manufacturing take one of two forms — missing an existing defect or incorrectly identifying a defect that does not exist (false positive). Misses tend to occur much more frequently than false alarms (See, 2012). Misses can lead to loss in quality, while false positives can cause unnecessary production costs and overall wastage.

Besides, manual inspection suffers from other limitations such as these

• There remains the fact that the human eye, while more technologically advanced than any mechanical camera, can also be easily fooled. Consider some of these weaknesses associated with visual inspection:

Human Vision alone is undependable — Optical illusions such as those to the left can demonstrate just how unreliable the human eye can be. This does not necessarily mean that manual inspection is totally useless, but that it would be unwise to depend entirely on it.

Imprecision of eyesight — The human eye is incapable of making precise measurements, especially on a very tiny scale. Even while comparing two similar objects, the eye might not notice that one is slightly smaller or larger than the other. This concept also applies to characteristics such as surface roughness, size, and any other factor that needs to be measured.

Cost of labour – Manual inspection remains a costly venture due to the appointment of (multiple) trained individuals. Cost-wise, manual inspection operators may be earning a yearly salary of $50,000 to $60,000.

A new age alternative

Automated Visual inspection can overcome these problems by making the whole procedure of visual inspection independent of any human involvement. Using automated systems typically surpasses the standard of manual inspection.

Using deep learning and machine vision, it is not only possible but quite achievable to build smart systems that perform thorough quality checks down to the finest details. We don’t require walking talking android robots to automate production like inspection. Minimal physical equipment is needed to automate the visual inspection process. Instead the process is made smarter due to the use of Deep Learning. This approach typically involves steps such as image acquisition, preprocessing, feature extraction, classification etc. This is also classified in broad group image recognition.

What is Deep Learning and how does it factor in ?

Deep learning technology uses neural networks containing thousands of layers which are adept at mimicking human level intelligence to distinguish anomalies, parts, and characters while tolerating natural variations in complex patterns. In this way, deep learning merges the adaptability of human visual inspection with the speed and robustness of a computerised system.

Deep learning teaches machines to do what comes naturally to humans: to learn by example. New, low-cost hardware has made it practical to deploy a multi-layered “deep” neural networks that mimic neuron networks in the human brain. This gives manufacturing technology amazing new abilities to recognize images, distinguish trends, and make intelligent predictions and decisions. Starting from a core logic developed during initial training, deep neural networks can continuously refine their performance as they are presented with new images, speech, and text.

So what is Machine Vision then ?

Machine vision is the technology and methods used to provide image-based automatic inspection. It is a system that uses visual computing technology to mechanically “see” the activities that take place one by one along the production line. The components of an automatic inspection system usually include lighting, a camera or other image acquiring device, a processor, software, and output devices.

Machine vision surpasses human vision at the quantitative and qualitative measurement of a structured scene because of its speed, accuracy, and repeatability. A machine vision system can easily assess object details too small to be seen by the human eye, and inspect them with greater reliability and lesser error. On a production line, machine vision systems can inspect hundreds or thousands of parts per minute reliably and repeatedly, far exceeding the inspection capabilities of humans.

A traditional automated system, while minimising costs and improving efficiency does not have the flexibility or tolerance for variation that human beings do. Manual inspectors are able to distinguish between subtle, cosmetic and functional flaws, and can interpret variations in part appearance that may affect perceived quality. Though limited in the rate at which they can process information, humans are uniquely able to conceptualise and generalize. Humans excel at learning by example and can differentiate what really matters when it comes to slight anomalies between parts. This begs the question of how machine vision can make the best choice, in many cases, for the qualitative interpretation of a complex, unstructured scene — especially those with subtle defects and unpredictable flaws.

Why Machine Vision and Deep learning go hand-in-hand for this scenario

Although machine vision systems tolerate some variation in a part’s appearance due to scaling, rotation, and pose distortion, complex surface textures and image quality issues introduce serious inspection challenges. Machine vision systems alone fail to assess the vast possibility of variation and deviation between very visually similar images.

Deep learning-based systems are well-suited for visual inspections that are more complex in nature: patterns that vary in subtle but tolerable ways. Deep learning is good at addressing complex surface and cosmetic defects, like scratches and dents on parts that are turned, brushed, or shiny. Whether used to locate, read, inspect, or classify features of interest, deep learning-based image analysis differs from traditional machine vision in its ability to conceptualize and generalize a part’s appearance.

And that’s not all

Here are more reasons to choose automated visual inspection when it comes to manufacturing:

Better Perception

Machine Vision has a very high optical resolution which depends upon the technology and equipment used for image acquisition.

Compared to human sight, machine vision has a ‘wider’ spectrum of visual perception with the ability to perform observations in the Ultraviolet, XRay and Infrared regions of the spectrum as well.

Faster — Observations as well as conclusions are made extremely fast, with the speed of a computer’s speed as measured in FLOPs and also, they result in precise calculations.

The system has all the power associated with higher processing speeds along with a potentially infinite memory capacity.

Reliable — The system is unbiased and programmable as required, following instructions without question.

Accurate — An automated system is capable of measuring absolute dimensions in a standardized manner.

Independent of Environment — Such a system can be deployed in dangerous and hazardous conditions or environments where human involvement may prove to be risky.

How to get started with Automated Visual Inspection

In terms of requirements, AVI does not really require much physical equipment. The equipment needed to start automating visual inspection can be split into hardware and software resources.

Hardware

These resources consist of primary equipments such as a camera, photometer, colorimeter and optional secondary equipment such as required for grading or sorting, which would be dependent on industry and automation processes.

We are essentially taking a picture and analysing the image, a camera is all you need!

Depending on the industry where it being use, the physical equipment can actually be categorized into three subsystems

• Feeding system — Spreads items evenly and moves them at a constant speed, so that the optical system could capture frames of individual items.
• Optical system — Consists of a specifically adjusted lighting source and a sensor (usually, a digital camera). The optical system captures images of inspected items so that the software can process and analyze them.
• Separation system — Removes defective items and/or grades and separates products into several categories according to their quality.

Software

The main requirement for AVI is the software layer, which at its core is the computer vision technology that helps inspect products or any object of interest for defects and absence/presence of certain parts.

The software part of an automated visual inspection system requires advanced image analysis algorithms and heavy programming. These algorithms process images to adjust their quality, locate interesting points and regions, and, finally decide based on the features found in these regions.

Deep learning technologies have enabled automated visual inspection systems that outperform human or traditional machine vision processes

Deep learning models have proved to be indispensable part of the software owing to the huge success that they have shown in solving inspection problems. They can be trained on thousands of images of, say, bolts, a deep-learning algorithm gradually learns to detect any meaningful deviations from the “standard” appearance of a bolt. Depending on your use case, your inspection problem could be solved using one or a combination of different tasks like Object Detection, Semantic Segmentation and Image Classification. It could also involve OCR models to read serial numbers or barcodes.

In order to maintain high image processing speeds, a trained deep learning model has to be generally deployed on high resource computers. For instance, a GPU is necessary to get results in real time.

Several factors play a role in the resultant accuracy and performance of the inspection model – lighting conditions, the number of products to inspect, types of defects to look for, size of the defects/objects, the resolution of the image to name a few. An automated visual inspection system thus requires a team of skilled R&D engineers capable of building such complex systems.

Each manufacturing unit has different and often unique data(images) – these could be because of the a different camera type, indoor illumination or the product itself. Therefore, the software part of AVI, is always a custom solution tailored to the specific inspection needs.

Nanonets helps you build and deploy deep learning models that are essential to develop the software to automate visual inspection

Experimenting with a new unexplored domain in order to improve business requires an efficient team with years of experience and a proven track record of building deep learning solutions for businesses.

There are many more factors to consider when training deep learning models, such as how you’re going to preprocess your data, define your model, and actually get a computer powerful enough to run the model.

Nanonets provides easy to use APIs to train and deploy deep learning models that are custom tailored to the specific needs of a business. It takes care of all of the heavy lifting needed to build a production ready high accuracy model – data augmentation, transfer learning and yes, hyperparameter optimization!

Once it finds the best model, Nanonets serves it on their cloud for you to test the model using their web interface or to integrate it into your program using 2 lines of code. Nanonets also provides Docker Images containing the trained model which enable you to deploy models on your own machines – this means your data/images never has to leave your premise/local network.

Some typical use cases where Nanonets can be used are:
• Quality control products before they leave production line
• Asses the quality of the harvest
• Verify insurance claims
• Inspect parts for defects in your production line
• Spot and identify defects in utility infrastructure

Want to automate visual inspection? lazy to code, don’t want to spend on GPUs? Head over to Nanonets and start building your model for free!

What can we take away from all of this ?

Artificial Intelligence is already revolutionizing our daily lives here and now. From Google search by image to complex industrial systems ensuring product quality — machine vision is making our lives easier undertaking the most mundane and complex tasks. And it is possible that very soon visual quality inspection tasks will be mostly machine-based, allowing humans to focus on more sophisticated tasks.

As technology becomes more commonplace, the costs of AI systems are continually going down while the efficiency is climbing without stopping for breath. The current trend in Automation in the industrial sector has given rise to a theme known as Industry 4.0 which includes cyber-physical systems, the Internet of things, cloud computing and cognitive computing. Industry 4.0 is commonly referred to as the fourth industrial revolution.

Industry 4.0 fosters what has been called a “smart factory”. Within modular structured smart factories, cyber-physical systems monitor physical processes, create a virtual copy of the physical world and make decentralized decisions. Over the Internet of Things, cyber-physical systems communicate and cooperate with each other and with humans in real-time both internally and across organizational services offered and used by participants of the value chain.

There is no doubt that Machine Vision and Deep Learning will be an integral part of this industrial revolution pushing global manufacturers to a new level of efficiency and productivity.

You might be interested in our latest posts on: