IDS Imaging Development Systems GmbH is a manufacturer of digital industrial cameras. It offers 2D and 3D cameras and services to help customers simplify image processing. More and more customers are no longer looking for “just” image-processing cameras; rather, they want flexible, complete solutions that are easy to operate. To enable IDS to go deep into the customer-specific application, it also works with partners. These include specialists such as urobots GmbH – a software developer specializing in “machine learning” and image processing for quality and assembly control.
The challenge: Error detection in chaos
Imagine the production of sealing rings, a typical mechanical engineering product. After production, the sealing rings fall onto an assembly line and are conveyed to quality control. They fall randomly and thus lie very differently on the belt: one lies flat, another on edge, still others overlap. In this mess, someone has to inspect the sealing rings for quality weaknesses such as burrs or cracks. This is no problem for a human being thanks to his intuition, nor for a robot, if it has been given the sight and knowledge of a human being. This can hardly be solved with classical programming; artificial intelligence is necessary.
Recently, urobots GmbH was to equip a pick-and-place robot with such capabilities. They had previously developed a PC-based artificial intelligence that can visually detect objects; but in this case, they wanted a cloud-based solution to flexibly respond to changes such as object sizes. In doing so, they relied on the support of IDS. The existing artificial intelligence model, which recognizes the position and orientation of objects in the images, had to be ported to the IDS NXT platform. This is a compact embedded vision system that allows neural networks to be executed directly on industrial cameras. Two challenges arose:
- The conventional solution runs on the PC’s graphics card, on the so-called Graphics Processing Unit (GPU). In contrast, the cloud-based solution runs directly on the camera, which is done with a different technology: the Field Programmable Gate Array (FPGA). The artificial intelligence had to be adapted and converted into an FPGA model using a tool.
- The software communicates with the robot by converting 2D to 3D coordinates. It is written in the Python programming language, while the IDS NXT app (which controls the camera) requires code in C++.
The solution: Intelligent robotics application thanks to "labeled" images
Thanks to urobots’ experience with the IDS NXT system, an intelligent robotics application was developed in just a few weeks. The FPGA of the IDS NXT embedded vision platform features an AI core (“deep ocean core”) specially developed by IDS that executes neural networks in a hardware-accelerated manner, allowing urobots’ AI application to run directly on the camera. IDS provides customers with two ways to add a neural network to cameras:
On the one hand, a neural network can be directly connected via the cloud-based software solution IDS NXT lighthouse to be “trained”. Customers don’t need any AI knowledge for this; they just need to upload suitable images, then label them, and then start the fully automatic training at the push of a button. Since all tools and workflows are precisely coordinated, the process is simple and requires no expertise. On the other hand, customers can also upload their own TensorFlow backend trained neural networks to the open platform. IDS provides special tools for the conversion on request. In both cases, the neural network then runs on the camera, enabling it to generate specific information or relay commands. For the urobots solution, two things mattered most:
- A neural network and an IDS NXT camera to detect the objects in the correct position.
- A robot that independently grasps and places the objects based on the image information
The result: PC-less system makes industrial PC superfluous
Through this smart solution, the competences are clearly distributed, because the customer focuses his attention on his product, while IDS and urobots concentrate on the AI algorithm of image processing. An additional advantage: Ethernet-based communication and the open IDS NXT platform mean that this app can also be adapted to other robot models.
Monotonous tasks such as sorting parts suit AI better than humans. It does not take breaks, works almost without errors and is therefore more efficient. The accuracy of the application is 99.9%. This allows manufacturers to reduce their complaint costs, which makes their customers more satisfied. Instead of the monotonous tasks, employees can devote themselves to “smarter” tasks in which they are superior to the robot. The IDS NXT camera makes the usually required PC obsolete thanks to FPGA, which speeds up the workflow and reduces energy consumption; in addition, the acquisition costs of the PC are eliminated. Last but not least, manufacturers can do without a team of experts for image processing.
This use case is not limited to the sorting of sealing rings, as it is easily transferable to other applications. Whether it’s detecting foreign matter in compost, sorting tools, or correctly aligning pepperoni on a richly topped pizza, an intelligent camera can do the job.
