High-tech company EvoLogics is using AI for real-time object detection to help operators of its autonomous survey robots. This allows them to identify objects in hard-to-interpret sonar image data faster and more accurately.
The challenge: Identifying objects under water
The news caused a stir in the early summer of 2021: Hundreds of electric scooters were lying at the bottom of the Rhine near Cologne, having been thrown in by rioters. Recovery is difficult: visibility in the water is poor, and the scooters are partly covered in mud. So far, divers have been able to discover only a few scooters. In similar cases, they are supported by autonomous survey vehicles, such as those being constructed by EvoLogics.
These devices do not work with cameras, but acoustically with side-scan sonar, which is also used for detection in submarines. Areas of application of the surface vehicle Sonobot of EvoLogics are, for example, the search for scrap and containers of all kinds as well as drowned persons. However, a sonar robot produces images that are difficult to interpret, and even experts with many years of experience cannot always evaluate them quickly and accurately.
In many cases, such as when searching for people, police, firefighters and aid organizations would like a faster solution. This includes getting the sonar as close to the bottom of the water as possible. That is why the company is working on a autonomous underwater robotwhich supports the operating personnel in object recognition. There are two important challenges to overcome:
- Communication with the operator on the water surface: radio and GPS do not work under water. Instead, EvoLogic uses underwater modems developed in-house that transmit digital data with sound. They can be integrated into the Internet of Things (IoT) in a similar way to radio modems.
- Identifying objects with artificial intelligence (AI) – automatically and in real time: This supports the operator so that he can evaluate the sonar images more quickly. However, because of the only low bandwidth of an acoustic modem, the AI solution must work in the robot itself.
The solution: Image detection (computer vision) analyzes sonar images better than humans do
Solutions for analyzing sonar images are developed by EvoLogics together with AI and cloud specialist Steadforce from Munich. For computer vision, developers use specific hardware from NVIDIA and train a deep neural network with deep learning algorithms. This requires large amounts of data that can normally only be processed easily in the cloud. But the bandwidth of communication via underwater modem is not sufficient, because sound under water only allows a data rate in the range of a few kilobits per second.
Therefore, the developers decided to split the solution: The training of the neural network is done in the cloud to make the most of their performance. The trained model, on the other hand, has less demanding requirements and is thus suitable for use on an edge device. It is installed in the robot’s on-board computer and processes the sonar data on site. This then transmits the sonar image and information about detected objects to the operator.
The operator now sees an image in which potentially important objects are already marked. This allows them to evaluate the image more quickly. The prerequisite: The neural network must first be trained to recognize specific object types. This is done using as much sample data as possible. The two development partners are using images from previous survey missions, among other things, and supplementing them with sonar images taken specifically for this purpose.
The data management and machine learning platform in the cloud continuously delivers the individually trained models. As a result, using organizations have a library of models for specific use cases, such as locating trash receptacles, cars, or boats.
The result: Flexible survey robots for all areas of application
The application of people search with sonar is only the concrete, first step towards more autonomy in marine robotics. With the AI solution, organizations and companies are able to adapt EvoLogics’ survey robots to any underwater objects, and other sensors can also be integrated. For example, it is possible to count the population of fish farms or to determine the number of fish in shoals or regions in the sea. To do this, users must customize the AI module with the appropriate training data.
The advantage of this solution is that as soon as the robot is in use, the resulting data can be collected and merged into a training data set. From a sufficient amount, the data can be used for post-training. This enables the robot to learn and improve from mission to mission.
