Are virtual reality and 3D applications old news? Absolutely not. Facebook is leading the way with its Meta brand: virtual reality is being used to create Second Life 2.0, our parallel universe on the Internet. This even goes as far as actually being able to feel things with IoT via VR gloves.
In podcast episode 60 of the IoT Use Case Podcast, we move from this altitude directly into practice. Because: These trends also move the industry. We learn from Siemens Large Drives, the world’s leading manufacturer of motors and inverters and inventor of the dynamo machine, and the two innovative companies 4Soft and Threedy.
Episode 60 at a glance (and click):
- [15:32] Challenges, potentials and status quo – This is what the use case looks like in practice
- [17:29] Solutions, offerings and services – A look at the technologies used
- [31:03] Results, Business Models and Best Practices – How Success is Measured
- [32:39] Transferability, scaling and next steps – Here’s how you can use this use case
Podcast episode summary
This podcast episode is about how 3D information is brought into the process chain. With Siemens Large Drives, we are operating in an area where a lot of power and drive force is needed: large pump systems, compressors, ventilation systems, rolling mills or ships. 3D streaming is expected to improve the efficiency of these plants and reduce emissions.
4Soft GmbH develops custom-fit software systems in the course of digital transformation. Threedy GmbH is a spin-off of the Fraunhofer Institute for Computer Graphics in Darmstadt. Threedy develops and markets the visual computing-as-a-service platform instant3Dhub. Together with Siemens Large Drives, they are rethinking industrial 3D applications with a focus on service.
Three use cases are discussed in this episode:
- Collaboration on the 3D model: enabling different users across the process chain to work together and directly
- The live visualization, especially also of IoT data: not just a product view, but also the data that is linked to it
- Aftersales with 3D data: “guided” repair by maintenance specialists
The interview guests for the 60th episode are: Thomas Barthel (Director Business Development, Threedy), Klaus Bergner (Managing Director, 4Soft) and Alexander Santiago (Product Owner @ Immsersive Experiences, Siemens AG).
Podcast interview
Klaus, you’re at 4Soft. Can you briefly say something about yourself, who you are, and what exactly you do at 4Soft?
Klaus
I am Klaus Bergner and one of the founders and managing directors of 4Soft. I’m a computer scientist myself: I studied computer science here at the TU Munich and then did my doctorate a long time ago. At that time also already graphics and 3D and so on. At that time it was about 2D, so UML and so on. We then spun off 4Soft in 1999, then as now as a project company. That is, we make custom applications for our customers, with a focus on engineering companies. The German OEMs, the suppliers, Siemens. We develop complex applications for them.
We’ll find out exactly what these are in more detail in a moment. Thomas, will you also briefly introduce yourself and what you do at Threedy – and the context to 4Soft? You belong together somewhere, don’t you?
Thomas
I’m Thomas, a computer-integrated-manufacturing graduate. I was allowed to study at the Heinz Nixdorf Institute at the university in Paderborn 25 years ago and got to know the first 3D cave there, which was built by the Fraunhofer Institute at that time. At that time, BMW simulated the first data in such a 3D cave; that was very expensive. For me, after 25 years in sales with various manufacturers, such as Tecnomatix, HP, Autodesk, I have come full circle to Fraunhofer IGD (Institute for Computer Graphics), because it is still about the topic of simulating 3D data. That is also my job here: I am a business developer at Threedy and am also responsible for the partner business. 4Soft is a very important partner for us because we develop pure technology. We need partners, like a 4Soft, who have a deep understanding of the process and can help our customers get the technology up and running.
“Customer” is the perfect segue to you, Alex. Could you also briefly introduce yourself, what exactly you do as a person, and also briefly classify – Siemens is a huge store – in which department you work?
Alex
Alexander Santiago, I have now been with Siemens AG for ten years. My background is in business informatics. About my background, started in Customer Service IT, then R&D and now Product Management. I am Product Owner for Immersive Experiences, i.e. everything around Augmented (AR) and Virtual Reality (VR), but also mobile technologies at Siemens Large Drives Applications (LDA). Located at Digital Business. You certainly need a few additions to be able to better classify the whole thing. First of all, as you mentioned, Siemens Large Drives Applications is the world’s leading manufacturer of HV motors and medium voltage converters. In addition, we have a digital portfolio in the context, including our digital companion platform SIDRIVE IQ.
Can you say more about the immersive experience?
Alex
Our department where we take care of Virtual and Augmented Reality – that means immersion in a new experience. We are responsible for taking our customers and our workforce from the old world – I’ll say 2D, the desktop world – to the new world.
You also said you are manufacturers of the engines and drives. How can I classify this with the SIDRIVE IQ? Where are the motors and drives used today everywhere
Alex
Especially in the areas and industrial applications where you need a lot of power and driving force. Large pumping equipment, compressors, ventilation systems, rolling mills. But it is also used in ships today. To name just a few. And in the same breath, we want to help our customers improve the efficiency of their plants and, at the end of the day, save on emissions. This already starts with the new plant, with modernization, but also with repairs, to constantly improve there.
Thomas, we are now talking about 3D data. Why is the topic important now and today and what is happening here at your market?
Thomas
At its core, it’s about increasing our customers’ digital capabilities. We have developed a technology, over 20 years, to bring the 3D information into the process chain. Why is 3D relevant? – We are currently experiencing an incredible hype on the subject of AR and VR. Metaverse is basically 2.0 – it’s been around for a while. But the technologies are available now. They are also, as they say, affordable. You can buy them cheaply and can improve the 3D experience. Threedy has a platform where we say, any size, any data, any device: We call it 3D streaming, and with that we bring the 3D data to every corner of the process chain. 3D data, that comes from brain research, we know that today: We humans live in a 3D interface. This means that our brain is used to working with 3D data. In this respect, this technology holds an uncanny utility value for users. KlausI can only agree with this. We talk to a lot of customers, and we see that 3D data is actually rapidly finding its way into their processes. Most of the time, they’ve been there for a long time – in design, in product development; CAD data has been around since time immemorial. But these are expert tools. And now I have new opportunities with Instant3Dhub to free that data from silos, to bring it to users. Then there are very many new use cases. Siemens LDA is a very good example and certainly a pioneer in many respects.
Perfect, we also want to talk about application examples today. I always try to put the whole thing into concrete use cases and practical projects. Thomas, Klaus, what use cases did you bring today?
Klaus
There is a plethora of use cases. In order not to lose the overview, we have picked out three examples. The first use case is collaboration on the 3D model: this means enabling different users across the process chain to work directly on a model and pass on comments and such. This is also the core of the project for Siemens. The second use case we will touch on is live visualization, especially of IoT data: This means that you don’t just see the pure product, but also the data that is linked to it. And that is also relevant in the third use case: The fact that 3D data can be used in aftersales, for example, to make repairs easier for maintenance specialists, i.e. to enable “guided” repairs.
Alex, can you pick us up a bit: How do you have to imagine your daily job today? What does it look like here on site and how does it work? You supply such a large drive to a pumping station – what needs to be done? What processes and jobs are involved?
Alex
Basically, we’re manufacturers of motors and converters that don’t just come off the shelf. In other words, in this case, cooperation with the customer is a key success factor. That’s where digital collaboration is a use case, as Klaus already alluded to; in particular, discussing and sharing 3D perspectives. You can think of it like this: Of course, products today are already designed according to a modular principle in a certain way. However, customer requirements are also added and adapted to this. That means you have the 3D model including all the connections, but also all the electrical circuits et cetera. So there are meetings about that: A customer with our order engineering, but also with our sales department. This is where we come in with Instant3Dhub in the form of SIDRIVE IQ X-ray; a product we will be launching soon. In it, you can collaborate and create complex 3D models that, in terms of richness and detail, are not easily rendered on any device. Think of it like this: Design CAD PCs have quite a lot of power under the hood – but we want to make that available to all users in the process chain! To take up Thomas’ slogan: Any size, any device. That means that we really transport this to smartphones, tablets, the normal notebook, but also – taking it to the next level, VR and AR – the whole thing into a HoloLens. That’s one of the big challenges. We have this wealth of information in the 3D model, you can’t actually render that on a computer as small as the HoloLens. – That’s where 3D’s technology comes in and helps us bring that into different experiences and, at the end of the day, make it available to everyone in the process chain so they can do their job in the best way possible.
That is, if I think of these day-to-day jobs – I have sales, I have sales, I have an inverter; there’s the engineering 3D data somewhere that I need for this outer shell of the drive. And then it goes into delivery. That means the engine is there, you look at where the connection points are; and then there’s the topic of services, which you just mentioned – that is, it’s a matter of working together on the 3D model with the customer. What do you discuss in the process? Of the individual parts, what is relevant to the collaboration?
Alex
I can illustrate this using the example of an inverter. It is the case that we can already use this, particularly in commissioning, in such a way that we can already present connection points, cable ducts, but also product features to the customer in a completely different way than in the past. This means it becomes much clearer to the customer how the product works – on a completely different level. And not only during commissioning, but also beforehand, when it takes place in contract engineering. And then comes the downstream process. That’s where we want to move, so that based on the 3D model that we have delivered, we can also do the overlay with augmented reality in the future, so that when a converter fails, we can then clearly identify which cell has failed, for example. This means that the plant engineer on site can also act quickly himself. In this way, the availability of the plant can be made as good as possible; for this purpose, we link 3D and IoT data in the event of service. So with the Instant3Dhub and SIDRIVE IQ X-ray, we are creating our basis for the future, so to speak.
Challenges, potentials and status quo - This is what the use case looks like in practice
Can you maybe summarize again what the challenges are here for you guys on a day-to-day basis?
Alex
It is indeed the case that we have a strong product variance with our products, but also customized products. Automating all of that, in a 3D pipeline, really challenges us, and we’re working on that at full speed right now.
Thomas, we have now talked about collaboration. What are the next steps here? Where are you going with this? Klaus has already mentioned a few use cases: Are these also your topics? Live visualization and also, towards service, guided repairs?
Thomas
Yes, maybe an idea to make that handy: Our credo is, after all, Google Maps for 3D data. This is how you have to think of it. That means I always have my big engineering models in my pocket on my phone, because our streaming technology does just that: we stream the data, no matter what size, to any device. This allows us to allow our customers to do exactly these things: live collaboration, as Alex put it. We can involve engineering ad hoc, with sales or with service or even with the customer. We can work directly on the model and exchange ideas here. Use cases are so many. BMW has now developed 25 use cases on the platform. This ranges from pure visualization to 3D animated service and repair guides.
Solutions, offerings and services - A look at the technologies used
Thomas, you had talked about an Instant3Dhub. How does that work exactly? What do I do with the platform and how is the interaction with 4Soft?
Thomas
The way to think of the technology platform is that we get the 3D data from the many expert tools that are available in the companies. But we don’t have to change them in any way, compress them or anything. But rather, we have developed our own calculation engine over 20 years, which can virtualize and accelerate the data. That means we access CATIA data or Step data or JT – these are the most common formats in the industry. We get those through the platform and stream them where they belong. We accelerate this visualization by a factor of 95 percent. This means that we provide customers with a technology where they can rethink many new use cases.
CATIA & Co: The bottom line is that this is a program that allows me to record such design data – the data of the hull of the engine, so to speak. For example, what is such data that you pull?
Thomas
We load not only the shell, but the complete component or assembly. We now handle over 40 formats used in the industry. This ranges from mechanical engineering, i.e. classic CAD data, to building data. We are able to visualize even large buildings – and we can mix it: We can visualize machines in a building, transporting every detail that the machine or component has, at this high speed.
A large assembly – that’s an insane amount of data! I also used to work in construction. When you want to rotate the part, it jerks and stalls the more details you load in. How do you deal with it? I have to make sure that this runs performantly, don’t I?
Thomas
This is exactly our trick, which we have developed over 20 years at the Fraunhofer Institute: An acceleration technology in which we completely recalculate the triangles in a geometry element and bring them here accelerated to the visualization. We experience it again and again with the customers, this “aha” effect, because they are used to the fact that when they load large assemblies, it is just not performant. Because we can load a complete vehicle, for example Mercedes Benz, with all details within 30 seconds and visualize it completely, on any end device. This is not a problem for us at all.
Alex, so for you that would be a whole assembly of a drive like that, right?
Alex
Right. In particular, the inverters, when you talk about how they come out of the CAD system, are a bit larger in size than motors. That’s where this USP comes in; that they say we don’t have to worry about this anymore. We now simply take the models as they are, we have all the information at our fingertips, and we can also-especially nice-access them in a web-based and scalable way. KlausThat is a very important point. This “we don’t care anymore.” We see our technology as a solution. We intercept the data, automate the transport to the end devices and the customer no longer has to worry about this very issue. But for this we need partners like 4Soft, who then help the customer to set up precisely this process chain.
You mentioned earlier that it’s also about getting that 3D data into the individual processes. Where exactly do you come into play here? Is that exactly for the processes
Thomas
That’s exactly how you can think of it. For us, the Instant3Dhub is a very powerful and innovative technology, and is actually an engine that then drives very many use cases. You can think of it a bit like an enabler that makes new things possible. Similar to perhaps a database. A database allows experts to store and query data very easily. But you can’t just put them in front of an end user raw. There they are overwhelmed when you have to learn the query language and so on. It’s a bit similar with Instant3Dhub, because the tool itself can do very, very much and because it’s not tailored to individual processes, of course, including integration into a tailored user interface, data integration and so on. If this is how you want to get the power that lies in the Instant3Dhub as an engine onto the road, that’s where we come in. And typically in the early stages. It is often a matter of first developing an idea in the first place: What use cases can actually be found in the company? Where is the most benefit? What new user groups are there? So under the heading of ideation for new applications, or working out a vision for new applications. It’s something we’re very happy to do because we know the technology and the possibilities that lie within it. If you then look further in the process, it is a matter of making the ideas concrete, analyzing the processes, the requirements with the customer. The keyword, design process-related changes and tickle out: What benefits the customer? What exactly does the customer and user need? We can then also illustrate this at very short notice with prototypes, proofs of concept. That’s where 3D and the Instant3Dhub are perfect, of course, because it’s inherently a visualization and users can actually see hands-on how it actually works. In this respect, a very, very nice, descriptive, handy technique.
End user would be in this scenario, probably also together with Siemens LDA, probably such a pumping station where someone gets access to the 3D data?
Klaus
That could be a customer end user, exactly. So someone who wants to buy a machine for a pumping station, that is, the buyer or engineer on site. But then, of course, users like the design engineer at Siemens, who looks to see if the thing will even fit into the pumping station, or do the connections have to be relocated? Is that possible? Or perhaps also the maintenance specialist, who then says, “if I move the connections there, I won’t be able to get to them at all” – you can actually get everyone together at one table and directly illustrate this and discuss the application.
A stupid question in between, since we are also talking about IoT here. One is collaboration on the 3D model. And on the other hand, it’s also about live visualization of IoT data. What exactly is IoT about this? Or where does that come into play?
Klaus
For me, 3D data is part of the inner core, the crystallization point of a virtual twin. That’s a key concept in IoT. It’s not just some block standing around in a machine hall somewhere. It’s a thing that’s networked with a whole lot of information, with data streams that are important for a whole lot of different people. Operations, maintenance experts. The design engineer, who then gets an impression of how my design is actually performing in the field. With the Instant3Dhub, we have the possibility to visualize these different IoT data on the living object. So live bringing the invisible data to life. ThomasMaybe I can also explain this again with the example of SIDRIVE IQ. We have SIDRIVE IQ at its core as an IoT platform for our drives. That means our inverters and motors are connected via a connectivity box and send their Time Series Data to the cloud. This means that if any error occurs, the plant engineer knows about it – he is notified. Our objective now is to go here and take the digital twin, link that up and put an iPad in the hand of the engineer on site in the plant or at the plant, for example, and then overlay that. “Now here at this point based on the reported IoT information, the cell has failed. This cell needs to be replaced.” That’s a chain, that’s where you can act extremely quickly and then provide the added value of reducing downtime extremely. At the end of the day, that means Euros for the customer: if the plant isn’t running, that’s downtime and that costs money as a result. And the better we can design this process by linking different technologies – including 3D visualization – the better we can serve the customer. KlausThis is also the case from our point of view. The greatest benefit usually comes from actually linking the 3D data with data from other systems or making it usable for other systems. That’s why process analysis is ONE topic. But data integration, data preparation, interfaces to other systems – that is actually usually, in the background so to speak, the most important and most time-consuming activity in the projects. So to consider, which data belong together? What quality must they have so that they are also suitable for maintenance, not just for production, for example. From which systems do they come and how are they then presented? Do I have them text-based in displays? Or, as Alex said, when I’m walking through the factory, through the hall, maybe the part lights up red when I look at it with the AR glasses. That then gives me much more stringent information.
I think that’s also the core of what’s new here: that you really do make this data usable across different systems and then both deal with the volume of data, but also ensure integration, which then creates the added value.
Thomas
To add perhaps one more example here. One of our customers is doing what is called 3D location of quality features in manufacturing. This means that the worker on the assembly line can use a pad to locate defects directly and mark them on the 3D model – and this information flows directly back into engineering. To me, that’s a very handy IoT use case, where the information is available where it’s needed and then flows right back from manufacturing to where it belongs.
Exactly, that the designer also knows how the design is used in the company – and how can that flow back to be able to optimize further? This is the loop.
ThomasFour weeks ago we were invited there in a meeting where exactly that takes place. An ad hoc engineering collaboration meeting between departments. You can discuss directly on the object and can distribute the information again.
I also think that there is a huge variety of use cases. Even if the listeners are in the process of say they have similar use cases – I think you’re open to discussing those as well. Contact information is provided in the show notes. To summarize. We talked about SIDRIVE IQ, which is the IoT platform for Siemens drives. Then we have the Instant3Dhub, which provides data integration and processing and everything that you described. This is what meshes together here. And then there’s a bit of hardware and connectivity on site, Alex mentioned, there’s a box that’s attached to the engine that ensures connectivity with the relevant systems, the cloud, the server. Do you have any additions
Klaus
Yes, the most important thing – from our point of view: The software, the application system that ties the whole thing together and shapes it – that we build – that’s what’s missing at this point for now. But clearly, the hardware is what first catches the eye and is then enriched.
Exactly, to connect in the end Instant3Dhub now with SIDRIVE IQ a piece, right? Making these use cases integrable in the first place.
Alex
Exactly, and that’s where we are, together with Klaus and his team, to bring the Instant3Dhub into our SIDRIVE-IQ world, and then under the name SIDRIVE-IQ-X-ray.
Results, Business Models and Best Practices - How Success is Measured
I’m always asked for catchy KPIs and best practices, really the business cases. Alex, what is the outcome for you guys? Have you achieved the goal you set for yourselves at the beginning? What might you have saved too?
Alex
Definitely. The time saved in the individual process steps is decisive here. There is very, very much meat on the bone, and in different places. At the end of the day, if it makes us faster, it also gets the plant up and running faster, which means the customer gets up and running faster. And if we save time, ultimately the customer is helped.
In the end, you also have a vivid representation of this data and can also realize exactly these feedback loops? You can say that the designer sees what’s going on in the company, and vice versa – such topics are also possible with such a database, aren’t they?
Alex
Yes, with the help of Instant3Dhub this is more easily linked. As mentioned at the beginning: It is the democratization of 3D data along the process chain. KlausThe greatest benefit usually comes from the fact that you can actually bring together different parts of the process chain, even different participants, and then avoid those long loops that occur when they talk to each other one after the other and perhaps in the end an error comes out that would no longer be correctable at all.
Transferability, Scaling, and Next Steps - Here's how you can use this use case.
This is a really exciting project. This use case is also transferable. For whom is this still exciting? Do you have any insights for us on where else to apply the theme?
Thomas
Here I can answer at least from our project point of view. We are currently experiencing very high demand from the classic industries, such as automotive engineering, but also building visualization and plant engineering. BASF is currently interested in visualizing large plants. We see a very clear trend there. And the use cases: In my view, what Alex has just described is one of the use cases that brings direct benefits for our target customers. That has to be said quite clearly. There are many use cases, including AR-driven ones, where people are trying with HoloLens – that’s getting better and better. But what Alex described is a classic industry use case that also offers high potential benefits.
A beautiful conclusion. I think there are a lot of use cases out there. Once again, I’d like to call on listeners: If you come from a wide variety of industries and are working on such topics, feel free to network with Klaus, Thomas and Alex and discuss your use cases. We are currently receiving an extremely large number of topics purely from the field of buildings. We should definitely do something together again to show what the added value is and what is possible.
