Quality Inspection Based on Deep Learning and a Data-Centric Approach

Nice to make the presentation. So we're talking again about some AI stuff, about how we can deploy it to production. So it's not in the cloud. It will be something on premise. It's something on an industrial PC. But you will see a lot of lessons learned, how we bring it to the production, how we deploy it.

About Miba, some of you know it, some of you not. We are an Austria family-run company, about 7,000 employees. We produce and develop products along the entire energy chain-- energy value chain.

And what it comes with this diversity of products is that we also have to inspect these products. So we want to deliver this high-quality products to our customers. And for that, we are going to inspect them automatically.

If we run a [INAUDIBLE] production, we started in the five, six years ago with parts from here-- so gears, [INAUDIBLE] gears, friction material, bearings. Now we are going more and more in this direction and are inspecting them.

But why are we doing this? The goal is to increase the productivity and reach these quality goals. For that, we are building this automated quality inspection systems. What you see here is the so-called smart IQ, smart Inspection of Quality. It's one of our in-house developed and built inspection system only for Miba.

To see here are-- is the digital twin of this. So it's something where we came up with synthetic data for our training. And what we are really concentrating is on inspecting, on running this with a deep learning approach. So we are training deep learning neural networks and highly focusing on a data-centric approach.

That means not model-centric, really concentrating on the data. I will explain this a little bit later. Basically, the presentation is two big points, I would say. One is I explain the quality inspection framework-- so how we run it, how we train it, how it's built up. And the second part of this presentation is about this data-centric approach.

So it's something what we already saw in the presentation before. When you remember, there was about this fleet management system. You can compare it. We have this online part. We are acquiring images, acquiring data. It could be 3D data. In this example, it's mostly some special images.

Then we have some kind of preprocessing, where we make a part detection, where we estimate the pose, and have some filters. So nothing special in this case. What really then is something which we introduce and which is different to the-- let's call it rule-based systems-- is this block. This block is classification and reasoning block.

And if you remember the last presentation, there was some similar block there. The only difference is we run this directly at the automation machine, not in the cloud. But more or less, it's something which you can compare. So that's nice to see that our others going with the same approach.

After we run here, mainly our deep learning networks, of course, we then determine some results, have some reasoning came up. Is it OK part, or is it a non-OK part? And then communicate with a PLC.

We are typically doing this over OPC-UA, sending our data, log data images to a database, or visualize it to the person there, to the guy who have the process engineer, the process guy there at the machine on an HMI, that he can identify, OK, what's going on? What's the current performance? Why are the parts now running out that's non-OK? Or why are they OK?

Next to this, what you see here on the upper to this online tool chain is our really important offline part. So to train these networks to came up with the data, which we, of course, acquire over there and store in the database, we need a block which runs offline. That's what every developer have. We have then this-- make there based on a MATLAB GUI, or a MATLAB Graphical User Interface.

We built some nice workflow-- we'll also show you later-- where we can really concentrate on the training-- on the data, sorry. And the training is more or less automated. So the model setup that we train then 20 models in parallel is automated. You can adjust some parameters for the training. The focus is more on the data. So to select the right data, and yeah, that's the next slide.

And if you then, of course, are done with the model training, you deploy your application. You can copy these neural networks or deploy it-- it's a nicer word for it-- to the application. And that's it.

And what we hear today also in the morning, the question is, how fast are you with this? Is it something which you need a day for it or that you need an hour for it? And in our case, the focus is really to make this model customization cycle quite fast.

So everything here-- checking the data, training the model, validating the model, testing the model-- is done here and should be in less than an hour. That's our approach. And that's then where we have an issue. It could be a new failure type. It could be a higher number of pseudoscrap, where we can really fast adapt our systems in a safe way and bring this trained model back to the running system. Good.

What you also see here is that this, what I explained, is MATLAB executable, is our backbone more or less. We have several toolboxes. But it's possible to, especially in the data acquisition, to include some CDLLs or run also some Python code in it. So that's not something which is only possible with MATLAB code. That's what we-- or I like on this approach. It's open to other systems also.

Some words to the hardware setup because we are not only software guys and concentrating on the software. We really have to seamless integrate this into the hardware. So we define or we spend a lot of time when we make a new inspection system defining the requirements, the cycle time, the position of the part, how fast the part could run under the camera, things like this. This is really important to have-- stable acquisition process.

The software setup I already mentioned. It's not running in the cloud because of lead time. So we don't have the time to send the images to the cloud, process it there, send it back. We run it on an industrial PC. It's quite not an expensive one. It's more cheap one, I would say. And it's an easy system. That we deploy it as an executable, I already mentioned. And what we are really focusing is on this fast updating of this networks.

Then next to this framework, second part of this presentation, is where I really want to share some lessons learned with you, is this data-centric approach. So that's nothing new, I would say. But when you remember 10 years ago, I think there was really the whole community in machine learning. AI was more a model-centric approach.

So you had the same data. Try to improve your model step by step. But now I think the whole community changes. And we also is that we are concentrating more on the data. So improving the data, getting out more of the data, and not spending so much time on the model.

What you see here is our-- what I explained before, for the developer, this offline tool, where you can run your trainings. You select your data. You more or less select some parameters, nothing special. You run the training, more trainings, more models in parallel. Then you evaluate the results and concentrate then on the identification of missing data or wrong labeled data.

So what could it be, missing data? It could be where I have troubles to correctly classify a part. I will see it there. So I will see, oh, this part is only in the minor group. So I have to optimize my augmentation to get more parts of it. Or I have to speak with the guys from the production. Hey, we need more of this non-OK parts, which is, as you can already know, it's not so easy.

So in this case, we are doing more augmentation to really came up with data where we know we have less or came up with synthetic data of this previous mentioned digital twin. So the most of the time, to repeat this, because this is really important from my point of view, is going through the images, see why does I have here a misclassification, and figure it out. What's the reason for it?

Solve it. Make again a loop. Train it again. And then what you see here, exporter could be deployed. That's it. The model now can be then copied, or it's automatically to the machine. And we can start the production again. Of course, we are doing, in the production then, manual tests to get 100% sure with a specific test parts. But that's it.

And what's, of course, also something which is good to know is this continuous improvement cycle. So when I'm done with such training, I close this application. After two months, I have to improve something because of some reasons. I open it, load the data and the configuration, and I can, in this case then, proceed there where I stopped the last time. And if another developer is doing this, it's also possible. So it's nothing where only one guy can work on it.

Another thing what is some lessons learned, I would say, what we are trying to do is to break down the problem. Or you can also call it decompose it. That means if you have a big part, don't train one model on it. Try to break it down. Make it easier.

With the teeth here, it's really obvious. You have this single teeth. You cut it out, train a small network which runs really fast, and you can train a better performed model with less data.

Another example here, where it's often not so easy to distinguish, when I cut out these smaller batches, it's often better to distinguish between these two classes. Of course, next to this, where we cut out specific areas, we are also trying to find anomalies. After we find these anomalies, we then run a classification. It could be not like in this case, a single class. It could be then multiclass classification, what helps us at the end then better figuring out what's the reason.

If I had a lot of this, for example, I know I have to find-- search the failure at the front end or at any specific machine. It could then be some-- I would say, this data-- the better this data at the end is for this classification, the better is then my performance at a predictive quality use case.

OK, some results, some discussions. After switching from this rule-based systems, where there are several suppliers out to this more model-based data-centric approach with the backbone of MATLAB, we were able to really fast get these systems running. With this example-based approach, let's say, it's easier to came down with the false positive rate. And yeah.

What we run since the last years are 10 individual deep learning applications. At the end, there are more systems, of course. But we're trying to have some standardized where we can run one deep learning application on different inspection stations. And the next goal is, of course, for us to scale this, to scale this over our 30 production plants. That's the plan for the next year.

What we see here is the inspected parts over the years. And we see also the decrease in pseudoscrap. What also is significant is, of course, the reduction of the pseudoscrap but also the reduction of the development time. So the faster we can adapt-- integrate such a system and adapt it, then the faster we can hand it over to production and produce it or inspect it, of course.

Comes to my last slide. What is now our scope for the next months or years, I would say, is to make it easier and simpler for a bigger group. So there are not always machine learning engineers out there or data scientists. It should be possible also to run for people who don't have a master's degree or something like this. It's really necessary to make it easier.

We want to increase the numbers of applications, of course, by reducing the development time and reducing the complexity. So the less neural networks at the end we have, the better it would be.

In addition, what we are doing is we want to not only make this quality inspection use case. We want to do something with this data. I mentioned it two slides ago. It was this predictive quality. So when we are inspecting the parts, the parts are already scrapped. We cannot make it good. It's not possible.

But if we can figure it out quite-- in nice classes, what's the reason for the parts, why they are bad, we can do some predictive improvements-- or not predictive. We can do improvements to avoid this scrap at the end.

And this nice sentence at the-- this last sentence is something-- the whole presentation is made from me. But the last sentence not. It's something which ChatGPT made. So I add this presentation and say, do you have any other advantage what I can share with the community? And it was this. I find it quite nice and add it here.

And what I also mentioned at the end is this great collaboration with the colleagues from MathWorks-- so with [INAUDIBLE] and Sebastian Bomberg, it's really great. It's really good. And also, together with the colleagues in Austria, who are joining or which are my colleagues, great to work in this team and to work on this machine learning, AI stuff. OK, thank you, guys.

[APPLAUSE]