AI and the Power of Simulation

And here is a great example of a young startup company Monarch who combined Simulink, that is model-based design, automatic code generation with artificial intelligence. This group came together in 2018, and only three years later, they deployed their first fully autonomous tractor in California. So this involved complex mechanics and advanced AI-based software making use of all the sensors and cameras and whatnot to allow the tractor to navigate autonomously, for example, through an orchard, as can be seen in this picture, with the goal to save water and reduce spraying of chemicals and so on.

Now, we are going to break this down into this rough sketch of a system model. We have a plant, some sensors or sensing equipment, which informs then our planning and scheduling, and finally, controlling what we want to do next. And let's see how we can use artificial intelligence in these four different applications or use cases.

The first would be to simplify the plant model, make it faster, for example, if you need real time capabilities, for example, for hardware in the loop simulation and tests. And second could be on the perception side where you want to create a so-called virtual or soft sensor. Think of you have data from test beds or certain prototypes that come with some types of sensing equipment mounted, but the overall fleet that is out in the field does not have that.

And you will still need these informations, which you cannot actually measure. Then you would resort to such virtual sensors. A third trend involves planning and controlling where we are going to have a look at reinforcement learning where an AI model can learn from simulations, for example, to make decisions, actually.

And our fourth and last trend is to use a digital twin or a plant model to synthesize data on which a subsequent AI model can be then trained on, in case we do not have the data or the needed quality of data which is required to train an AI model with a certain accuracy, for example. So why would you want to do this? It allows you to overcome certain challenges.

For example, models are too slow and cannot be run in real time. For example, to have a plant model run in real time to test your controls algorithms against that in a virtual hardware in the loop test. Sometimes models are unwieldly complex or inaccurate.

For example, modeling exhaust emissions is really complex. If you think of the chemistry, the physics happening there, it involves a lot of uncertainty. It's all just modeling. So oftentimes, a data-driven model can be even more accurate.

Or in the last case, you are trying to measure something which is difficult or costly to measure, in essence. So let's start with trend number one, reduced order modeling. So what reduced order modeling does is it's taken a higher order model-- this can be developed from first principles, according to theory, can be a physical model, such as a Simscape model, as in this picture here-- and measuring its inputs and outputs. And use this measured or historical data then to train a black box input-output model. In this case, a neural network, as we're going to see.

So why would you want to do that? Oftentimes, it's faster, more lightweight than the high fidelity physics-based models whenever you are mostly interested in speed, such as the HIL tests. And how would you do that?

Well, you have a plant model, a high-fidelity physics-based model, which you can use to run simulations and produce data. Or you have some real data from the field which you have recorded, and can use that to train the AI model. So in our demo today, we have a full vehicle simulation. And part of that is an engine model, a spark ignition engine model, which we are going to replace with a black box input-output AI-based model.

And this will speed up our simulation of the engine model by a factor of 10 to 50, depending on whether we start from a Simulink model, for example, or something that comes from TT power. And the overall simulation of the full vehicle still is up to 2 and 1/2 times faster. So what we're going to do is run a couple of simulations with the physics-based model and record the inputs, which would be the engine speed, ignition timing, throttle position, and the wastegate valve. And the outputs will be the engine torque, which then feeds the full vehicle system model from which we infer the velocity of the vehicle, for example.

So here's the workflow we're going to follow. We're going to ingest and prepare the data, model, design, and train an AI model, incorporate this into a Simulink implementation, the engine model integrated into the system model, and finally, generate the code for our real time capable hardware in the loop simulation. Let's jump to MATLAB then.

So here's the live script. Let me enlarge this quickly so you can better see. I'm going to run this section by section and comment on a few bits and pieces here. So we're going to ingest some pre-recorded simulation data. And let us quickly visualize this.

So we have, as I said before, engine torque as the output and the throttle position, wastegate valve, engine speed, and spark timing as the inputs. So that will be one output and one to three, four input signals. It's often good practice to split your data into a training and a validation data set so you can validate the performance of your AI model with new data it has not been presented with during training. OK, that's what we're doing here.

And also, we're going to standardize or normalize the input and output signals, which brings all parameters to more or less the same scale, which makes it easier for our model to converge, or the training to converge. So let us now design a neural network with just stacking up a bunch of so-called layers in an array. But let me show you this graphically.

There is a nice graphical tool, Deep Network Designer. Let me open this here. Where did it go? OK. And load this in.

So we have our input sequence layer with our four channels or four signals. Remember the-- what was it? Ignition timing, the engine speed, and so on and so forth. Then there is a so-called LSTM layer standing for Long Short-Term Memory. This is, as it says, something that looks back to the past inputs and has some memory or state.

And then a bunch of non-linearities introduced here. And finally, shrinking all the intermediate signals down to our one output signal, namely the engine torque. All right. And then we do train the network.

The training usually takes quite some time so I have recorded this and sped it up a bit here. We are tracking the root mean squared error. That is the output from our model compared with the actual output from the high-fidelity physics-based simulation. And hopefully, this goes down over the course of the iterations.

There is another dash dotted line where every other iteration, we are doing some validation data set, presenting our model with a validation data set. And if you see that this black line saturates or the error for the validation data set starts to rise, this is usually an indication of your model overfitting. So you should maybe tone down the complexity a bit or you need to acquire more data, try different training options. That's something to keep in mind.

So let's jump back to MATLAB now that our model is trained. We will save it in a MAT file, in a MATLAB format, and make use of this in the rest of our code. So we're going to see how our model performs.

We are looking at the root mean squared error again for our training data set. And also, the test data set which we had held out, held aside. And also, we're looking at the coefficient of determination. That is basically how well the actual and the predicted response signals correlate. So this ranges from 0 to 1 with 1 being best, and it looks like our model performs really well.

We can also have a look at the time series themselves. And here we see our ground truth data in blue, the output from the physical model, and the predictions from the deep neural net in red, which overlap nicely. Remember, this is just the engine model. If we go back, we will see next how to plug this into our Simulink/Simscape model.

There are specialized blocks to make use of neural nets. In this case, we're going to use the smart search function and start typing. Remember, this is a recurrent neural network that has some memory or state, so the block is called Stateful Predict, and we wire them up. And now, this is basically what we have just seen in MATLAB now again here in the Simulink scope how the physics-based model and the neural net compare.

So in the next step in our workflow, we now have swapped out the physics-based engine model. And now, we're going to take this sub model and integrate it with our full vehicle, full system model and see how the prediction of engine torque from the AI-based model now effect the behavior of the full vehicle. So here, we're going to see both the engine torque and the top part of the plot. And down below, the velocity of the vehicle based on the AI model in blue compared with the variant with the physics-based model in yellow. And they really match up nicely.

So now, we have designed and trained our AI-based model, have plugged this in the Simulink submodel, incorporated that into the full vehicle. And the last step in the workflow is now to generate the real time capable C code, which we will then run on our HIL test rig to, for example, test our motor control against this plant model. So we're going to use Simulink Coder and select C code.

There's a couple of optimizations. You can tweak the compilers output a little bit. And let's have a look at what our code for the LSTM, the neural net, actually looks like. And this is basically just a bunch of numbers representing the weights and biases, the parameters of our neural network.

And then we can move this over to our hardware-in-the-loop rig, our Speedgoat hardware, in that case, and again, compare the physics-based reference solution with the AI-based one. So this is really matching up nicely, the physics-based model and the AI-based one.

But remember, this now runs 2 and 1/2 times faster so this can really make a difference when you want to test your motor control against it, for example, in real time. So if you want to get going with this, here is an idea for a curriculum of trainings you could take centered around a two-day course of Simulink training on control system design and leaning into machine and deep learning with MATLAB for the AI part. And on the other hand, the real time testing and generating the code and moving it over to the Speedgoat hardware.

Of course, if there is a whole team or a smaller group, we will be always happy to customize the trainings to your special needs and maybe work with your own data and model during the course.

Now, let's move on to trend number two, virtual sensors. With virtual sensors, we either replace an existing sensor or an existing estimator, like a Kalman filter with AI. And why would we want to use AI instead of a physical sensor, for example? Well, the sensor could be too expensive, too noisy, or it might be degrading over time. So there are many reasons.

What we do is that we collect data from the sensor that we want to replace or the Kalman filter. And this will be the response in the AI algorithm. Along with that, we collect data from other already existing sensors that we believe correlate with the sensor of interest. We can then train the AI model using the other sensor measurements and use them to predict the sensor value of interest.

Let's take a look at a concrete example. Here, we have an electric motor from which we are interested in knowing a couple of different temperatures inside the motor. There is a way of getting those temperature measurements, and that is through a contactless infrared sensor. The sensor gives good and accurate results, but is expensive and not feasible to ship with the engines in production.

So what we can do is that we can put together a lab setup where we collect plenty of different kinds of sensor data, including the expensive infrared sensor. We can then have the temperature measurements from the infrared sensor as the outputs. And we see here the temperatures we are interested in, which are permanent magnet temperature and the stator choke, teeth, and binding temperature.

The inputs are existing cheaper sensor values that we know we want to have access to in the engines in production, such as ambient temperature, coolant temperature, voltage, current, and motor speed. So we are guessing that both thermal and electrical information is important to get the magnet temperature. And basically, what we believe is that there is a correlation between those physical properties and the temperatures we want to predict.

If there is a correlation, we should be able to predict those with AI. Here is the workflow that I will show you to go all the way from measured lab data to having something that can be tested in Simulink, and then deployed to, for example, an ECU or other embedded targets on the product. So let's hop into MATLAB to look at the data and how to create the AI model.

OK, all of the data collected from the lab are in one big CSV file. We read the data in and plot the top eight rows of the data. And yeah, all of the sensor values are in there, and some additional ones too.

You want to clean it up by extracting the inputs and outputs separately in different tables and visualize them as we did before. For that, we use one stackplot to get a feeling of what kind of data we are working with. So here we have four different temperatures that we want to estimate at the top and the sensor values we have as inputs are at the bottom.

Now, we step to the data pre-processing and feature engineering. Compared to the previous example where we hardly did any pre-processing, we will, in this case, extract some features from the data. So feature extraction is a step in which we want to enhance specific pieces of information in the data that we are interested in. And in this case, we are computing some kind of specific information, for example, the magnitude and reactive power, then the moving averages and the moving variances. Good.

In the next step, we prepare our training data by splitting the data into training, validation, and test data. The input dimension to our network is 66 because we have 66 different features and the raw data was only of five dimensions. The output size is the number of responses, in this case, four, because we want to estimate four different temperatures. OK, then let's use the Deep Network Designer app to take a look at the network.

In this architecture, we have two legs branching out, one with an LSTM layer just as the network in the other example and one with fully connected layers. The reason for this branching is that we wish for the LSTM layer to pick up on the slower moving temporal trends of the ambient temperature and the fully connected layers to pick up on the effect of heat conduction. All right, to train this network, we do exactly what we did when in the last example by using our architecture.

Here's the training feedback from the network. We still have a regression problem and we are looking at the root mean square error. One thing I could add here is that the blue line looks at the performance of the network on the training data itself. We are also seeing a black line on top of a blue, and this is the validation data performance.

So if the blue line and the black line starts diverging too much, that's a bit of a flag that we are overfitting. In this case, we might want to go back and do some changes to see if we can remedy the differences. Let's have a look into the results.

To the left here, we can see the actual lab tests in blue and the predicted temperatures in red. To the right, the plot shows the residuals or the differences between the two. And even though this is not a straight line as it would be in a perfect world, yeah, it has an error of well below 1%.

So we have some issues predicting low temperatures of the permanent magnet so we could either need more data, more features, or a slightly different neural network architecture to improve the model. Now, let's look at the model and its Simulink implementation. We'll just test the AI model block with the predictors and search from a saved MAT file.

From this scope here to the right, we have our four different temperatures being predicted. This component model is less than 50 kilobytes so it's not a very big model. As compared to the first example, we will, in this case, generate code that we wish to run on an embedded target processor that will ship with the product.

Now, we can, for example, deploy to an ECU if that is what we have, and this also opens up to run processor in the loop testing, which we can do in an iterative manner if we update the network. The largest portion of the generated code here are the fixed values for the weights and biases. As you can imagine, it is a very tedious job to write those by hand, and also, very error prone.

This was an example of estimating temperatures using AI instead of using an expensive infrared sensor. But what are some other use cases? For example, Coca-Cola developed a virtual pressure sensor with machine learning to improve beverage dispenser diagnostics. The Coca-Cola Freestyle beverage dispensers allow consumers to select from hundreds of different beverages via a touchscreen interface.

A key component of the dispenser is its Flow Control Module called FCM, which includes a solenoid actuated valve to regulate water flow. With no physical pressure sensor in the machine's water line, field technicians were unable to distinguish between an FCM failure and an upstream pressure loss, leading to unnecessary FCM replacements. So to improve field diagnostics, Coca-Cola engineers used MATLAB and Simulink to develop a machine learning based virtual sensor and deploy it to the dispenser's resource constraint microprocessor.

If you want to learn more about virtual sensors, there is the possibility to join our trainings in this field. We provide a machine and deep learning training with MATLAB in the scope of data preparation and AI model training, as well as the Computer Vision and sensor fusion plus object tracking with regards to implementing this in Simulink. If your objective is to embed the software component onto our hardware, we also provide training for the Embedded Coder for Production Code Generation. However, based on your need, there's also the option to request the training for specific workflows.

Great. Let's take a look at the reinforcement learning trend. Compared to the reduced order modeling and the virtual sensors, which are widely adopted in development and production, reinforcement learning is a bit less major when it comes to industry applications. You might have heard of reinforcement learning advances when it comes to beating the world's best Go player, but maybe less on how you would apply it to your controls problems.

With that said, reinforcement learning has a lot of potential. And there are many companies doing proof of concepts with reinforcement learning. So how does it work? What happens in reinforcement learning is that you train an agent, which you could consider the brain that should learn something, by interacting with a dynamic simulation environment and let it learn through a trial and error process.

The way the agent learns is that it tries some action, which interacts with the environment. And depending on the outcome we observe, we get positive and negative rewards. Let's take the case of a self-driving car.

If we hit another car, then we get a negative reward. The longer we manage to drive without hitting anybody, the more positive reward we get. It sounds easy, but it can take millions of simulation runs before the agent has learned this behavior. And also, the agent starts with no knowledge in the beginning, so it's not so surprising that it takes some time to learn it. So it's a complex thing.

In reinforcement learning, we create the data on the fly. So there's typically no step of collecting data first, and then train the AI model. If you would not have a simulation environment to train the reinforcement learning agent against, you would need to train it against the real environment. May that be a machine or something else, it would be extremely expensive and time consuming. So simulation is really the enabler in reinforcement learning.

The environment is typically the plant model. So you can reuse the models you already have. When does it make sense to look at reinforcement learning? So one is if the system you want to make decisions for is highly nonlinear or uncertain. And the other is if you wish for an end-to-end solution.

What would an end-to-end solution look like? Well, you would include all the steps required, like perception planning and scheduling and controls, all in one model. This differs a lot to what a traditional system would look like.

A traditional system would be more like this. We would first need to make sense of the sensors on the plant. Maybe we are detecting things from a camera sensor or understanding some behavior of a plant from a temperature sensor. Based on this information, we would do some planning. And that planning would need some controls to be executed.

All of this would, in a traditional system, be algorithms upon algorithms stacked on top of each other. And they would all be created individually as well. Using reinforcement learning, we can insert the raw measurement data without pre-processing and train it to give us the controls output, all in just one model.

One example of end-to-end reinforcement learning is the shipping example in our documentation on automatic parking. It's using only LiDAR and the current position as input to park at the target open space in our parking lot. Perception, planning, and control is done all in this one AI algorithm that we have created in MATLAB.

The model is trained using positive and negative feedback based on what happens in the environment, which in this case, is a parking lot modeled in Simulink. Hitting another car gives a lot of negative feedback. Using a very sharp steering angle is another thing that gives bad feedback. Through this feedback, the model will learn, just like a child, and update its state.

So now, let's look at the results of the simulating and updating the network thousands of times. Here we can see that the car moves into the state where it is only using the LiDAR, which is also when the reinforcement learning model kicks in. It's not great at parking yet, I think, but we could tweak the feedback to be a bit harder on overall oversteering a bit and train for a longer time.

But all in all, it is quite impressive that a network, which was not even very deep, managed to learn that. So using reinforcement learning on this problem would not be possible without a simulation environment like Simulink. It would just be too expensive to drive around a car, crashing into other cars thousands upon thousands of times.

For example, we Vitesco Technologies, which develops electrification technologies for all types of vehicles, has applied deep reinforcement learning for closed loop powertrain control. Powertrain control systems must be able to handle a huge variety of environmental conditions. Global climate change and more stringent emission laws require faster development time, including accelerated prototyping.

So they use Simulink to create a detailed model of a plant. The Reinforcement Learning Toolbox, enabled to quickly prototype, generate and optimize reinforcement learning agents considerably, reducing development time. So if you're interested now to learn more of reinforcement learning with MATLAB and Simulink, we provide a one-day course in which the workflow from the environment to the training, and then the subsequent deployment of a reinforcement learning agent, is discussed.

Of course, if there's more interest in this area, it is also possible to join the two-day courses of deep learning with MATLAB and/or the Control System Design with Simulink. And as mentioned before, it is always the option to request a training for a specific workflow.

And that brings us to our fourth and last trend when it comes to combining simulation model-based design with artificial intelligence. And that is data synthesis. And this ties in very closely with the concept of a digital twin.

If you don't know what a digital twin is, don't worry. It's actually quite similar to the plant models we have seen in the very beginning. But unlike the plant model, which is rather generic, a digital twin is fine-tuned to reproduce all the individual members of our fleet, as you saw, well out in the field. So we can track the status that our assets are in by comparing them with a digital twin.

And oftentimes, you can then use this digital twin to gain further insights of the health of your machinery. Or even predict how much longer you can operate a piece of equipment until it needs to undergo maintenance, for example. And that is actually what we are going to do here.

So we are going to start with a physics-based digital twin of this pump model and produce some data, with which we subsequently are going to train an AI model, which is going to infer information about the state that that pump is in. It could be that there is some failure states, for example, a degradation in a bearing causing an increased friction. There could be a blockage, a leakage at the plungers.

So just by looking at a couple of signals, we will be able to tell, hey, this and that component in that pump needs fixing. All right? So but why do you need the physics-based model in the first place? Well, oftentimes, we do not have enough data to train our AI model to begin with or we do not have the right data.

If you're thinking of this pump model, I was saying it could have some bearing fault, it could have a leakage, a blockage. So that's three independent states of fault, which may occur either individually or in arbitrary combinations of each other. Can you make sure you have data for this that would be eight different states and all the combinations? So a total of eight different states.

Can you make sure you have enough data of all of these? Maybe the bearing fault gets more and more pronounced as the bearing degrades over time. How can you capture that?

And oftentimes, besides not having this data, you will ever be unable to obtain it because it is too dangerous to drive the system to failure, or too costly. OK? So simulation can really be key to synthesize data which you cannot obtain in the field or you would not like to because it's, as I was saying, too costly or dangerous.

And on a side note, maybe your engineering departments already have plant models or the people working on the controls for that. And this is really a nice benefit to repurpose those existing models to synthesize data for your AI-based condition monitoring systems. And that's what we're going to do in this demo.

So we have this physical Simscape model of a triplex pump, right? This has been designed to the specifications that we got from the manufacturer. There were some uncertainties regarding the pressure, the delta P, when the valves would open and close.

So this was then fitted to measured data of a healthy pump. So our model is able to nicely reproduce the signals that a healthy pump would generate. And then, we can drill down into the model and start introducing our failures, our states of failures.

For example, we can artificially increase-- sorry, decrease the passage area to simulate a blockage. Or we could have a secondary path with a fluid sink that simulates the leakage. And we can tune the area of this leakage then.

And if we simulate and record this, as you see here, we are increasing gradually the friction coefficient. And the pressure signals look slightly different from one run to another. So every failure and combinations of failure stage have their unique signature. And our AI model, later on, is going to try and discriminate between the different conditions the machine is in.

So with the power of simulation, we can not just simply turn this on and off and combine different states, but we can do so gradually. And simulate, for example, how a bearing degrades over time, and then have an AI model predict the remaining useful life until you need to change to swap out this bearing. Here is a company, Krones in Germany, who have done exactly that.

They were developing a new packaging robot. And at the same time, wanted to equip this with some diagnostic. So the lack of real failure data is a common problem. Everybody's complaining we do not have enough data, not the right data. We do not have data from machinery running into failure.

And here is a company who did this data synthesis using a digital twin. That's a company Krones in Germany, and they were developing a new packaging robot, and also, a condition monitoring algorithm that would detect different types of failures. And as they were just developing this, they may have had a couple of data from their test bench, but not all the different states that this machine can find itself in.

So they developed a digital twin using Simscape and Simscape Multibody, and then started to inject those faults into the simulation model synthesized data and train their condition monitoring algorithm. If you want to learn more about anomaly detection condition monitoring and the prediction of remaining useful life of your pieces of equipment, there is a two-day course centered around predictive maintenance with MATLAB.

OK, great. So we have reached the end of today's talk. What are the conclusions? I would say that we have seen many promising applications in the intersection between AI and simulation. We saw cases where AI models are used to enhance simulation models to make them faster, more accurate, and so on.

And we also saw cases of simulation models being used to enhance AI models. What's nice about MATLAB and Simulink is that you can use one toolchain to do both.