MATLAB and Simulink with Python

My name is Sean De Wolski. I'm an Application Engineer at MathWorks. Today we're going to talk about integrating MATLAB and Simulink with Python. We'll first look at how MATLAB fits into the system architecture. Then we'll look at calling Python from MATLAB in Simulink and the opposite direction of calling MATLAB from Python, and finally, sharing our work that we've done in MATLAB as royalty-free Python libraries.

We're building a service so that users of any other tool can call MATLAB. Now it's important to note that I'm going to talk about Python today but the same workflows that we're going to cover also apply to C and C++, Java and .NET. So there are three ways to integrate MATLAB with Python. You can call Python from Matlab or reuse existing Python libraries, open source libraries that you found or things that your colleagues have developed.

You can call MATLAB from Python so you can take advantage of advantage of MATLAB's powerful analytics or simulation capabilities from a Python environment, and then we can share the work that we've done in MATLAB for Python users who don't have access to MATLAB. The first two buckets here are primarily for algorithm developers, so these would be users of definitely MATLAB and probably Python as well.

The latter here are not for users of MATLAB there. Targeted at people who need to use the work that you developed in MATLAB, So now let's get into it. How do you call Python from MATLAB in Simulink? We are going to use a cryptocurrency price predictor example today, so we'll be hitting a public API, using machine learning model trend in Matlab to make predictions about a cryptocurrency.

Purpose of this example is not to know anything about cryptocurrency. It's just an example that everybody can understand, so we're using that. So I'd like to start off with the Hello world of Python. And so to do that, I'm going to create a Python str, so I'm going to prepend the Python str with py dot and create a Python str. This is the Python str.

If I press Tab, that capitalized method is the Python STR capitalized method. If I want it back in MATLAB as a MATLAB string, I can do this and now I have it back in MATLAB. So at the simplest, you can take advantage of your Python capabilities by merely having Python installed and prepending whatever the Python capability is with py dot. That's how you use this interface.

What did I have to do to control this? Well, we can control the Python environment using a command called pyenv. This lets me specify the version of Python I'd like to use and the execution mode. The execution by default is in process and we recommend that you use this. Only switch to out of process if you're experience crashes with the interface.

Crashes can occur because of library conflicts, so for example some variants of TensorFlow can cause a library conflict and then crash everything, so you would use out of process for that. To take advantage of core Python capabilities, you can call py.math.sqrt. Obviously, I would never do this. I would just call it in MATLAB, but for the purpose of the example, you can see how this works.

You can create Python data types directly, so py.list, and you can see it's martialing the MATLAB cell array over to the Python And then there's a pyargs command that you can use to construct a Python dictionary.

Now, in general, the data marshaling is going to be the hardest part of using this interface. I'd like to just point into the doc page real quick, so that you can know where to look for it. So go to MATLAB, External Language Interfaces. We are calling Python from libraries from MATLAB. The opposite, which we'll tell you about later, is calling MATLAB from Python.

And if you go inside of here, we can see the various topics of how this works. And in general, I would recommend looking at this one, the How the Data Type Mapping Works between the two languages. So now I have a custom Python code. So this is a function called-- a Python file called dataLib.py and has a couple of functions in it-- getPriceData, which hits the Coinbase API. And then parseJson, which parses that response and allows me to select specific talents from it.

OK, so I'd like to take advantage of this. So to do that, I'll just use the Python interface again-- py.dataLib is the name of the Python file and then getPriceData, which is the function within it. And that returns to me a Python list. And from there, I'm going to select the things that I want and then call the parseJson, so that it can pass it and give me back the data as a Python tuple.

From there, we can grab out the constituents of that tuple, and we will see that it is now a MATLAB double. I've converted it to a double. And then I'll create a date time from that first time stamp. So this is a POSIX time. That's one of the things I was referring to you. Sometimes have to figure out how to deal with the data. So in this case, that timestamp is from POSIX time. So I'm going to convert it to a MATLAB date/time in the American New York time zone. And from here, we can go ahead and plot it.

Now one of the questions I often get as I'm showing this example as well-- for people who aren't aware of the MATLAB Live Editor, which is what I'm using, is that it looks awfully lot like a Jupyter Notebook. It is a notebook-style environment in MATLAB for writing MATLAB code. But it's also alive in that if I interact with it-- let me, for example, just make a few changes here-- then it will give me the option to update the codes. So the next time I run this, I get the same thing again. And it's interactive in this sense.

Now what else did I have to do to get this to work? There's one other thing that I had to do. So right now, my current folder is the main level of this project. If I run the Get Current Working Directory in Python, we'll see that it is this folder. So the Python API starts up in this folder. That dataLib.py actually lives in a slightly lower folder, right here.

OK. And that folder is not necessarily on the MATLAB path. So somewhere where I really strongly recommend using MATLAB Projects-- MATLAB Projects came out in the core MATLAB in 1980. They'd been in Simulink for a while before that. And they allow you to have shortcuts and startup functions to control the MATLAB environment. So here, I have a project startup, so every time this project starts, it's going to add the project path to the Python path. So that whatever I have access to in MATLAB, I would also have access to a Python. And that remedies is this problem. So I'm a strong advocate of Projects for everything, but this is one case where they really are powerful.

What do you need to do this? You need MATLAB R2014b or newer. We supported various versions of Python throughout time, adding support for 3.8 with our 20b release in September. And the architecture of MATLAB and Python both have to be 64-bit. So if you install 32-bit Python, then this interface will not work with it.

You can control the environment with pyenv. And you can manage the Python path by either adding to the Python path directly or changing our current directory in MATLAB to do the same. There's a File Exchange Submission as well that will add a folder to the Python path. So we manage the data type conversion for you from the common core Python data types. Starting in 18b, we do support marshaling directly to NumPy arrays via MemoryView, so you can construct a NumPy array directly. And then you can bring it back into MATLAB as a regular matrix or array in MATLAB. This is how that works. But you do need 18b for that capability.

Another common question that we get is, can you take a pandas data frame and convert it to a MATLAB table or timetable? Or vise versa-- can I construct a pandas data frame from a MATLAB table or timetable? And the answer is we can't do that directly. But there is an open-source file format called parquet that is optimized for high-speed reading of tabular data, like you would have in a table, timetable, or data frame.

And so we have readers and writers as well as a data store and big data interface via tall for that in MATLAB. And with a data frame, you can write it to parquet. So this is the recommend exchange format for getting data frames back and forth to MATLAB.

So what about calling Python from Simulink? You can do this using the Python API once again. I'm going to switch examples here to a Simulink example. Once again, I like Projects because they just set everything up for me.

So here, I have a simple PID class written in Python. You can see it has set methods to set the gains. And then it has a GenOut, which generates the output. It's the equivalent of step method in Simulink. And I would absolutely never recommend that you use Python for PID control. This is obviously somewhere where Simulink is superior. We're just using this as a simple example to show how it works.

So here I have the regular MATLAB PID block, and then I have the Python PID block. And the way that you add this is via a user-defined function. So you can add a MATLAB function or a MATLAB system, which give you a MATLAB code that you can call.

So now inside of here, because Simulink tries to generate C code for any MATLAB code and a MATLAB function block, we need to explicitly say, hey, we can't generate C code for Python, so we're going to flag that as coder.extrinsic, which says run it against the MATLAB Engine rather than trying to generate C code.

Now I find it's easiest to put all of the Python stuff inside of a function that's lower down. So have the MATLAB function simply do MATLAB only. Put all the Python stuff inside of a separate function. Flag that function as extrinsic. And then inside of here, use the Python interface just like you would do in MATLAB. So instantiate the PID controller on the first run, and then just adjust the gains on each iteration-- take a step.

OK, if I run the simulation, I'll see similar responses here. OK, so we can compare the responses. Now as I said, please don't do PID control with Python. The PID block in Simulink is far superior. There's all sorts of automated tuning capabilities that can handle saturation and wind up. So I definitely recommend using this instead. But this is to illustrate the example.

Now we're going to look at calling MATLAB from Python. To do this, we'll use the MATLAB Engine for Python. The MATLAB Engine is a client library that you import into your Python program that talks to MATLAB, behaving as a server process. Conceptually, it's fairly straightforward. You create or find an existing MATLAB session. You move data over to that MATLAB session or get data back from it.

And you can call MATLAB commands either synchronously or asynchronously in MATLAB. And we manage the data type conversions for you. Let's see an example. I'm going to start running this Python program because it does take a minute to run. And then I'll explain what's going on while it's running.

So we have a similar start to the script to what we were doing in MATLAB before. We'll get the price data and parse the Json response. And then we want to use the MATLAB machine learning model. So we'll import the MATLAB engine and then start a MATLAB. And here, I'm actually going to start it with the desktop. This will allow me to use my MATLAB as almost a visual debugger for my Python code. So that's actually a workflow in and of itself. I can move data over. I can plot it. I can now use the MATLAB apps interactively because I have a full desktop MATLAB. And that's what you see flashing in my Taskbar.

As I said before, I'm a big fan of using Projects to configure the environment. So we'll just open the MATLAB project the way we have before, which will take care of all of the path management. And you see that project opened here. Will move the data from the server over to MATLAB. And then call the predictPrice function with it.

All right, displaying the data-- that we just saw at the terminal here in VS Code. If we would like to move some data over, we can reference the workspace property of the MATLAB engine. So I'll take the coinPrice and validationData and migrate them over to MATLAB. And then I'll run this command to decode it in MATLAB. I can specify the number of output arguments to control how the function gets called.

Now additionally, this is where I can also specify if I wanted it to run asynchronously. So I could have it be async here as well. And then it would run asynchronously. We're simply waiting right now to close the MATLAB Engine. So when I type a key here, it'll close. We can go over here and investigate this beforehand.

So I have this full MATLAB session. You can see the data that were moved over. We can go ahead. We can plot the data and interact with it. I can change it. I could use it to turn in a machine learning model-- whatever it is I want to do, I have this MATLAB session available today. When I'm ready, I'll close it. That'll terminate the MATLAB Engine. And we'll see it disappear.

The MATLAB Engine is a package that you import into Python that allows you to interact with the MATLAB server process. There are a few additional things. For example, if you're running it asynchronously, you can poll it to see if it's done. And then you can handle exceptions as well. To install the MATLAB Engine, you need to run this setup install that comes with the MATLAB session.

I like to just have a little MATLAB helper function that does that for me. So I can just call this directly from MATLAB, and it will run it, so that I don't need to navigate into that directory myself. It does require running MATLAB as an admin. After that, then your Python code will be aware of the MATLAB engine.

Now once again, this does require a MATLAB license and MATLAB installed on the system to use this. So this is meant for someone who is using both Python and MATLAB. So what about for those folks who don't have MATLAB and only using Python, but we would like to expose our work to them? So the workflow here will be to take our MATLAB analytics and deploy it to Python users. We'll do that by taking whatever we've written in MATLAB and running it through the MATLAB Compiler SDK, which will spit out a Python package.

From there, we mount the Python package on the end-user machine. And it runs against the headless MATLAB Runtime that does not require a license to run. So it's royalty-free. You can sell it. And that Python package is encrypted as well. So if you'd like to protect your IP, you can do that too.

We'll use the Library Compiler app that comes with the MATLAB Compiler SDK. I'll get a library name, and I'll add the functions that I would like to export. In this case, I want to export that predictPrice function. It will do a dependency analysis to figure out what other files are required. I can put meta information around it and define a namespace.

I can also add a sample. Samples are examples of how you would use this interface in MATLAB. So you can have multiple samples for each function that you have or for the collection of functions that you would like to expose. I will add the existing sample here, so that it will generate a client Python version of this sample. We'll see that it did a dependency analysis and picked up the MATLAB files that are required as well as the machine learning model in this case.

Now this actually also requires the Python code. It requires dataLib.py. So I'll package that as well. When I package it, it takes a few minutes to run. So it's finished. We can our review of the log file and see that it package successfully. There's a warning about the path that we can ignore because the project is managing the path.

This creates three folders for me-- a Redistributable, which has the installer which will install the Runtime for end users; just the Python package, which comes with its own getting-started HTML file to help your Python users along. It'll show them how to install it and give them an example of the sample. And that sample shows the way that you would use the interface in Python.

Now what you'll notice here is that the workflow is very similar to what it was with the MATLAB Engine. We import the MATLAB Library, which is the name of this app-- MATLAB Library. And then we import MATLAB, initialize it, which starts the MATLAB Runtime. Dispatch our MATLAB function calls to it, and then terminate at the end.

Now paying this runtime initialization cost does take a little while. It's the equivalent of starting MATLAB. So typically, you wouldn't want to start it and then terminate it in this sense. If you have a Python application, your end users would want to initialize it just once and then maintain state with it, and keep it alive, so they can recall it every time without having to pay that startup cost.

And just to show the difference between the two, the MATLAB Engine code that we talked about before is very similar. It's just rather than-- we can't open the project. The project doesn't exist in the compiled world. But the actual MATLAB function calls are pretty much the same. Once again, the workspace concept doesn't exist in this Runtime. Its function calls only. So I can't do this. I can only use the compiled functions. Those are the only thing available to me.

This works really well for applications that you want to share with your colleagues who are Python users. There are some challenges with it, though. For example, that MATLAB Runtime has to be the same as the version of MATLAB that was compiled. So supporting multiple versions of MATLAB is difficult. Or supporting multiple versions of the Python-- of the library you've compiled-- can be difficult.

You saw that initialization time. You have to pay that every time you start it. So this is not something that can just be called really quickly the first time. And the end users have to have this package. So you have to have a way to redistribute it to them if you make change, fix bugs, et cetera.

So as you're trying to scale, we have a different solution that allows you to host MATLAB on a server or host your MATLAB-compiled artifacts on a server that then anybody can use. The advantages of this is that you manage the analytic centrally. So if you want to make an update, you'd just drop in the update. And the next time somebody goes to use it, it'll use your update. It's meant for high-throughput low-latency applications.

So if a request comes into one of these analytics, the round-trip time is sub-millisecond for a new operation. So it's meant for high throughput, high speed. And you can scale it to as many users as you want simply by purchasing more Production Server Workers, which are the actual MATLAB engines that do the work on the server and by scaling your hardware. So it's really meant for scaling analytics.

So this allows you to host a whole variety of different analytics. And anything that can see this server can call into it. So you can have the price increase, which is what I just did. But we could also have a database using some of the MATLAB analytics. You'd host things that are completely unrelated, and they can all live in this Production Server together. You can also have multiple versions of the Runtime. So I can deploy something in R2020a and R2020b and have a runtime for each, and it'll just run against the correct one.

Additionally, it's not just Python that I'm targeting now. It's anything that can make a REST call. And we do have thin-client libraries for the common languages. So the way that this works-- there's a MATLAB Production Server Compiler app that will let you deploy your analytics to a MATLAB Production Server. Here, I will add three functions-- getData, getPrediction, and predictPrice. Once again, it will do the dependency analysis. And I will need to add the Python dataLib.

OK. And one of the really nice things about this Production Server Compiler app is that I can have it turn my local MATLAB session into an emulated MATLAB Production Server. So I'll make it discoverable. And then I'll start it. And now from any client that can see my laptop here, we can test our MATLAB code against the actual real client. So I now have this client written in Python.

And we'll see that this client simply makes a HTTP request into my MATLAB Production Server, which is currently local. Though, it would be somewhere else for production. So it makes the request and then parses the response. So if I run this, we'll see that request come in on the right to my development test environment and MATLAB Production Server. And you can see that response gets fed right back over.

We can inspect exactly what things were as they came in. And we can even put breakpoints here to stop on error or to stop and function entry to make sure that the data types are what we expect. So if we were to run this, and we were getting some type of bug, I could come over here and say, well, what, exactly, is causing this? Let's look at the varargin.

So we can see exactly what the inputs are. I can walk through this with a debugger like I normally would do in MATLAB, except it's my Python client calling into it. Maybe the time zone or the time format is off or something, and so I just need to adjust that. But that's the kind of thing that's really good to catch right now while we're in a development test environment, rather than as we move a production. OK.

Now the purpose of this talk today is to discuss Python, so that's what I've been showing. But anything that can make an HTTP request can call into this. So I can have lightweight JavaScript, and a web page that can call into it. I could have a dashboard written in one of the dashboarding tools that could use MATLAB as the back end. Once again, as I push that-- as I push Refresh here, we should see another request come in. There it is. Oh, that's still pending because I didn't advance the breakpoint. So I'll advance the breakpoint here, let it continue, we'll see Python get its response back. And then that next request will come in-- web page on the left.

So in a real MATLAB Production Server, we wouldn't have these concurrency issues because it can feed multiple users based on how many workers you have. Here, my MATLAB is still single-session. Now we can actually even have this in MATLAB. So I can make an HTTP request from MATLAB. So we could have MATLAB calling a MATLAB Production Server as well if you just want to host your MATLAB analytic centrally. So this is how you would typically take MATLAB algorithms and scale them to lots of users.

So to summarize-- oh, I guess-- so now that we know it works, I can stop and close the test. And then I would package this, and it would give me the archive file that I would drop into a MATLAB Production Server to be running anywhere-- on the cloud, on-prem data center, et cetera.

So to summarize, we looked at reusing Python code in MATLAB, taking advantage of MATLAB capabilities from Python, and then sharing the work that we've done in MATLAB, either with desktop users of Python or to try and scale to the enterprise level with MATLAB Production Server.

There are additional resources available to you if you're interested in this. We have a MATLAB and Python Solutions page on our website. And typically, some of the places where MATLAB and Python are competitive are in the predictive analytics or deep learning space. So if you are currently using Python for this, and would like to look into MATLAB's capabilities, we try to focus on solving engineering problems with these tools. So we've built apps and tools to make it easy to take advantage of these technologies for engineering applications.

Additionally, one last thing that I'd like to call out is what happens when MATLAB crashes. If you send this report, it comes to MathWorks, and we try to find a potential solution automatically. If we can find one-- perhaps, it's a known graphics card bug or issue or something-- we'll provide the solution to you right there, so that you can fix it, take whatever steps are necessary. Maybe it's something that was fixed in an update. If it's something we haven't seen before, then it gets sent to Tech Support. And a Tech Support engineer will contact you within a day.

So this is one of the main advantages that MATLAB has over open-source software is that MathWorks backs up our software. And we're there to help you, so you don't have to go post around on Stack Overflow hoping the person who's developing something is still interested in it and available. Hope this was helpful. Have a great day.