Using MATLAB with Python - MATLAB
Video Player is loading.
Current Time 0:00
Duration 50:58
Loaded: 0.32%
Stream Type LIVE
Remaining Time 50:58
 
1x
  • Chapters
  • descriptions off, selected
  • en (Main), selected
    Video length is 50:58

    Using MATLAB with Python

    Overview

    MATLAB provides flexible, two-way integration with many programming languages, including Python. This allows different teams to work together and use MATLAB algorithms within production software and IT systems. This webinar will cover how to call MATLAB from Python and how to call Python libraries from MATLAB. 

    About the Presenters

    Heather Gorr holds a Ph.D. in Materials Science Engineering from the University of Pittsburgh and a Masters and Bachelors of Science in Physics from Penn State University. Since 2013, she has supported MATLAB users in the areas of mathematics, data science, deep learning, and application deployment. She currently acts as a Senior MATLAB Product Marketing Manager, specializing in data science, AI, and integrating MATLAB and Python code. Prior to joining MathWorks, she was a Research Fellow, focused on machine learning for prediction of fluid concentrations.

    Yann Debray acts as MATLAB Product Manager, focusing on the usages of MATLAB with Python. Prior to joining MathWorks in 2019, he has been working in the field of open-source scientific computing since 2014. He holds an Engineering Masters degree from the Arts & Métiers ParisTech Engineering School.

    Recorded: 16 Jan 2024

    All right. Heather, what's up?

    Hey. How's it going, Yann?

    Good, and you?

    Bon jour.

    Bon jour. So today, as usual, we'll be talking about MATLAB with Python.

    We're kind of the ones-- we do this a lot actually, because we're big fans of both, right? That's kind of the idea here. So I share lots of coding tips, and Yann does too. Go ahead and follow us. And we'll keep in touch hopefully. And we love to hear what sorts of things you're working on.

    I mean, I know for me, whenever I was working on my PhD, I was doing AI. I was in physics, but I did AI stuff. And so I used a lot of MATLAB, Python, C++, FORTRAN, like whatever I needed to along the way. What about you, Yann?

    So for me, I actually come more from the open source side. And I joined the MATLAB team right around almost three years ago now. Wow, time flies. And before that I was working on an open source scientific computing project. And that's part of the reason why I'm very interested in this conversation about the interoperability with Python and other open source frameworks.

    Exactly. And I think that's a great example of how invested the MathWorks is in open source projects, right? Like we're hiring people from competing frameworks and getting them to collaborate. So this is wonderful. Thanks, Yann once again for joining us--

    Thank you for having me.

    --for this great discussion.

    So what I suggest is first asking you the question, why would you want to be using MATLAB and Python together.

    And it actually kind of brings up a even bigger question of why you use any programming languages together. And a lot of times it's that you're familiar with the one, or you're really comfortable in that one, and you need to use some kind of library or some kind of package that someone else has created in a different language. Maybe you're not as comfortable, and you want to just kind of bring it in.

    So that's one of the reasons. We talk about MATLAB and Python in this webinar, but you could really-- most of the things that we're talking about, you could talk about with C++, with Java, again--

    .NET.

    --FORTRAN, .NET.

    FORTRAN.

    FORTRAN. Again, for those of you that are in the mathematics and physics community, you'll be familiar with the FORTRAN life. But the same things that we talk about are the same reasons. So for example, we have APIs that people have created that are really wonderful, like the climate Toolbox.

    I know you've worked a little bit more with that helping others that are using Python, using C++, and there's a lot of really great functionality that has been built in MATLAB that you can use in any language at all. So that's a great example.

    So yeah, this one example, I think it was built from someone at MathWorks to leverage the access to this climate data API that happened to be in Python in this case. But as you said, like it could be just another library in C, C++. So that's one example. Maybe another one? Speech command recognition--

    Oh, I like this one. Oh, this is near and dear to my heart of course. Because remember, I wrote the one that we deployed.

    Oh, yeah.

    And it was very sad JavaScript interface. But it used MATLAB. It used Python. And a lot of those things that-- again, there's so much work in both languages, over the years. And a lot of the work in MATLAB is through hardware and through these kind of low-level signal processing and speech command kind of things. And so it's really wonderful example to see it using both of those together.

    And we have another one, which is more in the space of simulating system, like battery, instead of charge estimation. I think that's a good one as well, where it goes all the way to simulation and generation.

    That's perfect. You can talk about that one.

    All right. I can think about that one.

    No, but I think it's actually really great, especially when we're talking about AI, because a lot of these examples are AI. Many times you don't have the data to train a proper model. So you can use Simulink. You can use these simulations in order to generate data and help out. So this is a great example.

    Yeah, definitely a good example to integrate this inside of a workflow. So maybe if I were to take a step back and just summarize the typical arguments we hear from users that want to be leveraging those two together, MATLAB and Python, we hear first reuse of existing code. We are about using functionalities that is coming from another language and just simply plain collaboration, right?

    Right. Yeah. And you see a paper that has something implemented, and this is just the scientific process, so I think that's a great example of this kind of collaboration.

    So what we will do today is take our usual air quality example. You can find the code on GitHub, on your GitHub repo. And we will be following along with you. I'll just go through the small story, the storytelling we've been putting together for this whole example.

    And I think it's a great example. We've been using this over and over again, and again, on my own GitHub, years ago. But it's a great example, because you get to see that whole process of that collaboration, and when you might want to collaborate with those different languages together.

    So it actually all starts with, in this case, me writing this Python code that says--

    He's the pie guy.

    So I'm the Python guy in this scenario. And you'll be the MATLAB person. And we'll be collaborating together. I'll be writing this small piece of code that will just get data from a web service, from a weather web service to process it a little bit.

    And then what you'll do first, is you'll be calling my code calling Python from MATLAB in order to get this data-- get access to the data.

    And again this is a really common thing that we see. Both Yann and I work with a lot of colleagues and a lot of different organizations that-- we have a team that are working in Python, or working in C, working in MATLAB. And everyone needs to take advantage of all this. We don't want to have to rewrite any code.

    Exactly.

    So that's exactly the game here that we're talking about, where Yann has already written this excellent Python code. I want to take advantage of it, also take advantage of some lovely MATLAB functionality, and then ultimately, we'll share it back into Python.

    Exactly. So that's the second step. Once ASR has developed an algorithm to predict the air quality based on the current weather, what I'll do is I will simply call MATLAB from Python in order to retrieve this and gradually go towards deployment. And we may say a few words at the end around how we will go from a desktop environment to more of a cloud system with Docker containers and such technologies in order to deploy it.

    And a lot of times, again, this is what we see, especially working with MathWorks, people want to take advantage and make use of their actual work. So you want to do this in a nice systematic way that maybe we want to share on the cloud. We want to have this nice website or put into a device.

    So we like to at least address this, because we know many MATLAB users are at least trying to take advantage of all this research in reality.

    Exactly. So the agenda for today-- what we will do is we will follow each of the step, first calling Python from MATLAB. You will do this part. Then I'll go to calling MATLAB from Python. We'll mention a few as options for deep learning and artificial intelligence. And then we'll go to deployment. How does that sound?

    Good plan.

    All right.

    Sounds good to me.

    So I'll hand it over to you, and you can share with us.

    Let's go for it. So I'm very excited. This is one of the things that I did during my PhD. During a lot of my research, I combined a lot of these languages together, especially back in the day. A lot of the deep learning and AI research was coming up all over the place. And a lot of it was in MATLAB. There was some in Python. There was some in C++. So I got pretty accustomed to this.

    So I'm going to take you through, and I'm opening up MATLAB 2023a largely because-- one thing that we wanted to mention is that Yann and I work very closely with the development teams. And we pass along all of the feedback that we hear from all of you. So note that in the chat. OK. Don't forget to share this with us.

    But we also are constantly developing things. And every release, something new and exciting is coming out. So it's usually a good idea to get on the latest release. And you can also do this through MATLAB Online. First things first, we're talking two languages. We've got to make sure they can connect, they can talk to one another.

    I'll be honest, getting MATLAB to find Python is very easy usually, because of the way things are installed. And MATLAB has a lot of awareness of your system. But one of the things to point out is that, with me, for example, I have about 17,000 versions of Python.

    So let's say I wanted to use 3.10 or 3.8 or something, I can-- OK.

    That's 3.8.

    That's not true. Maybe a different one. We'll just go with 3.9. Either way-- and it's actually useful, because if it can't find it, then you can actually pass in the direct location. And so this is one of the first things that people get a little hung up on. And we will provide some more info.

    And there's a nice video that goes through the installation and troubleshooting and things like that. But for right now, it works for me. This 3.9 in my Spotify demo-- that's your Spotify demo.

    That's my Spotify demo.

    Either way.

    Maybe just one word about what we just saw, because a lot of writing-- maybe people get to freak out about it. So here, since you are running in process, for you to change the version of Python that you'd be using, you need to restart MATLAB and run again this pyenv command with a new version.

    But for now, I think we can stick with--

    Exactly. And that's why I'm not doing it right now, because we're off to start over again. But that's the idea. This seems like a good enough version for me to use. And again, I have many, many versions on my machine, and I can point which one I want to that.

    And so the big thing to remember is that the syntax is basically just pie dot your library dot your function. So if, in Python, you were to do import math and then math dot square root, you just do that in one line. PI dot math dot square root. So it's good to remember that it's only going to import it once.

    So every time you call it, it's not importing that same thing. It's very smart behind the scenes. So I just like to-- I guess maybe we've been doing this for too long. I already know all the questions that you have. All right. So what other kinds of syntax things do we need to think about?

    Named arguments, tell me about it.

    Yes. And so for years-- I mean, this functionality has been available since 2014, where people have been calling MATLAB and Python together. Up until a few releases ago, it was done through this py args, because there was a different way of passing arguments in MATLAB.

    And so, of course, the teams have really recognized that there's so many different ways that people would want to pass name and values. And it's getting a lot easier now, especially whenever you're trying to cull Python. So you can just do it like this. No big deal.

    Ooh. Big deal. What is this error on there?

    So.

    Let's just read.

    I did it on purpose.

    You tricked us.

    Because I wanted to point out-- I mean, I don't know about you, but I make a lot of errors.

    Oh, yeah.

    And you can immediately see, here in the error message, that it's a Python error. And it's a type error. And so it's going to throw both. So you'll be able to-- especially, this is great for collaborating whenever you have MATLAB users aware of certain error messages, Python user is aware of certain error messages, it's a really great way to collaborate. And immediately you see what language is throwing at.

    So of course it has to be an integer. What does that mean? Well, if you're not aware, whenever you type just a 2 in MATLAB, it's a double. So it's double precision. It's like a float floating point precision. And when you type 2 in Python--

    [LAUGHING]

    You need to say which encoding it's encoded on.

    And this is bringing me to my point, that this is why we don't just willy nilly do this, because there is-- hold on.

    There's 8, 32, 16.

    64.

    U int. Do you want it to be unsigned? Do you want whatever. And so, because we have so many people that really, really care, we're precise about precision. And whenever it comes to integers like this, that's just something that you're going to see and you'll need to work with.

    But again, because we have so many people using MATLAB for hardware and that precision really matters. It matters a lot whether it's unsigned or signed or 8-bit or 16-bit. We don't like to just willy nilly make it some kind of integer.

    So tell me about the output that you get, the list here.

    This is great. Because if you take a look, we're used to seeing something like this in our workspace, where it's just a number. But in this case, we have a Python list. And so it's actually a different kind of object. So we could operate on it. We could convert it. We could index into it.

    So one of the things I do-- I know this is sort of-- I don't know. It's not the best way to do everything. But I think, especially if you're used to Python, you can get a sense of what you can do with this object, or what you can do with this thing that you're working with.

    And so you'll notice maybe that some of them are Python methods. Like the append and pop and those kinds of things are the things that you tend to do with a list in Python. But then you also have our--

    --cell array. I remember about the cell arrays. Good thing we can directly cast it into the stream now, right?

    You don't want to remember too much about it. But these are the more generic kind of conversions that you could use in MATLAB-- and then, of course, you're integers and logical and things like that.

    Well it makes sense. In this case, I think you have a list of strings, so you will be thrown an error if you try to convert it into a double or integral.

    Exactly. It's just it's "Hello, World!" with whatever I did to make it not readable. But it does also give you a sense of-- you will have some help. So it's not like-- I mean, you should still continue to listen to this webinar, but you will have a little bit of an indication of what data types you might want to convert it to immediately, as soon as you work with it.

    All right. So I guess this is my-- all right that was my figuring it out. And then I'm changing it to a string to bring that as a MATLAB string. And then I can work with it. And so again, this is one of those times where you want to kind of stop and assess what is happening. Do you need to convert it right away? Do you need to just access one element of it?

    And again, maybe some of us aren't used to thinking about this planning ahead as much, at least. I'm speaking for myself, because I just hack around in MATLAB. And so I'm like--

    We all do.

    --woo. What do I have now? So now I need to think, wait a second. I need to figure out when I need to convert this thing, what I need to convert. I need to step back and really assess. And so that's kind of what we'll talk about next. So we're going to use this weather API, because it's a great example. Again, Yann made this just a very nice straightforward Python code that imports the weather information.

    And then we want to be able to use it in MATLAB to predict our air quality. So first, we're going to get our access key, which we will share for free. But just like we saw before, py dot math dot blah, py dot weather dot get current weather. And then we pass in whatever we need.

    And then, just like we saw before, this is actually a Python dictionary. So again, this is where we need to stop and decide, what do we want out of this. So if we look at-- here's the information. We have all sorts of stuff here, latitude, longitude, descriptions, all sorts of things.

    So at this point, we say, do we need to convert. Do we want to continue to use it as a Python object, or do we need to just index out of it?

    And in this case, I provided you with a convenience function, pass current JSON. So you can keep it as a dictionary.

    Exactly. And so you can keep working with it. And that really helps, because one of those-- in the process, that data conversion or the data extraction tends to be something that you have to think about. And so the more you can work with it as is, the better. So you can see, it's 50 degrees. It's very humid and it's not France. I'm sorry.

    It's Boston.

    It's Boston. It's just-- it is what it is. And so again, this is where we want to think about what we want to do with it. So in this case, we actually have a machine learning model that has been trained on this data and needs this information in order to predict. So we can just start, right away, making it into a MATLAB thing, so that we can start using it in MATLAB. I don't know. Do you notice any patterns here?

    Yeah, you have some of the data types that were not automatically converted. So I see the integral. And I think we mentioned the fact that we cannot assume which type of integer you want. But then the strings-- I'm also interested in figuring out what do you want to do with those.

    And that's exactly right. And it's the same kind of strategy, the same kind of theory, where it's just-- I guess developers, not us, but we don't want to imply what data type you want to store this in in MATLAB. There are several ways, especially when it comes to the city, this likely will be a categorical instead of a string or a char. And if we just kind of implied that, it would just be a string or a char.

    And this way, you're able to think about it. And of course, the current time, that needs to be a datetime. And so of course it starts out in a string we need to convert that. And again, it's really pretty straightforward. Of course, if it's just a number, double-- no big deal --or single or whatever makes sense-- 16. Blah, blah, blah.

    And in this case, we actually created some code to detect what type of data it is, whenever it's coming in, and then do the appropriate thing. So we're converting it to a structure and then, eventually, converting datetimes and our text and things like that.

    So again, I think this is probably one of the most common questions that we get whenever we're combining languages, regardless of what it is. It's always that data type and that connection of, how do you represent something in one language versus the other. So always something to think about.

    So we wrote this code. You can check it out. And now we have things in a nice way that we can represent for machine learning. Again, very humid here, in case you notice us sweating. So I loved this part that you provided, because it's really showing some of the things that you really want to do with MATLAB and Python together.

    Yeah, arrays, right, like lists and arrays, that's the of--

    It's MATLAB. It's Matrix Laboratory. This is what you do. You don't work with scalars.

    And for forecasting, I guess it makes sense, right?

    Of course, we want to see the forecast. We want to compare it. We want to do our math and similar kind of thing. But of course, even changing it to a struct, you still have a struct of Python lists. So we want to make sure, again, we're converting, we're really thinking about what we need out of that information. And of course, again, it actually tells you right here nowadays, string, double cell, however you want to convert it, whatever makes sense.

    So that's really the approach, just taking that data. If you can, continue to work with it or extract information out of it or convert as you need it. And then of course, you can use all the lovely things that MATLAB and Python have to offer.

    A lot of our examples tend to do these--

    You need a timetable for that.

    Oh, oops. I did not run the-- I was getting ahead of myself. So I got a little timetable. This is awesome. And now I can retime it, or resample it.

    That's convenient.

    Very, very convenient.

    We can change the time step. Right now it's three hours, but maybe it'll be better to get it down to one hour.

    Yeah, perfect. Yes, exactly. And these are the kinds of things that I really value. I mean, both of us are savvy with both languages-- savvy enough to be dangerous. But this is one of the things I really value with MATLAB, of understanding what types of things you probably need to do with your data-- be it scientific engineering data --and making it very easy to experiment.

    I mean, I don't know about you, but I love modified akima cubic interpolation.

    I love it.

    Makima.

    It's making my day.

    But maybe linear is easier to understand.

    Yeah, that sounds more like me.

    That sounds better. I was very excited. I got to put makima in a blog post. So I peaked right then. But it's really nice, because again, you don't always know what you're working with whenever you are bringing in data, especially when you're working with different languages. So you can update this.

    We have another example here where we're removing trends. Taking a look at some of these, there's running Python code.

    Nice left.

    Yeah. Oh, yeah. I like that one. Yann made it. But again, many of those data things, like cleaning outliers, group computations, those kinds of things that people really value, just kind of going in and out. And now, let's say I want to share this with my pal, Yann. He's right there.

    So I--

    Make a prediction.

    Exactly. He wants a prediction. He has high expectations. You're the crazy MATLAB AI lady. Give me a prediction. So I'm predicting air quality based on just weather information, which is not calculated that way. It's just temperature, pressure, things like that.

    And it's wonderful because we can just go through this same process, take that new data that Yann has provided through his code, and now we can use our model and apply that appropriately.

    Good.

    It's good.

    Looks good.

    I didn't think it was. It's sort of humid. I don't know what air quality-- I don't know. I guess it's fine.

    We may need to revisit your model. But that's fine.

    OK. It was an example.

    It's maybe drifting over time a little bit.

    It was an example.

    Yeah, exactly this was trained in 2018. Things have changed.

    Things have changed.

    (WHISPERS) Climate change. But anyway. So now that we've gone through this whole flow, and we've gotten our prediction, then we want to prepare this to be able to call it in Python. So I want to pass this over to Yann, and make sure that we can call this in Python, so that eventually we could put this on the cloud. We can incorporate all of this into a nice website.

    So again, all of the steps that we just took. Anything else? What do we see here? We saw how to connect things together, how to configure the environments, and that syntax for the Python functions, where again it's just py dot math or py dot thing dot function, and then remembering the data types.

    There's a lot of information. We're going to share all of this with you, all the different data types between different languages, and then, of course, how to call the Python code, regardless of how it's organized to begin. All right. What do you think?

    Well I can certainly take it over from there. So you've provided me with a nice model with a function to call this model. And so what I will do now is I will call MATLAB from Python this time around.

    Perfect.

    And what I use here is a notebook, so a Jupyter Notebook, in this case with Python code in it. You can see here the version of Python that I'm using. What I will do is I will connect to the MATLAB engine. So for that, there are different ways you can start a new session of MATLAB.

    What I will do in here is actually share the already running session of MATLAB that I have. In order to do that, I'm going to go in MATLAB dot engine dot share engine.

    And it's a really common if you're working kind of side by side, like we've been, were developing in both languages. You can just share it. You don't have to start a new process.

    So here I'm getting the name of the engine. And I'm importing the MATLAB dot engine library, and making a connection to this session here. And it's instantaneous. So I get it in the variable--

    Just one more point too-- for the engine, API, and the compiler, we do provide a script to help connect everything. So I just wanted to point that out, as we're going along.

    You can also find a lot of the documentation here on our help. And you can see how to get it set up and working. So once you're there, the first thing you can do is just like, as I showed you, is test with a very simple function. So here again, we will be using the square root of 42. Let me make an error for you, so that you can see what happens.

    You're not supposed to tell them.

    Oh, sorry. I spoiled the fun.

    Shocking. It's an error.

    So here, an error. And it has to do-- here if we scroll down, it's a MATLAB execution error. It has to do with the undefined function square root for an input type of int integer 64.

    Yep. Classic data type problem.

    Classic again. And that was the reason why I was here using a double instead. And in this case, it's defined. And so I can simply execute it and print the result. So one small difference in behavior, as well, that you may have note here, is if I were to remove the print statement-- and it's not going to output the value of x, by default.

    This is a difference in behavior between the MATLAB and Python interpreters. MATLAB outputs by default, as a result when Python doesn't. So you need to be explicit about it. One case where you may have different numbers of output arguments. Here for instance, with the greatest common divisor, you can have is n argout equal 1 or n argout equal 2. And depending on here, what you say here is going to output a different value of X and a different size of X.

    So here, for instance, I get 5 and 0. And if I want here-- oh, it's interesting so X of 1, it's the second argument, right? This is the second element of this value X.

    We're just trying to remind you that you're working with two different languages.

    Exactly.

    And I don't know about you, but I always use the wrong brackets and the wrong--

    It's brackets--

    --string.

    --and starting index is at 0 or 1, is always a little--

    But you'll immediately know. I think it's good about the errors and the workflow. So you'll know where you're right, which one.

    Let me clear all of the output. I'm spoiling all of the fun otherwise. What I'll do is now I'll go inside of MATLAB and I will define a variable B equal 42. And when I come back, on this side, what I can do is get from the workspace m dot workspace B. And in here, is just giving me 42.0. So it has been defined on the MATLAB side. I'm getting it back as a double here.

    Here there's also a mention to displaying the desktop. It doesn't make sense in this case, because the desktop is already up and running, and I can switch between the two applications. But when you're running MATLAB as ls, and just calling MATLAB from Python ls, then you can simply also use this m dot desktop, and reminding that you need to specify n argout equals 0. So it's going to give you an error otherwise.

    So this is just to get started. Now, let's look at the example that we took previously with the current weather first. So we're going to get here sample data in London. So don't pay attention to the values, because they are just sample data. They were taken at some point in time in London, and they are in kelvin also.

    It's definitely 288 degrees Fahrenheit is not-- I mean, I'm probably a little--

    I mean, not the London that I know. So here, the first thing that I will do is I'll be using this predict air quality function that you provided me with. So I'm just doing m dot name of the function. I pass it the data. And here, I print the air quality. And it's good.

    It's good.

    All right. Now what I can do is also do a little more data preprocessing. So in this case, I'll take the forecast. And I'll do that in Munich, because the sample data is best in Munich. For the sample API. And here I'm getting, again--

    Just because you speak German too. You you're just trying to show off, because he speaks French and German.

    And mostly also because it's just a sample data. That's it. OK. So what I'll do is I'll pass things around, converting this if I-- here if you want to see data. What do I have? OK. So it's a dictionary. And I can ask it, OK, what are the keys in my dictionary. So those are the types of things you would normally do. And you'd be like, OK, so now I want to get the temperature.

    So what I'll do is I'll just simply type, data, and here, brackets, square brackets, temperature. Here you go. So here you have a list of numerical values. So what I'll do is I'll pass this list to the workspace and convert it into a MATLAB double.

    And part of the reason that we want to do that is because then we can use things like the apps. So one of the things that-- I mean, it sort of makes me lose sleep at night sometimes. But the classification and regression learner-- I spent years and years of my life working on one of those algorithms. And now there's like 30 algorithms for machine learning. And you can just hit a button.

    So these are exactly the kinds of things that people want to do with MATLAB. And you can just pull it right up. Or one of our other favorites is the--

    That I cannot help.

    Yes. Yes, that's a great one. That's the one that I intended to show to you. You also have the signal analyzer that you like.

    That was the one I was thinking. I was going to say signal analyzer, but you pull out the data cleaner.

    I'm going to show you the data cleaner.

    It's very similar.

    Yeah, I just want to stay in base MATLAB for now. So this is included in the base version of MATLAB, the data cleaner app. So it's just very convenient for me. What I'll do is, you may have seen me converting this into a timetable. And this is the type of data that is accepted by the data clean app. So this was me here doing an eval statement to create a timetable, as you can see here.

    So just a small line of code in order to convert in the right data type, in order to use the app.

    Yeah, this is great. And again, this is one of the most common things. We have a couple of examples, even using Simulink to simulate the data, and then bring it into Python, do some analysis, do whatever, and then bring it back into MATLAB to deploy it. So I think this is a great example, especially data cleaning, that's such a common thing. And here, as you're going to see, you can just click a few buttons and try some algorithms and see what happens.

    So here what I'll do first is exactly what you've done before. It's going to be retiming. So here I have a kind of a desk that is part of this app. And I can simply retime to, instead of every three hours of sampling, turn it into every one hour.

    And that's so common, because sensors tend to have different sampling rates, and you're always needing to combine them together. So it's a really important app for that sort of thing.

    So here it's going to be doing the retiming. And as you can see here with the data or visualization, it's going to turn from every three hours to every one hour, so fitting in between those values. And then what I will do next is just smoothing the data.

    So here you can see, it's a little rough. I just want it to be a little nicer. So I'm just going to simply use like a moving mean, and I can change the smoothing factor. So here you go, you see also the results here.

    And again, I think this is a great example of experimenting like this, where this is the first time you're looking at this data. Maybe you need to just kind of try things out and really try to understand this visual, and the app really helps a lot with that.

    And once I'm done, what I can do is simply export the output, the data to the workspace, or I can create a function. And in this case, that's what I'm going to do. I'm going to call it-- let's be creative. --free process. Very, very creative. And as you can see, it's just going to be generating the code. Here I want to replace the one that I had already created.

    And this function now. I can call it-- I can close the app. And I can call it, from Python. So here I'm going to first retrieve this timetables that I have in my MATLAB workspace. And I'm going to be calling the preprocess function on this timetable. I'm going to create a second timetable. And out of this table, I will simply write a Parquet file.

    And here I'm going to be using PANDAS in order to read the Parquet file and just plot it.

    And again, I think this is a great example, because there are certain situations where you want to go back and forth, where you want to have your app that you're exploring with, and then you want to bring it into Python. And then there are other times where you just want to call what you've already written and kind of go on with your process.

    So I think this is a great example of how to manage those different situations.

    So here I ran into an issue, because I had not-- oh, here you go and now I need matplotlib. So there's a whole bunch of libraries that I was missing. And you're going to be falling into this scenario as well. I want to execute these, but I don't have the library. So you just go back to your console and you pick install.

    I needed pyarrow. Now I need matplotlib. And you try again.

    Cross your fingers.

    So that's typically what just happens when you use Python, right? So we're good. And now we can finally plot the results. Here you go.

    Smooth. Very smooth.

    Very smooth. And once we're done, we can simply exit the MATLAB engine. And here we go. So if we were to go back to the slide and just summarize what we've seen, we've how to call MATLAB apps from Python. And we also saw how to connect to the MATLAB engine. We mentioned briefly that we can pip install it. So you can use pip Python package manager in order to install the MATLAB engine as of 22a or b, I believe.

    And you can specify which version of the engine you want. It's all documented in the documentation. So you can start a MATLAB process, or either connect to one that already exists. That's actually what I've done. And call functions, just simply as that.

    So before we move on to deployment, I want to mention a few additional options. And that is specifically for deep learning first. You have ways of working with TensorFlow and PyTorch by calling them, coexecuting MATLAB with Python to call TensorFlow or PyTorch.

    And you also have ways to import directly the model and export them. So we have import TensorFlow network. We also have a way now to export TensorFlow models. And we also have a way to export and import Onnx.

    Yeah, and I think this is great, because it shows we've been talking about coexecution, calling the two languages together. So as you're developing, you can call side-by-side. But then once the model is done and developed, you can just import it directly. So there are a few options for that.

    And of course, you can also check our Model Hub, the Deep Planning Model Hub to see if you already have some pretrained models that you can simply use.

    Very nice domain specific examples too.

    So this is for deep learning. And we also have a few specific things around data exchange. You can exchange with simple MAT files, because with SiPy, by you can also import read-and-write MAT files. But in this scenario, what I shared with you was the use of Parquet file.

    And as you have seen with some additional packages like pyarrow you have a way of reading and writing PANDAS that are framed from Parquet files. So for now, it's a very convenient way for us to exchange between tables in MATLAB and data frames in Python and PANDAS. So this is it for additional options. Now let's go to deployment, because we want to deploy our app.

    Yeah. And this is kind of the grand finale, where we won't go through the super details, but we'll give you all the hints and paint the picture of what you need to do to take those next few steps in sharing your application.

    So what I'll do here is I will use an app called the Library Compiler. And here what I will do is I will select the type of libraries that I want to compile. So if I go to the bottom of the list, there's a Python package here that I can select.

    Last, but not least.

    Last, but not least. And I will give it a name. I'm going to call it-- I'm just going to override the one that I already had, which was called AirQual, like this. I'm going to save it-- AirQual. --like this. And I will simply select the function that I want to export. So in this case, it's the predict AirQual function that you've developed for me.

    And here I'm simply going to be packaging this. So let's see.

    And it's nice that it really detects a lot of the other code that you need.

    Yeah, the dependencies.

    So if you have a large project, it'll detect a lot of that.

    So that's what you've seen popping up right now, is all of the dependencies to this function that I got from Azure, the air quality model dot MAT. There are some mentions about cities. And this function, the prepdata that you've used also to preprocess the data.

    So once I'm done with this, I can simply package it. It's going to create a few binaries. And it's going to open also my folder, my file system.

    And it's really great, because you can be the MATLAB developer or the domain scientist. And you don't really need to know about the next steps of IT. You really can just take this output and say, all right, here you go person that does IT. Let me know if you need anything else.

    So it really kind of helps with that hand off whenever you're building these large applications.

    So once I've generated this package, what I can do is simply open a command window. I could have done that also directly from my Visual Studio code environment. And I'm in the right folder for testing. I simply need to execute the Python setup, that py, with the function installed.

    And so what's it's going to do here-- as you can see, it's going to be installing this package. And now what I will do is call this Compiled Library from Python.

    And this is really helpful whenever you're trying to share this with somebody that doesn't need MATLAB installed on their machine. They just need to have the runtime. They just need to run that code. There's no need to have the desktop. So it's a really great option for that. And then we'll talk, of course, soon about the web deployment and other things where you definitely don't want to have all that stuff running in the background.

    So for now we will follow exactly the same process as before with the MATLAB engine. But in this case, as I just mentioned, we do not need to have a full MATLAB. We only need a runtime. And this would be the runtime that would be powering this air quality module.

    So here we import the module. And the first thing we do is we initialize it and assign it to this variable AQ. And now, instead of saying m dot predict AirQual, what we'll do instead is do AQ dot predict air quality with the data that we just got from the web service.

    And that's really helpful, because there are-- of course, we only showed one function. But there are times where you have many, many functions in that package. So maybe you're predicting, you're doing some other kind of forecasting, some other data cleaning, or whatever, and this makes it really easy to access all of those things.

    So here we go. We have a good air quality. And we can terminate the process. So we are done with the first way of deploying.

    And I think this is a great example-- again, I still think sometimes this can be still somewhat in the development phase, where you're getting to that point where you are using everyone's code, or you're trying to make use of it, but we're just about to get to that step where we want to share this more broadly, like on a web page. We want to type in a location and get the air quality and the information. And we want to use all of this work that we've done in this past 40 minutes or so.

    So one option is, as we just mentioned, to generate the Python library from those function. But following the exact same workflow, what we can also do is generate a Docker microservice with a Compiler SDK. And that's actually what this small emoji-- what I say is, oh that's what we need to put the full app with MATLAB and Python in production. So here if you go to this page, you'll see the documentation on how to do that.

    This was a direct quote from Yann's first day of work, whenever we first met and said, oh, this is a great weather example, but let's put it in production.

    Let's put it in production. And this is how it looks like-- here a small video of the running app in production. So they're saying here it's local to the user's machine. But here you can give it a new city. And once you hit Submit, it's going to go off in the background and request this web service to do the computing and getting back the air quality.

    And again, a great example of combining both MATLAB, Python-- well, OK. More than MATLAB, Python, because it's JavaScript, and Java, all sorts of things. So it's a great example of pulling everything together, all of your work, and presenting it for the world.

    So if you want to try it out yourself, go to our GitHub, and try it out, give a style, you can fork it. You can give us feedback, make pull requests, all of the types of things you do in open source. Maybe a bit of a conclusion, a summary from all of what we've seen today.

    First you showed us how to simply call py dot the name of the library dot the name of the function. That was to call Python from MATLAB. Then I was showing you how to call MATLAB from Python. And we saw a few additional options for deep learning and for that exchange.

    And finally, we've seen how to deploy MATLAB as a Python package or as a microservice. And that's the conclusion.

    That's great. Yeah, so hopefully we have covered most of what you're interested in. And we're happy to take any questions now. And we're happy to keep in touch. So thanks a lot for your attention. And we'll keep in touch.

    Thank you.

    Thanks, Yann.

    Download Code and Files