ChatGPT and Large Language Models with MATLAB
Learn how large language models (LLMs) work and how to build a transformer model in MATLAB®. See a demo of an LLM-based model for MATLAB and how you can use it in your work, including which prompts to use.
Published: 6 Dec 2023
Morning and thank you, Kshama for the wonderful introduction. And Thank you, Pune, for coming to MATLAB EXPO in such large numbers. I'm looking forward to taking you through the next 15 minutes talking a bit about MATLAB and ChatGPT.
Before I get started, how many of you have tried something using large language models, ChatGPT? Raise your hands. Anything you know. Of course, probably 90% in the hall. How many of you have used it for something productive in your work other than writing an email or your performance review? All right, not too bad, not too bad.
When asked this question probably a couple of months ago, it was just a handful, just one or two people. But this was probably 8%. My internal ChatGPT was able to figure that out. OK, let's get into this.
So, of course, AI is making its way into all stages of the development workflow of bringing vehicles to the market, whether it's in the R&D or also in the manufacturing space. So some examples here, Goshen, which is an energy solutions company, is using AI for battery state of health estimation. PathPartner, which is now a KPIT owned company, a KPIT group company, is using AI to develop radar based automotive packages specifically for pedestrian and vulnerable road user detection.
Toyota is using AI for doing predictive maintenance of their engine benches or engine test infrastructure. And Daihatsu is using AI for audio detection that will enable them to detect knock, engine knock, in their production, in their manufacturing process. So various different use cases of AI across the base-- and I'm sure many of you have also tried or also are using AI in your day to day work in bringing products to the market.
But now, we have this game changer, ChatGPT. And how is that going to change our R&D? So on the one side, of course, we could be thinking of integrating a ChatGPT feature into a vehicle, for example, as a voice assistant. That helps us figure out activities or coming from Bangalore, trying to figure out what we do when we're stuck in traffic.
But more than that-- and here's an example. This is an example from Mercedes Benz, who's using ChatGPT specifically or piloting ChatGPT specifically in their digital production ecosystem. And this is where they're enabling not only engineers, but also employees who do not have prior coding knowledge to gather data from the production line and make inferences from that data so that they can affect the strategic decision making process within their organization.
So this is a highly empowering technology, also. So let's get into this a little bit further. So what are large language models? A brief introduction to them, they're basically a type of artificial intelligence model. They're trained on huge amounts of data and they're typically really massive. There are millions of nodes, millions and trillions of nodes and parameters. Specifically, ChatGPT is based on what's known as a GPT 3.5 or 4, which is basically a human alignment based focused chat bot, developed by OpenAI.
And essentially it's a pre-trained transformer. So it's a type of large language model. So how do they work? So in contrast to actually planning out the complete work, the complete answer to a query, LLMs work by generating the output word by word. So they actually predict the next most likely word. So they take your question. They take your query. And they then will predict the next most likely word on the basis of that query. And they take that word and push that back into the model again to get the next most likely word, and so on and so forth, in a process that's called autoregressive generation.
So once again, they do not really understand the text. They do not really understand the query. They do not really understand what they're going to tell you. But what they're good at is they're great at recognizing patterns in the data that they were trained on. So going a bit further, this ability to being able to predict the next most likely word results in really, really strong or impressive functionality that we can use-- for example, for language itself for grammar, or for word knowledge, or also for doing sentiment analysis, basic arithmetic, or also programming.
But some things we cannot do this way. We do not have answers to current world knowledge because the models were trained on the internet or on the data in the internet from a time ago. If you ask ChatGPT, ChatGPT will not answer you where this conference is happening today. It cannot also do long arithmetic like this.
And as much as we like to know the answer of this third question, the winner of the 2023 World Cup, it cannot predict this. Some of us may be able to. There are still many limitations with large language models. And I'll just run through these briefly. So you've all probably heard about hallucination of models. And here is one example.
So if we have this query, how can I tokenize text in Matlab, it will give an answer. And that function that it mentions in that answer does not exist. So if you need help with your software, talk to us. We have technical support human beings who will answer your questions accurately. The output is also not always reliable. If you're trying to do something like translate the Matlab code that is given here into C code, it gives a result that looks good but unfortunately does not compile. You will not have this problem if you use Matlab coder.
OK, so those are some examples of where large language models are not good at. But let's come back to what they're great at. Once again, they're great at understanding natural language. And they're great at generating text and code and analyzing data. So they can help you solve specific problems. You can generate text, as all of those of you who have done that for generating emails, performance reviews, et cetera, et cetera, et cetera. They can do that very well.
They are great at classifying text. They are great at summarizing text or documents. And they are great at retrieving knowledge or retrieving information from various different publications and summarizing that, bringing that together, so that you have the information that you require to do your work.
So what are some approaches that you can use to leverage LLMs? One is straightforward prompt engineering, so query the LLM in various different ways. The second is retrieval augmented generation, also known as RAG. The third is actually fine tune the LLM for your actual usage. And I'll just talk about these briefly in the next few minutes. So let's go into prompt engineering. Sometimes it's just easy to use the chat interface.
So you ask the model a question like generate Matlab code for the sine values between A and B. And notice that we mentioned the code should be optimized and vectorized. So that's where you get optimized and vectorized code like this. So this process of iteratively querying the model and ensuring that you're getting the right results is called prompt engineering. And that refines your inputs to the AI model and refines the outputs that it's giving to you.
Part of this is you may want to do something with ChatGPT that may actually take a lot of time. For example, if you want to summarize a lot of documents, right now you need to copy paste each document individually into the interface, ask ChatGPT for the summary, and copy and paste the results back. This is time consuming and tedious. So what you can do is actually read each file inside a Matlab loop, request ChatGPT API-- we have an API-- for a summary for each document, and then automatically save each summary back to the file. So this is one way you can use ChatGPT. So especially where there's repetition in the work, there is a potential for using Matlab together with ChatGPT using the ChatGPT API.
Retrieval augmented generation, RAG. So let's say you have a certain query. And this is a query on, let's say from a fleet of vehicles. Model XYZ has a faulty whatever. What should I do? The information for this is not in ChatGPT's neural network. The answer to this is probably lying in your maintenance documents. Of course, you can upload now all these documents into the large language model. But that is potentially you'll have IP issues in doing so, and also it's not very effective or not very efficient.
What you can do is retrieve the relevant reports using text analytics toolbox and then feed these together with your query to the large language model to then get a very specific or focused answer. So if you require tailored answers to questions with your own data, this is one method that you can use, RAG.
Finally, you can fine tune large language models for your own use. Within Matlab we are shipping a few transformer models. There's BERT. For example, it's been in Matlab since release 2020A and you can fine tune this model specifically for your own requirements. Though just a caveat here is that this fine tuning can take a lot of resources, depending on how efficient you want the model to be. But you have that capability, as well.
And finally, I just introduce you to something new that we're working on. It's an early view, an early pilot. It's called ChatGPT playground. So what is it? It's actually a web app in which, on the left hand side, you can query a large language model with various different queries, for example, to generate code. The model will generate the code and you can directly plug that code into something that looks like the Live Editor and run that code directly in this interface. This is going to be available. Partly it's already available for some. If you want early access, please scan the QR code, fill out the form, and you'll get early access to this ChatGPT playground.
With that, thank you very much.