Resultados de

I'm working on training neural networks without backpropagation / automatic differentiation, using locally derived analytic forms of update rules. Given that this allows a direct formula to be derived for the update rule, it removes alot of the overhead that is otherwise required from automatic differentiation.
However, matlab's functionalities for neural networks are currently solely based around backpropagation and automatic differentiation, such as the dlgradient function and requiring everything to be dlarrays during training.
I have two main requests, specifically for functions that perform a single operation within a single layer of a neural network, such as "dlconv", "fullyconnect", "maxpool", "avgpool", "relu", etc:
  • these functions should also allow normal gpuArray data instead of requiring everything to be dlarrays.
  • these functions are currently designed to only perform the forward pass. I request that these also be designed to perform the backward pass if user requests. There can be another input user flag that can be "forward" (default) or "backward", and then the function should have all the necessary inputs to perform that operation (e.g. for "avgpool" forward pass it only needs the avgpool input data and the avgpool parameters, but for the "avgpool" backward pass it needs the deriviative w.r.t. the avgpool output data, the avgpool parameters, and the original data dimensions). I know that there is a maxunpool function that achieves this for maxpool, but it has significant issues when trying to use it this way instead of by backpropagation in a dlgradient type layer, see (https://www.mathworks.com/matlabcentral/answers/2179587-making-a-custom-way-to-train-cnns-and-i-am-noticing-that-avgpool-is-significantly-faster-than-maxpo?s_tid=srchtitle).
I don't know how many people would benefit from this feature, and someone could always spend their time creating these functionalities themselves by matlab scripts, cuDNN mex, etc., but regardless it would be nice for matlab to have this allowable for more customizable neural net training.
JH
JH
Última actividad el 14 de Mayo de 2025

outdated transformer models

Why is RoBERTa not available as a pretrained model? It is superior to BERT in many fields and has become more popular in the literature. For faster inference, you should offer DistilBERT, which is more modern than BERT but smaller/faster. The respository hasn't been updated in two years, which is a lifetime in the field of deep learning.
https://github.com/matlab-deep-learning/transformer-models
The MATLAB Online Training Suite has been updated in the areas of Deep Learning and traditional Machine Learning! These are great self-paced courses that can get you from zero to hero pretty quickly.
Deep Learning Onramp (Free to everyone!) has been updated to use the dlnetwork workflow, such as the trainnet function, which became the preferred method for creating and training deep networks in R2024a.
Machine Learning Onramp (Free to everyone) has been updated with the following enhancements:
  • Content streamlined to reduce the focus on data processing and feature extraction, and emphasize the machine learning workflow.
  • Course example simplified by using a sample of the original data.
  • Classification Learner used in the course where appropriate.
The rest of the updates are for subscribers to the full Online Training Suite
The Deep Learning Techniques in MATLAB for Image Applications learning path teaches skills to tackle a variety of image applications. It is made up of the following four short courses:
Two more deep learning short courses are also available:
The Machine Learning Techniques in MATLAB learning path helps learners build their traditional machine learning skill set:
MathWorks just released three new courses on Coursera liseted below. If you work with image or video data and are wanting to incorporate deep learning techniques into your workflow, this is a great opporutnity. The course creators monitor the discussion forums, so you can ask questions and get feedback on your work. Below are links to the three courses and a quick description of a project you'll complete in each.
  1. Introduction to Computer Vision for Deep Learning. You'll train a classifier to classify images of people signing the American Sign Language alphabet.
  2. Deep Learning for Object Detection. Move from just classification to finding object locations. You'll train a model to find different types of parking available on the MathWorks campus.
  3. Advanced Deep Learning Techniques for Computer Vision. You'll train anomaly detection models for medical images and use AI-assisted labeling auto label images.
christian
christian
Última actividad el 6 de Jul. de 2023

CNN for text classification

Hello! I am working on a project involving the use of CNNs for text classification. I found a very clear example on MathWorks that demonstrates how to transform text using an encoding process and provide it as input to a neural network. One of the initial layers of the network is the word embedding layer, which is responsible for capturing the semantic relationships between words. I'm wondering how this layer can work without directly using the words, but instead working with a numerical representation of them.
Thank you very much in advance to anyone who will reply to me.

Nowadays, many instructors are integrating AI to their courses. In a distance learning setting, the hardware students use to train their models vary. Training time of the deep learning models can be shortened with a pool of GPUs, CPUs or a pool of CPUs and GPUs locally or in the cloud. Accuracy of the results can depend on the hyperparameters used to train the models.

In MATLAB, Experiment Manager (introduced in R2020a) makes it easy to train networks with various hyperparameters and compare the results. Different models can be run in parallel by clicking on “Use Parallel” button in Experiment Manager App. But what if your computer has multiple CPU cores and a GPU? Would you be able to use Experiment Manager with a pool of CPUs and a GPU? The answer is “yes”. For example, my computer has 1 NVIDIA GPU and an 8 core CPU. To use all these computational resources, I typed these lines in my command window in MATLAB:

parpool(9);
spmd
if labindex==1
gpuDevice(1); %select GPU on worker 1
else
gpuDevice([]); %deselect GPU on remaining workers
end

Then, I clicked on “Use Parallel” button in Experiment Manager and used a modified setup function in Experiment Manager to change the execution environment from CPU to GPU:

if isempty(parallel.gpu.GPUDeviceManager.instance.SelectedDevice)
options=trainingOptions(~,"ExecutionEnvironment",'cpu');
else
options=trainingOptions(~,"ExecutionEnvironment",'gpu');
end

Default training options automatically use an NVIDIA GPU if there is one, and specific hardware can be selected using “ExecutionEnvironment” in the trainingOptions.

If you like to learn more about the fundamentals of parallel computing, check out “Parallel Computing Fundamentals” on our documentation and gain some hands-on experience with Parallel Computing through "Parallel Computing Hands-On Workshop" .

Boby S
Boby S
Última actividad el 13 de Abr. de 2020

Mouse video tracking and deep learning

Hi I want to track a animal in my recorded video. I tried computer vision toolbox but it is not very accurate for this type of tracking. The recording is from top and the animal runs in a maze. I want to track body and head. The next step is classifying the movements in video using deep learning but again we do not have a trained network.