version 1.0.0.0 (3.52 KB) by
Alireza

RBF Neural Networks (center and distribution of activation functions are selected using K-means)

A RBFNN is an artificial neural network that uses radial basis functions as activation functions. Figure shows the structure of the RBFNN. The RBFNN is three layered feed-forward neural network. The first layer is linear and only distributes the input signal, while the next layer is nonlinear and uses Gaussian functions. The third layer linearly combines the Gaussian outputs. Only the tap weights between the hidden layer and the output layer are modified during training.

RBFNN have 5 parameters for optimization:

1- The weights between the hidden layer and the output layer.

2- The activation function.

3- The center of activation functions.

4- The distribution of center of activation functions.

5- The number of hidden neurons.

The weights between the hidden layer and the output layer are calculated by using Moore-Penrose generalized pseudo-inverse. This algorithm overcomes many issues in traditional gradient algorithms such as stopping criterion, learning rate, number of epochs and local minima. Due to its shorter training time and generalization ability, it is suitable for real-time applications.

The radial basis function selected is usually a Gaussian kernel for pattern recognition application.

Generally the center and distribution of activation functions should have characteristic similar to data. Here, the center and width of Gaussians are selected using Kmeans clustering algorithm.

Based on universal approximation theory center and distribution of activation functions are not deterministic if the numbers of hidden neurons being sufficient enough, one can say that the single hidden layer feed-forward network with sufficient number of hidden neurons can approximate any function to any arbitrary level of accuracy.

---------------------------------------------------------------------------------

This code is used in the following papers:

1. A. Asvadi, M. Karami, Y. Baleghi, “Efficient Object Tracking Using Optimized K-means Segmentation and Radial Basis Function Neural Networks,” International Journal of Information and Communication Technology Research (IJICT), vol. 4, no. 1, pp. 29-39, December 2011.

2. A. Asvadi, M. Karami, Y. Baleghi, H. Seyyedi, “Improved Object Tracking Using Radial Basis Function Neural Networks,” in: Proceedings of 7th Iranian Machine Vision and Image Processing (MVIP2011), Tehran, Iran, November 2011.

---------------------------------------------------------------------------------

"mvnrnd" from "Statistics and Machine Learning Toolbox" is used for generating sample data points.

Alireza (2020). Radial Basis Function Neural Networks (with parameter selection using K-means) (https://www.mathworks.com/matlabcentral/fileexchange/52580-radial-basis-function-neural-networks-with-parameter-selection-using-k-means), MATLAB Central File Exchange. Retrieved .

Created with
R2011a

Compatible with any release

**Inspired by:**
K-means clustering

**Inspired:**
KM-RBF Tracker, RBF Neural Networks with random selection of parameters

Find the treasures in MATLAB Central and discover how the community can help you!

Start Hunting!Create scripts with code, output, and formatted text in a single executable document.

ali zigmondVery helpful and concise manner of implementing an RBF neural network.

Better to be aware of the comment of "Alan Yu" on "9 Dec 2017" that is listed below.

tina niniHow can I use this code "Radial Basis Function neural networks with parameter selection using K-means" for facial recognition ? I want to train the network first with a set of trainnind data then simulate it with a set of test data ? please help

If this code does not work for that, can anyone tell me what neural network with radial basis function can work for my problem? thank you

Dzakiyyah Salma Damayantiabdulkader helwanDear Mr Alireza ,

how can i use your code "Radial Basis Function Neural Networks with parameter selection using K-means" for regression task ?

Regards

Alan YuThere is a problem in rbfn_test.m. If the values of output Y are close, then it would produce the same classification(wrong result).

My code to fix the problem is as follows.

function Y = rbfn_test(F, W, K, MU, SIGMA)

%RBFN_TEST Summary of this function goes here

% Detailed explanation goes here

%% Test RBF

N = size(F,1); % n = Number of Pixels

H = zeros(N,K);

for k = 1:K

h = RBFKernel(F,MU,SIGMA,N,k);

H(:,k) = h;

end

Y = H*W; % Output

[M,I] = max(Y,[],2);

IV = ind2vec(I');

Y = IV';

end

Besides, you use the diagonal matrices to represent the kernel widths SIGMA(rbfn_train.m). Is there any theory behind this formula?

Alan YuGood work!

Can you give more explanation how to find kernel function of RBF? That is, give more description on RBFKernel.m.

Felipe GiuntiniI'm learning Matlab and RBF now. My question is: how can I use this code with data from a csv, consisting of three columns?

Thanks!

Alireza@pradeep s v it is due to random initialization of the k-means clustering algorithm

pradeep s vaccuracy is varying for every time running without any change

usama mirsir this code give me an eror in this line [data1,Y1,Xt1]=generate_two_moons(rad,width,dist,(num_samp./2),seed,0)

random_data=[data1,Y1]';random_seq = randperm(num_samp);

it said Error in Untitled (line 15)

[data1,Y1,Xt1]=generate_two_moons(rad,width,dist,(num_samp./2),seed,0)

please solve this issue

Manjari GautamFadya jawnum_tr = 3000;

num_te = 500;

num_hd = 20;

num_ou = 1;

num_in = 2;

num_hd = 30;

num_samp = num_tr+num_te;

width = 4;

dist = -4;

seed=3000;

rand('seed',seed);

[data1,Y1,Xt1]=half double_moons(rad,width,dist,(num_samp./2),)

random_data=[data1,Y1]';random_seq = randperm(num_samp);random_data = random_data(:,random_seq);

X = random_data(1:2,1:num_tr)';

[indx,centers] = kmeans(X,num_hd);

pos_vector = indx';

c = centers';

%Calculating the variance for each center

for i = 1 : num_hd;

cluster_number=(i);

variance = 0;

counter2=0;

for j = 1 : num_tr;

if cluster_number==pos_vector(j)

sssss = sqrt(sum((random_data(1:2,j)'-c(:,i)').^2))

variance = sssss + variance

counter2=counter2+1;

end

end

sigma(i) = variance/counter2;

end

%Parameters For RLS

w = ones(num_ou,num_hd);

lambda=0.9

P= (lambda)^(-1)*eye(num_hd);

spread = sigma;

IN = random_data(1:2,1:num_tr);

T = random_data(3,1:num_tr);

P= (lambda)^(-1)*eye(num_hd);

w = ones(num_ou,num_hd);

num_train_min=10;

for epoch =1:50;

for m = 1:(num_tr-num_tr./10);

for i = 1:num_hd;

g(i,:) = exp(-(x-c(:,i))'*(x-c(:,i))/(2*spread(i)^2));

end

alpha = d - w*g;

w_diffrence = ((P*g)*alpha' )';

if (norm(w_diffrence) < 0.01) && (m >= num_train_min)

break;

end

w = w + w_diffrence ;

P=P-((P*g/(g'*P*g))*g'*P) ;

end

end

Fadya jaw%k-mean/rls

num_tr = 3600;

num_te = 400;

num_hd = 20;

num_ou = 1;

num_in = 2;

num_hd = 30;

num_samp = num_tr+num_te;

rad = 10;

width = 6;

dist = -6;

seed=30000000;

rand('seed',seed);

[data1,Y1,Xt1]=generate_two_moons(rad,width,dist,(num_samp./2),seed,0)

random_data=[data1,Y1]';random_seq = randperm(num_samp);random_data = random_data(:,random_seq);

X = random_data(1:2,1:num_tr)';

[indx,centers] = kmeans(X,num_hd);

pos_vector = indx';

c = centers';

%Calculating the variance for each center

for i = 1 : num_hd;

cluster_number=(i);

variance = 0;

counter2=0;

for j = 1 : num_tr;

if cluster_number==pos_vector(j)

sssss = sqrt(sum((random_data(1:2,j)'-c(:,i)').^2))

variance = sssss + variance

counter2=counter2+1;

end

end

sigma(i) = variance/counter2;

end

%Parameters For RLS

w = ones(num_ou,num_hd);

lambda=0.9

P= (lambda)^(-1)*eye(num_hd);

spread = sigma;

IN = random_data(1:2,1:num_tr);

T = random_data(3,1:num_tr);

P= (lambda)^(-1)*eye(num_hd);

w = ones(num_ou,num_hd);

num_train_min=10;

for epoch =1:50;

for m = 1:(num_tr-num_tr./10);

for i = 1:num_hd;

g(i,:) = exp(-(x-c(:,i))'*(x-c(:,i))/(2*spread(i)^2));

end

alpha = d - w*g;

w_diffrence = ((P*g)*alpha' )';

if (norm(w_diffrence) < 0.01) && (m >= num_train_min)

break;

end

w = w + w_diffrence ;

P=P-((P*g/(g'*P*g))*g'*P) ;

end

end

AlirezaThanks Charles for mentioning that. Yes, mvnrnd is used for generating sample data points for classification.

Charles HarrisonYou may want to mention that your set of functions requires mvnrnd which is part of the Statistics and Machine Learning Toolbox.