activation of different states in neural network

hello i want to ask if it is possible to create a neural network in mat lab that is able to have at least 4 states which gets activated randomly based on what input we may give.Suppose I have states A,B,C and D. And suppose If as input i am giving images sequentially some states gets activated randomly based on what type of image input it is or even handwriting input.

Answer 1

[Greg Heath]

In that case you should use Gaussian hidden nodes

 help radbas
 doc radbas

so that one or more will recognize only a particular type of input. Clustering the data beforehand to define the Gaussians will help.

Then after you have determined the feature recognition layer, train a linear layer from the outputs that will yield the final answer.

Learning is not as effortless as using fitnet or patternnet. However, see if you can use NEWRB, NEWRBE or NEWGRNN.

I have spent many years designing these types of nets for the military. Unfortunately I was not able to take any of my work when I retired. So...

Good Luck

and

Thank you for formally accepting my answer

Greg

[Newman]

Dear sir,by clustering r u saying clustering using Gaussian mixture models?I am using a very large data set of almost a million images.How to apply clustering there effectively.It will take significant amount of space and time I think.

You are trying to say that i need to cluster the dataset where each cluster defines a specific property of an image (beauty,dull, etc).Can these be done specifically or clustering is done randomly?(I mean the properties of a certain clusters are defined randomly by the algorithm and not by me as it is in the brackets above.)

Secondly the clustered data is feed to the first hidden layer which is a feature recognition layer right? After that the out put of this layer will go as an input to the linear layer that will train it to recognize faces in the data set.

But I am unable to understand how can the outputs of the feature layer be used for recognition in the linear.As the feature layer only gives the states that get activated as output and not as images as input to be trained for the linear layer.

please tell on how to proceed towards the code part as it looks kind of complex.

[Greg Heath]

% activation of different states in neural network % Asked by AK on 22 Apr 2016 % http://www.mathworks.com/matlabcentral/answers/280508

% Dear sir,by clustering r u saying clustering using Gaussian mixture % models?

Yes

% I am using a very large data set of almost a million images. % How to apply clustering there effectively.It will take significant % amount of space and time I think.

The most relevant numbers are the dimensions of your input and output spaces. Typically, an I-dimensional input space or O-dimen- sional output space can be be adequately represented by 30 or more random or equally spaced vectors per dimension.

% You are trying to say that i need to cluster the dataset where each % cluster defines a specific property of an image (beauty,dull, etc).

That would be ideal. However, features will be correlated. So it is not quite as simple as that. You need enough clusters to adequately represent the combination of features that interest you.

% Can these be done specifically or clustering is done randomly? % (I mean the properties of a certain clusters are defined randomly % by the algorithm and not by me as it is in the brackets above.)

It depends on number of data points, dimensionality, and a priori knowledge of what the features are. For example, you may just want to create a sample of vectors that define your features and use them to initialize a clustering of a significantly sized subset of your million examples

% Secondly the clustered data is feed to the first hidden layer which % is a feature recognition layer right? After that the out put of this % layer will go as an input to the linear layer that will train it to % recognize faces in the data set.

No! The cluster centers form the hidden layer and can represent elliptically shaped clusters. You'll have to search Google to find references on elliptical basis function NNs. However, I would first try radial basis functions as used in MATLAB's NEWRB, NEWRBE and NEWGRNN.

% But I am unable to understand how can the outputs of the feature % layer be used for recognition in the linear. As the feature layer only % gives the states that get activated as output and not as images as % input to be trained for the linear layer.

I'm pretty sure that, like MLPs, radial basis function NNs are universal approximators. Although the clustering of the hidden layer might be unsupervised or semi-supervised, the output layer is learned via supervised learning.

I'm not sure if there are MATLAB functions that can handle the following (so you might have to write some of your own). I would start with trying to solve a c-classes classification problem with the simple radial basis function classifier NEWRBE and {0,1} c-dimensional unit vectors as targets. Then quickly move on to using NEWRB which automatically chooses a subset of input vectors to be centers of hidden node radial basis functions.

The problem is that the basis functions are radial and not elliptical. However, If you use RADBAS in PATTERNNET you will have elliptical basis functions in the hidden layer. The shape of the ellipses are learned .. which means you will have no control over their shape. Nevertheless you should get the correct answer.

Be careful of what I have written above. Much of it was developed in Fortran eons ago and I was not allowed to take it with me when I retired. Also, there are some things which might not be readily available in MATLAB.

Hope this helps.

Greg

Answer 2

[Greg Heath]

That makes no sense to me. Outputs are not activated randomly. For a given net there is a nonrandom relationship between input and output. If the input is random, the output may seem random. However, it has a non-random relationship to the input.

Hope this helps.

Thank you for formally accepting my answer

Greg

[Newman]

Dear sir these are not outputs but some parts of the neural network architecture which acts as an intermediate state before the net work gives it outputs. for eg if an image of a flower is beautiful the state A should get activated indicating the beauty of the flower