Decision level fusion from parallel classifier

I have three sets of features for predicting the same outputs (normal or abnormal). However, rather than training everything at once by concatenating the features, I'd like to train them independently and then fuse the results (level fusion).

How can I perform it in random forest/kNN/any other classification?

Please assist me by providing a brief example or description.

[MAHMOUD EID]

Have you found an answer?

[VINOTHINI S]

Yes. majority voting based decision we can use

[MAHMOUD EID]

please share an example how to fuse the decisions ?

Answer 1

[Hari]

Hi Vinothini,

I understand that you have three sets of features for predicting the same outputs and you want to train them independently and then fuse the results using level fusion in a classification algorithm such as random forest or "kNN".

To achieve this, you can use “fitcknn” function for “kNN” or “TreeBagger” function for random forest from the Statistics and Machine Learning Toolbox. Depending on your fusion method you can use appropriate MATLAB functions such as “mode” or perform element-wise operations for averaging.

Here's an example using random forest classifiers:

1. Split your dataset into three subsets, each corresponding to one set of features.

% Split the dataset into subsets based on feature sets
subset1 = yourData(:, featuresSet1);
subset2 = yourData(:, featuresSet2);
subset3 = yourData(:, featuresSet3);

2. Train a separate classifier (e.g., random forest or “kNN”) on each subset of features.

% Train separate random forest classifiers on each subset
rf1 = TreeBagger(numTrees, subset1, labels);
rf2 = TreeBagger(numTrees, subset2, labels);
rf3 = TreeBagger(numTrees, subset3, labels);

3. Obtain predictions from each classifier for new data points.

% Obtain predictions from each classifier
predictions1 = rf1.predict(newData);
predictions2 = rf2.predict(newData);
predictions3 = rf3.predict(newData);

4. Combine the predictions using a fusion method, such as majority voting or weighted averaging, to obtain the final prediction.

% Perform fusion (e.g., majority voting)
finalPrediction = mode([predictions1, predictions2, predictions3], 2);

By following the above steps, you can train separate classifiers on each subset of features and fuse the results using level fusion. Remember to choose an appropriate fusion method based on the nature of your data and the specific classification task.

Refer to the documentation of “TreeBagger” and “classificationkNN” algorithms for more details on training and prediction methods.

Random Forest: https://www.mathworks.com/help/stats/treebagger.html

k-Nearest Neighbors (kNN): https://www.mathworks.com/help/stats/classificationknn.html

Hope this helps!