what is the best strategy/toolbox to filter out / smoothen random large spikes on data.

Hi,

I have a specific physical value that is measured by multiple sensors in an extremely disruptive atmosphere. These signals are then to be filtered and combined until the combined signal is as closely represented to the original as possible. As of now, using a moving average (and other filters), the final combined signal I get is very close to the actual signal, except at some specific points. As shows in the figure below

The blue line is the actual physical value and the red and yellow line are the filtered components through 2 designs I have created till now.

What would be the best toolbox / strategy to filter out these remaining random spikes?

Any help would be appreciated.

Thanks.

Answer 1

[John D'Errico]

This is not a MATLAB question, since there is no such "best" tool that will solve your problems.

You don't have isolated spikes, in an otherwise simple curve. You have SOME isolated spikes, and then you have entire regions where there is crap happening. The point is, this curve does not have any type of homogeneous noise structure. Instead, it appears that in some regions, the noise variance just gets huge.

So if you want to remove the spike-ish noise, I'd just look for regions where the noise is large, then do more smoothing in that area, or drop the points completely in those regions, interpolating from the neighbors.

[Craig Kavan Pinto]

Yes, I can see that. The reason i ask for a toolbox/stratergy is because I would require to buy the toolbox before I could use it to analyse/filter the data at those points.

Answer 2

[Star Strider]

Probably the easiest is a Savitzky-Golay filter (the sgolayfilt function).

If you want to be less empirical, take the fft (link) of your data, and design an appropriate time-domain filter (lowpass or bandpass) to eliminate the high-frequency noise. The easiest way to do that is with the designfilt function. Use the freqz function to be certain the filter does what you want it to, and the filtfilt function to do the actual filtering.

All of the functions (except fft) are in the Signal Processing Toolbox.

[Craig Kavan Pinto]

Thanks, I shall try this after getting the toolbox, and get back to you after that.

[Star Strider]

My pleasure.

The Wavelet Toolbox is also an option for de-noising signals. This works best for broadband noise. The approach you use depends on what you want to do and how often you in intend to use the other Toolbox functions.

I have both Toolboxes, although limited recent experience with wavelets. If you attach your data as a ‘.mat’ file, I can experiment to see what strategy might work best.