Issues with ML Pattern Recognition After Bandpass Filtering

Started by TarekSAR 11 months ago4 replieslatest reply 11 months ago159 views
Hello everyone,

We've been working on a machine learning project for pattern recognition, using time-domain features such as kurtosis, mean, standard deviation, variance, skewness, and peak-to-peak values.


Initially, we trained our data after applying a high-pass filter at 1 kHz. The results were satisfactory.
Upon performing a spectral analysis last week, we discovered that our region of interest lay between 1 kHz and 3 kHz.
When testing our pattern recognition system this week, the model's performance deteriorated significantly. Analyzing the data revealed a strong signal component at 8 kHz.

Steps Taken:

We decided to apply a bandpass filter between 1 kHz and 3 kHz to focus on our identified region of interest, expecting our time-domain features to be more relevant.
We trained a new model using the bandpass-filtered data.
However, the model's performance in recognizing patterns was not up to par.
As an additional experiment:

We applied the 1 kHz to 3 kHz bandpass filter on the dataset originally trained with 1 kHz high-pass filtering.
Yet again, we faced recognition performance issues.
We're somewhat puzzled as to why our ML system is underperforming after these filtering operations. Any insights or suggestions would be highly appreciated.
[ - ]
Reply by neiroberAugust 9, 2023


It would help if you could provide more info about your data:  how many samples, sample rate, and a plot of the (unfiltered) data.  A spectral plot would also be helpful.  Is your bandpass filter FIR or IIR?  What are the filter coefficients?  Also, a plot of the filter's magnitude and group delay response would be helpful.



[ - ]
Reply by MichaelRWAugust 9, 2023

What happened to the 8 kHz region?

[ - ]
Reply by omersayliAugust 9, 2023


First we don't know the properties of your signal. Your parameters may or may not capture relevant information. The frequency range of interest is important, you have to be sure that your bandpass filter range includes the signal. 

Finally your pattern recognition/machine learning algorithm has a role, how you trained it and what algorithm you use are important. 

[ - ]
Reply by dudelsoundAugust 9, 2023

From the little information about the signal you supply I would simply guess that your assumption about the region of interest is wrong - obviously the low-pass throughs away relevant discrimination information, so maybe the component at 8kHz does significantly contribute information.