The noise enters the system mostly in the receiver RF, which is between the two matched filters. Remember that the 3dB point of the matched filter is at the symbol rate, i.e., there is some rolloff happening within the symbol rate spectrum. So usually what is done is to use the symbol rate as the ENBW, regardless of the % EBW rolloff, and I've never had trouble using this, even calibrating systems down to a few tenths of a dB to theoretical performance.
If you really want to get anal about it, though, you can do an assessment of the filter shape with a density evaluation at samples in the frequency domain and compute the overall ENBW. You have to do this when building an SNR test set with a noise generator if you are measuring the applied noise and power separately with a power meter in order to compute the noise psd from the total measurement. It's kind of a pain, but can be done. It really isn't needed if you're just doing sims.
Thank you very much Eric. I would contend that the accuracy will depend on the roll-off factor. Would you agree to this statement?
In your second paragraph, you mention "measuring the applied noise and power", did you mean signal (what kind of signal) power? Also, how are you able to take the measurements separately?
In practice, no, the roll-off doesn't matter because with raised-cosine matched filters the 3dB point is always at the symbol rate. This is where the inflection of the cosine function is, so if the EBW is larger, it's also removing more of the in-band energy to move it out there. I hope that makes sense.
In a calibrated SNR test fixture it is typical to generate noise separate from the signal under test. That noise power has to be measured, separate from the signal power, in order to measures SNR or Eb/No or whatever. If you want an accurate noise spectral density measurement with a power meter measuring the noise power, then you need to account for the spectral shape of the noise.
Another way to do it is just stick a spectrum analyzer on it and measure the psd of the signal and the noise separately. There's more than one way to get there, but you do need to mind the details.
One approach is to compute the area under the magnitude-squared curve of the filter frequency response. If this value is A, then the ENBW is also A. A common rule of thumb is that ENBW is the 3-dB bandwidth of the filter.
Thank you Kishan. I ended up using MATLAB's built-in function enbw() which I believe is doing exactly what you suggested. For window W:
Let us know how it differs from the symbol rate.
Pretty much equal to a few hundredth of a dB