Signal.welch function: f and pxx outcome is 0
Question:
I have an array of BOLD activity of length 240. A recording took place every 2 seconds (you can download a part of the data here (the file can be opend normally with a .txt editor). I want to analyse this time series with the signal.welch function.
f, pxx = signal.welch(data, fs=0.5, window='hanning', nperseg=50, noverlap=25, scaling='density', average='mean')
It gives me the following error
ValueError: noverlap must be less than nperseg.
When I set noverlap = None , no error shows up, but f equals 0 and pxx is an array of 0s.
Thank you very much for your suggestions!!
Answers:
If I read the data file with, for example,
data = np.loadtxt('rest1_LeftInsula.1D')
data will be a 1-d numpy array with shape (240,), and the call
f, pxx = signal.welch(data, fs=0.5, window='hanning', nperseg=50, noverlap=25, scaling='density', average='mean')
works with no errors.
If I reshape data to be an array with shape (240, 1), and pass that to welch (which can be done by indexing it as data[:, None] or with the reshape method data.reshape((240, 1))), I get the same error that you reported:
In [11]: f, pxx = signal.welch(data[:,None], fs=0.5, window='hanning', nperseg=5, 0, noverlap=25, scaling='density', average='mean')
[...]/scipy/signal/spectral.py:1964: UserWarning: nperseg = 50 is greater than input length = 1, using nperseg = 1
warnings.warn('nperseg = {0:d} is greater than input length '
---------------------------------------------------------------------------
ValueError Traceback (most recent call last)
<ipython-input-11-0e5235350cfd> in <module>
----> 1 f, pxx = signal.welch(data[:,None], fs=0.5, window='hanning', nperseg=50, noverlap=25, scaling='density', average='mean')
[...]/scipy/signal/spectral.py in welch(x, fs, window, nperseg, noverlap, nfft, detrend, return_onesided, scaling, axis, average)
446
447 """
--> 448 freqs, Pxx = csd(x, x, fs=fs, window=window, nperseg=nperseg,
449 noverlap=noverlap, nfft=nfft, detrend=detrend,
450 return_onesided=return_onesided, scaling=scaling,
[...]/scipy/signal/spectral.py in csd(x, y, fs, window, nperseg, noverlap, nfft, detrend, return_onesided, scaling, axis, average)
580
581 """
--> 582 freqs, _, Pxy = _spectral_helper(x, y, fs, window, nperseg, noverlap, nfft,
583 detrend, return_onesided, scaling, axis,
584 mode='psd')
[...]/scipy/signal/spectral.py in _spectral_helper(x, y, fs, window, nperseg, noverlap, nfft, detrend, return_onesided, scaling, axis, mode, boundary, padded)
1756 noverlap = int(noverlap)
1757 if noverlap >= nperseg:
-> 1758 raise ValueError('noverlap must be less than nperseg.')
1759 nstep = nperseg - noverlap
1760
ValueError: noverlap must be less than nperseg.
The problem is that, by default, welch is applied along the last axis of the input array. If that array has shape (240, 1), welch attempts to apply the calculation to each "row" of the 2-d array. But each row has length 1, which is too small for the given values of nperseg and noverlap, and that leads to the (somewhat cryptic) error.
You haven’t shown how you read the file and created data, but I suspect it is creating an array (or some other data structure, such as a Pandas DataFrame) that has shape (240, 1).
To fix this, you can "flatten" the data into a 1-d array (e.g. pass data.ravel() to welch), or pass the argument axis=0 to welch to tell it to act along the first dimension instead of the last. If you do the latter, be aware that the shape of pxx will be (26, 1), instead of the shape (26,) that you get when data has shape (240,).
I have an array of BOLD activity of length 240. A recording took place every 2 seconds (you can download a part of the data here (the file can be opend normally with a .txt editor). I want to analyse this time series with the signal.welch function.
f, pxx = signal.welch(data, fs=0.5, window='hanning', nperseg=50, noverlap=25, scaling='density', average='mean')
It gives me the following error
ValueError: noverlap must be less than nperseg.
When I set noverlap = None , no error shows up, but f equals 0 and pxx is an array of 0s.
Thank you very much for your suggestions!!
If I read the data file with, for example,
data = np.loadtxt('rest1_LeftInsula.1D')
data will be a 1-d numpy array with shape (240,), and the call
f, pxx = signal.welch(data, fs=0.5, window='hanning', nperseg=50, noverlap=25, scaling='density', average='mean')
works with no errors.
If I reshape data to be an array with shape (240, 1), and pass that to welch (which can be done by indexing it as data[:, None] or with the reshape method data.reshape((240, 1))), I get the same error that you reported:
In [11]: f, pxx = signal.welch(data[:,None], fs=0.5, window='hanning', nperseg=5, 0, noverlap=25, scaling='density', average='mean')
[...]/scipy/signal/spectral.py:1964: UserWarning: nperseg = 50 is greater than input length = 1, using nperseg = 1
warnings.warn('nperseg = {0:d} is greater than input length '
---------------------------------------------------------------------------
ValueError Traceback (most recent call last)
<ipython-input-11-0e5235350cfd> in <module>
----> 1 f, pxx = signal.welch(data[:,None], fs=0.5, window='hanning', nperseg=50, noverlap=25, scaling='density', average='mean')
[...]/scipy/signal/spectral.py in welch(x, fs, window, nperseg, noverlap, nfft, detrend, return_onesided, scaling, axis, average)
446
447 """
--> 448 freqs, Pxx = csd(x, x, fs=fs, window=window, nperseg=nperseg,
449 noverlap=noverlap, nfft=nfft, detrend=detrend,
450 return_onesided=return_onesided, scaling=scaling,
[...]/scipy/signal/spectral.py in csd(x, y, fs, window, nperseg, noverlap, nfft, detrend, return_onesided, scaling, axis, average)
580
581 """
--> 582 freqs, _, Pxy = _spectral_helper(x, y, fs, window, nperseg, noverlap, nfft,
583 detrend, return_onesided, scaling, axis,
584 mode='psd')
[...]/scipy/signal/spectral.py in _spectral_helper(x, y, fs, window, nperseg, noverlap, nfft, detrend, return_onesided, scaling, axis, mode, boundary, padded)
1756 noverlap = int(noverlap)
1757 if noverlap >= nperseg:
-> 1758 raise ValueError('noverlap must be less than nperseg.')
1759 nstep = nperseg - noverlap
1760
ValueError: noverlap must be less than nperseg.
The problem is that, by default, welch is applied along the last axis of the input array. If that array has shape (240, 1), welch attempts to apply the calculation to each "row" of the 2-d array. But each row has length 1, which is too small for the given values of nperseg and noverlap, and that leads to the (somewhat cryptic) error.
You haven’t shown how you read the file and created data, but I suspect it is creating an array (or some other data structure, such as a Pandas DataFrame) that has shape (240, 1).
To fix this, you can "flatten" the data into a 1-d array (e.g. pass data.ravel() to welch), or pass the argument axis=0 to welch to tell it to act along the first dimension instead of the last. If you do the latter, be aware that the shape of pxx will be (26, 1), instead of the shape (26,) that you get when data has shape (240,).