Time Series Augmentation
Question:
How to apply “Magnitude Warping” data augmentation on time series dataset? Is there any sample code in pytorch?
Also can we apply the image augmentation techniques like RandomGrayscale, GaussianBlur and Normalize to time series dataset?
Trying with the below code for magnitude Warping on Swelltrain dataset of size (1887, 95):
sigma = 0.2
knot = 4
def GenerateRandomCurves(X, sigma=0.2, knot=4):
xx = (np.ones((X.shape[1],1))*(np.arange(0,X.shape[0], (X.shape[0]-1)/(knot+1)))).transpose()
yy = np.random.normal(loc=1.0, scale=sigma, size=(knot+2, X.shape[1]))
x_range = np.arange(X.shape[0])
cs_x = CubicSpline(xx[:,0], yy[:,0])
cs_y = CubicSpline(xx[:,1], yy[:,1])
cs_z = CubicSpline(xx[:,2], yy[:,2])
return np.array([cs_x(x_range),cs_y(x_range),cs_z(x_range)]).transpose()
fig = plt.figure(figsize=(15,4))
for ii in range(8):
ax = fig.add_subplot(2,4,ii+1)
ax.plot(GenerateRandomCurves(Swelltrain, sigma))
plt.axis([0,3600,-0.5,2])
def DA_MagWarp(X, sigma):
return X * GenerateRandomCurves(X, sigma)
fig = plt.figure(figsize=(15,4))
for ii in range(8):
ax = fig.add_subplot(2,4,ii+1)
ax.plot(DA_MagWarp(Swelltrain, sigma))
ax.set_xlim([0,3600])
ax.set_ylim([-1.5,1.5])
But getting the error:
ValueError: Unable to coerce to DataFrame, shape must be (1887, 95): given (1887, 3)
The code works fine for 3 columns. My columns are 95…where should I make the changes from 3 to 95…not clear?
Answers:
The GenerateRandomCurves code was fixed to output 3 dimensions. The following code outputs all dimensions.
Memory management wise, the list append is probably not optimal, but it demonstrates the point.
def GenerateRandomCurves(X, sigma=0.2, knot=4):
xx = (np.ones((X.shape[1],1))*(np.arange(0,X.shape[0], (X.shape[0]-1)/(knot+1)))).transpose()
yy = np.random.normal(loc=1.0, scale=sigma, size=(knot+2, X.shape[1]))
x_range = np.arange(X.shape[0])
cs = []
for i in range(X.shape[1]):
cs.append(CubicSpline(xx[:,i], yy[:,i])(x_range))
return np.array(cs).transpose()
On image augmentation techniques
Images are comprised of 2 spacial dimensional(2D) arrays of pixels, 3 dimensional color structures.
Spacial relationships are easy to define.
The color relationship is much harder to define: Simply put, since each channel, RGB, contains mostly duplicate information, both independently and when transformed to other spaces, they are related.
GaussianBlur:
You must look at your feature set. Do you have spacial 2D or 3D relationships?
Pandas windowing mean when combined with np.Roll() can provide a similar effect.
Window size effects the approximate weak frequency response(not a true convolution since phase is not taken into account) represented in the output.
RandomGrayscale:
This is the fixed image equivalent of layer dropout + a jitter step.
Normalize:
I hope you normalize all data per feature.
Normalization across related features is a preprocessing step.
How to apply “Magnitude Warping” data augmentation on time series dataset? Is there any sample code in pytorch?
Also can we apply the image augmentation techniques like RandomGrayscale, GaussianBlur and Normalize to time series dataset?
Trying with the below code for magnitude Warping on Swelltrain dataset of size (1887, 95):
sigma = 0.2
knot = 4
def GenerateRandomCurves(X, sigma=0.2, knot=4):
xx = (np.ones((X.shape[1],1))*(np.arange(0,X.shape[0], (X.shape[0]-1)/(knot+1)))).transpose()
yy = np.random.normal(loc=1.0, scale=sigma, size=(knot+2, X.shape[1]))
x_range = np.arange(X.shape[0])
cs_x = CubicSpline(xx[:,0], yy[:,0])
cs_y = CubicSpline(xx[:,1], yy[:,1])
cs_z = CubicSpline(xx[:,2], yy[:,2])
return np.array([cs_x(x_range),cs_y(x_range),cs_z(x_range)]).transpose()
fig = plt.figure(figsize=(15,4))
for ii in range(8):
ax = fig.add_subplot(2,4,ii+1)
ax.plot(GenerateRandomCurves(Swelltrain, sigma))
plt.axis([0,3600,-0.5,2])
def DA_MagWarp(X, sigma):
return X * GenerateRandomCurves(X, sigma)
fig = plt.figure(figsize=(15,4))
for ii in range(8):
ax = fig.add_subplot(2,4,ii+1)
ax.plot(DA_MagWarp(Swelltrain, sigma))
ax.set_xlim([0,3600])
ax.set_ylim([-1.5,1.5])
But getting the error:
ValueError: Unable to coerce to DataFrame, shape must be (1887, 95): given (1887, 3)
The code works fine for 3 columns. My columns are 95…where should I make the changes from 3 to 95…not clear?
The GenerateRandomCurves code was fixed to output 3 dimensions. The following code outputs all dimensions.
Memory management wise, the list append is probably not optimal, but it demonstrates the point.
def GenerateRandomCurves(X, sigma=0.2, knot=4):
xx = (np.ones((X.shape[1],1))*(np.arange(0,X.shape[0], (X.shape[0]-1)/(knot+1)))).transpose()
yy = np.random.normal(loc=1.0, scale=sigma, size=(knot+2, X.shape[1]))
x_range = np.arange(X.shape[0])
cs = []
for i in range(X.shape[1]):
cs.append(CubicSpline(xx[:,i], yy[:,i])(x_range))
return np.array(cs).transpose()
On image augmentation techniques
Images are comprised of 2 spacial dimensional(2D) arrays of pixels, 3 dimensional color structures.
Spacial relationships are easy to define.
The color relationship is much harder to define: Simply put, since each channel, RGB, contains mostly duplicate information, both independently and when transformed to other spaces, they are related.
GaussianBlur:
You must look at your feature set. Do you have spacial 2D or 3D relationships?
Pandas windowing mean when combined with np.Roll() can provide a similar effect.
Window size effects the approximate weak frequency response(not a true convolution since phase is not taken into account) represented in the output.
RandomGrayscale:
This is the fixed image equivalent of layer dropout + a jitter step.
Normalize:
I hope you normalize all data per feature.
Normalization across related features is a preprocessing step.