Apply a function per matrix cell for multiple matrices numpy

Question:

I’ll try to simplify my actual problem.
Let’s say I have 3 matrices with similar shape:

m1
array([[0.1 , 0.12, 0.32],
       [0.22, 0.15, 0.16],
       [0.42, 0.52, 0.62],
       [0.44, 0.35, 0.36]])

m2
array([[0.12 , 0.144, 0.384],
       [0.264, 0.18 , 0.192],
       [0.504, 0.624, 0.744],
       [0.528, 0.42 , 0.432]])

m3
array([[0.106 , 0.1272, 0.3392],
       [0.2332, 0.159 , 0.1696],
       [0.4452, 0.5512, 0.6572],
       [0.4664, 0.371 , 0.3816]])

Each matrix has a corresponded date, which I convert to an integer and call it delta, so:

delta_m1  = 2
delta_m2 = 5
delta_m3 = 10

What I would like to do is to apply a function to each cell-based-time-series, meaning that if I consider all cells at position (0,0), then m1_cell is 0.1, m2_cell is 0.12, and m3_cell is 0.106. So now I have a cell-based-time-series where the time (the x-axis) being the deltas (2,5,10) and the y-axis is the value of the cells (0.1,0.12,0.106). I would like to apply a smooth function (that takes two 1d-arrays and outputs 1d-array) that takes this cell-based-time-series, meaning takes X=(2,5,10) and y=(0.1,0.12,0.106) and outputs a new value per date/delta/matrix, so, for example, an output could be (0.09,0.1,0.8), where each value corresponds to the cell at (0,0) for each matrix. The whole process I’ve described refers to only one cell. I would like to apply it to all cells. Then I can create new matrices based on the results. I guess I can simply iterate over each cell over each matrix, and apply the function, but that seems super not efficient.
How can I achieve that in a pythonic efficient way?

EDIT

The function I would like to apply on each cell-based-time-series:

def local_regression(x0, X, Y, k):
    # add bias term
    x0 = np.r_[1, x0]
    X = np.c_[np.ones(len(X)), X]
    
    # fit model: normal equations with kernel
    xw = X.T * radial_kernel(x0, X, k)
    beta = np.linalg.pinv(xw @ X) @ xw @ Y
    
    # predict value
    return x0 @ beta

def radial_kernel(x0, X, k):
    return np.exp(np.sum((X - x0) ** 2, axis=1) / (-2 * k * k))

def smooth(x,y):
    return [local_regression(x[p],x,y,k) for p in range(len(x))]

So for example:

y = np.array([2,5,10])
x = np.array([0.1,0.12,0.106])
smooth(x,y)
>>> array([5.14556944, 6.34810085, 5.50632924])

Where x is an array of integers (though it can be changed to float) and y is an array of float. k is a parameter of the function and here k=20

Asked By: user88484

||

Answers:

The issue with broadcasting is primarily because you are using a list comprehension in the last function. What happens because of that is that your function only operates on the last dimension (axis=-1). This means you are forced to transpose your inputs even with np.vectorize and then transpose them back.

Plus, because of this you will have to explicitly mention the signature in the np.vectorize as well.

Check the code below. I have marked the areas where I have made changes.

#Stack all the matrices (because similar shape)
M = np.stack([m1, m2, m3])                   #(3, 4, 3)
D = np.stack([delta_m1, delta_m2, delta_m3]) #(3,)

##################### FUNCTION ######################
k=20

def local_regression(x0, X, Y, k):
    # add bias term
    x0 = np.r_[1, x0]
    X = np.c_[np.ones(len(X)), X]
    
    # fit model: normal equations with kernel
    xw = X.T * radial_kernel(x0, X, k)
    beta = np.linalg.pinv(xw @ X) @ xw @ Y
    
    # predict value
    return x0 @ beta

def radial_kernel(x0, X, k):
    return np.exp(np.sum((X - x0) ** 2, axis=1) / (-2 * k * k))

def smooth(x,y):
    return np.array([local_regression(x[p],x,y,k) for p in range(len(x))]) #<-----

smooth_v = np.vectorize(smooth, signature='(i),(i)->(i)') #<----
##################################################

smooth_v(D[None, None,:], M.transpose(2,1,0)).transpose(2,1,0)  #<----- 
#First transpose brings axis=0 to axis=-1, and the second transpose reverses it.
array([[[0.10713757, 0.12856508, 0.34284022],
        [0.23570265, 0.16070635, 0.17142011],
        [0.44997779, 0.55711536, 0.66425293],
        [0.4714053 , 0.37498149, 0.38569525]],

       [[0.10847502, 0.13017003, 0.34712007],
        [0.23864505, 0.16271253, 0.17356004],
        [0.4555951 , 0.56407012, 0.67254514],
        [0.4772901 , 0.37966258, 0.39051008]],

       [[0.11017182, 0.13220618, 0.35254982],
        [0.242378  , 0.16525773, 0.17627491],
        [0.46272164, 0.57289345, 0.68306527],
        [0.484756  , 0.38560136, 0.39661854]]])
Answered By: Akshay Sehgal