How to use Numpy to multiply all elements of matrix x by all elements of matrix y?
Question:
I am using Numpy as part of a neural network, and when updating the weights I am struggling to implement a step in a natural way.
The step works for an input rho_deltas (shape: (m,)) and self.node_layers[i-1].val (shape: (n,)) and outputs self.previous_edge_layer[i - 1] (shape: (m,n))
It should be such that self.previous_edge_layer[i - 1][j][k] == rho_deltas[j] * self.node_layers[i - 1].vals[k]
Example working inputs and outputs here.
(I’ll try to update these so it is easier to copy and paste for testing your methods.)
I have managed to get it working well like:
self.previous_edge_layer[i - 1] = np.array([rho_delta * self.node_layers[i - 1].vals for rho_delta in rho_deltas])
However, it feels to me as though there is a Numpy operator/function that should be able to do this without the iteration over the full list. My inclination is matrix multiplication (@) however, I have not been able to get this to work. Or perhaps, dot product (*), however for n != m this fails.
Furthermore, I struggled to come up with a useful name for this question so feel free to rename so something better :).
Answers:
Matrix multiplication is the right idea: the preliminary is to form matrices from your 1D vectors. We need 2D matrices here, even though one dimension will be of size 1. Something like this:
import numpy as np
rho_deltas = np.array([7.6, 12.3, 11.1]) # example data with m = 3
layer_vals = np.array([1.5, 20.9, -3.5, 7.0]) # example with n = 4
rho_deltas_row_mat = rho_deltas.reshape(-1, 1) # m rows, 1 column
layer_vals_col_mat = layer_vals.reshape(1, -1) # 1 row, n columns
res = rho_deltas_row_mat @ layer_vals_col_mat
print(res.shape)
print(all(res[j][k] == rho_deltas[j] * layer_vals[k] for j in range(rho_deltas.shape[0]) for k in range(layer_vals.shape[0])))
prints:
(3, 4)
True
Alternatively, you could reshape both of them to row matrices and use transposition, something like:
rho_deltas_row_mat = rho_deltas.reshape(-1, 1)
layer_vals_row_mat = layer_vals.reshape(-1, 1)
res = rho_deltas_row_mat @ layer_vals_row_mat.T
Based on the link you provided you can use Numpy meshgrid function to repeat two array based on each others dimension and then simply multiply them element wise.
The following will do what you want (tested it on your example and produced same results)
import numpy as np
a = np.array([1,2,3])
b = np.array([10,20,30,40,50])
bv, av = np.meshgrid(b,a) # returns repetition of one arraya by the other one's dimension.
# av = [[1 1 1 1 1]
# [2 2 2 2 2]
# [3 3 3 3 3]]
# bv = [[10 20 30 40 50]
# [10 20 30 40 50]
# [10 20 30 40 50]]
c = av*bv
# c = [[ 10 20 30 40 50]
# [ 20 40 60 80 100]
# [ 30 60 90 120 150]]
Similar result can also be achieved with Numpy einsum function if you are familiar with einstein sum and notation.
I am using Numpy as part of a neural network, and when updating the weights I am struggling to implement a step in a natural way.
The step works for an input rho_deltas (shape: (m,)) and self.node_layers[i-1].val (shape: (n,)) and outputs self.previous_edge_layer[i - 1] (shape: (m,n))
It should be such that self.previous_edge_layer[i - 1][j][k] == rho_deltas[j] * self.node_layers[i - 1].vals[k]
Example working inputs and outputs here.
(I’ll try to update these so it is easier to copy and paste for testing your methods.)
I have managed to get it working well like:
self.previous_edge_layer[i - 1] = np.array([rho_delta * self.node_layers[i - 1].vals for rho_delta in rho_deltas])
However, it feels to me as though there is a Numpy operator/function that should be able to do this without the iteration over the full list. My inclination is matrix multiplication (@) however, I have not been able to get this to work. Or perhaps, dot product (*), however for n != m this fails.
Furthermore, I struggled to come up with a useful name for this question so feel free to rename so something better :).
Matrix multiplication is the right idea: the preliminary is to form matrices from your 1D vectors. We need 2D matrices here, even though one dimension will be of size 1. Something like this:
import numpy as np
rho_deltas = np.array([7.6, 12.3, 11.1]) # example data with m = 3
layer_vals = np.array([1.5, 20.9, -3.5, 7.0]) # example with n = 4
rho_deltas_row_mat = rho_deltas.reshape(-1, 1) # m rows, 1 column
layer_vals_col_mat = layer_vals.reshape(1, -1) # 1 row, n columns
res = rho_deltas_row_mat @ layer_vals_col_mat
print(res.shape)
print(all(res[j][k] == rho_deltas[j] * layer_vals[k] for j in range(rho_deltas.shape[0]) for k in range(layer_vals.shape[0])))
prints:
(3, 4)
True
Alternatively, you could reshape both of them to row matrices and use transposition, something like:
rho_deltas_row_mat = rho_deltas.reshape(-1, 1)
layer_vals_row_mat = layer_vals.reshape(-1, 1)
res = rho_deltas_row_mat @ layer_vals_row_mat.T
Based on the link you provided you can use Numpy meshgrid function to repeat two array based on each others dimension and then simply multiply them element wise.
The following will do what you want (tested it on your example and produced same results)
import numpy as np
a = np.array([1,2,3])
b = np.array([10,20,30,40,50])
bv, av = np.meshgrid(b,a) # returns repetition of one arraya by the other one's dimension.
# av = [[1 1 1 1 1]
# [2 2 2 2 2]
# [3 3 3 3 3]]
# bv = [[10 20 30 40 50]
# [10 20 30 40 50]
# [10 20 30 40 50]]
c = av*bv
# c = [[ 10 20 30 40 50]
# [ 20 40 60 80 100]
# [ 30 60 90 120 150]]
Similar result can also be achieved with Numpy einsum function if you are familiar with einstein sum and notation.