Overflow error when multiplying matrix with vector

Question

When I multiply v by the matrix called transition below n times I get unexpected results. When n = 101, I get v = [-2003791491, -2065138873, 1087163732, 12302843, -363547669, 31628264] but it can’t have negative values because all initial numbers in v are positive and the transition matrix is positive.

import numpy as np  

transition =  [[1, 1, 1, 0, 0, 0],
               [1, 0, 0, 0, 0, 0],
               [0, 1, 0, 0, 0, 0],
               [1, 1, 1, 1, 1, 1],
               [0, 0, 0, 1, 0, 0],
               [0, 0, 0, 0, 1, 0]]

v = [2,1,1,3,1,0]

n = 101
for i in range(n-2): 
    v = np.matmul(transition,v)

print(v.tolist()) 
print(sum(v)%(1000000000 + 7))

Can anyone explain how to solve this issue?

Asked By: Andy Wang

||

Source

Answer 1

The issue is that you are multiplying the matrix by itself n-2 times, which will cause the elements in the resulting vector to grow exponentially. If the elements in the matrix are positive, the resulting vector will also contain positive numbers, but they may become very large. The modulo operation with (1000000000 + 7) is not affecting it. The result are exceeding the valid range of the integers. One possible solution is to use a different data type, such as numpy’s float64, to store the elements in the matrix and vector, or to use a different method to keep the elements in the resulting vector within a reasonable range.

Answered By: Ahsan Aman

Answer 2

You can take a look at what happens with code that looks like this:

t = np.array(transition, 'int32')
v = np.array(v, 'int32')
m = []
while (v >= 0).all():
    m.append(v.max())
    v = t @ v
print(f'Overflow after {len(m)} iterations')
plt.plot(m)

The increase is exponential: since the largest number approximately doubles every iteration, it is plausible that a 32-bit integer will overflow after ~30 iterations. In fact the overflow takes exactly 30 iterations.

You can fix this by using a 64-bit integer, but then it will take ~60 iterations. Using floating point numbers will increase your dynamic range to ~2^1024, but that will lose precision in the lower digits, since you can only have 53 bits of precision in a double.

To harness Python’s infinite-precision integers, you can rewrite the transition matrix as a a set of sums:

def transition(v):
    return [sum(v[:3]), v[0], v[1], sum(v), v[3], v[4]]

for _ in range(99):
    v = transition(v)

The result is

[331800673921785084815380861,
 180396380815100901214157639,
 98079530178586034536500564,
 20795180176044632893334206971,
 11194550262857170904126108651,
 6025699644510475468432120808]

sum(v) gives 38625706668327751286458475494, which is ~4e28, or ~2**95. Even using uint64 would not be useful here, as expected.

The final result is

>>> sum(v) % 1000000007
823829994

Answered By: Mad Physicist

Overflow error when multiplying matrix with vector

Question:

Answers: