Time complexity of recursion of multiplication
Question:
What is the worst case time complexity (Big O notation) of the following function for positive integers?
def rec_mul(a:int, b:int) -> int:
if b == 1:
return a
if a == 1:
return b
else:
return a + rec_mul(a, b-1)
I think it’s O(n) but my friend claims it’s O(2^n)
My argument:
The function recurs at any case b times, therefor the complexity is O(b) = O(n)
His argument:
since there are n bits, ab value can be no more than (2^n)-1,
therefor the max number of calls will be O(2^n)
Answers:
In every (valid) case, the value of "b" will be the number of iterations, AKA O(n).
You can always check the number of iterations by adding a global counter variable in the function that increments each time it’s called.
You are both right.
If we disregard the time complexity of addition (and you might discuss whether you have reason to do so or not) and count only the number of iterations, then you are both right because you define:
n = b
and your friend defines
n = log_2(b)
so the complexity is O(b) = O(2^log_2(b)).
Both definitions are valid and both can be practical. You look at the input values, your friend at the lengths of the input, in bits.
This is a good demonstration why big-O expressions mean nothing if you don’t define the variables used in those expressions.
Your friend and you can both be right, depending on what is n. Another way to say this is that your friend and you are both wrong, since you both forgot to specify what was n.
Your function takes an input that consists in two variables, a and b. These variables are numbers. If we express the complexity as a function of these numbers, it is really O(b log(ab)), because it consists in b iterations, and each iteration requires an addition of numbers of size up to ab, which takes log(ab) operations.
Now, you both chose to express the complexity in function of n rather than a or b. This is okay; we often do this; but an important question is: what is n?
Sometimes we think it’s "obvious" what is n, so we forget to say it.
- If you choose
n = max(a, b) or n = a + b, then you are right, the complexity is O(n).
- If you choose
n to be the length of the input, then n is the number of bits needed to represent the two numbers a and b. In other words, n = log(a) + log(b). In that case, your friend is right, the complexity is O(2^n).
Since there is an ambiguity in the meaning of n, I would argue that it’s meaningless to express the complexity as a function of n without specifying what n is. So, your friend and you are both wrong.
Background
A unary encoding of the input uses an alphabet of size 1: think tally marks. If the input is the number a, you need O(a) bits.
A binary encoding uses an alphabet of size 2: you get 0s and 1s. If the number is a, you need O(log_2 a) bits.
A trinary encoding uses an alphabet of size 3: you get 0s, 1s, and 2s. If the number is a, you need O(log_3 a) bits.
In general, a k-ary encoding uses an alphabet of size k: you get 0s, 1s, 2s, …, and k-1s. If the number is a, you need O(log_k a) bits.
What does this have to do with complexity?
As you are aware, we ignore multiplicative constants inside big-oh notation. n, 2n, 3n, etc, are all O(n).
The same holds for logarithms. log_2 n, 2 log_2 n, 3 log_2 n, etc, are all O(log_2 n).
The key observation here is that the ratio log_k1 n / log_k2 n is a constant, no matter what k1 and k2 are… as long as they are greater than 1. That means f(log_k1 n) = O(log_k2 n) for all k1, k2 > 1.
This is important when comparing algorithms. As long as you use an "efficient" encoding (i.e., not a unary encoding), it doesn’t matter what base you use: you can simply say f(n) = O(lg n) without specifying the base. This allows us to compare runtime of algorithms without worrying about the exact encoding you use.
So n = b (which implies a unary encoding) is typically never used. Binary encoding is simplest, and doesn’t provide a non-constant speed-up over any other encoding, so we usually just assume binary encoding.
That means we almost always assume that n = lg a + lg b as the input size, not n = a + b. A unary encoding is the only one that suggests linear growth, rather than exponential growth, as the values of a and b increase.
One area, though, where unary encodings are used is in distinguishing between strong NP-completeness and weak NP-completeness. Without getting into the theory, if a problem is NP-complete, we don’t expect any algorithm to have a polynomial running time, that is, one bounded by O(n**k) for some constant k when using an efficient encoring.
But some algorithms do become polynomial if we allow a unary encoding. If a problem that is otherwise NP-complete becomes polynomial when using an unary encoding, we call that a weakly NP-complete problem. It’s still slow, but it is in some sense "faster" than an algorithm where the size of the numbers doesn’t matter.
What is the worst case time complexity (Big O notation) of the following function for positive integers?
def rec_mul(a:int, b:int) -> int:
if b == 1:
return a
if a == 1:
return b
else:
return a + rec_mul(a, b-1)
I think it’s O(n) but my friend claims it’s O(2^n)
My argument:
The function recurs at any case b times, therefor the complexity is O(b) = O(n)
His argument:
since there are n bits, ab value can be no more than (2^n)-1,
therefor the max number of calls will be O(2^n)
In every (valid) case, the value of "b" will be the number of iterations, AKA O(n).
You can always check the number of iterations by adding a global counter variable in the function that increments each time it’s called.
You are both right.
If we disregard the time complexity of addition (and you might discuss whether you have reason to do so or not) and count only the number of iterations, then you are both right because you define:
n = b
and your friend defines
n = log_2(b)
so the complexity is O(b) = O(2^log_2(b)).
Both definitions are valid and both can be practical. You look at the input values, your friend at the lengths of the input, in bits.
This is a good demonstration why big-O expressions mean nothing if you don’t define the variables used in those expressions.
Your friend and you can both be right, depending on what is n. Another way to say this is that your friend and you are both wrong, since you both forgot to specify what was n.
Your function takes an input that consists in two variables, a and b. These variables are numbers. If we express the complexity as a function of these numbers, it is really O(b log(ab)), because it consists in b iterations, and each iteration requires an addition of numbers of size up to ab, which takes log(ab) operations.
Now, you both chose to express the complexity in function of n rather than a or b. This is okay; we often do this; but an important question is: what is n?
Sometimes we think it’s "obvious" what is n, so we forget to say it.
- If you choose
n = max(a, b)orn = a + b, then you are right, the complexity isO(n). - If you choose
nto be the length of the input, thennis the number of bits needed to represent the two numbersaandb. In other words,n = log(a) + log(b). In that case, your friend is right, the complexity isO(2^n).
Since there is an ambiguity in the meaning of n, I would argue that it’s meaningless to express the complexity as a function of n without specifying what n is. So, your friend and you are both wrong.
Background
A unary encoding of the input uses an alphabet of size 1: think tally marks. If the input is the number a, you need O(a) bits.
A binary encoding uses an alphabet of size 2: you get 0s and 1s. If the number is a, you need O(log_2 a) bits.
A trinary encoding uses an alphabet of size 3: you get 0s, 1s, and 2s. If the number is a, you need O(log_3 a) bits.
In general, a k-ary encoding uses an alphabet of size k: you get 0s, 1s, 2s, …, and k-1s. If the number is a, you need O(log_k a) bits.
What does this have to do with complexity?
As you are aware, we ignore multiplicative constants inside big-oh notation. n, 2n, 3n, etc, are all O(n).
The same holds for logarithms. log_2 n, 2 log_2 n, 3 log_2 n, etc, are all O(log_2 n).
The key observation here is that the ratio log_k1 n / log_k2 n is a constant, no matter what k1 and k2 are… as long as they are greater than 1. That means f(log_k1 n) = O(log_k2 n) for all k1, k2 > 1.
This is important when comparing algorithms. As long as you use an "efficient" encoding (i.e., not a unary encoding), it doesn’t matter what base you use: you can simply say f(n) = O(lg n) without specifying the base. This allows us to compare runtime of algorithms without worrying about the exact encoding you use.
So n = b (which implies a unary encoding) is typically never used. Binary encoding is simplest, and doesn’t provide a non-constant speed-up over any other encoding, so we usually just assume binary encoding.
That means we almost always assume that n = lg a + lg b as the input size, not n = a + b. A unary encoding is the only one that suggests linear growth, rather than exponential growth, as the values of a and b increase.
One area, though, where unary encodings are used is in distinguishing between strong NP-completeness and weak NP-completeness. Without getting into the theory, if a problem is NP-complete, we don’t expect any algorithm to have a polynomial running time, that is, one bounded by O(n**k) for some constant k when using an efficient encoring.
But some algorithms do become polynomial if we allow a unary encoding. If a problem that is otherwise NP-complete becomes polynomial when using an unary encoding, we call that a weakly NP-complete problem. It’s still slow, but it is in some sense "faster" than an algorithm where the size of the numbers doesn’t matter.