Runtime of dynamic programming versus it's naive recursive counterpart

Question

The problem follows:

You are playing a game where you start with n sticks in a pile (where n is a positive integer),
and each turn you can take away exactly 1, 7, or 9 sticks from the pile (if there are fewer than 9
sticks left, you can’t choose that option, and similarly with 7). You want to know the minimum
number of turns needed to reach 0 sticks.

And we’re asked to implement a naive recursive solution and a O(n) dynamic programming solution, here’s what I have:

def take(pile, turns, cur):
    if cur > pile:
        return float("inf")    
    if cur == pile:
        return turns
    else: 
        res = 1 + min(take(pile, turns, cur+1), take(pile, turns,cur+7), take(pile, turns, cur+9))
        return res

And my DP solution

def takeDP(pile, turns, cur):
    if cur > pile:
        return float("inf")
    if mem[cur] != float("inf"):
        return mem[cur]
    if cur == pile:
        return turns
    else: 
        res =  1 + min(takeDP(pile, turns, cur+9), takeDP(pile, turns,cur+7), takeDP(pile, turns, cur+1))
        if res < mem[cur]:
            mem[cur] = res 
        return res

When timing each, takeDP is much faster but I’m trying to bridge the gap of understanding the complexities of each. The best angle of attack I can think of is to compare the reduced recursion tree of takeDP compared to take‘s, but I can’t really express how take‘s O(3^n) is optimized to O(n) enough to justify it in words. By virtue of this I’m also not entirely confident in my takeDP solution being correct.

Asked By: PhunkyPhil

||

Source

Answer 1

Note the following observations, which are true for both functions:

The only variable that changes between different recursive calls is cur;
Initially cur is equal to 0;
cur can never be higher than n, the number of sticks in the pile.

From the above observations, you can conclude that there really are only n+1 different calls that are ever made:

take(n, 0, 0), take(n, 0, 1), take(n, 0, 2), take(n, 0, 3), ..., take(n, 0, n)

So, if more than n+1 calls are made in total, it must mean that the extra calls are redundant, i.e., recalculating things that have already been calculated before.

The DP version makes sure that everytime a new different call is made, its return value is stored in array mem. And if two identical calls are made, then the second call is going to be interrupted immediately by the if mem[cur] != float("inf"): clause.

So, with the DP version, there can be at most n+1 calls that are not interrupted by the if mem[cur] != float("inf"): clause. And each of these n+1 calls spawn at most 3 calls, so you can immediately conclude that there will not be more than 4n+4 calls in total. In fact, a closer analysis would say that there is exactly 3n+4 calls. But anyway, even if you’re only convinced that there cannot be more than 4n+4 calls, that’s enough to conclude: the time-complexity of the DP version is O(n).

With the naive-recursive version though, all we can say is that each recursive call spawns 3 new recursive calls, down to a depth of recursion of n+1, and therefore the total number of recursive calls is about 3^n. Actually, each recursive branch has a depth somewhere between n and n/9, because cur is incremented by 1, 7 or 9. So we can conclude that the time-complexity of the naive-recursive version is somewhere between 3^n and 3^(n/9) ≈ 1.13^n. Anyway, even without a closer analysis, that’s enough to conclude that the time-complexity of the naive-recursive version is exponential.

Answered By: Stef

Runtime of dynamic programming versus it's naive recursive counterpart

Question:

Answers: