Skip to main content
  1. References/
  2. Algo Reference/
  3. Canonicals/

Topic 12: 1D DP

·· 10702 words· 43–72 min read

Key-Intuition: Many dynamic programming problems can be reduced to 1D subproblems involving optimal substructure and overlapping subproblems.

Typically, they involve some sort of optimisation problem that we’re interested in and we’re usually concerned with some order statistic instead of permuting or finding combinations here – e.g. min sum vs configuration that gives the min sum.

Counting problems may be optimised with this as well, especially if there’s a whole host of overlapping sub-problems, the calculation for which we can do smartly.

Read the notes below for a run-down.

Canonical Question Types #

Linear State Transitions (Fibonacci-like) #

  • Progression by fixed window of previous states
  • most of them can consider state roll-ups to reduce the space usage – it’s clearer depending on the general recursive formula we get (how \(k + 1\) relates to \(k\))

Problems #

  1. Climbing Stairs (70)

    Both topdown recursive and bottom up iterative are easy to write, we notice that we can use rolling variables here because there only is a need to keep track of a single historical value.

    NB: it’s better to work towards the rolled-up version because it’s clearer that way.

     1
     2
     3
     4
     5
     6
     7
     8
     9
    10
    11
    12
    13
    14
    15
    16
    17
    18
    19
    20
    21
    22
    
        # bottom up iterative 2-var rolling approach
        class Solution:
            def climbStairs(self, n: int) -> int:
                if n <= 2:
                    return n
    
                prev, curr = 1, 2
                for _ in range(3, n+1):
                    prev, curr = curr, prev + curr
    
                return curr
    
        # top down recursive using builtin cache decorator
        from functools import cache
    
        class Solution:
            @cache
            def climbStairs(self, n: int) -> int:
                if n <= 2:
                    return n
    
                return self.climbStairs(n - 1) + self.climbStairs(n - 2)
  2. Min Cost Climbing Stairs (746)

    Same as previous, now with a cost applied remember the cost is for reaching that step so reach this step == paid the cost of jumping from previous steps; it’s NOT “cost of landing on this step”. You do not need to land on the last step; you need the minimum cost of getting past the last step, so from either the last or second last step.

    We realise that we only need one historical state, so a 2-var rolling approach works well for the iterative implementation.

     1
     2
     3
     4
     5
     6
     7
     8
     9
    10
    11
    12
    13
    14
    15
    16
    17
    18
    19
    20
    21
    22
    23
    24
    25
    
        # bottom up recursive:
        class Solution:
            def minCostClimbingStairs(self, cost: List[int]) -> int:
                if len(cost) == 0:
                    return 0
                if len(cost) == 1:
                    return cost[0]
    
                prev, curr = cost[0], cost[1]
                for i in range(2, len(cost)):
                    prev, curr = curr, min(prev, curr) + cost[i]
    
                return min(prev, curr)
    
        # top down recursive approach:
        from functools import cache
        class Solution:
            def minCostClimbingStairs(self, cost: List[int]) -> int:
                @cache
                def min_cost(i):
                    if i < 2: return cost[i]
                    return cost[i] + min(min_cost(i-1), min_cost(i-2))
                n = len(cost)
    
                return min(min_cost(n-1), min_cost(n-2))
  3. Fibonacci Number (509)

    Straight up classic.

     1
     2
     3
     4
     5
     6
     7
     8
     9
    10
    11
    12
    
        class Solution:
            def fib(self, n: int) -> int:
                if n == 0:
                    return 0
                if n == 1:
                    return 1
    
                prev, curr = 0, 1
                for _ in range(2, n + 1):
                    prev, curr = curr, prev + curr
    
                return curr

Non-Adjacent Selection (House Robber type) #

  • Select a subset of elements under adjacency constraints (max sum, choose or skip)

Problems #

  1. House Robber (198)

    It’s a max sum and for each element, we either choose it or we skip it. Just need to build up our answer from small to big. We naturally realise that we only need to keep track of 2 states, hence the rolling variables are only 2.

     1
     2
     3
     4
     5
     6
     7
     8
     9
    10
    11
    12
    13
    14
    15
    16
    17
    18
    19
    20
    21
    22
    23
    24
    25
    26
    27
    28
    29
    30
    
        # iterative bottom up 2-rolling variables approach.
        class Solution:
            def rob(self, nums: List[int]) -> int:
                if not nums:
                    return 0
    
                if len(nums) == 1:
                    return nums[0]
    
    
                prev, curr = nums[0], max(nums[1], nums[0])
                for i in range(2, len(nums)):
                    this = nums[i]
                    prev, curr = curr, max(
                        nums[i] + prev,  # choose this house to rob
                        curr # don't choose this house to rob
                        )
    
                return curr
    
        # top down recursive solution
        from functools import cache
        class Solution:
            def rob(self, nums: List[int]) -> int:
                n = len(nums)
                @cache
                def dp(i):
                    if i >= n: return 0
                    return max(nums[i] + dp(i+2), dp(i+1))
                return dp(0)
  2. House Robber II (213)

    The key idea here is that to reduce it to the earlier version, we consider 2 mutually exclusive events (that only exist because of the circular nature). The rest is pretty much the same. Extracting things into a helper function is useful.

    In most cases, we’re interested in how our problem is an evolution of a related problem. Here, we can try to make a circular array behave like a linear array precisely because we can break it down into 2 mutually exclusive cases (first house considered vs first house NOT considered.)

     1
     2
     3
     4
     5
     6
     7
     8
     9
    10
    11
    12
    13
    14
    15
    16
    17
    18
    19
    20
    21
    22
    23
    24
    25
    26
    27
    28
    29
    
        class Solution:
            def rob(self, nums: List[int]) -> int:
                if not nums:
                    return 0
                if len(nums) == 1:
                    return nums[0]
    
                def rob_straight(houses):
                    if len(houses) == 0:
                        return 0
    
                    if len(houses) == 1:
                        return houses[0]
    
                    prev, curr = houses[0], max(houses[1], houses[0])
                    for i in range(2, len(houses)):
                        this = houses[i]
                        prev, curr = curr, max(
                            houses[i] + prev, # chose this house to rob
                            curr
                        )
    
                    return curr
    
                # get 1D maxes for the staight-lined version:
                return max(
                    rob_straight(nums[1:]), # case 1: we choose last house, forgo first house
                    rob_straight(nums[:len(nums) - 1]), # case 2: we choose first house, forgo last house
                )

Maximum Subarray/Product/Range #

  • Contiguous region optimal (sum or product)
  • We can consider this because of the associative nature of the operations
  • For contiguous subarray sums, Kadane is usually a first-thought, works with negative numbers as well.

Problems #

  1. Maximum Product Subarray (152)

    We just want the max product from a subarray within the array given, it would smell like an inclusion / exclusion kind of problem (either start fresh or continue accumulating) initially. It technically is, because at each step, we choose whether to continue from before or start anew.

    We might be also tempted to approach it like a continguous region and 2D DP-table it. However, it’s not always true that we HAVE to pick 2D approaches when we are trying to keep track of contiguous regions, we should consider other properties / characteristics of the problem.

    The key insight is that we need to keep track of both min and max at each step because the numbers can be negative and two negatives may give us a positive.

    On further observation, we then realise that we do not need explicit two pointers or a 2D DP table because the maximum product subarray ending at index i depends only on max/min products ending at i-1. By tracking just those two variables at each step, you integrate the contribution of all contiguous subarrays ending there and keep the global max in \(O(n)\) time and \(O(1)\) space.

    So a 1D DP table is sufficient:

     1
     2
     3
     4
     5
     6
     7
     8
     9
    10
    11
    12
    13
    14
    15
    16
    17
    18
    19
    20
    
        # 1D DP table
        class Solution:
            def maxProduct(self, nums: List[int]) -> int:
                n = len(nums)
                curr_max, curr_min = nums[0], nums[0]
                result = curr_max
    
                for idx in range(1,len(nums)):
                    num = nums[idx]
                    # negative multiplication, will swap the min and max values
                    if num < 0:
                        curr_min, curr_max = curr_max, curr_min
    
                    # max of (starting fresh, or accumulating)
                    curr_max = max(num, curr_max * num)
                    curr_min = min(num, curr_min * num)
    
                    result = max(result, curr_max)
    
                return result
  2. Maximum Subarray (53) – Kadane algo At each index I track the best subarray sum ending here — it’s either this element alone, or this element appended to the best ending at the previous index. I keep a global max of all these. One pass, O(1) space.

    Recognition Signals
    • “contiguous subarray” + optimise some cumulative score
    • No constraint on minimum length (if there were, you’d need a deque/prefix-sum variant)
    • Values can be negative (otherwise a simple total sum works)
    Anti-signals (when NOT to use the Kadane approach)
    • Subsequences (non-contiguous) — use a different DP state
    • Need the actual subarray indices — Kadane still works but you track start/end pointers
    • Circular array — use Kadane + total_sum minus min_subarray (LC 918)
    • At-most-K length constraint — need sliding window with monotonic deque (LC 862)

    There’s a couple of ways to do this, though Kadane-algo should be the first-reach algo for us:

    M1: Realising that it’s DP, specifically Kadane Algo
     1
     2
     3
     4
     5
     6
     7
     8
     9
    10
    11
    12
    13
    14
    15
    16
    17
    18
    
       class Solution:
           def maxSubArray(self, nums: List[int]) -> int:
               """
               This is straight up just Kadane Algo:
               1. we care about inclusion or exclusion.
               2. elements can be negative
    
               Thinking of it as a state-tracing 1D DP isn't that far off though. We can realise that it's Kadane-by focusing on the substructures and then emphasising the contiguious nature of the problem.
    
               Fastest recognition heuristic: "contiguous subarray" + "maximise sum" + values can be negative → Kadane, immediate.
               """
               best_ending_here = nums[0]
               global_best = nums[0]
               for num in nums[1:]:
                   best_ending_here = max(best_ending_here + num, num) # build onto, include num OR start fresh
                   global_best = max(global_best, best_ending_here)
    
               return global_best
    M2: divide and conquer approach, which we see the problem as having 2 parts: left and right partschinite

    It’s recursive,

     1
     2
     3
     4
     5
     6
     7
     8
     9
    10
    11
    12
    13
    14
    15
    16
    17
    18
    19
    20
    21
    22
    23
    24
    25
    26
    27
    28
    29
    30
    31
    32
    33
    34
    35
    36
    37
    38
    39
    40
    41
    42
    43
    
       from typing import List
    
       class Solution:
           def maxSubArray(self, nums: List[int]) -> int:
               def helper(left: int, right: int) -> int:
                   # Base case: single element
                   if left == right:
                       return nums[left]
    
                   mid = left + (right - left) // 2
    
                   # Maximum subarray in left half
                   left_max = helper(left, mid)
    
                   # Maximum subarray in right half
                   right_max = helper(mid + 1, right)
    
                   # Maximum subarray crossing mid
                   cross_max = max_crossing_subarray(left, mid, right)
    
                   # Return overall max
                   return max(left_max, right_max, cross_max)
    
               def max_crossing_subarray(left: int, mid: int, right: int) -> int:
                   # Find max subarray sum crossing mid, including mid
    
                   left_sum = float('-inf')
                   curr_sum = 0
                   for i in range(mid, left - 1, -1):
                       curr_sum += nums[i]
                       if curr_sum > left_sum:
                           left_sum = curr_sum
    
                   right_sum = float('-inf')
                   curr_sum = 0
                   for i in range(mid + 1, right + 1):
                       curr_sum += nums[i]
                       if curr_sum > right_sum:
                           right_sum = curr_sum
    
                   return left_sum + right_sum
    
               return helper(0, len(nums) - 1)

    There’s also some common misconceptions:

    if stuck @ state definition

    ask “what does it mean for a subarray to end at position i?” It means every element from some start index to i is included — no gaps. So the only state you need is: “what is the maximum sum subarray that ends exactly here?” That’s one number, not two.

    how to decide whether to extend or restart?

    extending is prev_ending_here + curr; restarting is curr alone. You take the max. If the running sum was negative, restarting always wins.

    stuck @ global ans update?

    after every element, compare ending_here to global_best. One line

    Finally, there are ways to transform this, Kadane is simpler to imagine its transformations

    transformations
    • Circular array (LC 918): the optimal subarray can wrap. Kadane gives you the non-wrapping case; total_sum minus min_subarray (run Kadane with inverted signs) gives the wrapping case. Your circular-subarray work in canonicals.org covers this.

    • All elements positive: Kadane degenerates — best_ending_here never resets, so the answer is always the total sum. Knowing this degenerate case sharpens the intuition for why negatives are the crux.

    • Online/streaming: Kadane is already online — you only need O(1) state. No issue. If you additionally need to answer “max subarray ending exactly here” as a query, you emit best_ending_here after each step.

    • 2D generalisation (max sum rectangle, LC 363): compress row pairs into a 1D column-sum array, then run Kadane. This is a non-trivial but direct extension.

Combination/Subsequence Counting #

  • Number of ways to achieve a target by combining elements, usually counting answers not optimizing them
  • combination counting is about iterating over the options as the outer loops so that we don’t end up doing double counting
  • when doing this, the order of traversing each option-dimension is the same.

Problems #

  1. Decode Ways (91) This is a counting question, we just need to build onto previous subproblems’ counts. So for the iterative version, let dp[i] the number of ways of decoding up to index one. Remember to include values for the trivial empty string as well (which is why we fill up to idx n + 1).

    we go left to right because we can use the previously built values for the future ones (right values can be built using left values).

    Applying a more frameworked approach to understanding this
    • Subproblems: Number of ways to decode substring s[:i].
    • Overlapping subproblems: Yes, decoding s[:i] depends on s[:i-1] and s[:i-2].
    • Choices at each subproblem: Decode single digit or two digits.
    • DP Relation: \[ dp[i] = \mathbb{1}[s_{i-1} \neq 0] \cdot dp[i-1] + \mathbb{1}[10 \leq s_{i-2} s_{i-1} \leq 26] \cdot dp[i-2] \]
    • Independence: Each choice leads to subproblems that do not interfere.

    The recurrence relation tells us that we only need to consider 2 states each time, so we can use a 2-rolling-variable optimisation for the space.

    iterative approach,without the rolling variable optimisation and the equivalent form of rolling it up into 2-vars
     1
     2
     3
     4
     5
     6
     7
     8
     9
    10
    11
    12
    13
    14
    15
    16
    17
    18
    19
    20
    21
    22
    23
    24
    25
    26
    27
    28
    29
    30
    31
    32
    
            # iterative without the rolling variable optimisation: =>
            class Solution:
                def numDecodings(self, s: str) -> int:
                    # If the input is empty or starts with '0', there are no valid decodings.
                    if not s or s[0] == '0':
                        return 0
    
                    L = len(s)
    
                    # dp[i] will store the number of ways to decode the substring s[:i]
                    # We use length L + 1 to handle the base case for an empty substring
                    dp = [0] * (L + 1)
    
                    # Base cases:
                    dp[0] = 1  # There's one way to decode an empty string (do nothing)
                    dp[1] = 1  # Since s[0] != '0', there's one way to decode the first character
    
                    # Loop through the string from position 2 onwards (1-based indexing in dp)
                    for i in range(2, L + 1):
                        # Check if single digit decode is possible (last one digit)
                        # If s[i-1] is not '0', then we can decode it as a letter on its own
                        if s[i - 1] != '0':
                            dp[i] += dp[i - 1]  # Add the ways up to the previous position
    
                        # Check if two-digit decode is possible (last two digits)
                        two_digit = int(s[i - 2:i])
                        # If the two-digit number is between 10 and 26 inclusive, it can be decoded as a letter
                        if 10 <= two_digit <= 26:
                            dp[i] += dp[i - 2]  # Add the ways up to i - 2 position
    
                    # The answer is the number of ways to decode the entire string s[:L]
                    return dp[L]
    Code Snippet 1: iterative form, no rolling variable optimisation
     1
     2
     3
     4
     5
     6
     7
     8
     9
    10
    11
    12
    13
    14
    15
    16
    17
    18
    19
    20
    21
    22
    23
    24
    25
    26
    27
    28
    29
    30
    31
    
            # so the 2 rolling variable optimisation is straightforward for us to see:
            # note the change in the iteration range because of the use of the roll-up variable.
            class Solution:
                def numDecodings(self, s: str) -> int:
                    # Edge case: empty string or starting with '0' means no possible decoding
                    if not s or s[0] == '0':
                        return 0
    
                    prev = 1  # dp[0], ways to decode empty string
                    curr = 1  # dp[1], ways to decode string of length 1 (assuming s[0] != '0')
    
                    for i in range(1, len(s)):
                        temp = 0  # ways to decode up to position i+1 (1-based indexing)
    
                        # Check if single digit decode is valid (s[i] != '0')
                        if s[i] != '0':
                            temp += curr  # Can decode single char, so add ways till previous char
    
                        # Check if two-digit decode is valid (10 <= s[i-1:i+1] <= 26)
                        two_digit = int(s[i-1:i+1])
                        if 10 <= two_digit <= 26:
                            temp += prev  # Can decode two chars together
    
                        # If no valid decode for this position, temp will be 0 -> return 0 later
                        if temp == 0:
                            return 0
    
                        # Shift dp states for next iteration
                        prev, curr = curr, temp
    
                    return curr
    Code Snippet 2: converting it into a rolled up variable form

    Here’s the rolled up solution:

     1
     2
     3
     4
     5
     6
     7
     8
     9
    10
    11
    12
    13
    14
    15
    16
    17
    18
    19
    20
    21
    22
    23
    24
    
       class Solution:
           def numDecodings(self, s: str) -> int:
               # the main choice is whether i can be a one-fer or two-fer, if at all
               # dp[i] = ith place how many
               # recurrence: A) cannot 1fer and cannot 2-fer B) can 1 fer but not 2fer C: can 1fer or 2fer
    
               n = len(s)
               if n == 1:
                   return 0 if s == '0' else  1
    
               if n >= 2 and 1 <= int(s[:2]) <= 9:
                   return 0
    
               prev, curr = 1, 1
               for i in range(1, n): # NOTE: here, i represents the calculation up to idx i of string s. so if we calculate i, means s[:i + 1] has been calculated.
                   can_2fer = 10 <= int(s[i - 1: i + 1]) <= 26 # this accounts for the leadings zeros cases also, since it's a space over a bunch of digits adn we're only concerned if it has a 2fer value (which is bound by the double digits ascii charset for alphabetical chars)
                   can_1fer = 1 <= int(s[i]) <= 9
    
                   if not (can_2fer or can_1fer):
                       return 0
    
                   prev, curr = curr, (prev if can_2fer else 0) + (curr if can_1fer else 0)
    
               return curr
    We can choose to solve this via a dfs approach as well since this is about exploring paths.
     1
     2
     3
     4
     5
     6
     7
     8
     9
    10
    11
    12
    13
    14
    15
    16
    17
    18
    19
    20
    21
    22
    23
    24
    
            # of course, we have the recursive solution as well:
            from functools import cache
    
            class Solution:
                def numDecodings(self, s: str) -> int:
                    n = len(s)
    
                    @cache
                    def dfs(i):
                        if i == n:
                            return 1  # Finished decoding successfully
                        if s[i] == '0':
                            return 0  # Invalid
    
                        # Single digit decode
                        count = dfs(i + 1)
    
                        # Two digit decode if valid
                        if i + 1 < n and 10 <= int(s[i:i+2]) <= 26:
                            count += dfs(i + 2)
    
                        return count
    
                    return dfs(0)

    Also this is a good demonstration of the iteration ranges when we use rolling variable optimisations. See this table for more info:

    MethodState/IndexingRangeRationale
    DP Array1-based, dp base2 to L+1 (n+1)Need the two previous for `dp[i]`, and there’s a dummy base at index 0
    2-Variable Roll0-based, positions1 to L-1Only need to look back two steps; base cases set up before loop
  2. Combination Sum IV (377) This is such a juke. This is badly named. Look at the inputs and realise that it’s actually an unbounded knapsack problem where we need to do permutation counting (not combination counting).

     1
     2
     3
     4
     5
     6
     7
     8
     9
    10
    11
    12
    13
    14
    15
    16
    17
    18
    19
    20
    
       class Solution:
           def combinationSum4(self, nums: List[int], target: int) -> int:
               """
               Combination counting! -- but this is a juke, it's actually permutation counting
               It's an unbounded knapsack.
    
               dp[i] = the number of permutations that add up to i -- it's 1-indexed so the answer will be at dp[target]
               we iterate the outer loop over the nums since it's combination counting
    
               loop order doesn't matter here because we can take any number for any number of times
               """
               dp = [0] * (target + 1)
               dp[0] = 1 # k 1 way to make nothing
    
               for partial_sum in range(1, target + 1): # targets outer: permutation counting
                   for num in nums:
                       if partial_sum - num >= 0:
                           dp[partial_sum] += dp[partial_sum - num]
    
               return dp[target]

Longest Increasing Subsequence #

The use of patience sort is niche, but relevant to LIS problems.

The honest answer: patience sorting (bisect on a tails array) is applicable to a narrower family than it might seem.

The mechanism is specifically: maintain a sorted structure of “best reachable endpoints” and binary search to place each new element.

That only helps when the problem reduces to tracking an ordered frontier of candidates. Here’s the full enumeration of where it genuinely applies in LC.

Direct applications — same core mechanism

Longest Increasing Subsequence variants. LC 300 is the archetype. Any variant that asks for LIS length on a transformed sequence falls here directly:

][LC 1964 Find the Longest Valid Obstacle Course]]: LIS with non-strict inequality (bisect_right instead of bisect_left).

Increasing subsequence with transformed keys. Any problem where you can reduce to “find LIS of some derived sequence” is a candidate. The reduction step is the work; patience sorting is just the execution.

Indirect applications — patience sorting intuition but different execution

Minimum number of chains / piles needed to cover a sequence. By Dilworth’s theorem, the minimum number of non-increasing subsequences needed to partition a sequence equals the length of the longest strictly increasing subsequence. This is the theoretical link between patience sorting (which minimises pile count) and LIS. In practice this surfaces as:

  • LC 435 Non-overlapping Intervals: minimum removals to make non-overlapping = n minus maximum non-overlapping count. Greedy by earliest end-time. Not patience sorting directly, but the same Dilworth duality underlies it.
  • LC 406 Queue Reconstruction by Height: not patience sorting, but the greedy insertion logic has structural similarity.

Stock / greedy problems with a monotone frontier. Problems that maintain a sorted list of “active candidates” and binary search to update them:

  • LC 1671 Minimum Number of Removals to Make Mountain Array: requires LIS from left and LDS from right at each index. Patience sorting used twice as a subroutine.
  • LC 2111 Minimum Operations to Make Array K-Increasing: split into k interleaved subsequences, find LIS of each. Patience sorting per subsequence.

Where it does NOT apply despite seeming similar

  • LCS (Longest Common Subsequence): no sorted frontier, different recurrence. O(n²) DP or O(n log n) via reduction to LIS only when one sequence has distinct elements.
  • Longest Bitonic Subsequence: needs LIS + LDS at every index, patience sorting as subroutine but not directly.
  • Any problem where you need the actual subsequence (not just length): patience sorting needs backpointer augmentation and loses its implementation simplicity. The O(n²) DP is often cleaner when reconstruction is required.
  • Problems with non-total ordering on elements (e.g. 2D dominance without a sort trick): bisect requires a linear order, so patience sorting breaks unless you engineer one.

The recognition signal

Patience sorting is the right tool when: (1) you need the length of a longest chain under some strict or non-strict ordering, (2) elements can be linearly ordered or reduced to a linear order by preprocessing, and (3) O(n log n) matters (otherwise O(n²) DP is simpler and more flexible). The `bisect_left` vs `bisect_right` choice controls strict vs non-strict inequality in the chain — that’s the only tuning knob.

The cleaner way to think about it for your canonicals: patience sorting is not a separate canonical family, it’s an O(n log n) implementation technique for the LIS canonical. The canonical is “longest chain under partial order.” Patience sorting is one execution strategy. The other is O(n²) DP. Know when each is appropriate.

Problems #

  1. Longest Increasing Subsequence (300) This is a classic question, we have the slow 1D DP option and the patience sorting option to work with. The patience sorting option is useful for this because of the requirement that we need strictly increasing subsequences, so we’re almost playing the game of patience there. We just need to keep track of tails (smallest number in existing stacks) and spawn a new stack if the insertion index for it is not within the existing stacks. patience sorting version only works if we want to find the length of LIS, not the LIS itself (have to mod it).

    The optimal patience sort approach works like so, in \(O(nlogn)\) time
     1
     2
     3
     4
     5
     6
     7
     8
     9
    10
    11
    12
    13
    14
    15
    16
    17
    18
    19
    20
    21
    22
    23
    
        import bisect
    
        class Solution:
            def lengthOfLIS(self, nums: List[int]) -> int:
                tails = []  # tails[i] will hold the smallest ending number of an increasing subsequence of length i+1 (tail of a stack of cards)
    
                for x in nums:
                    # Find the insertion point for x in tails to maintain sorted order
                    # This corresponds to the leftmost pile top >= x (or new pile if none found)
                    idx = bisect.bisect_left(tails, x)
    
                    # CASE 1: add new pile:
                    # If x is larger than all pile tops, create a new pile by appending it
                    if idx == len(tails):
                        tails.append(x)
                    else:
                        # CASE 2: replace within existing stack:
                        # Otherwise, replace the existing pile top with x
                        # This keeps tails optimized to have smallest possible tail elements
                        tails[idx] = x
    
                # Number of piles equals length of longest increasing subsequence
                return len(tails)

    The patience sort version is optimal, but we can also employ a 1D-DP approach for this. dp[i] is the length of LIS for string formed by first i chars inclusive. This runs in \(O(n^{2})\)

     1
     2
     3
     4
     5
     6
     7
     8
     9
    10
    11
    
       class Solution:
           def lengthOfLIS(self, nums: List[int]) -> int:
               n = len(nums)
               # all init base to 1
               dp = [1] * (n)
               for right in range(1, n):
                   for left in range(right):
                       # we have this window and we need to find out if nums[right] can be incremented (if any smaller elements exist in the window)
                       if nums[right] > nums[left]:
                           dp[right] = max(dp[right], dp[left] + 1)
               return max(dp)
  2. TODO LC 673 Number of LIS

  3. TODO LC 1964 Find the Longest Valid Obstacle Course

Stock Trading and State-Machine DP #

  • This canonical group was supposed to revolve mainly around the use of FSMs in solving some problems – the stock trading family of problems ends up being a really good reference for this .

  • Multi-state progression: hold, rest, cooldown, buy/sell with constraints

Problems #

  1. Best Time to Buy and Sell Stock with Cooldown (309)

    Here, cooldown adds a “memory” requirement: you must know the previous state’s action to decide the current. Thus, this problem essentially requires DP/state-machine or similar approach instead of simple greedy.

    We know the there can be 3 possible states for every day, we have to track them all and comb through, on first instinct, we end up needing one array per state, but we can inspect it and know that we can just use 3 rolling variables instead.

    without the space optimisation
     1
     2
     3
     4
     5
     6
     7
     8
     9
    10
    11
    12
    13
    14
    15
    16
    17
    18
    19
    20
    21
    22
    23
    24
    25
    26
    27
    28
    29
    
        # M1: without the space optimisations
        class Solution:
            def maxProfit(self, prices: List[int]) -> int:
                if not prices:
                    return 0
    
                n = len(prices)
    
                # init 3-state dp:
                hold, sold, rest = [0] * n, [0] * n, [0] * n
    
                # init first vals, only hold matters, we can't sell that day
                hold[0] -= prices[0]
    
                for i in range(1, n):
                    hold[i] = max(
                        hold[i - 1], # was holding yesterday also:
                        rest[i - 1] - prices[i] # cooldown ytd, can buy today
                    )
                    sold[i] = hold[i - 1] + prices[i] # held until ytd, sold today
                    rest[i] = max(
                        rest[i - 1], # rested ytd also
                        sold[i - 1], # sold ytd, rested today
                    )
    
                return max(
                    sold[n - 1], # sold on the last day
                    rest[n - 1] # rested on the last day
                )
    Adding in the state rollups:
     1
     2
     3
     4
     5
     6
     7
     8
     9
    10
    11
    12
    13
    14
    15
    16
    17
    18
    19
    20
    21
    22
    23
    24
    25
    26
    27
    28
    29
    
        # M2: with the space optimisations:
        class Solution:
            def maxProfit(self, prices: List[int]) -> int:
                if not prices:
                    return 0
    
                n = len(prices)
    
                # init 3-state dp:
                hold, sold, rest = -prices[0], 0, 0
    
                for i in range(1, n):
                    # snapshot the old vals:
                    p_hold, p_sold, p_rest = hold, sold, rest
    
                    hold = max(
                        p_hold, # was holding yesterday also:
                        p_rest - prices[i] # cooldown ytd, can buy today
                    )
                    sold = p_hold + prices[i] # held until ytd, sold today
                    rest = max(
                        p_rest, # rested ytd also
                        p_sold, # sold ytd, rested today
                    )
    
                return max(
                    sold, # sold on the last day
                    rest # rested on the last day
                )

Knapsack/Subset Sum Variants #

  • In knapsack problems, we’re interested in whether to take or not take things (and put them in our knapsack.)

  • Knapsack problems can be bounded (limited by the number of times a choice can be made) or unbounded choices (unlimited), partitioning, subset formation

  • within this, we might try to do counting (of combinations…), optimising some statistic based on counts, or try to determine a predicate outcome (e.g. whether a pattern can be formed)

Problems #

  1. Coin Change (322)

    This is a optimisation problem AND there’s more than one way to do the coin changing (so equivalent paths in the decision-tree) and that’s why it’s more of a DP than a greedy approach that we adopt..

    Applying the DP framework:

    StepExplanationStatus/Notes
    1. Define SubproblemsNumber of coins needed to form amount <= target amountdp[i] = min coins needed for amount i
    2. Check Overlapping Subproblemsdp[i] depends on solutions of dp[i - coin] for each coinYes, overlapping subproblems exist
    3. Identify Choices Per SubproblemChoose any coin from coins that fits into current amountMultiple choices per state
    4. Confirm Optimal SubstructureOptimal for amount i computed from optimal solutions for smaller amountsYes, minimum over choices is optimal
    5. Compute Solution (Top-Down / Bottom-Up)Bottom-up iterative or top-down memoized recursion both applicableBoth implemented correctly
    6. (Optional) Construct Actual SolutionCan store chosen coins to reconstruct pathNot implemented in provided solutions

    This question is a DP because we are dealing with arbitrary denominations and with unlimited amounts of each coin. If we had been given a fixed amount, then it would have been a greedy solution instead because we would have to pick the best choice each time (locally optimal) to give the “biggest jump”.

    Also note that we’re intentionally doing a 1-idx approach so that dp[i] = min number of ways to make up value = i.

     1
     2
     3
     4
     5
     6
     7
     8
     9
    10
    11
    12
    13
    
        # iterative bottom up approach
        class Solution:
            def coinChange(self, coins: List[int], amount: int) -> int:
                # This init of inf avoids negative values by using infinity, making code cleaner.
                dp = [float('inf')] * (amount + 1)
                dp[0] = 0
    
                for amt in range(1, amount + 1):
                    for coin in coins:
                        if coin <= amt:
                            dp[amt] = min(dp[amt], dp[amt - coin] + 1)
    
                return dp[amount] if dp[amount] != float('inf') else -1
     1
     2
     3
     4
     5
     6
     7
     8
     9
    10
    11
    12
    13
    14
    15
    16
    17
    18
    19
    20
    21
    
        # recursive topdown approach
        from functools import cache
        class Solution:
            def coinChange(self, coins: List[int], amount: int) -> int:
                @cache
                def helper(amount):
                    if amount == 0:
                        return 0
    
                    if amount < 0:
                        return -1
    
                    options = [helper(amount - denom) for denom in coins if
                               amount - denom >= 0 and helper(amount - denom) != -1]
    
                    if not options:
                        return -1
                    else:
                        return min(options) + 1
    
                return helper(amount)
  2. Coin Change II (518) :: 2D in naive implementation, 1D rollup easy – is a combination counting problem

    The key idea here is to solve an unbounded variant of the knapsack problem (unbounded because each coin (choice) can be used as many times as desired). The main part of this problem is to realise that it is indeed a 2D or whatever implementation, however, we can easily roll it up into a 1D array, we attempt to use each coin and update the partial sums we can create and in that way each coin can be used as many times as possible.

    dp[i] is the number of ways to make up that amount = i. Also realise that this a combination counting and we need to absolutely avoid double counting (and caring about permutations) that’s why the outer loop is on coins. We introduce the coin denominations one by one because we only care about combinations.

    This allows us to introduce coin by coin (combination) instead of caring about when they’re introduced (permutations).

    Loop order is left to right because we can use the left sub-answers to build the right sub-answers.

     1
     2
     3
     4
     5
     6
     7
     8
     9
    10
    11
    12
    13
    14
    15
    
        class Solution:
            def change(self, amount: int, coins: List[int]) -> int:
                dp = [0] * (amount + 1)
    
                # trivial base: 1 way to have empty combi
                dp[0] = 1
    
                # iterates through each coin denomination. This ordering ensures combinations are counted without double counting combinations (because they are permuted differently) (crucial for problems counting combinations).
    
                for coin in coins:
                    # use the same denom as many times as possible:
                    for amt in range(coin, amount + 1):
                        dp[amt] += dp[amt - coin]
    
                return dp[amount]
    Code Snippet 3: bottom-up, iterative solution
    by the way, this is how we can see the non-flattened 2D approach to connect the dots on how to flatten it into 1D

    2D definition:

    • dp[i][j] = number of ways to make up amount j using the first i coins (i.e., coins ) (slice)
    • This is the traditional “table” version before you flatten.

    So, base cases:

    • For any i, dp[i][0] = 1 (There is exactly one way to make amount 0: take nothing.)
    • For dp[0][j] = 0 when j > 0 (With 0 coins, can’t form any positive amount.)

    So our state transitions look like this:

    dp[i][j] = dp[i−1][j] + dp[i][j−coins[i−1]] if j≥coins[i−1]

    dp[i][j]=dp[i−1][j] if j<coins[i−1]

    which gives the following 2D version of the code

     1
     2
     3
     4
     5
     6
     7
     8
     9
    10
    11
    
        n = len(coins)
        dp = [[0] * (amount + 1) for _ in range(n + 1)]
        for i in range(n + 1):
            dp[i][0] = 1
    
        for i in range(1, n + 1):          # using first i coins
            for j in range(amount + 1):     # for each target amount
                if j >= coins[i-1]:
                    dp[i][j] = dp[i-1][j] + dp[i][j - coins[i-1]]
                else:
                    dp[i][j] = dp[i-1][j]

    and our answer would be at dp[n][target]

    In the 1D flattening:

    • We roll up the i dimension (coins) into an update rule by ensuring we process coins one at a time and, for each coin, accumulate number of ways for all amounts in order.

    • The key is: for each coin, update dp[amt] += dp[amt - coin] iteratively for all amt ≥ coin.

    That’s why we get:

    1
    2
    3
    4
    5
    
       dp = [0] * (amount + 1)
       dp[0] = 1
       for coin in coins: # this is the "row" progression (when compared to the 2D version)
           for amt in range(coin, amount + 1):
               dp[amt] += dp[amt - coin]

    The outer loop on coins is equivalent to the 2D DP’s row progression—each new coin can only build on results found so far, not reorder the combinations.

  3. Target Sum (494) – bounded 0/1 Knapsack Problem

    We can solve this using a recursive approach, or we can rephrase the question into one that is a bounded knapsack problem where the LHS = RHS.

    the recursive approach:

    The idx here within the helper is a choice index, it’s the current element that we have to make a choice on, we’re once again going from right to left of the options.

     1
     2
     3
     4
     5
     6
     7
     8
     9
    10
    11
    12
    13
    14
    15
    16
    17
    18
    19
    20
    21
    22
    23
    
            # M1: Recursive solution:
            from functools import cache
            class Solution:
               def findTargetSumWays(self, nums: List[int], target: int) -> int:
    
                   @cache
                   def helper(idx, target):
                       if idx == 0:
                           a, b = nums[idx], -nums[idx]
    
                           # trivial edge case when target = 0 then both will work and that's 2 wayss
                           if a == target and b == target:
                               return 2
    
                           return 1 if a == target or b == target else 0
    
                       num = nums[idx]
                       # choose this:
                       pos_ways, neg_ways = helper(idx - 1, target + num), helper(idx - 1, target - num)
    
                       return pos_ways + neg_ways
    
                   return helper(len(nums) - 1, target)

    3 things to take note:

    1. the way we convert it into a bounded knapsack problem — it’s because we realise that it’s an inclusion / exclusion decision that we’re doing AND we’re building up to a value.
    2. the problem being a combination sum problem, therefore we have to iterate over the options to avoid double counting
    3. and the order of traversal of the range (right to left) – this is because we only have a bounded 0/1 inclusion – so going right to left only allows us to do single inclusions (or exclusions) and avoids any sort of double counting / reuse of an element.
       1
       2
       3
       4
       5
       6
       7
       8
       9
      10
      11
      12
      13
      14
      15
      16
      17
      18
      19
      20
      21
      22
      23
      24
      25
      
                 # M2: Iterative bottom up with space optimisation (1D array rollup that allows us to reduce space usage)
                 class Solution:
                    def findTargetSumWays(self, nums: List[int], target: int) -> int:
                        # rephrasing the problem into a bounded knapsack problem.
                        # we want to have LHS = RHS the target will also be split
                        num_sum = sum(nums)
                        total = (num_sum + target)
                        if is_impossible:=(num_sum < abs(target) or total % 2 != 0):
                            return 0
      
                        knap_target = total // 2
                        # we want to select subset such that we will reach knap_target.
                        # dp[i] = num ways to reach target val = i
                        dp = [0] * (knap_target + 1)
                        dp[0] = 1
      
                        # NOTE [1]: we wanna find the number of combis here, so we control the order of inclusion of nums so that we don't end up doing permutation_sum type and end up with double counting
                        for num in nums:
                            # NOTE [2]: the order here matters and we need to go from right to left (big to small) to avoid double counting.
                            # for target_sum in range(num, knap_target + 1): # so this will be wrong
                            for target_sum in range(knap_target, num - 1, -1):
                                gap = target_sum - num
                                dp[target_sum] += dp[gap]
      
                        return dp[knap_target]

0/1 Knapsack and Bounded Subset Choices #

  1. Loop structure matters a lot here for knapsack-style DP questions:

    1. Bounded (0/1) or unbounded? \(\Rightarrow\) 0/1: inner loop backward \(\Rightarrow\) unbounded: inner loop forward

    2. Counting combinations or permutations? \(\Rightarrow\) combinations: items (nums) in outer loop, targets in inner \(\Rightarrow\) permutations: targets in outer loop, items in inner These two decisions are independent and compose.

      • Target Sum is 0/1 (backward inner) + combination count (nums outer).

      • Coin Change II (LC 518) is unbounded (forward inner) + combination count (coins outer).

        Getting these wrong produces a solution that passes most cases but fails on inputs with repeated elements.

  2. permutation vs combination count

    Table 1: 2axes: boundedness vs permutation/combination counting
    counting objectivebounded 0/1 each item onceunbounded (items reusable)
    combinationnums outer, targets backwardnums outer, targets forward
    permutationtargets outer, nums backward *targets outer, nums forward

    The * cell (bounded permutation) is rare and mostly doesn’t appear in LC, but the table is complete.

    [permutation count:]

    • If you loop: for target in range(...): for num in nums:
      • each target sum is built up by trying all nums in sequence for each partial target. The same subset {a, b} can be counted as (a then b) and (b then a) separately. You get permutation count.

    [combination count:]

    • If you loop: for num in nums: for target in range(…):
      • you fix the order in which nums are introduced. A subset can only be assembled in the order nums appears in the outer loop. {a, b} is only ever counted once. You get combination count.
  3. The generalisation worth internalising: the 0/1 backward-sweep rule extends to as many dimensions as you have independent resource constraints.

    • Two budgets → 2D table, both loops backward.
    • Three budgets → 3D table, all three loops backward. The rule doesn’t change; only the number of nested loops scales.

Problems #

  1. Partition Equal Subset Sum (416)

    Yet another classic, we are asked to return True if there’s a partition of an array into 2 subsets such that we have equal sums. This is just a slight rewording of the classic 0/1 knapsack problem (because we either include or exclude choice that’s why 0/1). It’s a reword because if we find one subset, then we have also found the other subset (equalling to half of what we found). For implementation, a 1D array approach is already a space optimised rolling approach. We need to be careful about how we build up values to avoid the double counting, which is why we need to work backwards on the dp array index for each num case (so as to not double-count multiple uses of num).

     1
     2
     3
     4
     5
     6
     7
     8
     9
    10
    11
    12
    13
    14
    15
    16
    17
    18
    19
    20
    21
    22
    23
    24
    
        from typing import List
    
        class Solution:
            def canPartition(self, nums: List[int]) -> bool:
                total_sum = sum(nums)
                if total_sum % 2 != 0:
                    return False  # sum is odd, cannot split equally
    
                target = total_sum // 2
    
                dp = [False] * (target + 1) # dp[i] is True if value = i can be achieved from summing elements within a subset.
                dp[0] = True
    
                # NB: inclusion / exclusion of number is classic for 0/1 bounded knapsack, that's why this is the outer for loop
                for num in nums:
                    # iterate backwards to avoid reuse of the same item multiple times in the same iteration
                    for i in range(target, num - 1, -1):
                        dp[i] = (
                            dp[i] # exclusion case
                            or
                            dp[i - num] # inclusion
                        )
    
                return dp[target]
  2. Target Sum (494) – bounded 0/1 Knapsack Problem We can solve this using a recursive approach, or we can rephrase the question into one that is a bounded knapsack problem where the LHS = RHS.

    the recursive approach:

    The idx here within the helper is a choice index, it’s the current element that we have to make a choice on, we’re once again going from right to left of the options.

     1
     2
     3
     4
     5
     6
     7
     8
     9
    10
    11
    12
    13
    14
    15
    16
    17
    18
    19
    20
    21
    22
    23
    
            # M1: Recursive solution:
            from functools import cache
            class Solution:
               def findTargetSumWays(self, nums: List[int], target: int) -> int:
    
                   @cache
                   def helper(idx, target):
                       if idx == 0:
                           a, b = nums[idx], -nums[idx]
    
                           # trivial edge case when target = 0 then both will work and that's 2 wayss
                           if a == target and b == target:
                               return 2
    
                           return 1 if a == target or b == target else 0
    
                       num = nums[idx]
                       # choose this:
                       pos_ways, neg_ways = helper(idx - 1, target + num), helper(idx - 1, target - num)
    
                       return pos_ways + neg_ways
    
                   return helper(len(nums) - 1, target)
    Why is the outer loop is on the nums and the inner loops is on the target values – combinations vs permuatations

    This controls whether you count combinations or permutations.

    [permutation count:]

    • If you loop: for target in range(...): for num in nums:
      • each target sum is built up by trying all nums in sequence for each partial target. The same subset {a, b} can be counted as (a then b) and (b then a) separately. You get permutation count.

    [combination count:]

    • If you loop: for num in nums: for target in range(…):
      • you fix the order in which nums are introduced. A subset can only be assembled in the order nums appears in the outer loop. {a, b} is only ever counted once. You get combination count.

    This problem asks for ways, which is combination count. So nums is the outer loop. This is not specific to target sum — it is the general rule for any “count subsets” DP.

    3 things to take note:

    1. the way we convert it into a bounded knapsack problem — it’s because we realise that it’s an inclusion / exclusion decision that we’re doing AND we’re building up to a value.
    2. the problem being a combination sum problem, therefore we have to iterate over the options to avoid double counting
    3. and the order of traversal of the range (right to left) – this is because we only have a bounded 0/1 inclusion – so going right to left only allows us to do single inclusions (or exclusions) and avoids any sort of double counting / reuse of an element.
       1
       2
       3
       4
       5
       6
       7
       8
       9
      10
      11
      12
      13
      14
      15
      16
      17
      18
      19
      20
      21
      22
      23
      24
      25
      
            # M2: Iterative bottom up with space optimisation (1D array rollup that allows us to reduce space usage)
                 class Solution:
                    def findTargetSumWays(self, nums: List[int], target: int) -> int:
                        # rephrasing the problem into a bounded knapsack problem.
                        # we want to have LHS = RHS the target will also be split
                        num_sum = sum(nums)
                        total = (num_sum + target)
                        if is_impossible:=(num_sum < abs(target) or total % 2 != 0):
                            return 0
      
                        knap_target = total // 2
                        # we want to select subset such that we will reach knap_target.
                        # dp[i] = num ways to reach target val = i
                        dp = [0] * (knap_target + 1)
                        dp[0] = 1
      
                        # NOTE [1]: we wanna find the number of combis here, so we control the order of inclusion of nums so that we don't end up doing permutation_sum type and end up with double counting
                        for num in nums:
                            # NOTE [2]: the order here matters and we need to go from right to left (big to small) to avoid double counting.
                            # for target_sum in range(num, knap_target + 1): # so this will be wrong
                            for target_sum in range(knap_target, num - 1, -1):
                                gap = target_sum - num
                                dp[target_sum] += dp[gap]
      
                        return dp[knap_target]
  3. Ones and Zeroes (474) This question is a classic 0/1 bounded knapsack but with extra conditions to govern the choices, it’s the choices that effectively act as a “cost” (in the framing of the knapsack problem, where a choice has a cost). For each str in strs, we try to include or exclude it, we can only use a str once that’s why it’s a 0/1 knapsack. It’s very important for us to do the right to left sweeping else we won’t be able to enforce the single use constraint that is at the heart of this 0/1 family of problems.

    solution with extended working
     1
     2
     3
     4
     5
     6
     7
     8
     9
    10
    11
    12
    13
    14
    15
    16
    17
    18
    19
    20
    21
    22
    23
    24
    25
    26
    27
    28
    29
    30
    31
    32
    33
    34
    35
    36
    37
    
       class Solution:
           def findMaxForm(self, strs: List[str], m: int, n: int) -> int:
               """
               We know that this is a 0/1 bounded knapsack problem, it's the unique costs that we need to account for.
    
               Let's do the 2D version first
    
    
               dp[i][j] =  size of largest subset with i0s and j1s  (1-indexed)
    
               So the ans will be at dp[m][n]
    
               Since it's a subset, it's a combination that we're focused on so it's combination counting.
               """
    
               dp = [[0] * (n + 1) for _ in range(m + 1)]
               dp[0][0] = 0
    
               # since we're doing combination counting, we go through the options:
               for s in strs:
                   zero_count = s.count('0')
                   one_count = len(s) - zero_count
                   if zero_count > m or one_count > n: # if out of bounds, can't try including this.
                       continue
    
                   # now we do the 0/1 inclusion, it's on two different dimensions, so we right to left on both
                   for balance_m in range(m, zero_count - 1, -1):
                       for balance_n in range(n, one_count - 1, -1):
                           gap_m, gap_n = balance_m - zero_count, balance_n - one_count
                           dp[balance_m][balance_n] = max (
                               # exclusion case: keep the best subset achievable without this string
                               dp[balance_m][balance_n],
                               # accumulate using this:
                               1 + dp[gap_m][gap_n]
                           )
    
               return dp[m][n]
    complexity analysis

    Time and Space Complexity

    Time complexity: O(L×m×n) where LL is the number of strings in strs, and m,nm,n are the allowed zeros and ones respectively. LL iterations for all strings, and within each, an O(mn) iteration for dp updates.

    Space complexity: O(m×n) for the dp table.

    Given constraints (m,n≤100m,L≤600), this is efficient enough for typical inputs.

    1
    2
    3
    4
    5
    6
    7
    8
    9
    
       def findMaxForm(self, strs, m, n):
           dp = [[0] * (n + 1) for _ in range(m + 1)]
           for s in strs:
               zeros = s.count('0')
               ones = len(s) - zeros
               for i in range(m, zeros - 1, -1):
                   for j in range(n, ones - 1, -1):
                       dp[i][j] = max(dp[i][j], 1 + dp[i - zeros][j - ones])
           return dp[m][n]
    Code Snippet 4: extra conditions and dimensions still get traversed right to left

Non-standard State Representation / Compression DP e.g. bitmasking ⭐️ #

  • When it comes to state compression, we’ve seen the dimensional flattening that happens when we roll-up variables, that’s a decent bucket. There’s other approaches to, such as:
    • using bitmasks to represent state
    • rethinking what state we should be keeping track of and using something like a map for it
  • State represented by bitmask or difference for subset/assignment/exact grouping

Problems #

  1. Number of Unique Good Subsequences (1987)

    If we explore the non compressed version, we know that it’s a subsequence counting, so dp[i] = (# good seqs ending with 0, # good seqs ending with 1, has_encountered_0_before)

    The has_encountered_zero_before just helps us to know if "0" can be a good subsequence or not.

    As for the state transitions:

    1. if curr is 1: then we can end it with 1 for whatever we are accumulating so far

    2. if curr is 0: then we can only add that to those that end with 1 (to avoid leading zeroes)

      Then we realise that we can just a triple of rolling variables based on the state transitions.

       1
       2
       3
       4
       5
       6
       7
       8
       9
      10
      11
      12
      13
      14
      15
      16
      17
      18
      
                 class Solution:
                     def numberOfUniqueGoodSubsequences(self, binary: str) -> int:
                         MOD = 10**9 + 7
                         count_end_with_0 = 0
                         count_end_with_1 = 0
                         has_zero = False
      
                         for ch in binary:
                             if ch == '1':
                                 # Append '1' to all subsequences (ending with 0 or 1) + start new subsequence '1'
                                 count_end_with_1 = (count_end_with_0 + count_end_with_1 + 1) % MOD
                             else:
                                 # Append '0' only to subsequences ending with '1'
                                 has_zero = True
                                 count_end_with_0 = (count_end_with_0 + count_end_with_1) % MOD
      
                         # sum subsequences ending with 0 and 1 plus subsequence "0" if exists
                         return (count_end_with_0 + count_end_with_1 + (1 if has_zero else 0)) % MOD
  2. ⭐️Tallest Billboard (956) (represent via map)

    step-by-step building up to the ans intuition

    This feels less like magic if we start with the brute force and watch what structure emerges.

    [ 1. consider the brute force ]

    1. for each rod, we have 3 options: (so \(n\) rods give \(3^{n}\) assignments)

      1. don’t use it
      2. add to the left pile that is being built
      3. add to the right pile that is being built
    2. in the brute force approach, the answer is the height of the shorter pile when both piles are of equal height \(\implies\) largest value of min(left, right) where left === right

    3. brute force won’t be enough because we have \(n\) at most 20 and that’s in the billions \(\implies\) we need to look for pruned approaches – there’s a lot of duplicate effort beacuse of overlapping subproblems

    [ 2.Rethink the state representation ]

    1. what’s important to track?
      • NOT: which pile a specific rod went to – so the choices are irrelevant to track
      • YES: r/s between the two pile heights is what is important – so each rod, has 3 decisions :
        1. accumulate discrepancy (add to larger, make it a bigger gap between smaller and larger)
        2. reduce the discrepancy (add to smaller, so that diff is reduced)
        3. don’t use it – discrepancy remains as is
    2. how to track?
      • diff and height of smaller stack is sufficient \(\implies\) 2 variables are sufficient.
      • so state is just a (diff, short)
      • given sum of rod heights based on inputs is at most 20 * 1000 = 20000, then we can have 20k distinct d-values
      • For each d, we want the max possible short length – that’s how we track it
      • compression:
        • instead of : tracking every (d, short) pair
        • we shall track: dp[d] = max short achievable for this difference \(\implies\) 1 number per difference value.
    3. state space collapse is from \(3^{n}\) to \(O(sum of rods)\)

    [ 3. Understanding the Transitions ]

    1. if existing state is dp[d] = smaller.
      • somewhere in the rods processed so far, we have assignment where tall - short = d and short = smaller, so tall = smaller + d.
    2. for new rod in consideration: 3 options
      1. skip it dp[d] unchanged
      2. add to taller pile – gap widens so new state: dp[d+ rod] = smaller
      3. add to shorter pile: we have 2 sub-cases
        1. rod < d : gap shorter, no flipping

        2. rod >= d: flipping of short and tall

          we can collapse this into: new_diff = abs(d - rod), new_smaller = smaller + min(d, rod)

    [ 4. Filling order caveats ]

    1. to avoid mixing up the state across two iterations, use the copy trick to separate reads and writes.
    general building up to the main answer

    The unlock sequence:

    1. Brute force is 3ⁿ. What am I tracking? Left sum and right sum — two numbers.

    2. Can I compress? I don’t need both absolute values.

      I need their difference and one of them.

      Pick the smaller one because that’s what the answer asks for (answer is dp[0]).

    3. What’s the state space of differences? Bounded by total rod length. Much smaller than 3ⁿ.

    4. Write dp as a map from difference to max-smaller. Transitions follow from “what happens to (d, smaller) when I add a rod to tall, short, or neither.”

    This is wild, the first thing to notice is that many subsets share the same difference between them so we can consider keeping track of the differences that we find as we build things. That’s why dp{k:v} where k is the difference, d, between the sums of the two subsets formed so far and v is the max sum of the smaller subset amongst the two for this difference, d.

     1
     2
     3
     4
     5
     6
     7
     8
     9
    10
    11
    12
    13
    14
    15
    16
    17
    18
    19
    20
    21
    22
    23
    24
    25
    26
    27
    28
    29
    30
    31
    32
    
        class Solution:
            def tallestBillboard(self, rods: List[int]) -> int:
                # dp{k:v} where k = d = diff b/w left and right poles (subsets); v = max sum of the smaller subset for the difference
    
                dp = {0:0} # trivial when both subsets are empty
    
                for rod in rods:
                    # for each diff we encounter so far, we can do 3 things:
    
                    # A: do not use the rod
                    # B: use the rod, add it to the "taller" subset => this updates the differences from d to d + rod, no change to the smaller subset sum
                    # C: use the rod, add it to the "shorter" subset => this updates the  diff to be |d - r| (short and tall may swap places)
                    # eventually, answer will be max value within dp[0]
    
                    copy = dp.copy() # copying keeps the changes unaffected across multiple cases. separates reads and writes.
                    for d, smaller in dp.items():
                        # case A: no use:
                        copy[d] = max(copy.get(d, 0), smaller) # keep the bigger val
                        # case B: add to taller
                        copy[d + rod] = max(copy.get(d + rod, 0), 0, smaller)
                        # case C: add to the shorter one:
                        diff = abs(d - rod)
                        if d < rod:
                            updated_smaller = smaller + d
                        else:
                            updated_smaller = smaller + rod
    
                        copy[diff] = max(updated_smaller, copy.get(diff, 0))
    
                    dp = copy
    
                return dp.get(0, 0)

Word Segmentation and Dictionary DP #

  • Segment string with a dictionary, reconstruct valid sentences

Problems #

  1. Word Break (139) all about segmentability

    1. We are asked for whether a solution exists, not asked for the best solution – we can explore non-greedy approaches. Also, we realise that we need to build up to an optimal answer for the question, so that’s why we go left to right on the string-indices.

    2. This is about creating valid segments and there’s a lookup that we would need to do for validity checks. We are asked to find out whether we can segment for this version of the problem.

    3. The substructure property is as such,

      1. if my last segment I can make from the right is legit, and the immediate left of that is also a valid segment then I’m fine with the whole segmentation process.
      2. dp[i] keeps track of whether a valid segment ends at ith idx (follow pythonic convention, exclusive range).

      The j marker exists because for s[i] to be a segmentable point, we need to look to its left to:

      1. find a segment point (j) that is valid AND
      2. the remaining slice has to be a valid word (i.e. s[j:i] to be a valid word)
       1
       2
       3
       4
       5
       6
       7
       8
       9
      10
      11
      12
      13
      14
      15
      16
      17
      
                 class Solution:
                 def wordBreak(self, s: str, wordDict: List[str]) -> bool:
                     wordSet = set(wordDict)  # O(1) lookup
                     n = len(s)
                     dp = [False] * (n + 1) # dp[i] = True if first i characters are segmentable
                     dp[0] = True  # empty substring is segmentable
      
                     max_word_length = max(map(len, wordDict)) if wordDict else 0
      
                     for i in range(1, n + 1):
                         # Limit left boundary (j) to speed up checks (optimization)
                         for j in range(max(0, i - max_word_length), i):
                             if dp[j] and s[j:i] in wordSet:
                                 dp[i] = True
                                 break  # early break for efficiency, since we know that dp[i] is already a legit segmentation.
      
                     return dp[n]
    4. max_word_length optimisation:

      • Since we know that the search space of words is fixed, we know that the values for j are bounded, so we can be strict about how many possibilities to search.
        • The justification is: s[j:i] can only be in the wordset if its length is at most max_word_length, so any j further left than i - max_word_length can never produce a valid word.
  2. Word Break II (140)

    This is a mix of approaches, it’s somewhat tedius but not difficult to consider if we think calmly and strategise based on the information that we need. Word Break I answers a yes/no question. Word Break II asks for all valid segmentations. The moment the answer is a list of all solutions, backtracking is implicated — you cannot avoid enumerating the decision tree. DP alone cannot accumulate variable-length structured outputs like sentences.

    1. phase 1 (DP pre-processing) To create sentences, we need to consider segmentability, and that’s what Word Break I (139) deals with – so we have a way to use DP on this. That becomes a pre-processing step for us, and the DP array ends up being an intermediate info that we use.

    2. phase 2: backtracking

      For the sentence accumulation, realise that it’s an accumulation, we want all the configs – it’s going to be a costly algo, like backtracking.

      • – that it’s a matter of exploring decision curves (break here as a fork between multiple sentences?…) so it’s a backtracking that we need to do. Every segmentable index is a n-ary choice – each possible word is a start there. Branching factor is bounded by the number of words in the dictionary whose prefix matches the current position

The backtracking can be assisted by a helper DP array or we may directly rely on memoised calls to a backtrack function.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
         # V2: bottom up, dp-assisted
         from typing import List
         from functools import cache

         class Solution:
         def wordBreak(self, s: str, wordDict: List[str]) -> List[str]:
            """
            This question is a mix of multiple approaches.

            {preprocessing}

            1. we need to know about "segmentability" i.e. can i segment at this location?  ==> this is what we can use a DP for

            2. need info on where I could have started from given ith idx is the end-segment. ==> need to know valid_starts

            {gathering the sentences}
            1. we start from the right possible index for spaces, then we explore the different approaches and gather a path.

            this is a decision tree traversal, so it's dfs-like / backtracking that we're doing here
            """
            n = len(s)
            wordset = set(wordDict)

            # so dp[i] ==> I can segment at ith idx (for insertion point)
            dp = [False] * (n + 1)
            dp[0] = True # vacuous truth

            # valid starts is going to be a list of lists, it means that ith idx is the end, and keeps track of where we could have started from.
            valid_starts = [[] for _ in range(n + 1)]

            for i in range(1, n + 1): # i is the right exclusive idx for slicing, hence goes till i = n
                for j in range(i): # j is the left inclusive idx for slicing
                    if dp[j] and s[j:i] in wordset:
                        dp[i] = True
                        valid_starts[i].append(j)

            if not dp[n]: # early returns for can't segment
                return []


            """
            After preprocessing, now we can backtrack:
            """
            @cache
            def backtrack(end):
                # end states:
                if end == 0: # i.e. reached the starting idx
                    return [""]

                sentences = []
                for start in valid_starts[end]:
                    word = s[start:end]
                    prefixes = backtrack(start)
                    for prefix in prefixes:
                        if prefix:
                            sentences.append(f"{prefix} {word}")
                        else: # for the empty string case @ terminus (starting idx = end = 0)
                            sentences.append(word)
                return sentences

            return backtrack(n)

and this is the merely memoised approach, which is easier to think of first:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
         from functools import cache

         class Solution:
         def wordBreak(self, s: str, wordDict: List[str]) -> List[str]:
            wordSet = set(wordDict)

            @cache
            def backtrack(start_idx):
                # end-cases:
                if start_idx == len(s):
                    return [""] # trivially, we return a sentence with empty string

                res = [] # sentences to form starting with start_idx
                for end in range(start_idx + 1, len(s) + 1):
                    word = s[start_idx:end]
                    if not word in wordSet:
                        continue

                    # we can backtrack now:
                    choices = backtrack(end)
                    for sub in choices:
                        if sub:
                            res.append(f"{word} {sub}")
                        else: # current word is last word
                            res.append(word) # this is the last word in the sentence
                return res

            return backtrack(0)

DP with Rolling Variables / Space Optimization #

  • Reduce storage to minimal multi-var tracking
  • typically rolling of variables is something we use to do space usage optimisations – so this category isn’t exactly a self-standing one, that’s why the problems below have been out-linked to other sites
  • it’s easier to intuit how to do the space-saving once we inspect the recurrence relationship, especially the generic one that shows how the \(k + 1\) case relates to the \(k\) case.

Problems #

  1. Climbing Stairs (70) :: optimised
  2. Min Cost Climbing Stairs (746) :: rolling two-variable method
  3. Maximum Product Subarray (152) :: track max/min in constant space

SOE #

  1. Rolling variable optimisations typically also involve changes to the iteration range, be careful on that.

    see this elaboration

    The difference in the iteration range between the DP array version and the two-variable (“rollup”) version comes down to two things: indexing conventions and how you track the “state” across iterations.

    1. Array DP Version (dp array)

      • The dp array is of length L+1, where dp[i] means: “number of ways to decode the substring s[:i]”.
      • You initialize dp = 1 (empty string), and dp = 1 if the first character is valid.[1]
      • The loop runs from i = 2 to L (i.e., range(2, L+1)), and you check the last one or two digits ending at index i-1 and i-2.
      • This is 1-based with a “dummy” base at dp, so you need to go all the way to L.
    2. Two-Variable Rolling Version

      • Instead of keeping a full array, you only keep enough history for two previous states (prev, curr) — which correspond to dp[i-2] and dp[i-1], respectively.
      • You start at i = 1 and go to len(s)-1 (so range(1, len(s))), since:
      • At each index i, you’re considering the substring ending at s[i], which, with Python’s slicing, aligns with how you retrieve the digits for single-digit and two-digit checks: s[i] and s[i-1:i+1].
      • The initialization lines (prev = curr = 1) handle the base cases for the empty prefix and the first digit, so your loop can safely start at index 1 (second character of s).

    Why the ranges differ:

    • DP Array: Needs to cover all possible string positions in a 1-based approach due to the extra initial base state.
    • Rolling Variables: Tracks only the minimal required info, and because of how states are rolled forward, the effective start point is different (starts after the very first character).

    Summary Table

    MethodState/IndexingRangeRationale
    DP Array1-based, dp base2 to L+1 (n+1)Need the two previous for `dp[i]`, and there’s a dummy base at index 0
    2-Variable Roll0-based, positions1 to L-1Only need to look back two steps; base cases set up before loop

    This convention is common in rolling/compressed DP: the rolling index starts later because the base cases (`prev`, `curr`) are set up before the loop, and you always have at least two characters to look back on. This minimizes storage but shifts the loop start point.

    References and further reading:

    • AlgoMonster explanation[1]
    • Tom VanAntwerp Leetcode DP breakdown[2]
    • YouTube walkthroughs[4]

    1 2 3 4 5 6 7 8

  2. Incorrect base case initialisation

  3. Wrong iteration order leading to overwriting valid states

  4. Misinterpretation of state meaning or transition relationship

  5. Multiplying negative numbers:

    • if we’re seeking to find out an eventual max, we need to keep track of both min negative number and max positive number if we care about their products

      for product-related accumulators, remember that we would likely have to handle both positive and negative accumulators (high, low) because if we do negative operands, then the negative value (low) might flip and become a large positive!

      this is seen in the “Maximum Product Subarray” question.

Decision Flow #

  1. Is problem about sequential optimal choice? \(\Rightarrow\) DP
  2. Want space-optimized solution? → Rolling array techniques
  3. subarray related (contiguous) and operation in consideration is associative \(\Rightarrow\) optimal substructure shown \(\Rightarrow\) explore the substructure property, what needs to be tracked? If it’s left and right boundaries \(\Rightarrow\) 2D DP, else might be a smaller dimension too
  4. Complex constraints or multiple states? → Multi-dimensional DP
  5. “contiguous subarray” + “maximise sum” + values can be negative → Kadane, immediate.

Style, Tricks and Boilerplate #

Style #

  • I think in general, it’s a good idea to just do the first-reach 2D tables and then look for space optimisations.

  • When working with 2D tables, just define the length/breadth symbols being used via comments to prevent getting confused.

    I think it’s VERY IMPORTANT to define the meanings of i and j and the table dimension meanings when writing out the answer so that we don’t end up wasting time rethinking that part or worse, getting confused.

    the determining of filling order and state transitions also becomes easier.

  • General rule of thumb for space optimisations The general rule of thumb (for flattening 2D into 1D state tracking)

    • 0/1 knapsack (each item once) \(\implies\) iterate capacity/target backwards so you don’t reuse the same item in one iteration. (this is for the 1D array rollup)

    • Unbounded knapsack (items unlimited) \(\implies\) iterate forwards so that the newly updated states can be reused immediately for multiple copies of the same item.

    • Combination counting with unlimited reuse (coin change) \(\implies\) coins outer loop, amount inner loop forward.

    • Combination counting with one-use-only (subset sum) \(\implies\) nums outer loop, amount inner loop backward.

Tricks #

  1. DP questions often init array with 1 extra to account for empty / vacuous truths

    You created dp = [[] for _ in range(L)] but often DP arrays are sized L+1 to handle empty prefix easily.

    However, remember that if we find a way to do rolling variables, it may not be the same iteration range.

  2. edge-padding is useful when handling sentinel values like at border-elements. This can be seen in the “Burst Balloons” question. Also kind of related to the largest rectangle question that we have seen before.

  3. avoiding the double counting when doing state compressions

    1. for cases like the 0/1 bounded knapsack, when we’re using arrays, we end up with the heuristic that “we iterate from right to left so that we can avoid double counting – because it’s a 0/1 inclusion / exclusion that we care about”.

    2. for other datastructures for the state management:

      • a dictionary \(\implies\) there’s a need to separate the reads from the writes during the iteration.

        example: tallest billboard (956) where we use a dictionary, we need to do the dictionary copying so that reads and writes are separated across iteration boundary
         1
         2
         3
         4
         5
         6
         7
         8
         9
        10
        11
        12
        13
        14
        15
        16
        17
        18
        19
        20
        21
        22
        23
        24
        25
        26
        27
        28
        29
        30
        31
        32
        
                 class Solution:
                     def tallestBillboard(self, rods: List[int]) -> int:
                         # dp{k:v} where k = d = diff b/w left and right poles (subsets); v = max sum of the smaller subset for the difference
        
                         dp = {0:0} # trivial when both subsets are empty
        
                         for rod in rods:
                             # for each diff we encounter so far, we can do 3 things:
        
                             # A: do not use the rod
                             # B: use the rod, add it to the "taller" subset => this updates the differences from d to d + rod, no change to the smaller subset sum
                             # C: use the rod, add it to the "shorter" subset => this updates the  diff to be |d - r| (short and tall may swap places)
                             # eventually, answer will be max value within dp[0]
        
                             copy = dp.copy() # copying keeps the changes unaffected across multiple cases. separates reads and writes.
                             for d, smaller in dp.items():
                                 # case A: no use:
                                 copy[d] = max(copy.get(d, 0), smaller) # keep the bigger val
                                 # case B: add to taller
                                 copy[d + rod] = max(copy.get(d + rod, 0), 0, smaller)
                                 # case C: add to the shorter one:
                                 diff = abs(d - rod)
                                 if d < rod:
                                     updated_smaller = smaller + d
                                 else:
                                     updated_smaller = smaller + rod
        
                                 copy[diff] = max(updated_smaller, copy.get(diff, 0))
        
                             dp = copy
        
                         return dp.get(0, 0)

Tricks relating to recurrence formulae #

  • circular to linear array is a matter of entertaining 2 mutually exclusive events.

    in the case of house robber II vs house robber I, we realise that it’s a matter of reducing the problem from circular to straight arrays. Literally just array splicing in the context of the contraints given.

  • In the case of Longest Palindromic Substring, it’s a matter of tracking contiguous regions which makes it feel 2-dimensional.

    What are the 2 dimensions? Can we find equivalent but different ways of representing the state equations? Yes We can let the substrings (continuous) be defined by left,right pointers. We can ALSO let it be defined by left, length instead.

Python #

  1. memoise functions using the @cache decorator, all arguments must be hashable.
  2. for string, char counting is easy, just do something like "hello".count('h')

Boilerplate #

Patience Sorting for Longest Increasing Subsequence (\(O(nlogn)\)) #

When doing LIS problems, we can find a linear time solution by using Patience sorting.

The easiest way to reason this is to just think about keeping track of piles of numbers that are increasing subsequences and just tracking the tail of that number.

A tail is the smallest value in that pile of numbers.

So we usually have 2 options for a particular number:

  1. it gets added to an existing pile

    • it may possibly become a new tail within an existing pile
  2. it starts a new pile

    • if the new number doesn’t fit with any of the existing tails (existing min values for the existing piles) then it should start a pile of its own.

Finally, we know that we can join up at least one element from each pile to form the longest increasing subsequence.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
import bisect
class Solution:
    def lengthOfLIS(self, nums: List[int]) -> int:
        tails = []
        for x in nums:
            # left most place to keep x within tails
            # this picks the currently accumulating pile / stacks
            # so tails[k] is giong t obe the smallest possible value of increasing subsequence of length (k + 1)
            idx = bisect.bisect_left(tails, x)
            if should_create_new_stack:=(idx == len(tails)): # can create a new stack
                tails.append(x)
            else:  # replace within an existing stack's min (represents top of the stack)
                tails[idx] = x

        return len(tails)

The expand around center algo is relatively intuitive, the manacher’s algo is optimal but requires some mental gymnastics.

Expand Around Center Algorithm
  • Intuition
    • Every palindrome can be defined by its “center.”
    • For each character (and the space between characters), treat it as a center and expand outwards as long as the substring is a palindrome.
    • Handles both odd and even-length palindromes by considering:
      • A single character as center (odd length)
      • A pair between characters as center (even length).
  • Process
    1. For each index `i` in string `s`, try to expand around:
      • Center at `i` (for odd-length palindromes)
      • Center between `i` and `i+1` (for even-length palindromes)
    2. While the substring `s[left:right+1]` remains a palindrome `s[left] == s[right]`, expand outwards.
    3. Track start and length of the longest palindromic substring found[1][2][4][5][6].
  • Time/Space Complexity
    • Time: O(n²) (expands for each possible center)
    • Space: O(1)
  • Pythonic Example
        def longestPalindrome(s):
            n = len(s)
            start, max_len = 0, 1
    
            def expand_around_center(left, right):
                while left >= 0 and right < n and s[left] == s[right]:
                    left -= 1
                    right += 1
                return left + 1, right - 1
    
            for i in range(n):
                l1, r1 = expand_around_center(i, i)
                l2, r2 = expand_around_center(i, i + 1)
                if r1 - l1 + 1 > max_len:
                    start, max_len = l1, r1 - l1 + 1
                if r2 - l2 + 1 > max_len:
                    start, max_len = l2, r2 - l2 + 1
            return s[start:start+max_len]
  • Why it Works
    • For string of length n, there are (2n-1) centers (n odd, n-1 even).
    • Expanding around each yields all possible palindromic substrings.
Manacher’s Algorithm (Linear Time)
  • Intuition
    • Fastest known algorithm: finds the longest palindromic substring in O(n) time and space[7][10].
    • Uses the idea of palindromic symmetry and pre-processing.
    • Treats all palindromes as odd length by inserting ‘#’ or special chars between every real char and at boundaries.
  • Process (Summary)
    1. Preprocess string: insert special chars (e.g., ‘#’ between, ‘^’ at start, ‘\(’ at end) to handle both odd/even-length centers uniformly (`“aba”` → `"^#a#b#a#\)"`).
    2. For each position in the new string, use previously calculated palindrome lengths to attempt to expand the palindrome centered at that position.
    3. Maintain a center `C` and a right boundary `R` of the current rightmost palindrome.
    4. For every new center, mirror its value using the palindrome around `C` if possible, and then try to expand further.
    5. Track the longest palindrome as you go[7][10].
  • Time/Space Complexity
    • Time: O(n)
    • Space: O(n)
  • Benefits/Drawbacks
    • Extremely fast for very large inputs
    • More complex to implement (hard for coding interviews unless specifically requested)
    • Usually not needed for n ≤ 1000, but valuable to be aware of for advanced optimization[1].
  • Pseudocode
        def manacher(s):
            T = '^#' + '#'.join(s) + '#$'
            n = len(T)
            P = [0] * n
            C = R = 0
            for i in range(1, n-1):
                mirr = 2*C - i
                if i < R:
                    P[i] = min(R - i, P[mirr])
                # Expand palindrome centered at i
                while T[i + 1 + P[i]] == T[i - 1 - P[i]]:
                    P[i] += 1
                # Update center and right edge
                if i + P[i] > R:
                    C, R = i, i + P[i]
            # Extract longest palindrome
            max_len = max(P)
            center = P.index(max_len)
            start = (center - max_len) // 2
            return s[start:start + max_len]

Notes #

Fundamentals #

  • optimisation problems \(\implies\) desire is to find an optimal solution instead of the optimal solution
  • the an is because the same min/max can be found due to other solutions
  • seems like the terminoloy and its origins is a little funny (see post1 and post2):
    • “Dynamic Programming” just a phrase to describe the multi-stage process of framing the solution
    • perhaps should be seen as a “collection of optimisations” instead
    • the main recipe:
      1. Generate a naive recursive algorithm.
      2. Memoize it by saving partial results in an array.
      3. Invert control, so the table of results is explicity assembled bottom-up instead of as a byproduct of top-down recursion.
      4. Analyze the dependencies among the partial results, and if possible, simplify the saved results to avoid storing unnecessary data.

General Implementation Structure #

Thinking along these element gives the generic structure that we can expect

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
# Top-down recursive dynamic programming
def dp(state1, state2, ...):
    for choice in all possible choices:
        # The state changes after making the choice
        result = find_optimal(result, dp(state1, state2, ...))
    return result

# Bottom-up iterative dynamic programming
# Initialize base case
dp[0][0][...] = base case
# Perform state transitions
for state1 in all possible values of state1:
    for state2 in all possible values of state2:
        for ...
            dp[state1][state2][...] = find_optimal(choice1, choice2, ...)
here’s an example, recursive solution with memoisation for the coin change problem
 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
class Solution:
    def __init__(self):
        self.memo = []

    def coinChange(self, coins: List[int], amount: int) -> int:
        self.memo = [-666] * (amount + 1)
        # Initialize the memo with a special value that won't be
        # picked, representing it has not been calculated
        return self.dp(coins, amount)

    def dp(self, coins, amount):
        if amount == 0: return 0
        if amount < 0: return -1
        # Check the memo to prevent repeated calculations
        if self.memo[amount] != -666:
            return self.memo[amount]

        res = float('inf')
        for coin in coins:
            # Calculate the result of the subproblem
            subProblem = self.dp(coins, amount - coin)
            # Skip if the subproblem has no solution
            if subProblem == -1: continue
            # Choose the optimal solution from the subproblems and add one
            res = min(res, subProblem + 1)
        # Store the calculation result in the memo
        self.memo[amount] = res if res != float('inf') else -1
        return self.memo[amount]
and here’s the bottom up iterative version of it:
 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
class Solution:
    def coinChange(self, coins: List[int], amount: int) -> int:
        # The array size is amount + 1, and the initial value is also amount + 1
        dp = [amount + 1] * (amount + 1)

        dp[0] = 0
        # base case
        # The outer for loop is traversing all possible values of all states
        for i in range(len(dp)):
            # The inner for loop is finding the minimum value among all choices
            for coin in coins:
                # The subproblem has no solution, skip
                if i - coin < 0:
                    continue
                dp[i] = min(dp[i], 1 + dp[i - coin])

        return -1 if dp[amount] == amount + 1 else dp[amount]

Elements of Dynamic Programming: Key Indicators of Applicability #

  • [1] Optimal Sub-Structure

    • Defining “Optimal Substructure”

      A problem exhibits optimal substructure if an optimal solution to the problem contains within it optimal solutions to sub-problems.

      • typically a signal that DP (or Greedy Approaches) will apply here.
      • this is something to discover about the problem domain
    • Common Pattern when Discovering Optimal Substructure

      1. solution involves making a choice e.g. choosing an initial cut in the rod or choosing what k should be for splitting the matrix chain up

        \(\implies\) making the choice also determines the number of sub-problems that the problem is reduced to

      2. Assume that the optimal choice can be determined, ignore how first.

      3. Given the optimal choice, determine what the sub-problems looks like \(\implies\) i.e. characterise the space of the subproblems

        1. keep the space as simple as possible then expand

        2. typically the substructures vary across problem domains in 2 ways:

          1. [number of subproblems] – this shows how the subproblems are composed

            how many sub-problems does an optimal solution to the original problem use?

            • rod cutting: # of subproblems = 1 (of size n-i) where i is the chosen first cut length

            • matrix split:

              # of subproblems = 2 (left split and right split)

          2. [number of subproblem choices]

            how many choices are there when determining which subproblems(s) to use in an optimal solution. i.e. now that we know what to choose, how many ways are there to make that choice?

            • rod cutting: # of choices for i = length of first cut, n choices

            • matrix split: # of choices for where to split (idx k) = j - i

        3. Naturally, this is how we analyse the runtime: # of subproblems * # of choices @ each sub-problem

      4. Reason to yourself if and how solving the subproblems that are used in an optimal solution means that the subproblem-solutions are also optimal. This is typically done via the cut-and-paste argument.

    • Caveat: Subproblems must be independent

      • “independent subproblems”

        solution to one subproblem doesn’t affect the solution to another subproblem for the same problem

        example: Unweighted Shortest Path (independent subproblems) vs Unweighted Longest Simple Path (dependent subproblems)

        Consider graph \(G = (V,E)\) with vertices \(u,v \notin V\)

        • Unweighted shortest path: we want to find path \(u\overset{p}{\leadsto}v\) then we can break it down into subpaths \(u\overset{p1}{\leadsto}w\overset{p2}{\leadsto}v\) (optimal substructure). If path, p, is optimal from \(u\leadsto{v}\) then p1 must be shortest path from \(u\leadsto{w}\) and likewise p2 must be shortest path from \(w\leadsto{v}\) (cut-and-paste argument) Here, we know that the subproblems are independent.

        • Unweighted Longest Simple Path: the subproblems can have paths that overlap (not subproblem overlap), hence it’s not independent because the vertices that appear can appear in the solution to the second subproblem

    • Terminology: Optimal substruture vs optimal solution

      optimal substructure is a property that enables efficient computation of the optimal value, while the solution is the specific configuration that yields that optimal value

      a description
      You're right that in dynamic programming algorithms for optimization problems, the goal is usually to find **an** optimal solution rather than **the** unique optimal solution. Here's how the optimal substructure and the actual solution differ:
      
      ### Optimal Substructure
      
      The optimal substructure refers to the property that the optimal solution to a problem can be constructed from the optimal solutions to its subproblems. This means that if we know the optimal solutions to the subproblems, we can efficiently determine the optimal solution to the overall problem.
      
      For example, in the rod cutting problem, if we know the maximum revenue for rods of length 1 to n-1, we can efficiently determine the maximum revenue for a rod of length n by considering cutting the rod at each possible length and taking the maximum revenue.
      
      ### The Solution
      
      The actual solution, on the other hand, refers to the specific configuration or set of choices that yields the optimal value. In the rod cutting problem, the solution would be the specific way to cut the rod that results in the maximum revenue.
      
      The key difference is that the optimal substructure allows us to efficiently compute the optimal value, but it doesn't necessarily specify the exact solution. There may be multiple solutions that yield the same optimal value.
      
      For example, in the rod cutting problem, there may be multiple ways to cut the rod that result in the same maximum revenue. The optimal substructure allows us to efficiently compute the maximum revenue, but it doesn't specify which particular cutting configuration to choose.
      
      So in summary, the optimal substructure is a property that enables efficient computation of the optimal value, while the solution is the specific configuration that yields that optimal value. Dynamic programming algorithms leverage the optimal substructure to efficiently compute the optimal value, but the actual solution may not be uniquely determined.
      
      Citations:
      [1] https://library.fiveable.me/key-terms/programming-for-mathematical-applications/optimal-substructure
      [2] https://afteracademy.com/blog/optimal-substructure-and-overlapping-subproblems/
      [3] https://www.geeksforgeeks.org/optimal-substructure-property-in-dynamic-programming-dp-2/
      [4] https://labuladong.gitbook.io/algo-en/i.-dynamic-programming/optimalsubstructure
      [5] https://stackoverflow.com/questions/33563230/optimal-substructure-in-dynamic-programing
      [6] https://en.wikipedia.org/wiki/Optimal_substructure
      [7] https://www.cs.cmu.edu/~15451-f22/lectures/lec09-dp1.pdf
      [8] https://www.javatpoint.com/optimal-substructure-property
  • [2] Overlapping Sub-Problems

    • DP should be disambiguated from Divide & Conquer approaches:
      • for Divide&Conquer, choosing a division point generates brand new problems that likely don’t overlap
      • for DP, the space of sub-problems is small and hence if done purely recursively, the same operations likely get computed multiple times
    • [Practical point] Reconstructing an optimal solution

      Since one way to look at DP is table-filling, we typically have:

      1. table of costs
      2. table of optimal choices

      the table of optimal choices can be built from the table of costs, but we want to avoid recalculating the optimal choices

    • Memoisation as an alternative

      • the textbook used sees this as an alternative and memoised tables as the aux DS for recursive, top-down approaches
      • memoisation also exploits the observation that the space of the subproblems is small and hence overlapping

Designing State Transitions and Relying on Mathematical Induction #

Ref

The article suggests that it’s better to see the dp design as aspects of a proof by induction:

Assume the previous answers are known, use mathematical induction to correctly deduce and transition states, and ultimately arrive at the solution.

We assume we have the information and see how an optimal solution to the subproblem can be used to find the answer to the larger problem (similar to \(k^{th}\) case and \((k+1)^{th}\) case induction step).

For that to happen, the main dp array should be instrumental in giving us our answer. If it isn’t there’s a need to identify and build more auxiliary dp arrays for this.

Disambiguating DP vs Greedy since both rely on substructures #

  • The core question is about proof technique, not code shape

    Both DP and Greedy require optimal substructure. That is the shared foundation. The distinction is not “does this problem have optimal substructure” but rather “which additional property does it have on top of that?”

    • DP applies when you have optimal substructure + overlapping subproblems, but you cannot prove that a locally optimal choice is always globally safe. So you enumerate all relevant subproblem solutions, cache them, and take the best. The correctness proof is typically a cut-and-paste argument: assume the global optimal does not use the optimal subproblem solution, substitute it in, show you do no worse. Contradiction.

    • Greedy applies when you have optimal substructure + the greedy choice property: a locally optimal choice is always part of some globally optimal solution. You can prove this without enumerating alternatives. The correctness proof is either “greedy stays ahead” (show your greedy is never behind any other algorithm at each step) or an exchange argument (show any optimal solution can be transformed into the greedy one without loss).

    The distinction is therefore: do you need to enumerate alternatives, or can you prove you never need to? DP = enumerate. Greedy = prove you don’t have to.

  • Definition and Approach

    • Greedy Strategy:
      • A greedy algorithm makes the most optimal choice at each step with the hope that these local optimum choices will lead to a global optimum solution. It does not consider the overall impact of the current choice on future decisions.
      • Greedy algorithms are typically simpler and faster to implement, as they do not require storing results of subproblems. However, they do not guarantee an optimal solution for all problems.
    • Dynamic Programming:
      • Dynamic programming is a technique used to solve complex problems by breaking them down into simpler subproblems. It is based on the principles of optimal substructure and overlapping subproblems.
      • DP guarantees an optimal solution by considering all possible cases and storing the results of subproblems to avoid redundant computations. This can typically lead to higher time and space complexity compared to greedy algorithms.
  • Optimal Substructure

    • Greedy Algorithms:
      • Greedy algorithms rely on the greedy choice property, which means that a locally optimal choice leads to a globally optimal solution. However, this property does not hold for all problems, which is why greedy algorithms may fail to find the optimal solution in some cases.
    • Dynamic Programming:
      • Dynamic programming explicitly uses the optimal substructure property, where an optimal solution to a problem can be constructed from optimal solutions of its subproblems. DP ensures that all relevant subproblems are considered, leading to a guaranteed optimal solution.
  • The greedy choice property

    A problem has the greedy choice property if: making the locally optimal choice at each step is provably consistent with some globally optimal solution, independent of future choices.

    This is stronger than optimal substructure. Optimal substructure says “the optimal global solution uses optimal sub-solutions.” The greedy choice property adds: “I know which sub-solution to pick right now, without solving anything else first.”

    Proof techniques:

    • Greedy stays ahead: show that after each step, your greedy solution is at least as good as any other solution on the same prefix. By induction, it stays ahead through the end.
    • Exchange argument: take any optimal solution O. Show that if O differs from your greedy solution G at some step, you can swap O’s choice for G’s choice and produce a solution that is no worse. Therefore G is also optimal.
  • Subproblem Reuse

    • Greedy Strategy:
      • Greedy algorithms typically do not reuse solutions to subproblems. Each decision is made independently based on the current state without revisiting previous choices.
        • there are exceptional cases:

          Kadane’s algorithm reuses dp[i-1] at every step. It just compresses the table to a single variable because only the previous step is ever needed. Subproblem reuse is present; the table is optimised away. The presence or absence of an explicit cache is an implementation detail, not a conceptual criterion.

    • Dynamic Programming:
      • DP explicitly solves and caches solutions to overlapping subproblems. This reuse of subproblem solutions is a key feature of dynamic programming, allowing for efficient computation.
  • Complexity and Efficiency

    This is a consequence of the above, not an independent criterion. Greedy tends to be faster because it makes one pass without enumerating alternatives.

    DP’s cost is (# of subproblems) × (#choices per subproblem).

    When the greedy choice property holds, the “# choices per subproblem” term collapses to 1, which is exactly why greedy is faster — it’s DP where you’ve proved you only ever need to consider one branch.

    • Greedy Algorithms:
      • Greedy algorithms are generally more efficient in terms of time complexity due to their straightforward approach. They often run in linear or logarithmic time, depending on the problem.
      • They are suitable for problems where local optimization leads to global optimization, such as in minimum spanning trees or certain shortest path problems.
    • Dynamic Programming:
      • DP can be less efficient than greedy algorithms in terms of time complexity, especially if every possible solution is computed. However, proper use of memoization can mitigate this.
      • DP is applicable to problems with overlapping subproblems and optimal substructure, such as the Fibonacci sequence, longest common subsequence, or knapsack problems.
  • * Canonical examples aligned to the framework

    ProblemHas OSPHas GCPMethodWhy
    0/1 KnapsackyesnoDPTaking heaviest value item first fails
    Fractional KnapsackyesyesGreedyValue/weight ratio ordering provably optimal
    Shortest path (Dijkstra)yesyesGreedyExchange argument on relaxed edges
    Longest Common SubseqyesnoDPNo local rule determines which char to match
    Activity SelectionyesyesGreedyEarliest finish time, exchange argument
    Kadane (Max Subarray)yesyesBothDP recurrence and exchange argument both hold

    OSP = optimal substructure property GCP = greedy choice property

  • Examples

    • Greedy Algorithms:
      • Common examples include:
        • Fractional Knapsack Problem: Where you can take fractions of items.
        • Minimum Spanning Tree: Algorithms like Prim’s and Kruskal’s.
        • Huffman Coding: For optimal prefix codes.
    • Dynamic Programming:
      • Common examples include:
        • 0/1 Knapsack Problem: Where you cannot take fractions of items.
        • Longest Common Subsequence: Finding the longest subsequence present in both sequences.
        • Matrix Chain Multiplication: Finding the most efficient way to multiply a given sequence of matrices.
  • A framework of thought for Greedy Algos

    Formal Framework for Understanding and Applying Greedy Algorithms

    Here’s a step-by-step template/framework for analyzing and designing greedy algorithms:
    1. Problem Identification:

      • Does the problem ask for an optimal subset or sequence?

      • Are you trying to maximize/minimize something subject to constraints?

    2. Greedy Choice Property:

      • Can you make a locally optimal choice at each step that is part of some global optimal solution?

      • Example greedy choices could be earliest finishing time, shortest duration, smallest cost, etc.

    3. Optimal Substructure:

      • Does the problem exhibit optimal substructure?

      • That is, does an optimal solution to the problem contain optimal solutions to subproblems?

    4. Candidate Greedy Strategies:

      • List out all possible “greedy” choices you might try (e.g., earliest start, earliest finish, shortest length).

      • Test how intuitive or promising each strategy looks with small examples.

    5. Proof or Intuition of Correctness:

      • Use “Greedy stays ahead” or “Exchange argument” proof techniques to argue your greedy choice leads to optimal solution.

      • Greedy stays ahead: show your greedy algorithm is never worse than any other solution at each step.

      • Exchange argument: show that any optimal solution can be converted to your greedy one without loss.

    6. Design and Implementation:

      • Implement your approach carefully.

      • Usually involves sorting based on greedy criteria.

      • Iterate through candidates, making decisions based on the greedy choice.

    7. Analyze Complexity:

      • Time complexity will often involve sorting (\(O(n log n)\)) plus linear scan.

      • Space complexity is often \(O(1)\) or \(O(n)\).

  • Decision flow: which framework to reach for first

    1. Does the problem have optimal substructure? (Almost always yes for optimisation problems.) If no, neither applies — look elsewhere.

    2. Can you state a greedy choice property and sketch an exchange argument or greedy-stays-ahead proof in under a minute?

      • Yes → use Greedy. It will be faster to implement and O(n) or O(n log n) almost always.
      • No, or not sure → default to DP. Enumerate subproblems, write the recurrence, optimise the cache later.
    3. If you chose Greedy but get a wrong answer on a counterexample, the greedy choice property failed. Fall back to DP — do not attempt to patch the greedy.

  • Conclusion

    In summary, while both greedy strategies and dynamic programming involve the concept of optimal substructure, they differ significantly in their approach to problem-solving. Greedy algorithms make decisions based on local optimality without revisiting previous choices, whereas dynamic programming systematically considers all possible subproblems and their solutions to ensure a globally optimal result. Understanding these differences allows for better problem-solving strategies and the appropriate application of each method based on the problem’s characteristics.

Misconceptions #

Correct: Since it’s optimisation, we can consider DP for max/min objectives #

  1. the defining characteristics is more of the recursive structure to the problems (hence the optimal substructure characteristic)

  2. also when it comes to a max / min instead of an accum, it’s usually the case that our state-transition also includes a max / min operation

TODO Pending tasks [0/0] #

  1. Niche patterns from CP algos:
    1. Subset over Sum DP (this is a bitmasking pattern where the subsets are bitmask subsets.)