Is there a danger in letting a generator run for a very long time?

Question:

I am designing code to run certain enumeration simulations. I’ll put the code at the end, but I don’t think my question really should concern the details of this program. In principle I think my question could apply just as well to this much more minimal example script:

class Generator:
    def __init__(self, n):
        self.current = 0
        self.limit = n
        self.end_reached = False
    def next(self):
        if self.current == self.limit:
            self.end_reached = True
        if self.end_reached:
            return None

        # Here the point is to have an example where 
        # the variable could potentially store a very large 
        # object.  Will Python's garbage collection, or whatever,
        # wisely use space, or could this be dangerous for 
        # the computer's memory?
        out = [i for i in range(self.limit)]
        self.current += 1
        return out

example = Generator(a_really_big_number)
count = 0
while not example.end_reached:
    count += 1
    example.next()

If I input an extremely large n and left it running all day, is there a risk that it would damage my computer? Do I need to put in some kind of break manually or will the compiler do it for me? Especially because the actual application I’m building this for, almost by design, experiences "combinatorial explosion" I will often be at risk of a fairly innocuous sounding input leads to a gigantic number of computations, so I’ve been considering whether I need to also build a "generator runner" to automatically insert breaks after some iterations, or memory events, or something like that to keep my computer safe.


I would imagine this is irrelevant but I’m running this on a Windows 10 computer, using VSCode IPythonNotebooks. Python 3.8.13.


Here is the actual generator that I’m building, in case it’s useful to see the real application.

class CProd_Generator:
    '''Generator for the Cartesian product of a list of sets (but sets are represented as lists).'''
    def __init__(self, setList):
        if len(setList) == 0:
            raise Exception("Empty setList on construction")
        self.setList = setList
        self.current_index = [0 for i in range(len(setList))]
        self.max_index = [len(setList[i])-1 for i in range(len(setList))]
        self.end_reached = False

    def step(self, property):
        '''Get next element to be generated, if it satisfies the property.'''
        sl = self.setList
        ci = self.current_index
        mi = self.max_index
        n = len(sl)

        out = tuple([sl[i][ci[i]] for i in range(n)])

        i = 0
        while ci[i] == mi[i]:
            ci[i] = 0
            if (i:=i+1) == n:
                self.end_reached = True
                break
        if i < n:
            ci[i] += 1

        if property(out):
            return out
        return None
    
    def next(self, property=lambda x: True):
        while not self.end_reached:
            if (out := self.step(property)) != None:
                return out
        return None
    
    def reset(self): 
        self.current_index = [0 for i in range(len(self.setList))]
        self.end_reached = False


Asked By: Addem

||

Answers:

out is a local variable in the next() method. When the method returns, the variable goes away.

If the caller doesn’t save all the returned values, they’ll be garbage collected.

There’s nothing in your generator that saves all of the values that are returned. You’re not saving them in instance variables, for example.

Answered By: Barmar