Can a generator wrapper be used to rate limit a method call?

Question

Suppose I have a method that repeatedly gets called in a loop.

for user in users:
     send_email(user)

But since I want to reduce congestion, I want something like.

for user in rate_limit(per_min=50, users):
     send_email(user)

Where the rate_limit function simply sleeps and starts yielding from users again once the rate limit as been reached.

# something like this, where it just sleeps (until next min/sec cycle) 
# once the number of iterations per_min or per_sec have been achieved.

def rate_limit(per_min=None, per_sec=None, objects):
    ...

The issue I am facing is that I don’t know how to maintain the state (such as the amount of time that has passed or the number of emails sent) in the current form of the method. I would greatly appreciate any suggestions or ideas on how to address this problem.

Asked By: AlanSTACK

||

Source

Answer 1

You’re probably looking for time.sleep():

Yield an object
Sleep
Repeat from step 1

For example:

import time

def rate_limit(objects, per_min=None):
    sleep_time = 60 / per_min if per_min else 0
    for obj in objects:
        yield obj
        time.sleep(sleep_time)

Problem is I have no clue how to preserve state (in the sense of time elapsed/emails sent) in this form.

This will automatically preserver the state, as yield simply pauses the function and continues after the function is called again. If you want to know the spent time and number of mails sent, you can simply add a counter:

def rate_limit(objects, per_min=None):
    sleep_time = 60 / per_min if per_min else 0
    for count, obj in enumerate(objects):
        yield obj
        print('Sent email {0} at {1} seconds.'.format(count, count * sleep_time))
        time.sleep(sleep_time)

Answered By: Markus Meskanen

Answer 2

Just use time.sleep this will suspend the current thread for a given number of seconds.

Answered By: rmorshea

Can a generator wrapper be used to rate limit a method call?

Question:

Answers: