Execute statement every N iterations in Python

Question:

I have a very long loop, and I would like to check the status every N iterations, in my specific case I have a loop of 10 million elements and I want to print a short report every millionth iteration.

So, currently I am doing just (n is the iteration counter):

if (n % 1000000==0):
    print('Progress report...')

but I am worried I am slowing down the process by computing the modulus at each iteration, as one iteration lasts just few milliseconds.

Is there a better way to do this? Or shouldn’t I worry at all about the modulus operation?

Asked By: Andrea Zonca

||

Answers:

It’s fast enough that I wouldn’t worry about it.

If you really wanted to speed it up, you could do this to avoid the modulus

if (n == 1000000):
    n = 0
    print('Progress report...')
Answered By: corsiKa

Is it really slowing down? You have to try and see for yourself. It won’t be much of a slowdown, but if we’re talking about nanoseconds it may be considerable. Alternatively you can convert one 10 million loop to two smaller loops:

m = 1000000
for i in range(10):
    for i in range(m):
        // do sth
    print("Progress report")
Answered By: pajton

I’d do some testing to see how much time your modulus calls are consuming. You can use timeit for that. If your results indicate a need for time reduction, another approach which eliminates your modulus calculation:

for m in xrange(m_min, m_max):
    for n in xrange(n_min, n_max):
        #do_n_stuff
    print('Progress report...')
Answered By: GreenMatt

How about keeping a counter and resetting it to zero when you reach the wanted number? Adding and checking equality is faster than modulo.

printcounter = 0

# Whatever a while loop is in Python
while (...):   
    ...
    if (printcounter == 1000000):
        print('Progress report...')
        printcounter = 0
    ...
    printcounter += 1

Although it’s quite possible that the compiler is doing some sort of optimization like this for you already… but this may give you some peace of mind.

Answered By: AndrewKS

Something like that ? :

for n in xrange(1000000,11000000,1000000):
    for i in xrange(n-1000000,n):
        x = 10/2
    print 'Progress at '+str(i)

result

Progress at 999999
Progress at 1999999
Progress at 2999999
Progress at 3999999
Progress at 4999999
Progress at 5999999
Progress at 6999999
Progress at 7999999
Progress at 8999999
Progress at 9999999

.

EDIT

Better:

for n in xrange(0,10000000,1000000):
    for i in xrange(n,n+1000000):
        x = 10/2
    print 'Progress at '+str(i)

And inspired from pajton:

m = 1000000
for n in xrange(0,10*m,m):
    for i in xrange(n,n+m):
        x = 10/2
    print 'Progress at '+str(i+1)

I prefer this that I find more immediately readable than the pajton’s solution.
It keeps the display of a value depending of i

Answered By: eyquem

It’s difficult to know how your system will optimize your code without testing.

You could simplify the relational part by realizing that zero is evaluated as false.

if(not N % 10000000)
   do stuff
Answered By: Lt Dax

This makes the inner loop lean, and m does not have to be divisible by interval.

m = 10000000
interval = 1000000
i = 0
while i < m:
    checkpoint = min(m, i+interval)
    for j in xrange(i, checkpoint):
        #do something
    i = checkpoint
    print "progress"
Answered By: Janne Karila

1. Human-language declarations for x and n:

let x be the number of iterations that have been examined at any given time.
let n be the multiple of iterations upon which your code will executed.

Example 1: "After x iterations, how many times was n done?"

Example 2: "It is the xth iteration and the action has occurred every nth time, so far."

2. What we’re doing:

The first code block (Block A) uses only one variable, x (defined above), and uses 5 (an integer) rather than the variable n (defined above).

The second code block (Block B) uses both of the variables (x and n) that are defined above. The integer, 5, will be replaced by the variable, n. So, Block B literally performs an action at each nth iteration.

Our goal is to do one thing on every iteration and two things on every nth iteration.
We are going through 100 iterations.

m. Easy-to-understand attempt:

Block A, minimal variables:

for x in 100:
    #what to do every time (100 times in-total): replace this line with your every-iteration functions.
    if x % 5 == 0:
        #what to do every 5th time: replace this line with your nth-iteration functions.

Block B, generalization.

n = 5
for x in 100:
    #what to do every time (100 times in-total): replace this line with your every-iteration functions.
    if x % n == 0:
        #what to do every 5th time: replace this line with your nth-iteration functions.

Please, let me know if you have any issues because I haven’t had time to test it after writing it here.

3. Exercises

  1. If you’ve done this properly, see if you can use it with the turtle.Pen() and turtle.forward() function. For example, move the turtle forward 4 times and then right and forward once?
  2. See if you can use this program with the turtle.circle() function. For example, draw a circle with radius+1 4 times and a circle of a new color with radiut+1 once?
  3. Check out the reading (seen below) to attempt to improve the programs from exercise 1 and 2. I can’t think of a good reason to be doing this: I just feel like it might be useful!

About modulo and other basic operators:
https://docs.python.org/2/library/stdtypes.html
http://www.tutorialspoint.com/python/python_basic_operators.htm

About turtle:
https://docs.python.org/2/library/turtle.html
https://michael0x2a.com/blog/turtle-examples

Answered By: Wolfpack'08

When I’m doing timing/reports based on count iterations, I just divide my counter by the desired iteration and determine if the result is an integer. So:

    if n/1000000 == int(n/1000000):
        print(report)
Answered By: Kyle Hurst
Categories: questions Tags: ,
Answers are sorted by their score. The answer accepted by the question owner as the best is marked with
at the top-right corner.