Python iterate over each 100 elements

Question

I don’t know if this is a good way to optimize, but basically I am using python inside a 3D app to create random colors per object. And the code I have works well with objects within 10k polygons. But it crashes in 100k polygons. Is there a way to do it by chunks in the loop, basically I have the for loop and using an if statement to filter the first 100. But then I need another 100, and another 100, etc. How can I write that? Maybe with a time sleep between each. It’s not going to be faster but at least won’t possible crash the program. Thanks.

for i, n in enumerate(uvShellIds):


#code can only perform well within sets of 100 elements


limit = 100 #?

if 0 <= i <= 100:

    #do something
    print(n)

# now I need it to work on a new set of 100 elements
#if 101 <= i <= 200:

#(...keep going between sets of 100...)

My current code :

import maya.OpenMaya as om
import maya.cmds as cmds
import random
 
def getUvShelList(name):
    selList = om.MSelectionList()
    selList.add(name)
    selListIter = om.MItSelectionList(selList, om.MFn.kMesh)
    pathToShape = om.MDagPath()
    selListIter.getDagPath(pathToShape)
    meshNode = pathToShape.fullPathName()
    uvSets = cmds.polyUVSet(meshNode, query=True, allUVSets =True)
    allSets = []
    for uvset in uvSets:
        shapeFn = om.MFnMesh(pathToShape)
        shells = om.MScriptUtil()
        shells.createFromInt(0)
        # shellsPtr = shells.asUintPtr()
        nbUvShells = shells.asUintPtr()
 
        uArray = om.MFloatArray()   #array for U coords
        vArray = om.MFloatArray()   #array for V coords
        uvShellIds = om.MIntArray() #The container for the uv shell Ids
 
        shapeFn.getUVs(uArray, vArray)
        shapeFn.getUvShellsIds(uvShellIds, nbUvShells, uvset)
 
        # shellCount = shells.getUint(shellsPtr)
        shells = {}
        for i, n in enumerate(uvShellIds):
            
            
            #print(i,n)

            limit = 100

            if i <= limit:
            
                if n in shells:
                    # shells[n].append([uArray[i],vArray[i]])
                    shells[n].append( '%s.map[%i]' % ( name, i ) )
                else:
                    # shells[n] = [[uArray[i],vArray[i]]]
                    shells[n] = [ '%s.map[%i]' % ( name, i ) ]

        allSets.append({uvset: shells})
    
    for shell in shells:
        
        selection_shell = shells.get(shell)
        cmds.select(selection_shell)
        #print(shells.get(shell))
        facesSel = cmds.polyListComponentConversion(fromUV=True, toFace=True)
        cmds.select(facesSel)
        r = [random.random() for i in range(3)]
        cmds.polyColorPerVertex(facesSel,rgb=(r[0], r[1], r[2]), cdo=1 )
        cmds.select(deselect=1)
 
getUvShelList( 'polySurface359' )

Asked By: johancc

||

Source

Answer 1

I use this generator to "chunkify" my long-running operations in python into smaller batches:

def chunkify_list(items, chunk_size):
    for i in range(0, len(items), chunk_size):
        yield items[i:i+chunk_size]

With this defined, you can write your program something like this:

items = [1,2,3,4,5 ...]
for chunk in chunkify_list(items, 100):
    for item in chunk:
        process_item(item)
    sleep(delay)

Now, I’m not going to guarantee that sleep will actually solve your problems, but this lets you handle your data one chunk at a time.

Answered By: David Culbreth

Answer 2

You can use islice from itertools to chunk.

from itertools import islice

uvShellIds = list(range(1000))
iterator = iter(uvShellIds)

while True:
    chunk = list(islice(iterator, 100))

    if not chunk:
        break

    print(chunk) # chunk contains 100 elements you can process

I don’t know how well it fits in your current code but, below is how you can process the chunks:

from itertools import islice

uvShellIds = list(range(1000))
iterator = iter(uvShellIds)
offset = 0

while True:
    chunk = list(islice(iterator, 100))

    if not chunk:
        break

    # Processing chunk items
    for i, n in enumerate(chunk):
        # offset + i will give you the right index referring to the uvShellIds variable
        # Then , perform your actions
        if n in shells:
            # shells[n].append([uArray[i],vArray[i]])
            shells[n].append( '%s.map[%i]' % ( name, offset + i ) )
        else:
            # shells[n] = [[uArray[i],vArray[i]]]
            shells[n] = [ '%s.map[%i]' % ( name, offset + i ) ]

    offset += 100
    # Your sleep can come here

The snippet above should replace your for i, n in enumerate(uvShellIds): block.

As @David Culbreth’s answer stated, I’m not sure the sleep will be of help, but I left a comment on where you can place it.

Answered By: tobihans

Python iterate over each 100 elements

Question:

Answers: