django how can I use multiprocessing in a management command

Question

how can I use multiprocessing in a django management command? I can’t figure out how to start the process inside of my websites django context

management command

users = users.objects.filter(is_active=True)

with ProcessPoolExecutor(max_workers=5) as executor:
   processes = []
   for user in users:
       p = executor.submit(complex_user_calculation,user)
       processes.append(p)

   for f in concurrent.futures.as_completed(processes):
       result = f.result()
       if result:
           print(result)

when I run this management command I get this error

    raise AppRegistryNotReady("Apps aren't loaded yet.")
django.core.exceptions.AppRegistryNotReady: Apps aren't loaded yet.
...
  File "C:UsersuserAppDataLocalProgramsPythonPython310libconcurrentfutures_base.py", line 390, in __get_result
    raise self._exception
concurrent.futures.process.BrokenProcessPool: A process in the process pool was terminated abruptly while the future was running or pending.

Asked By: user14665310

||

Source

Answer 1

You need to initialise your Django application in each subprocess to ensure your apps, models, etc are all loaded. Pass django.setup as your pool initialiser

with ProcessPoolExecutor(max_workers=5, initializer=django.setup) as executor:
    ...

Answered By: Iain Shelvington

django how can I use multiprocessing in a management command

Question:

Answers: