Do processes need to be stopped manually
Question:
I’m new to multiprocessing in Python so I’m in doubt. My first idea was to use threads, but then I read about GIL and moved to multiprocessing.
My question is, when I start a process like this:
t1 = Process(target=run, args=lot)
t1.start()
do I need to stop it somehow from the main process, or they shutdown when the run() method is finished?
I know that things like join() exist, but I’m scheduling a job every n minutes and start a couple of processes in parallel, and this procedure goes until stopped, so I don’t really need to wait for processes to finish.
Answers:
Yes when t1.start() happens it executes the method which is specified in target(i.e run). Once its completed it exits automatically.
You can check this by checking the running process eg in linux use below command,
"ps -aux |grep python" or "ps -aux |grep "program_name.py"
when your target is running count will be more.
To wait until a process has completed its work and exited, use the join() method. But in your case its not required
more example are here : https://pymotw.com/2/multiprocessing/basics.html
GIL is not a big problem when doing little computation. Problems arise when doing computationally intensive tasks like networking or reading files while execution of a program is hung (hanged), which is where control flow is given to the kernel until input/output operation is performed. Once at that point, another thread can run in python.
If you are doing CPU-intensive programming, go with multiprocessing instead of GIL.
The join() method is used for thread synchronization, so when the main thread relies on data processed by another thread, it is important to use join(). When the main thread isn’t reliant on the data that child threads process, join() is unnecessary. Your operating system will handle things like closing child processes in a safe manner.
EDIT: check this discussion for more details.
I’m new to multiprocessing in Python so I’m in doubt. My first idea was to use threads, but then I read about GIL and moved to multiprocessing.
My question is, when I start a process like this:
t1 = Process(target=run, args=lot)
t1.start()
do I need to stop it somehow from the main process, or they shutdown when the run() method is finished?
I know that things like join() exist, but I’m scheduling a job every n minutes and start a couple of processes in parallel, and this procedure goes until stopped, so I don’t really need to wait for processes to finish.
Yes when t1.start() happens it executes the method which is specified in target(i.e run). Once its completed it exits automatically.
You can check this by checking the running process eg in linux use below command,
"ps -aux |grep python" or "ps -aux |grep "program_name.py"
when your target is running count will be more.
To wait until a process has completed its work and exited, use the join() method. But in your case its not required
more example are here : https://pymotw.com/2/multiprocessing/basics.html
GIL is not a big problem when doing little computation. Problems arise when doing computationally intensive tasks like networking or reading files while execution of a program is hung (hanged), which is where control flow is given to the kernel until input/output operation is performed. Once at that point, another thread can run in python.
If you are doing CPU-intensive programming, go with multiprocessing instead of GIL.
The join() method is used for thread synchronization, so when the main thread relies on data processed by another thread, it is important to use join(). When the main thread isn’t reliant on the data that child threads process, join() is unnecessary. Your operating system will handle things like closing child processes in a safe manner.
EDIT: check this discussion for more details.