where to put freeze_support() in a Python script?
Question:
I am confused about using freeze_support() for multiprocessing and I get a Runtime Error without it. I am only running a script, not defining a function or a module. Can I still use it? Or should the packages I’m importing be using it?
Here is the documentation.
Note that the specific issue is about scikit-learn calling GridSearchCV which tries to spawn processes in parallel. I am not sure if my script needs to be frozen for this, or the some code that’s called (from the Anaconda distro). If details are relevant to this question, please head over to the more specific question.
Answers:
On Windows all of your multiprocessing-using code must be guarded by if __name__ == "__main__":
So to be safe, I would put all of your the code currently at the top-level of your script in a main() function, and then just do this at the top-level:
if __name__ == "__main__":
main()
See the “Safe importing of main module” sub-section here for an explanation of why this is necessary. You probably don’t need to call freeze_support at all, though it won’t hurt anything to include it.
Note that it’s a best practice to use the if __name__ == "__main__" guard for scripts anyway, so that code isn’t unexpectedly executed if you find you need to import your script into another script at some point in the future.
I am confused about using freeze_support() for multiprocessing and I get a Runtime Error without it. I am only running a script, not defining a function or a module. Can I still use it? Or should the packages I’m importing be using it?
Here is the documentation.
Note that the specific issue is about scikit-learn calling GridSearchCV which tries to spawn processes in parallel. I am not sure if my script needs to be frozen for this, or the some code that’s called (from the Anaconda distro). If details are relevant to this question, please head over to the more specific question.
On Windows all of your multiprocessing-using code must be guarded by if __name__ == "__main__":
So to be safe, I would put all of your the code currently at the top-level of your script in a main() function, and then just do this at the top-level:
if __name__ == "__main__":
main()
See the “Safe importing of main module” sub-section here for an explanation of why this is necessary. You probably don’t need to call freeze_support at all, though it won’t hurt anything to include it.
Note that it’s a best practice to use the if __name__ == "__main__" guard for scripts anyway, so that code isn’t unexpectedly executed if you find you need to import your script into another script at some point in the future.