What is the global interpreter lock (GIL) in CPython?
Question:
What is a global interpreter lock and why is it an issue?
A lot of noise has been made around removing the GIL from Python, and I’d like to understand why that is so important. I have never written a compiler nor an interpreter myself, so don’t be frugal with details, I’ll probably need them to understand.
Answers:
Suppose you have multiple threads which don’t really touch each other’s data. Those should execute as independently as possible. If you have a “global lock” which you need to acquire in order to (say) call a function, that can end up as a bottleneck. You can wind up not getting much benefit from having multiple threads in the first place.
To put it into a real world analogy: imagine 100 developers working at a company with only a single coffee mug. Most of the developers would spend their time waiting for coffee instead of coding.
None of this is Python-specific – I don’t know the details of what Python needed a GIL for in the first place. However, hopefully it’s given you a better idea of the general concept.
Python’s GIL is intended to serialize access to interpreter internals from different threads. On multi-core systems, it means that multiple threads can’t effectively make use of multiple cores. (If the GIL didn’t lead to this problem, most people wouldn’t care about the GIL – it’s only being raised as an issue because of the increasing prevalence of multi-core systems.) If you want to understand it in detail, you can view this video or look at this set of slides. It might be too much information, but then you did ask for details π
Note that Python’s GIL is only really an issue for CPython, the reference implementation. Jython and IronPython don’t have a GIL. As a Python developer, you don’t generally come across the GIL unless you’re writing a C extension. C extension writers need to release the GIL when their extensions do blocking I/O, so that other threads in the Python process get a chance to run.
Whenever two threads have access to the same variable you have a problem.
In C++ for instance, the way to avoid the problem is to define some mutex lock to prevent two thread to, let’s say, enter the setter of an object at the same time.
Multithreading is possible in python, but two threads cannot be executed at the same time
at a granularity finer than one python instruction.
The running thread is getting a global lock called GIL.
This means if you begin write some multithreaded code in order to take advantage of your multicore processor, your performance won’t improve.
The usual workaround consists of going multiprocess.
Note that it is possible to release the GIL if you’re inside a method you wrote in C for instance.
The use of a GIL is not inherent to Python but to some of its interpreter, including the most common CPython.
(#edited, see comment)
The GIL issue is still valid in Python 3000.
Let’s first understand what the python GIL provides:
Any operation/instruction is executed in the interpreter. GIL ensures that interpreter is held by a single thread at a particular instant of time. And your python program with multiple threads works in a single interpreter. At any particular instant of time, this interpreter is held by a single thread. It means that only the thread which is holding the interpreter is running at any instant of time.
Now why is that an issue:
Your machine could be having multiple cores/processors. And multiple cores allow multiple threads to execute simultaneously i.e multiple threads could execute at any particular instant of time..
But since the interpreter is held by a single thread, other threads are not doing anything even though they have access to a core. So, you are not getting any advantage provided by multiple cores because at any instant only a single core, which is the core being used by the thread currently holding the interpreter, is being used. So, your program will take as long to execute as if it were a single threaded program.
However, potentially blocking or long-running operations, such as I/O, image processing, and NumPy number crunching, happen outside the GIL. Taken from here. So for such operations, a multithreaded operation will still be faster than a single threaded operation despite the presence of GIL. So, GIL is not always a bottleneck.
Edit: GIL is an implementation detail of CPython. IronPython and Jython don’t have GIL, so a truly multithreaded program should be possible in them, thought I have never used PyPy and Jython and not sure of this.
Why Python (CPython and others) uses the GIL
From http://wiki.python.org/moin/GlobalInterpreterLock
In CPython, the global interpreter lock, or GIL, is a mutex that prevents multiple native threads from executing Python bytecodes at once. This lock is necessary mainly because CPython’s memory management is not thread-safe.
How to remove it from Python?
Like Lua, maybe Python could start multiple VM, But python doesn’t do that, I guess there should be some other reasons.
In Numpy or some other python extended library, sometimes, releasing the GIL to other threads could boost the efficiency of the whole programme.
I want to share an example from the book multithreading for Visual Effects. So here is a classic dead lock situation
static void MyCallback(const Context &context){
Auto<Lock> lock(GetMyMutexFromContext(context));
...
EvalMyPythonString(str); //A function that takes the GIL
...
}
Now consider the events in the sequence resulting a dead-lock.
βββββ¦βββββββββββββββββββββββββββββββββββββββββ¦βββββββββββββββββββββββββββββββββββββββ
β β Main Thread β Other Thread β
β ββββ¬βββββββββββββββββββββββββββββββββββββββββ¬βββββββββββββββββββββββββββββββββββββββ£
β 1 β Python Command acquires GIL β Work started β
β 2 β Computation requested β MyCallback runs and acquires MyMutex β
β 3 β β MyCallback now waits for GIL β
β 4 β MyCallback runs and waits for MyMutex β waiting for GIL β
βββββ©βββββββββββββββββββββββββββββββββββββββββ©βββββββββββββββββββββββββββββββββββββββ
Python doesn’t allow multi-threading in the truest sense of the word. It has a multi-threading package but if you want to multi-thread to speed your code up, then it’s usually not a good idea to use it. Python has a construct called the Global Interpreter Lock (GIL).
https://www.youtube.com/watch?v=ph374fJqFPE
The GIL makes sure that only one of your ‘threads’ can execute at any one time. A thread acquires the GIL, does a little work, then passes the GIL onto the next thread. This happens very quickly so to the human eye it may seem like your threads are executing in parallel, but they are really just taking turns using the same CPU core. All this GIL passing adds overhead to execution. This means that if you want to make your code run faster then using the threading package often isn’t a good idea.
There are reasons to use Python’s threading package. If you want to run some things simultaneously, and efficiency is not a concern, then it’s totally fine and convenient. Or if you are running code that needs to wait for something (like some IO) then it could make a lot of sense. But the threading library wont let you use extra CPU cores.
Multi-threading can be outsourced to the operating system (by doing multi-processing), some external application that calls your Python code (eg, Spark or Hadoop), or some code that your Python code calls (eg: you could have your Python code call a C function that does the expensive multi-threaded stuff).
Python 3.7 documentation
I would also like to highlight the following quote from the Python threading documentation:
CPython implementation detail: In CPython, due to the Global Interpreter Lock, only one thread can execute Python code at once (even though certain performance-oriented libraries might overcome this limitation). If you want your application to make better use of the computational resources of multi-core machines, you are advised to use multiprocessing or concurrent.futures.ProcessPoolExecutor. However, threading is still an appropriate model if you want to run multiple I/O-bound tasks simultaneously.
This links to the Glossary entry for global interpreter lock which explains that the GIL implies that threaded parallelism in Python is unsuitable for CPU bound tasks:
The mechanism used by the CPython interpreter to assure that only one thread executes Python bytecode at a time. This simplifies the CPython implementation by making the object model (including critical built-in types such as dict) implicitly safe against concurrent access. Locking the entire interpreter makes it easier for the interpreter to be multi-threaded, at the expense of much of the parallelism afforded by multi-processor machines.
However, some extension modules, either standard or third-party, are designed so as to release the GIL when doing computationally-intensive tasks such as compression or hashing. Also, the GIL is always released when doing I/O.
Past efforts to create a βfree-threadedβ interpreter (one which locks shared data at a much finer granularity) have not been successful because performance suffered in the common single-processor case. It is believed that overcoming this performance issue would make the implementation much more complicated and therefore costlier to maintain.
This quote also implies that dicts and thus variable assignment are also thread safe as a CPython implementation detail:
Next, the docs for the multiprocessing package explain how it overcomes the GIL by spawning process while exposing an interface similar to that of threading:
multiprocessing is a package that supports spawning processes using an API similar to the threading module. The multiprocessing package offers both local and remote concurrency, effectively side-stepping the Global Interpreter Lock by using subprocesses instead of threads. Due to this, the multiprocessing module allows the programmer to fully leverage multiple processors on a given machine. It runs on both Unix and Windows.
And the docs for concurrent.futures.ProcessPoolExecutor explain that it uses multiprocessing as a backend:
The ProcessPoolExecutor class is an Executor subclass that uses a pool of processes to execute calls asynchronously. ProcessPoolExecutor uses the multiprocessing module, which allows it to side-step the Global Interpreter Lock but also means that only picklable objects can be executed and returned.
which should be contrasted to the other base class ThreadPoolExecutor that uses threads instead of processes
ThreadPoolExecutor is an Executor subclass that uses a pool of threads to execute calls asynchronously.
from which we conclude that ThreadPoolExecutor is only suitable for I/O bound tasks, while ProcessPoolExecutor can also handle CPU bound tasks.
Process vs thread experiments
At Multiprocessing vs Threading Python I’ve done an experimental analysis of process vs threads in Python.
Quick preview of the results:
In other languages
The concept seems to exist outside of Python as well, applying just as well to Ruby for example: https://en.wikipedia.org/wiki/Global_interpreter_lock
It mentions the advantages:
- increased speed of single-threaded programs (no necessity to acquire or release locks on all data structures separately),
- easy integration of C libraries that usually are not thread-safe,
- ease of implementation (having a single GIL is much simpler to implement than a lock-free interpreter or one using fine-grained locks).
but the JVM seems to do just fine without the GIL, so I wonder if it is worth it. The following question asks why the GIL exists in the first place: Why the Global Interpreter Lock?
What is a global interpreter lock and why is it an issue?
A lot of noise has been made around removing the GIL from Python, and I’d like to understand why that is so important. I have never written a compiler nor an interpreter myself, so don’t be frugal with details, I’ll probably need them to understand.
Suppose you have multiple threads which don’t really touch each other’s data. Those should execute as independently as possible. If you have a “global lock” which you need to acquire in order to (say) call a function, that can end up as a bottleneck. You can wind up not getting much benefit from having multiple threads in the first place.
To put it into a real world analogy: imagine 100 developers working at a company with only a single coffee mug. Most of the developers would spend their time waiting for coffee instead of coding.
None of this is Python-specific – I don’t know the details of what Python needed a GIL for in the first place. However, hopefully it’s given you a better idea of the general concept.
Python’s GIL is intended to serialize access to interpreter internals from different threads. On multi-core systems, it means that multiple threads can’t effectively make use of multiple cores. (If the GIL didn’t lead to this problem, most people wouldn’t care about the GIL – it’s only being raised as an issue because of the increasing prevalence of multi-core systems.) If you want to understand it in detail, you can view this video or look at this set of slides. It might be too much information, but then you did ask for details π
Note that Python’s GIL is only really an issue for CPython, the reference implementation. Jython and IronPython don’t have a GIL. As a Python developer, you don’t generally come across the GIL unless you’re writing a C extension. C extension writers need to release the GIL when their extensions do blocking I/O, so that other threads in the Python process get a chance to run.
Whenever two threads have access to the same variable you have a problem.
In C++ for instance, the way to avoid the problem is to define some mutex lock to prevent two thread to, let’s say, enter the setter of an object at the same time.
Multithreading is possible in python, but two threads cannot be executed at the same time
at a granularity finer than one python instruction.
The running thread is getting a global lock called GIL.
This means if you begin write some multithreaded code in order to take advantage of your multicore processor, your performance won’t improve.
The usual workaround consists of going multiprocess.
Note that it is possible to release the GIL if you’re inside a method you wrote in C for instance.
The use of a GIL is not inherent to Python but to some of its interpreter, including the most common CPython.
(#edited, see comment)
The GIL issue is still valid in Python 3000.
Let’s first understand what the python GIL provides:
Any operation/instruction is executed in the interpreter. GIL ensures that interpreter is held by a single thread at a particular instant of time. And your python program with multiple threads works in a single interpreter. At any particular instant of time, this interpreter is held by a single thread. It means that only the thread which is holding the interpreter is running at any instant of time.
Now why is that an issue:
Your machine could be having multiple cores/processors. And multiple cores allow multiple threads to execute simultaneously i.e multiple threads could execute at any particular instant of time..
But since the interpreter is held by a single thread, other threads are not doing anything even though they have access to a core. So, you are not getting any advantage provided by multiple cores because at any instant only a single core, which is the core being used by the thread currently holding the interpreter, is being used. So, your program will take as long to execute as if it were a single threaded program.
However, potentially blocking or long-running operations, such as I/O, image processing, and NumPy number crunching, happen outside the GIL. Taken from here. So for such operations, a multithreaded operation will still be faster than a single threaded operation despite the presence of GIL. So, GIL is not always a bottleneck.
Edit: GIL is an implementation detail of CPython. IronPython and Jython don’t have GIL, so a truly multithreaded program should be possible in them, thought I have never used PyPy and Jython and not sure of this.
Why Python (CPython and others) uses the GIL
From http://wiki.python.org/moin/GlobalInterpreterLock
In CPython, the global interpreter lock, or GIL, is a mutex that prevents multiple native threads from executing Python bytecodes at once. This lock is necessary mainly because CPython’s memory management is not thread-safe.
How to remove it from Python?
Like Lua, maybe Python could start multiple VM, But python doesn’t do that, I guess there should be some other reasons.
In Numpy or some other python extended library, sometimes, releasing the GIL to other threads could boost the efficiency of the whole programme.
I want to share an example from the book multithreading for Visual Effects. So here is a classic dead lock situation
static void MyCallback(const Context &context){
Auto<Lock> lock(GetMyMutexFromContext(context));
...
EvalMyPythonString(str); //A function that takes the GIL
...
}
Now consider the events in the sequence resulting a dead-lock.
βββββ¦βββββββββββββββββββββββββββββββββββββββββ¦βββββββββββββββββββββββββββββββββββββββ
β β Main Thread β Other Thread β
β ββββ¬βββββββββββββββββββββββββββββββββββββββββ¬βββββββββββββββββββββββββββββββββββββββ£
β 1 β Python Command acquires GIL β Work started β
β 2 β Computation requested β MyCallback runs and acquires MyMutex β
β 3 β β MyCallback now waits for GIL β
β 4 β MyCallback runs and waits for MyMutex β waiting for GIL β
βββββ©βββββββββββββββββββββββββββββββββββββββββ©βββββββββββββββββββββββββββββββββββββββ
Python doesn’t allow multi-threading in the truest sense of the word. It has a multi-threading package but if you want to multi-thread to speed your code up, then it’s usually not a good idea to use it. Python has a construct called the Global Interpreter Lock (GIL).
https://www.youtube.com/watch?v=ph374fJqFPE
The GIL makes sure that only one of your ‘threads’ can execute at any one time. A thread acquires the GIL, does a little work, then passes the GIL onto the next thread. This happens very quickly so to the human eye it may seem like your threads are executing in parallel, but they are really just taking turns using the same CPU core. All this GIL passing adds overhead to execution. This means that if you want to make your code run faster then using the threading package often isn’t a good idea.
There are reasons to use Python’s threading package. If you want to run some things simultaneously, and efficiency is not a concern, then it’s totally fine and convenient. Or if you are running code that needs to wait for something (like some IO) then it could make a lot of sense. But the threading library wont let you use extra CPU cores.
Multi-threading can be outsourced to the operating system (by doing multi-processing), some external application that calls your Python code (eg, Spark or Hadoop), or some code that your Python code calls (eg: you could have your Python code call a C function that does the expensive multi-threaded stuff).
Python 3.7 documentation
I would also like to highlight the following quote from the Python threading documentation:
CPython implementation detail: In CPython, due to the Global Interpreter Lock, only one thread can execute Python code at once (even though certain performance-oriented libraries might overcome this limitation). If you want your application to make better use of the computational resources of multi-core machines, you are advised to use
multiprocessingorconcurrent.futures.ProcessPoolExecutor. However, threading is still an appropriate model if you want to run multiple I/O-bound tasks simultaneously.
This links to the Glossary entry for global interpreter lock which explains that the GIL implies that threaded parallelism in Python is unsuitable for CPU bound tasks:
The mechanism used by the CPython interpreter to assure that only one thread executes Python bytecode at a time. This simplifies the CPython implementation by making the object model (including critical built-in types such as dict) implicitly safe against concurrent access. Locking the entire interpreter makes it easier for the interpreter to be multi-threaded, at the expense of much of the parallelism afforded by multi-processor machines.
However, some extension modules, either standard or third-party, are designed so as to release the GIL when doing computationally-intensive tasks such as compression or hashing. Also, the GIL is always released when doing I/O.
Past efforts to create a βfree-threadedβ interpreter (one which locks shared data at a much finer granularity) have not been successful because performance suffered in the common single-processor case. It is believed that overcoming this performance issue would make the implementation much more complicated and therefore costlier to maintain.
This quote also implies that dicts and thus variable assignment are also thread safe as a CPython implementation detail:
Next, the docs for the multiprocessing package explain how it overcomes the GIL by spawning process while exposing an interface similar to that of threading:
multiprocessing is a package that supports spawning processes using an API similar to the threading module. The multiprocessing package offers both local and remote concurrency, effectively side-stepping the Global Interpreter Lock by using subprocesses instead of threads. Due to this, the multiprocessing module allows the programmer to fully leverage multiple processors on a given machine. It runs on both Unix and Windows.
And the docs for concurrent.futures.ProcessPoolExecutor explain that it uses multiprocessing as a backend:
The ProcessPoolExecutor class is an Executor subclass that uses a pool of processes to execute calls asynchronously. ProcessPoolExecutor uses the multiprocessing module, which allows it to side-step the Global Interpreter Lock but also means that only picklable objects can be executed and returned.
which should be contrasted to the other base class ThreadPoolExecutor that uses threads instead of processes
ThreadPoolExecutor is an Executor subclass that uses a pool of threads to execute calls asynchronously.
from which we conclude that ThreadPoolExecutor is only suitable for I/O bound tasks, while ProcessPoolExecutor can also handle CPU bound tasks.
Process vs thread experiments
At Multiprocessing vs Threading Python I’ve done an experimental analysis of process vs threads in Python.
Quick preview of the results:
In other languages
The concept seems to exist outside of Python as well, applying just as well to Ruby for example: https://en.wikipedia.org/wiki/Global_interpreter_lock
It mentions the advantages:
- increased speed of single-threaded programs (no necessity to acquire or release locks on all data structures separately),
- easy integration of C libraries that usually are not thread-safe,
- ease of implementation (having a single GIL is much simpler to implement than a lock-free interpreter or one using fine-grained locks).
but the JVM seems to do just fine without the GIL, so I wonder if it is worth it. The following question asks why the GIL exists in the first place: Why the Global Interpreter Lock?