How are exceptions implemented under the hood?
Question:
Just about everyone uses them, but many, including me simply take it for granted that they just work.
I am looking for high-quality material. Languages I use are: Java, C, C#, Python, C++, so these are of most interest to me.
Now, C++ is probably a good place to start since you can throw anything in that language.
Also, C is close to assembly. How would one emulate exceptions using pure C constructs and no assembly?
Finally, I heard a rumor that Google employees do not use exceptions for some projects due to speed considerations. Is this just a rumor? How can anything substantial be accomplished without them?
Thank you.
Answers:
setjmp() and longjmp() usually.
Exception catching does have a non-trivial cost, but for most purposes it’s not a big deal.
In his book C Interfaces and Implementations: Techniques for Creating Reusable Software, D. R. Hanson provides a nice implementation of exceptions in pure C using a set of macros and setjmp/longjmp. He provides TRY/RAISE/EXCEPT/FINALLY macros that can emulate pretty much everything C++ exceptions do and more.
The code can be perused here (look at except.h/except.c).
P.S. re your question about Google. Their employees are actually allowed to use exceptions in new code, and the official reason for the ban in old code is because it was already written that way and it doesn’t make sense to mix styles.
Personally, I also think that C++ without exceptions isn’t the best idea.
C/C++ compilers use the underlying OS facilities for exception handling. Frameworks like .Net or Java also rely, in the VM, on the OS facilities. In Windows for instance, the real heavy lifting is done by SEH, the Structured Exception Handling infrastructure. You should absolutely read the old reference article: A Crash Course on the Depths of Win32™ Structured Exception Handling.
As for the cost of not using exceptions, they are expensive but compared to what? Compared to return error codes? After you factor in the cost of correctness and the quality of code, exceptions will always win for commercial applications. Short of few very critical OS level functions, exceptions are always better overall.
An last but not least there is the anti-pattern of using exceptions for flow control. Exceptions should be exceptional and code that abuses exceptions fro flow control will pay the price in performance.
Regarding performance – sparse use of exceptions will probably have negligible effects, but do not abuse them.
I have personally seen Java code which performed two orders of magnitude worse than it could have (took about x100 the time) because exceptions were used in an important loop instead of more standard if/returns.
Here is a common way C++ exceptions are implemented:
http://www.codesourcery.com/public/cxx-abi/abi-eh.html
It is for the Itanium architecture, but the implementation described here is used in other architectures as well. Note that it is a long document, since C++ exceptions are complicated.
Here is a good description on how LLVM implements exceptions:
http://llvm.org/docs/ExceptionHandling.html
Since LLVM is meant to be a common intermediate representation for many runtimes, the mechanisms described can be applied to many languages.
C++ code at Google (save for some Windows-specific cases) don’t use exceptions: cfr the guidelines, short form: “We do not use C++ exceptions”. Quoting from the discussion (hit the arrow to expand on the URL):
Our advice against using exceptions is
not predicated on philosophical or
moral grounds, but practical ones.
Because we’d like to use our
open-source projects at Google and
it’s difficult to do so if those
projects use exceptions, we need to
advise against exceptions in Google
open-source projects as well. Things
would probably be different if we had
to do it all over again from scratch.
This rule does not apply to Google code in other languages, such as Java and Python.
Some runtimes like the Objective-C runtime have zero-cost 64-bit exceptions. What that means is that it doesn’t cost anything to enter a try block. However, this is quite costly when the exception is thrown. This follows the paradigm of “optimize for the average case” – exceptions are meant to be exceptional, so it is better to make the case when there are no exceptions really fast, even if it comes at the cost of significantly slower exceptions.
Exceptions are just a specific example of a more general case of advanced non-local flow control constructs. Other examples are:
- notifications (a generalization of exceptions, originally from some old Lisp object system, now implemented in e.g. CommonLisp and Ioke),
- continuations (a more structured form of
GOTO, popular in high-level, higher-order languages),
- coroutines (a generalization of subroutines, popular especially in Lua),
- generators à la Python (essentially a restricted form of coroutines),
- fibers (cooperative light-weight threads) and of course the already mentioned
GOTO.
(I’m sure there’s many others I missed.)
An interesting property of these constructs is that they are all roughly equivalent in expressive power: if you have one, you can pretty easily build all the others.
So, how you best implement exceptions depends on what other constructs you have available:
- Every CPU has
GOTO, therefore you can always fall back to that, if you must.
- C has
setjmp/longjmp which are basically MacGyver continuations (built out of duct-tape and toothpicks, not quite the real thing, but will at least get you out of the immediate trouble if you don’t have something better available).
- The JVM and CLI have exceptions of their own, which means that if the exception semantics of your language match Java’s/C#’s, you are home free (but if not, then you are screwed).
- The Parrot VM as both exceptions and continuations.
- Windows has its own framework for exception handling, which language implementors can use to build their own exceptions on top.
A very interesting use case, both of the usage of exceptions and the implementation of exceptions is Microsoft Live Lab’s Volta Project. (Now defunct.) The goal of Volta was to provide architectural refactoring for Web applications at the push of a button. So, you could turn your one-tier web application into a two- or three-tier application just by putting some [Browser] or [DB] attributes on your .NET code and the code would then automagically run on the client or in the DB. In order to do that, the .NET code had to be translated to JavaScript source code, obviously.
Now, you could just write an entire VM in JavaScript and run the bytecode unmodified. (Basically, port the CLR from C++ to JavaScript.) There are actually projects that do this (e.g. the HotRuby VM), but this is both inefficient and not very interoperable with other JavaScript code.
So, instead, they wrote a compiler which compiles CIL bytecode to JavaScript sourcecode. However, JavaScript lacks certain features that .NET has (generators, threads, also the two exception models aren’t 100% compatible), and more importantly it lacks certain features that compiler writers love (either GOTO or continuations) and that could be used to implement the above-mentioned missing features.
However, JavaScript does have exceptions. So, they used JavaScript Exceptions to implement Volta Continuations and then they used Volta Continuations to implement .NET Exceptions, .NET Generators and even .NET Managed Threads(!!!)
So, to answer your original question:
How are exceptions implemented under the hood?
With Exceptions, ironically! At least in this very specific case, anyway.
Another great example is some of the exception proposals on the Go mailing list, which implement exceptions using Goroutines (something like a mixture of concurrent coroutines ans CSP processes). Yet another example is Haskell, which uses Monads, lazy evaluation, tail call optimization and higher-order functions to implement exceptions. Some modern CPUs also support basic building blocks for exceptions (for example the Vega-3 CPUs that were specifically designed for the Azul Systems Java Compute Accelerators).
The key thing an exception implementation needs to handle is how to return to the exception handler once an exception has been thrown. Since you may have made an arbitrary number of nested function calls since the try statement in C++, it must unwind the call stack searching for the handler. However implemented, this must incur the code size cost of maintaining sufficient information in order to perform this operation (and generally means a table of data for calls that can take exceptions). It also means that the dynamic code execution path will be longer than simply returning from functions calls (which is a fairly inexpensive operation on most platforms). There may be other costs as well depending on the implementation.
The relative cost will vary depending on the language used. The higher-level language used, the less likely the code size cost will matter, and the information may be retained regardless of whether exceptions are used.
An application where the use of exceptions (and C++ in general) is often avoided for good reasons is embedded firmware. In typical small bare metal or RTOS platforms, you might have 1MB of code space, or 64K, or even smaller. Some platforms are so small, even C is not practical to use. In this kind of environment, the size impact is relevant because of the cost mentioned above. It also impacts the standard library itself. Embedded toolchain vendors will often produce a library without exception capability, which has a huge impact on code size. Highly optimizing compilers may also analyze the callgraph and optimize away needed call frame information for the unwind operation for considerable space reduction. Exceptions also make it more difficult to analyze hard real-time requirements.
In more typical environments, the code size cost is almost certainly irrelevant and the performance factor is likely key. Whether you use them will depend on your performance requirements and how you want to use them. Using exceptions in non-exceptional cases can make an elegant design, but at a performance cost that may be unacceptable for high performance systems. Implementations and relative cost will vary by platform and compiler, so the best way to truly understand if exceptions are a problem is to analyze your own code’s performance.
The best paper ever written on the implementation of exceptions (under the hood) is Exception Handling in CLU by Barbara Liskov and Alan Snyder. I have referred to it every time I’ve started a new compiler.
For a somewhat higher-level view of an implementation in C using setjmp and longjmp, I recommend Dave Hanson’s C Interfaces and Implementations (like Eli Bendersky).
Just about everyone uses them, but many, including me simply take it for granted that they just work.
I am looking for high-quality material. Languages I use are: Java, C, C#, Python, C++, so these are of most interest to me.
Now, C++ is probably a good place to start since you can throw anything in that language.
Also, C is close to assembly. How would one emulate exceptions using pure C constructs and no assembly?
Finally, I heard a rumor that Google employees do not use exceptions for some projects due to speed considerations. Is this just a rumor? How can anything substantial be accomplished without them?
Thank you.
setjmp() and longjmp() usually.
Exception catching does have a non-trivial cost, but for most purposes it’s not a big deal.
In his book C Interfaces and Implementations: Techniques for Creating Reusable Software, D. R. Hanson provides a nice implementation of exceptions in pure C using a set of macros and setjmp/longjmp. He provides TRY/RAISE/EXCEPT/FINALLY macros that can emulate pretty much everything C++ exceptions do and more.
The code can be perused here (look at except.h/except.c).
P.S. re your question about Google. Their employees are actually allowed to use exceptions in new code, and the official reason for the ban in old code is because it was already written that way and it doesn’t make sense to mix styles.
Personally, I also think that C++ without exceptions isn’t the best idea.
C/C++ compilers use the underlying OS facilities for exception handling. Frameworks like .Net or Java also rely, in the VM, on the OS facilities. In Windows for instance, the real heavy lifting is done by SEH, the Structured Exception Handling infrastructure. You should absolutely read the old reference article: A Crash Course on the Depths of Win32™ Structured Exception Handling.
As for the cost of not using exceptions, they are expensive but compared to what? Compared to return error codes? After you factor in the cost of correctness and the quality of code, exceptions will always win for commercial applications. Short of few very critical OS level functions, exceptions are always better overall.
An last but not least there is the anti-pattern of using exceptions for flow control. Exceptions should be exceptional and code that abuses exceptions fro flow control will pay the price in performance.
Regarding performance – sparse use of exceptions will probably have negligible effects, but do not abuse them.
I have personally seen Java code which performed two orders of magnitude worse than it could have (took about x100 the time) because exceptions were used in an important loop instead of more standard if/returns.
Here is a common way C++ exceptions are implemented:
http://www.codesourcery.com/public/cxx-abi/abi-eh.html
It is for the Itanium architecture, but the implementation described here is used in other architectures as well. Note that it is a long document, since C++ exceptions are complicated.
Here is a good description on how LLVM implements exceptions:
http://llvm.org/docs/ExceptionHandling.html
Since LLVM is meant to be a common intermediate representation for many runtimes, the mechanisms described can be applied to many languages.
C++ code at Google (save for some Windows-specific cases) don’t use exceptions: cfr the guidelines, short form: “We do not use C++ exceptions”. Quoting from the discussion (hit the arrow to expand on the URL):
Our advice against using exceptions is
not predicated on philosophical or
moral grounds, but practical ones.
Because we’d like to use our
open-source projects at Google and
it’s difficult to do so if those
projects use exceptions, we need to
advise against exceptions in Google
open-source projects as well. Things
would probably be different if we had
to do it all over again from scratch.
This rule does not apply to Google code in other languages, such as Java and Python.
Some runtimes like the Objective-C runtime have zero-cost 64-bit exceptions. What that means is that it doesn’t cost anything to enter a try block. However, this is quite costly when the exception is thrown. This follows the paradigm of “optimize for the average case” – exceptions are meant to be exceptional, so it is better to make the case when there are no exceptions really fast, even if it comes at the cost of significantly slower exceptions.
Exceptions are just a specific example of a more general case of advanced non-local flow control constructs. Other examples are:
- notifications (a generalization of exceptions, originally from some old Lisp object system, now implemented in e.g. CommonLisp and Ioke),
- continuations (a more structured form of
GOTO, popular in high-level, higher-order languages), - coroutines (a generalization of subroutines, popular especially in Lua),
- generators à la Python (essentially a restricted form of coroutines),
- fibers (cooperative light-weight threads) and of course the already mentioned
GOTO.
(I’m sure there’s many others I missed.)
An interesting property of these constructs is that they are all roughly equivalent in expressive power: if you have one, you can pretty easily build all the others.
So, how you best implement exceptions depends on what other constructs you have available:
- Every CPU has
GOTO, therefore you can always fall back to that, if you must. - C has
setjmp/longjmpwhich are basically MacGyver continuations (built out of duct-tape and toothpicks, not quite the real thing, but will at least get you out of the immediate trouble if you don’t have something better available). - The JVM and CLI have exceptions of their own, which means that if the exception semantics of your language match Java’s/C#’s, you are home free (but if not, then you are screwed).
- The Parrot VM as both exceptions and continuations.
- Windows has its own framework for exception handling, which language implementors can use to build their own exceptions on top.
A very interesting use case, both of the usage of exceptions and the implementation of exceptions is Microsoft Live Lab’s Volta Project. (Now defunct.) The goal of Volta was to provide architectural refactoring for Web applications at the push of a button. So, you could turn your one-tier web application into a two- or three-tier application just by putting some [Browser] or [DB] attributes on your .NET code and the code would then automagically run on the client or in the DB. In order to do that, the .NET code had to be translated to JavaScript source code, obviously.
Now, you could just write an entire VM in JavaScript and run the bytecode unmodified. (Basically, port the CLR from C++ to JavaScript.) There are actually projects that do this (e.g. the HotRuby VM), but this is both inefficient and not very interoperable with other JavaScript code.
So, instead, they wrote a compiler which compiles CIL bytecode to JavaScript sourcecode. However, JavaScript lacks certain features that .NET has (generators, threads, also the two exception models aren’t 100% compatible), and more importantly it lacks certain features that compiler writers love (either GOTO or continuations) and that could be used to implement the above-mentioned missing features.
However, JavaScript does have exceptions. So, they used JavaScript Exceptions to implement Volta Continuations and then they used Volta Continuations to implement .NET Exceptions, .NET Generators and even .NET Managed Threads(!!!)
So, to answer your original question:
How are exceptions implemented under the hood?
With Exceptions, ironically! At least in this very specific case, anyway.
Another great example is some of the exception proposals on the Go mailing list, which implement exceptions using Goroutines (something like a mixture of concurrent coroutines ans CSP processes). Yet another example is Haskell, which uses Monads, lazy evaluation, tail call optimization and higher-order functions to implement exceptions. Some modern CPUs also support basic building blocks for exceptions (for example the Vega-3 CPUs that were specifically designed for the Azul Systems Java Compute Accelerators).
The key thing an exception implementation needs to handle is how to return to the exception handler once an exception has been thrown. Since you may have made an arbitrary number of nested function calls since the try statement in C++, it must unwind the call stack searching for the handler. However implemented, this must incur the code size cost of maintaining sufficient information in order to perform this operation (and generally means a table of data for calls that can take exceptions). It also means that the dynamic code execution path will be longer than simply returning from functions calls (which is a fairly inexpensive operation on most platforms). There may be other costs as well depending on the implementation.
The relative cost will vary depending on the language used. The higher-level language used, the less likely the code size cost will matter, and the information may be retained regardless of whether exceptions are used.
An application where the use of exceptions (and C++ in general) is often avoided for good reasons is embedded firmware. In typical small bare metal or RTOS platforms, you might have 1MB of code space, or 64K, or even smaller. Some platforms are so small, even C is not practical to use. In this kind of environment, the size impact is relevant because of the cost mentioned above. It also impacts the standard library itself. Embedded toolchain vendors will often produce a library without exception capability, which has a huge impact on code size. Highly optimizing compilers may also analyze the callgraph and optimize away needed call frame information for the unwind operation for considerable space reduction. Exceptions also make it more difficult to analyze hard real-time requirements.
In more typical environments, the code size cost is almost certainly irrelevant and the performance factor is likely key. Whether you use them will depend on your performance requirements and how you want to use them. Using exceptions in non-exceptional cases can make an elegant design, but at a performance cost that may be unacceptable for high performance systems. Implementations and relative cost will vary by platform and compiler, so the best way to truly understand if exceptions are a problem is to analyze your own code’s performance.
The best paper ever written on the implementation of exceptions (under the hood) is Exception Handling in CLU by Barbara Liskov and Alan Snyder. I have referred to it every time I’ve started a new compiler.
For a somewhat higher-level view of an implementation in C using setjmp and longjmp, I recommend Dave Hanson’s C Interfaces and Implementations (like Eli Bendersky).