How do I avoid catastrophic cancellation for small numbers in f(x) = (1-cos(x))/x**2 in Python 3.7?
Question:
How do I avoid catastrophic cancellation for small numbers in
f(x) = (1-cos(x))/x**2 in Python 3.7?
This is what I tried so far (the key, I know, is some trigonometric identity that enables you to avoid the cancellation, and I also know, having used L’Hopital’s rule, that the limit→0 for f(x) is 0.5, so the correct program output is something very close to 0.5, which is what you do get if you use x = 1.2e-4 for example, but you get cancellation with smaller numbers like 1.2e-8, and I need to make it so this doesn’t happen).
from math import *
def f(x): #these are all the same function using different identities
a = (1-(sin(x)/tan(x)))/(x**2)
b = (1-(sin(2*x)/(2*sin(x))))/(x**2)
c = (1-((1-((tan(x/2))**2))/(1+(tan(x/2))**2)))/(x**2)
d = (sin(x)**2+cos(x)**2-cos(x))/(x**2)
e = (sin(x)**2+cos(x)**2-(sin(2*x)/(2*sin(x))))/(x**2)
return a, b, c, d, e
print(k(1.2e-8))
#Output: (0.0, 0.7709882115452477, 0.0, 0.0, 0.0) - whereas need 0.5000...
Answers:
Like this:
sin(x)/x * tan(x/2)/x
It does the job right up to the end, x = 1e-308 is still OK.
Unfortunately I cannot offer much insight into why it works well.
THe problem is the limited precision of float and double. You have to use arithmetic with more precision such as mpfr. It can be used in Python through binding such as
https://pypi.org/project/gmpy2/
Here is an example where I’m using it through a more high level environment called Sagemath: I’m using 100 bits of precision:
sage: R = RealField(100) // 100 bits of precision
sage: def f(x): #these are all the same function using different identities
....: a = (1-(sin(x)/tan(x)))/(x**2)
....: b = (1-(sin(2*x)/(2*sin(x))))/(x**2)
....: c = (1-((1-((tan(x/2))**2))/(1+(tan(x/2))**2)))/(x**2)
....: d = (sin(x)**2+cos(x)**2-cos(x))/(x**2)
....: e = (sin(x)**2+cos(x)**2-(sin(2*x)/(2*sin(x))))/(x**2)
....: return a, b, c, d, e
....:
sage: f(R(1.2e-8))
(0.50000000000000264647624775223,
0.49999999999999716827551705076,
0.49999999999999716827551705076,
0.49999999999999716827551705076,
0.49999999999999169007478634929)
Use numpy.float128 instead. numpy is the standard for data analysis and more complex math. It can be installed with the following command in the terminal.
pip install numpy
from numpy import *
def f(x): #these are all the same function using different identities
a = (1-(sin(x)/tan(x)))/(x**2)
b = (1-(sin(2*x)/(2*sin(x))))/(x**2)
c = (1-((1-((tan(x/2))**2))/(1+(tan(x/2))**2)))/(x**2)
d = (sin(x)**2+cos(x)**2-cos(x))/(x**2)
e = (sin(x)**2+cos(x)**2-(sin(2*x)/(2*sin(x))))/(x**2)
return a, b, c, d, e
print(f(float128(1.2e-8)))
This prints
(0.5003141275115400736, 0.49956120933620291774, 0.49993766842387149567, 0.49993766842387149567, 0.49956120933620291774)
You can conditionally return the limit for small x, like this:
import matplotlib.pyplot as plt
from numpy import *
epsilon=1e-8
def f(x):
if x<epsilon
return 0.5
return (1-sin(x)/tan(x))/x**2
#note: same as (1-cos(x))/x**2
x=arange(0,6,0.01)
y=vectorize(f)
plt.plot(x,y(x))
plt.show()
The curve plotted looks smooth
Note: I prefer numpy to math. Vectorize makes it possible to call the function with an array (not very efficient, but it is easy to use).
How do I avoid catastrophic cancellation for small numbers in f(x) = (1-cos(x))/x**2 in Python 3.7?
You can use a half-angle identity:
1 − cos(x) = 2 sin²(x/2).
(Historical trigonometers will recognize this as the versine function, or twice the haversine function, which is useful and tabulated for this purpose!)
If you compute cos(x) as an approximation to cos(x) for x near 0, then catastrophic cancellation in the subtraction 1 − cos(x) may dramatically amplify any error in that approximation—possibly giving a nonsense answer, if cos(x) is rounded to 1 so (1 - cos(x))/x**2 is rounded to 0 when it should be near 1/2.
In contrast, if you rewrite it as
(1 − cos(x))/x² = 2 sin²(x/2)/x²
using the half-angle identity, and then evaluate 2*(sin(x/2)/x)**2 for x near zero, you are composing functions on domains where they are well-conditioned.
In binary floating-point, dividing and multiplying by two is always exact except in case of overflow or underflow, and sin and squaring are both well-conditioned near zero, so the relative error is reasonably small even with intermediate rounding and error in the math library’s sin procedure.
You can also use another half-angle identity,
1 − cos(x) = sin(x) tan(x/2),
as another answer suggests—both sin and tan are well-conditioned near zero, so it will also give a good approximation.
But it costs more to compute sin and tan than to just compute sin, and I see no obvious advantage in error at least for inputs near zero.
To handle x = 0 without conditionals, if you have a sinc procedure which approximates the function sinc(x) = sin(x)/x naturally extended with the limit sinc(0) = 0, you could also rewrite it as:
(1 − cos(x))/x² = 2 sin²(x/2)/x² = (1/2) sin²(x/2)/(x/2)² = (1/2) sinc²(x/2).
How do I avoid catastrophic cancellation for small numbers in
f(x) = (1-cos(x))/x**2 in Python 3.7?
This is what I tried so far (the key, I know, is some trigonometric identity that enables you to avoid the cancellation, and I also know, having used L’Hopital’s rule, that the limit→0 for f(x) is 0.5, so the correct program output is something very close to 0.5, which is what you do get if you use x = 1.2e-4 for example, but you get cancellation with smaller numbers like 1.2e-8, and I need to make it so this doesn’t happen).
from math import *
def f(x): #these are all the same function using different identities
a = (1-(sin(x)/tan(x)))/(x**2)
b = (1-(sin(2*x)/(2*sin(x))))/(x**2)
c = (1-((1-((tan(x/2))**2))/(1+(tan(x/2))**2)))/(x**2)
d = (sin(x)**2+cos(x)**2-cos(x))/(x**2)
e = (sin(x)**2+cos(x)**2-(sin(2*x)/(2*sin(x))))/(x**2)
return a, b, c, d, e
print(k(1.2e-8))
#Output: (0.0, 0.7709882115452477, 0.0, 0.0, 0.0) - whereas need 0.5000...
Like this:
sin(x)/x * tan(x/2)/x
It does the job right up to the end, x = 1e-308 is still OK.
Unfortunately I cannot offer much insight into why it works well.
THe problem is the limited precision of float and double. You have to use arithmetic with more precision such as mpfr. It can be used in Python through binding such as
https://pypi.org/project/gmpy2/
Here is an example where I’m using it through a more high level environment called Sagemath: I’m using 100 bits of precision:
sage: R = RealField(100) // 100 bits of precision
sage: def f(x): #these are all the same function using different identities
....: a = (1-(sin(x)/tan(x)))/(x**2)
....: b = (1-(sin(2*x)/(2*sin(x))))/(x**2)
....: c = (1-((1-((tan(x/2))**2))/(1+(tan(x/2))**2)))/(x**2)
....: d = (sin(x)**2+cos(x)**2-cos(x))/(x**2)
....: e = (sin(x)**2+cos(x)**2-(sin(2*x)/(2*sin(x))))/(x**2)
....: return a, b, c, d, e
....:
sage: f(R(1.2e-8))
(0.50000000000000264647624775223,
0.49999999999999716827551705076,
0.49999999999999716827551705076,
0.49999999999999716827551705076,
0.49999999999999169007478634929)
Use numpy.float128 instead. numpy is the standard for data analysis and more complex math. It can be installed with the following command in the terminal.
pip install numpy
from numpy import *
def f(x): #these are all the same function using different identities
a = (1-(sin(x)/tan(x)))/(x**2)
b = (1-(sin(2*x)/(2*sin(x))))/(x**2)
c = (1-((1-((tan(x/2))**2))/(1+(tan(x/2))**2)))/(x**2)
d = (sin(x)**2+cos(x)**2-cos(x))/(x**2)
e = (sin(x)**2+cos(x)**2-(sin(2*x)/(2*sin(x))))/(x**2)
return a, b, c, d, e
print(f(float128(1.2e-8)))
This prints
(0.5003141275115400736, 0.49956120933620291774, 0.49993766842387149567, 0.49993766842387149567, 0.49956120933620291774)
You can conditionally return the limit for small x, like this:
import matplotlib.pyplot as plt
from numpy import *
epsilon=1e-8
def f(x):
if x<epsilon
return 0.5
return (1-sin(x)/tan(x))/x**2
#note: same as (1-cos(x))/x**2
x=arange(0,6,0.01)
y=vectorize(f)
plt.plot(x,y(x))
plt.show()
The curve plotted looks smooth
Note: I prefer numpy to math. Vectorize makes it possible to call the function with an array (not very efficient, but it is easy to use).
How do I avoid catastrophic cancellation for small numbers in f(x) = (1-cos(x))/x**2 in Python 3.7?
You can use a half-angle identity:
1 − cos(x) = 2 sin²(x/2).
(Historical trigonometers will recognize this as the versine function, or twice the haversine function, which is useful and tabulated for this purpose!)
If you compute cos(x) as an approximation to cos(x) for x near 0, then catastrophic cancellation in the subtraction 1 − cos(x) may dramatically amplify any error in that approximation—possibly giving a nonsense answer, if cos(x) is rounded to 1 so (1 - cos(x))/x**2 is rounded to 0 when it should be near 1/2.
In contrast, if you rewrite it as
(1 − cos(x))/x² = 2 sin²(x/2)/x²
using the half-angle identity, and then evaluate 2*(sin(x/2)/x)**2 for x near zero, you are composing functions on domains where they are well-conditioned.
In binary floating-point, dividing and multiplying by two is always exact except in case of overflow or underflow, and sin and squaring are both well-conditioned near zero, so the relative error is reasonably small even with intermediate rounding and error in the math library’s sin procedure.
You can also use another half-angle identity,
1 − cos(x) = sin(x) tan(x/2),
as another answer suggests—both sin and tan are well-conditioned near zero, so it will also give a good approximation.
But it costs more to compute sin and tan than to just compute sin, and I see no obvious advantage in error at least for inputs near zero.
To handle x = 0 without conditionals, if you have a sinc procedure which approximates the function sinc(x) = sin(x)/x naturally extended with the limit sinc(0) = 0, you could also rewrite it as:
(1 − cos(x))/x² = 2 sin²(x/2)/x² = (1/2) sin²(x/2)/(x/2)² = (1/2) sinc²(x/2).