warning about too many open figures
Question:
In a script where I create many figures with fix, ax = plt.subplots(...), I get the warning RuntimeWarning: More than 20 figures have been opened. Figures created through the pyplot interface (matplotlib.pyplot.figure) are retained until explicitly closed and may consume too much memory.
However, I don’t understand why I get this warning, because after saving the figure with fig.savefig(...), I delete it with fig.clear(); del fig. At no point in my code, I have more than one figure open at a time. Still, I get the warning about too many open figures. What does that mean / how can I avoid getting the warning?
Answers:
Use .clf or .cla on your figure object instead of creating a new figure. From @DavidZwicker
Assuming you have imported pyplot as
import matplotlib.pyplot as plt
plt.cla() clears an axis, i.e. the currently active axis in the current figure. It leaves the other axes untouched.
plt.clf() clears the entire current figure with all its axes, but leaves the window opened, such that it may be reused for other plots.
plt.close() closes a window, which will be the current window, if not specified otherwise. plt.close('all') will close all open figures.
The reason that del fig does not work is that the pyplot state-machine keeps a reference to the figure around (as it must if it is going to know what the ‘current figure’ is). This means that even if you delete your ref to the figure, there is at least one live ref, hence it will never be garbage collected.
Since I’m polling on the collective wisdom here for this answer, @JoeKington mentions in the comments that plt.close(fig) will remove a specific figure instance from the pylab state machine (plt._pylab_helpers.Gcf) and allow it to be garbage collected.
Here’s a bit more detail to expand on Hooked’s answer. When I first read that answer, I missed the instruction to call clf() instead of creating a new figure. clf() on its own doesn’t help if you then go and create another figure.
Here’s a trivial example that causes the warning:
from matplotlib import pyplot as plt, patches
import os
def main():
path = 'figures'
for i in range(21):
_fig, ax = plt.subplots()
x = range(3*i)
y = [n*n for n in x]
ax.add_patch(patches.Rectangle(xy=(i, 1), width=i, height=10))
plt.step(x, y, linewidth=2, where='mid')
figname = 'fig_{}.png'.format(i)
dest = os.path.join(path, figname)
plt.savefig(dest) # write image to file
plt.clf()
print('Done.')
main()
To avoid the warning, I have to pull the call to subplots() outside the loop. In order to keep seeing the rectangles, I need to switch clf() to cla(). That clears the axis without removing the axis itself.
from matplotlib import pyplot as plt, patches
import os
def main():
path = 'figures'
_fig, ax = plt.subplots()
for i in range(21):
x = range(3*i)
y = [n*n for n in x]
ax.add_patch(patches.Rectangle(xy=(i, 1), width=i, height=10))
plt.step(x, y, linewidth=2, where='mid')
figname = 'fig_{}.png'.format(i)
dest = os.path.join(path, figname)
plt.savefig(dest) # write image to file
plt.cla()
print('Done.')
main()
If you’re generating plots in batches, you might have to use both cla() and close(). I ran into a problem where a batch could have more than 20 plots without complaining, but it would complain after 20 batches. I fixed that by using cla() after each plot, and close() after each batch.
from matplotlib import pyplot as plt, patches
import os
def main():
for i in range(21):
print('Batch {}'.format(i))
make_plots('figures')
print('Done.')
def make_plots(path):
fig, ax = plt.subplots()
for i in range(21):
x = range(3 * i)
y = [n * n for n in x]
ax.add_patch(patches.Rectangle(xy=(i, 1), width=i, height=10))
plt.step(x, y, linewidth=2, where='mid')
figname = 'fig_{}.png'.format(i)
dest = os.path.join(path, figname)
plt.savefig(dest) # write image to file
plt.cla()
plt.close(fig)
main()
I measured the performance to see if it was worth reusing the figure within a batch, and this little sample program slowed from 41s to 49s (20% slower) when I just called close() after every plot.
If you intend to knowingly keep many plots in memory, but don’t want to be warned about it, you can update your options prior to generating figures.
import matplotlib.pyplot as plt
plt.rcParams.update({'figure.max_open_warning': 0})
This will prevent the warning from being emitted without changing anything about the way memory is managed.
The following snippet solved the issue for me:
class FigureWrapper(object):
'''Frees underlying figure when it goes out of scope.
'''
def __init__(self, figure):
self._figure = figure
def __del__(self):
plt.close(self._figure)
print("Figure removed")
# .....
f, ax = plt.subplots(1, figsize=(20, 20))
_wrapped_figure = FigureWrapper(f)
ax.plot(...
plt.savefig(...
# .....
When _wrapped_figure goes out of scope the runtime calls our __del__() method with plt.close() inside. It happens even if exception fires after _wrapped_figure constructor.
This is also useful if you only want to temporarily suppress the warning:
import matplotlib.pyplot as plt
with plt.rc_context(rc={'figure.max_open_warning': 0}):
lots_of_plots()
import matplotlib.pyplot as plt
plt.rcParams.update({'figure.max_open_warning': 0})
If you use this, you won’t get that error, and it is the simplest way to do that.
matplotlib by default keeps a reference of all the figures created through pyplot. If a single variable used for storing matplotlib figure (e.g "fig") is modified and rewritten without clearing the figure, all the plots are retained in RAM memory. Its important to use plt.cla() and plt.clf() instead of modifying and reusing the fig variable. If you are plotting thousands of different plots and saving them without clearing the figure, then eventually your RAM will get exhausted and program will get terminated. Clearing the axes and figures have a significant impact on memory usage if you are plotting many figures. You can monitor your RAM consumption in task manager (Windows) or in system monitor (Linux). First your RAM will get exhausted, then the OS starts consuming SWAP memory. Once both are exhausted, the program will get automatically terminated. Its better to clear figures, axes and close them if they are not required.
In a script where I create many figures with fix, ax = plt.subplots(...), I get the warning RuntimeWarning: More than 20 figures have been opened. Figures created through the pyplot interface (matplotlib.pyplot.figure) are retained until explicitly closed and may consume too much memory.
However, I don’t understand why I get this warning, because after saving the figure with fig.savefig(...), I delete it with fig.clear(); del fig. At no point in my code, I have more than one figure open at a time. Still, I get the warning about too many open figures. What does that mean / how can I avoid getting the warning?
Use .clf or .cla on your figure object instead of creating a new figure. From @DavidZwicker
Assuming you have imported pyplot as
import matplotlib.pyplot as plt
plt.cla() clears an axis, i.e. the currently active axis in the current figure. It leaves the other axes untouched.
plt.clf() clears the entire current figure with all its axes, but leaves the window opened, such that it may be reused for other plots.
plt.close() closes a window, which will be the current window, if not specified otherwise. plt.close('all') will close all open figures.
The reason that del fig does not work is that the pyplot state-machine keeps a reference to the figure around (as it must if it is going to know what the ‘current figure’ is). This means that even if you delete your ref to the figure, there is at least one live ref, hence it will never be garbage collected.
Since I’m polling on the collective wisdom here for this answer, @JoeKington mentions in the comments that plt.close(fig) will remove a specific figure instance from the pylab state machine (plt._pylab_helpers.Gcf) and allow it to be garbage collected.
Here’s a bit more detail to expand on Hooked’s answer. When I first read that answer, I missed the instruction to call clf() instead of creating a new figure. clf() on its own doesn’t help if you then go and create another figure.
Here’s a trivial example that causes the warning:
from matplotlib import pyplot as plt, patches
import os
def main():
path = 'figures'
for i in range(21):
_fig, ax = plt.subplots()
x = range(3*i)
y = [n*n for n in x]
ax.add_patch(patches.Rectangle(xy=(i, 1), width=i, height=10))
plt.step(x, y, linewidth=2, where='mid')
figname = 'fig_{}.png'.format(i)
dest = os.path.join(path, figname)
plt.savefig(dest) # write image to file
plt.clf()
print('Done.')
main()
To avoid the warning, I have to pull the call to subplots() outside the loop. In order to keep seeing the rectangles, I need to switch clf() to cla(). That clears the axis without removing the axis itself.
from matplotlib import pyplot as plt, patches
import os
def main():
path = 'figures'
_fig, ax = plt.subplots()
for i in range(21):
x = range(3*i)
y = [n*n for n in x]
ax.add_patch(patches.Rectangle(xy=(i, 1), width=i, height=10))
plt.step(x, y, linewidth=2, where='mid')
figname = 'fig_{}.png'.format(i)
dest = os.path.join(path, figname)
plt.savefig(dest) # write image to file
plt.cla()
print('Done.')
main()
If you’re generating plots in batches, you might have to use both cla() and close(). I ran into a problem where a batch could have more than 20 plots without complaining, but it would complain after 20 batches. I fixed that by using cla() after each plot, and close() after each batch.
from matplotlib import pyplot as plt, patches
import os
def main():
for i in range(21):
print('Batch {}'.format(i))
make_plots('figures')
print('Done.')
def make_plots(path):
fig, ax = plt.subplots()
for i in range(21):
x = range(3 * i)
y = [n * n for n in x]
ax.add_patch(patches.Rectangle(xy=(i, 1), width=i, height=10))
plt.step(x, y, linewidth=2, where='mid')
figname = 'fig_{}.png'.format(i)
dest = os.path.join(path, figname)
plt.savefig(dest) # write image to file
plt.cla()
plt.close(fig)
main()
I measured the performance to see if it was worth reusing the figure within a batch, and this little sample program slowed from 41s to 49s (20% slower) when I just called close() after every plot.
If you intend to knowingly keep many plots in memory, but don’t want to be warned about it, you can update your options prior to generating figures.
import matplotlib.pyplot as plt
plt.rcParams.update({'figure.max_open_warning': 0})
This will prevent the warning from being emitted without changing anything about the way memory is managed.
The following snippet solved the issue for me:
class FigureWrapper(object):
'''Frees underlying figure when it goes out of scope.
'''
def __init__(self, figure):
self._figure = figure
def __del__(self):
plt.close(self._figure)
print("Figure removed")
# .....
f, ax = plt.subplots(1, figsize=(20, 20))
_wrapped_figure = FigureWrapper(f)
ax.plot(...
plt.savefig(...
# .....
When _wrapped_figure goes out of scope the runtime calls our __del__() method with plt.close() inside. It happens even if exception fires after _wrapped_figure constructor.
This is also useful if you only want to temporarily suppress the warning:
import matplotlib.pyplot as plt
with plt.rc_context(rc={'figure.max_open_warning': 0}):
lots_of_plots()
import matplotlib.pyplot as plt
plt.rcParams.update({'figure.max_open_warning': 0})
If you use this, you won’t get that error, and it is the simplest way to do that.
matplotlib by default keeps a reference of all the figures created through pyplot. If a single variable used for storing matplotlib figure (e.g "fig") is modified and rewritten without clearing the figure, all the plots are retained in RAM memory. Its important to use plt.cla() and plt.clf() instead of modifying and reusing the fig variable. If you are plotting thousands of different plots and saving them without clearing the figure, then eventually your RAM will get exhausted and program will get terminated. Clearing the axes and figures have a significant impact on memory usage if you are plotting many figures. You can monitor your RAM consumption in task manager (Windows) or in system monitor (Linux). First your RAM will get exhausted, then the OS starts consuming SWAP memory. Once both are exhausted, the program will get automatically terminated. Its better to clear figures, axes and close them if they are not required.