Open Jupyter notebook without render images and without popolating dataframes – safe mode loading preventing out of memory state
Question:
Under Windows 10 I have a Jupyter notebook that I am not able anymore to open because the browser reaches the "out of memory" state. The system resources tool confirms this: reaching 85% of used RAM the process is stopped. The last time I used the notebook there were a lot of rendered charts and some heavy dataframe populated. The feeling is that they are still there and once the notebook is opened again Jupyter tries to load also the full data and all the images. I need at least an easy way to not lose all the code in between. Is there a way to open the notebook in a "safe mode" so that images and dataframe are not loaded?
Answers:
You can copy your notebook to a new file and then clean the output from that. The clean version should then allow you to open it and access your code easily if indeed the output stored was the problem. (The copying is important because you don’t want to clobber your original .ipynb file containing the intact output as it probably was an effort to come by.)
To see nbclean in action stripping output from a test notebook:
Go here and click launch binder under ‘Example’.
In the session that comes up run the following in a notebook cell:
import nbclean as nbc
path_original_notebook = './test_notebooks/test_notebook.ipynb'
path_save = './test_notebooks/'
ntbk = nbc.NotebookCleaner(path_original_notebook)
ntbk.clear('output')
# Now we'll save the notebook to inspect
ntbk.save(path_save + 'test_notebook_cleaned.ipynb')
Then using Jupyter compare the file test_notebook.ipynb to test_notebook_cleaned.ipynb.
Then adapt the code to the copy of your notebook. (If you want to do it on your machine, you’ll need to run %pip install nbclean in a cell in your notebook first. You may have to do it on your machine or a more substantial remote system than MyBinder provides if your file is too large to upload to sessions running via MyBinder.) The clean version hopefully opens just fine for you.
nbclean is based on nbformat (find intro at top of here), which would also allow you to do this, I have some examples of various applications of nbformat with code among answers here and here.
Recipe for using jupyter nbconvert to clear outputs
There’s also clean_ipynb and nb-clean.
UPDATE: The OP found nb-clean nice because could install using conda or pip and use single line to act on the notebook:
nb-clean clean notebook.ipynb
(Because that command runs in place, make sure you use a copy if you care about what was among the output.)
Alternatively, you can convert the notebook to a script using Jupytext on the command line. That would allow you to access the code in it as well.
Under Windows 10 I have a Jupyter notebook that I am not able anymore to open because the browser reaches the "out of memory" state. The system resources tool confirms this: reaching 85% of used RAM the process is stopped. The last time I used the notebook there were a lot of rendered charts and some heavy dataframe populated. The feeling is that they are still there and once the notebook is opened again Jupyter tries to load also the full data and all the images. I need at least an easy way to not lose all the code in between. Is there a way to open the notebook in a "safe mode" so that images and dataframe are not loaded?
You can copy your notebook to a new file and then clean the output from that. The clean version should then allow you to open it and access your code easily if indeed the output stored was the problem. (The copying is important because you don’t want to clobber your original .ipynb file containing the intact output as it probably was an effort to come by.)
To see nbclean in action stripping output from a test notebook:
Go here and click launch binder under ‘Example’.
In the session that comes up run the following in a notebook cell:
import nbclean as nbc
path_original_notebook = './test_notebooks/test_notebook.ipynb'
path_save = './test_notebooks/'
ntbk = nbc.NotebookCleaner(path_original_notebook)
ntbk.clear('output')
# Now we'll save the notebook to inspect
ntbk.save(path_save + 'test_notebook_cleaned.ipynb')
Then using Jupyter compare the file test_notebook.ipynb to test_notebook_cleaned.ipynb.
Then adapt the code to the copy of your notebook. (If you want to do it on your machine, you’ll need to run %pip install nbclean in a cell in your notebook first. You may have to do it on your machine or a more substantial remote system than MyBinder provides if your file is too large to upload to sessions running via MyBinder.) The clean version hopefully opens just fine for you.
nbclean is based on nbformat (find intro at top of here), which would also allow you to do this, I have some examples of various applications of nbformat with code among answers here and here.
Recipe for using jupyter nbconvert to clear outputs
There’s also clean_ipynb and nb-clean.
UPDATE: The OP found nb-clean nice because could install using conda or pip and use single line to act on the notebook:
nb-clean clean notebook.ipynb
(Because that command runs in place, make sure you use a copy if you care about what was among the output.)
Alternatively, you can convert the notebook to a script using Jupytext on the command line. That would allow you to access the code in it as well.