VScode Jupyter Notebook crash in cell

Question

I get this error when I run sklearn to train on a very large dataset. If the dataset is small, it works, but if it is above a threshold, the kernel crashes.

Error:

info 16:24:11.630: Process Execution: > ~/miniconda3/envs/auto-sklearn/bin/python -m pip list
> ~/miniconda3/envs/auto-sklearn/bin/python -m pip list
info 16:24:11.712: Process Execution: > ~/miniconda3/envs/auto-sklearn/bin/python -c "import ipykernel; print(ipykernel.__version__); print("5dc3a68c-e34e-4080-9c3e-2a532b2ccb4d"); print(ipykernel.__file__)"
> ~/miniconda3/envs/auto-sklearn/bin/python -c "import ipykernel; print(ipykernel.__version__); print("5dc3a68c-e34e-4080-9c3e-2a532b2ccb4d"); print(ipykernel.__file__)"
info 16:24:11.910: Process Execution: > ~/miniconda3/envs/auto-sklearn/bin/python -m ipykernel_launcher --ip=127.0.0.1 --stdin=9011 --control=9009 --hb=9008 --Session.signature_scheme="hmac-sha256" --Session.key=b"39e77f25-eae0-4712-8a1a-628305c2ff03" --shell=9010 --transport="tcp" --iopub=9012 --f=/home/baraa/.local/share/jupyter/runtime/kernel-v2-261323AOvTtMclSsgz.json
> ~/miniconda3/envs/auto-sklearn/bin/python -m ipykernel_launcher --ip=127.0.0.1 --stdin=9011 --control=9009 --hb=9008 --Session.signature_scheme="hmac-sha256" --Session.key=b"39e77f25-eae0-4712-8a1a-628305c2ff03" --shell=9010 --transport="tcp" --iopub=9012 --f=/home/baraa/.local/share/jupyter/runtime/kernel-v2-261323AOvTtMclSsgz.json
info 16:24:11.910: Process Execution: cwd: ~/Documents/Python/Testing/Search
cwd: ~/Documents/Python/Testing/Search
info 16:24:12.237: ipykernel version & path 6.21.2, ~/miniconda3/envs/auto-sklearn/lib/python3.8/site-packages/ipykernel/__init__.py for /home/baraa/miniconda3/envs/auto-sklearn/bin/python
info 16:24:13.131: Started Kernel auto-sklearn (Python 3.8.16) (pid: 263555)
info 16:24:13.182: Process Execution: > ~/miniconda3/envs/auto-sklearn/bin/python ~/.vscode/extensions/ms-toolsai.jupyter-2023.1.2010391206/pythonFiles/printJupyterDataDir.py
> ~/miniconda3/envs/auto-sklearn/bin/python ~/.vscode/extensions/ms-toolsai.jupyter-2023.1.2010391206/pythonFiles/printJupyterDataDir.py
error 16:25:00.281: Disposing session as kernel process died ExitCode: undefined, Reason: /home/baraa/miniconda3/envs/auto-sklearn/lib/python3.8/site-packages/traitlets/traitlets.py:2548: FutureWarning: Supporting extra quotes around strings is deprecated in traitlets 5.0. You can use 'hmac-sha256' instead of '"hmac-sha256"' if you require traitlets >=5.
  warn(
/home/baraa/miniconda3/envs/auto-sklearn/lib/python3.8/site-packages/traitlets/traitlets.py:2499: FutureWarning: Supporting extra quotes around Bytes is deprecated in traitlets 5.0. Use '39e77f25-eae0-4712-8a1a-628305c2ff03' instead of 'b"39e77f25-eae0-4712-8a1a-628305c2ff03"'.
  warn(

info 16:25:00.284: Dispose Kernel process 263555.
error 16:25:00.284: Raw kernel process exited code: undefined
error 16:25:00.301: Error in waiting for cell to complete [Error: Canceled future for execute_request message before replies were done
    at t.KernelShellFutureHandler.dispose (/home/baraa/.vscode/extensions/ms-toolsai.jupyter-2023.1.2010391206/out/extension.node.js:2:33213)
    at /home/baraa/.vscode/extensions/ms-toolsai.jupyter-2023.1.2010391206/out/extension.node.js:2:52265
    at Map.forEach (<anonymous>)
    at y._clearKernelState (/home/baraa/.vscode/extensions/ms-toolsai.jupyter-2023.1.2010391206/out/extension.node.js:2:52250)
    at y.dispose (/home/baraa/.vscode/extensions/ms-toolsai.jupyter-2023.1.2010391206/out/extension.node.js:2:45732)
    at /home/baraa/.vscode/extensions/ms-toolsai.jupyter-2023.1.2010391206/out/extension.node.js:17:139244
    at Z (/home/baraa/.vscode/extensions/ms-toolsai.jupyter-2023.1.2010391206/out/extension.node.js:2:1608939)
    at Kp.dispose (/home/baraa/.vscode/extensions/ms-toolsai.jupyter-2023.1.2010391206/out/extension.node.js:17:139221)
    at qp.dispose (/home/baraa/.vscode/extensions/ms-toolsai.jupyter-2023.1.2010391206/out/extension.node.js:17:146518)
    at process.processTicksAndRejections (node:internal/process/task_queues:96:5)]
warn 16:25:00.301: Cell completed with errors {
  message: 'Canceled future for execute_request message before replies were done'
}
info 16:25:00.302: Cancel all remaining cells true || Error || undefined

I tried reinstall ipykernal, downgrading traitlets and Pyzmq==19.0.2, update python version, reinstalling miniconda, choosing different environment… still to no avail

Asked By: Baraa Zaid

||

Source

Answer 1

Canceled future for execute_request message before replies were done

This problem is usually caused by insufficient memory. You can try to add the following code to your file:

import os
os.environ["KMP_DUPLICATE_LIB_OK"]="TRUE"

Answered By: MingJie-MSFT

Answer 2

The problem was with my Swap. I was using Linux, and for Linux I needed to manually specify the Swap needed so as when the kernel requires more memory, it reverts to using Swap memory. That way, it doesn’t crash.
Instructions on doing so can be found here:
https://www.cloudsigma.com/adding-swap-space-on-ubuntu-20-04-a-tutorial/

Answered By: Baraa Zaid

VScode Jupyter Notebook crash in cell

Question:

Answers: