Python kernel crash in Jupyter Notebook when calling TSNE.fit_transform()

Question

I have an output of sklearn‘s tf-idf which I want to visualize with T-SNE. However, when calling fit_transform on sklearn‘s T-SNE object, I get the error message:

"Canceled future for execute_request message before replies were done

The Kernel crashed while executing code in the the current cell or a
previous cell. Please review the code in the cell(s) to identify a
possible cause of the failure. Click here for more info. View Jupyter
log for further details."

Why is this happening? Code below.

dense = np.array(
   [[0.        , 0.        , 0.        , 0.        , 0.        ],
   [0.        , 0.        , 0.        , 0.        , 0.        ],
   [0.        , 0.        , 0.        , 0.        , 0.        ],
   [0.        , 0.        , 0.        , 0.        , 0.        ],
   [0.        , 0.        , 0.        , 1.        , 0.        ],
   [0.        , 0.        , 0.        , 1.        , 0.        ],
   [0.70710678, 0.70710678, 0.        , 0.        , 0.        ],
   [0.        , 0.        , 0.70710678, 0.        , 0.70710678],
   [0.70710678, 0.70710678, 0.        , 0.        , 0.        ],
   [0.        , 0.        , 0.        , 0.        , 0.        ],
   [0.        , 0.        , 0.70710678, 0.        , 0.70710678]])

from sklearn.manifold import TSNE
tsne = TSNE(n_components = 2, verbose = 1, perplexity = 50, n_iter = 1000)
results = tsne.fit_transform(dense)

Asked By: Jean Broc

||

Source

Answer 1

I wasn’t able to reproduce the error in a Google Colab. It works fine on my end, with the following output:

[t-SNE] Computing 10 nearest neighbors...
[t-SNE] Indexed 11 samples in 0.000s...
[t-SNE] Computed neighbors for 11 samples in 0.009s...
[t-SNE] Computed conditional probabilities for sample 11 / 11
[t-SNE] Mean sigma: 1125899906842624.000000
[t-SNE] KL divergence after 250 iterations with early exaggeration: 39.474655
[t-SNE] KL divergence after 1000 iterations: 0.268328

I’ve found an old thread on GitHub that may address the problem. It is a Mac related issue, but I don’t know what OS does your machine has.

There is a chance that they fixed the error in newer versions of sklearn, so my first suggestion is to try upgrading, if you haven’t already.

If the issue still persists, since the problem may be due to a dependency that sklearn uses (and even if you do not have a Mac, you still have a problem), I would recommend using a different library. I know about python-bhtsne that can be used in a similar way as sklearn’s.

Answered By: ClaudiaR

Python kernel crash in Jupyter Notebook when calling TSNE.fit_transform()

Question:

Answers: