Same script computes different results on Matlab and Python Question: I am trying to implement softmax function but weirdly I am getting two different outputs on MATLAB and on Python: MATLAB script: function sm = softmax(Y) e_y = exp(Y – max(Y)) sm = e_y / sum(e_y) which computes ten times 0.1 as a column vector …
PyTorch equivalent to tf.nn.softmax_cross_entropy_with_logits and tf.nn.sigmoid_cross_entropy_with_logits Question: I found the post here. Here, we try to find an equivalence of tf.nn.softmax_cross_entropy_with_logits in PyTorch. The answer is still confusing to me. Here is the Tensorflow 2 code import tensorflow as tf import numpy as np # here we assume 2 batch size with 5 classes preds …
Should I use softmax as output when using cross entropy loss in pytorch? Question: I have a problem with classifying fully connected deep neural net with 2 hidden layers for MNIST dataset in pytorch. I want to use tanh as activations in both hidden layers, but in the end, I should use softmax. For the …
softmax_loss function: Turn the loop into matrix operation Question: I am now learning the stanford cs231n course. When completing the softmax_loss function, I found it is not easy to write in a full-vectorized type, especially dealing with the dw term. Below is my code. Can somebody optimize the code. Would be appreciated. def softmax_loss_vectorized(W, X, …
In TensorFlow, why a m*n matrix can add n * 1 matrix? Question: I am very new to python and TensorFlow, recent days I met a problem when I study “MNIST For ML Beginners”(https://www.tensorflow.org/get_started/mnist/beginners). In this tutorial, we use y = tf.nn.softmax(tf.matmul(X, W) + b) to get our outputs. My question is, for example, X …
Compute a Jacobian matrix from scratch in Python Question: I’m trying to implement the derivative matrix of softmax function (Jacobian matrix of Softmax). I know mathematically the derivative of Softmax(Xi) with respect to Xj is: where the red delta is a Kronecker delta. So far what I have implemented is: def softmax_grad(s): # input s …
Numerically stable softmax Question: Is there a numerically stable way to compute softmax function below? I am getting values that becomes Nans in Neural network code. np.exp(x)/np.sum(np.exp(y)) Asked By: Abhishek Bhatia || Source Answers: There is nothing wrong with calculating the softmax function as it is in your case. The problem seems to come from …
CS231n: How to calculate gradient for Softmax loss function? Question: I am watching some videos for Stanford CS231: Convolutional Neural Networks for Visual Recognition but do not quite understand how to calculate analytical gradient for softmax loss function using numpy. From this stackexchange answer, softmax gradient is calculated as: Python implementation for above is: num_classes …
How to implement the Softmax function in Python Question: From the Udacity’s deep learning class, the softmax of y_i is simply the exponential divided by the sum of exponential of the whole Y vector: Where S(y_i) is the softmax function of y_i and e is the exponential and j is the no. of columns in …
