finding the maximum of a function using jax Question: I have a function which I would like to find its maximum by optimizing two of its variables using Jax. The current code that I have currently, which does not work, reads import jax.numpy as jnp import jax import scipy import numpy as np def temp_func(x,y,z): …
How does TensorFlow perform algorithms so fast? Question: I wanted to solve a linear regression problem by creating by myself the Adam optimization algorithm and performing it on my dataset. However, the problem is solved with an acceptable loss in around 100 epochs, while a tenth than my loss is computed by TensorFlow in just …
parameter b diverges when calculating gradient descent for linear regression Question: I have two arrays x and y: x = np.arange(-500, 500, 3) y = 2.5*x+5 I want to perform a linear regression by calculating gradient descent I implemented a gradient descent function, def gradient_descent(x, y, alpha, epochs): w = 0.0 b = 0.0 n …
Simple Neural Network Using Pytorch Question: I want to build Simple Neural Network with pytorch. And I want to teach this network. the network has y = w(weight) * x + b(bias) with w = 3 and b = 0 so I have the data x = [1,2,3,4,5,6] y = [3,6,9,12,15,18] But I have some …
Pytorch mat1 and mat2 must have the same dtype mlp Question: So i am trying to do a function that trains an mlp using PyTorch. My code is as follows : def mlp_gradient_descent(x,y , model , eta = 1e-6 , nb_iter = 30000) : loss_descent = [] dtype = torch.float device = torch.device("cpu") x = …
tf.gradients to tf.GradientTape Question: I have the following code in one part of my program: inverse = tf.gradients(x_conv, x, x_conv)[0] reconstruction_loss = tf.nn.l2_loss(inverse – tf.stop_gradient(x)) where x_conv is a Tensor (float32) of shape (384, 24, 1051) and x is a Tensor (float32) with shape (4, 3, 32, 4201). I am trying to change from using …
Getting infinity values from gradient descent Question: I’m trying to implement a multivariable linear regression with gradient descent but when I try this: # Starting values w = np.ones(3) # The number of features is 3 b = float(0) def gradient_descent(): global w global b learning_rate = 0.0001 for i in range(x_train.shape[0]): prediction = np.dot(x_train[i], …
Gradient Descent with objective loss backpropagation does not minimize loss Question: I have this code below for my implementation of a gradient descent: class Neuron(): def __init__(self): #Generate random weight self.weights = [np.random.rand() for _ in range(4)] self.learning_rate = 0.1 @staticmethod def activation_func(z): return ((math.e ** z) – (math.e ** (-z)))/((math.e ** z) + (math.e …
How to do gradient clipping in pytorch? Question: What is the correct way to perform gradient clipping in pytorch? I have an exploding gradients problem. Asked By: Gulzar || Source Answers: Reading through the forum discussion gave this: clipping_value = 1 # arbitrary value of your choosing torch.nn.utils.clip_grad_norm(model.parameters(), clipping_value) I’m sure there is more depth …
pytorch how to set .requires_grad False Question: I want to set some of my model frozen. Following the official docs: with torch.no_grad(): linear = nn.Linear(1, 1) linear.eval() print(linear.weight.requires_grad) But it prints True instead of False. If I want to set the model in eval mode, what should I do? Asked By: Qian Wang || Source …
