pytorch

How to replace this naive code with scaled_dot_product_attention() in Pytorch?

How to replace this naive code with scaled_dot_product_attention() in Pytorch? Question: Consider a code fragment from Crossformer: def forward(self, queries, keys, values): B, L, H, E = queries.shape _, S, _, D = values.shape scale = self.scale or 1./sqrt(E) scores = torch.einsum("blhe,bshe->bhls", queries, keys) A = self.dropout(torch.softmax(scale * scores, dim=-1)) V = torch.einsum("bhls,bshd->blhd", A, values) …

Total answers: 1

ImportError: Using the Trainer with PyTorch requires accelerate = 0.20.1

ImportError: Using the Trainer with PyTorch requires accelerate = 0.20.1 Question: Please help me when I tried to use it in my Google Colab for transformers error: ImportError: Using the Trainer with PyTorch requires accelerate=0.20.1: Please run pip install transformers[torch] or pip install accelerate -U` NOTE: If your import is failing due to a missing …

Total answers: 2

How to use pretrained encoder for customized Unet

How to use pretrained encoder for customized Unet Question: if you have a standard Unet encoder such as resnet50, then it’s easy to add pertaining to it. for example: ENCODER = ‘resnet50’ ENCODER_WEIGHTS = ‘imagenet’ CLASSES = class_names ACTIVATION = ‘sigmoid’ # could be None for logits or ‘softmax2d’ for multiclass segmentation # create segmentation …

Total answers: 1

How to remove some labels of a pytorch dataset

How to remove some labels of a pytorch dataset Question: I have a torchvision.datasets object. I only want to keep some labels and delete the others. For example, if my dataset is CFAR10 like this trainset = torchvision.datasets.CIFAR10(root=’./data’, train=True, download=True) I will have 10 labels. I only want to keep the first three labels and …

Total answers: 1

Pytorch – mat1 and mat2 shapes cannot be multiplied (3328×13 and 9216×4096)

Pytorch – mat1 and mat2 shapes cannot be multiplied (3328×13 and 9216×4096) Question: I’m a beginner to pytorch and I’m trying to extract activations from 7 layers for the pre-trained alexnet model. Getting this error when trying to extract activations from the fc6 layer: image of the error Here is my code for looping over …

Total answers: 1

How to convert a pytorch Dataset object to a pandas dataframe?

How to convert a pytorch Dataset object to a pandas dataframe? Question: I have a pandas dataframe called df that I split into train/test sets using the random_split function from torch library. import pandas as pd from torch.utils.data import Dataset, random_split df = pd.read_csv(‘some_txt_file.txt’, sep= ‘ ‘) len_test = len(df) // 10 len_train = len(df) …

Total answers: 1

train pytorch CNN to specific image style

train pytorch CNN to specific image style Question: I’m trying to train pytorch CNN model to transform my images into specific style. I found tutorial for this at https://pytorch.org/tutorials/advanced/neural_style_tutorial.html, but this will transform the image in multiple iterations and I need to have a model that do the same thing to use it in C++ …

Total answers: 1

self() as function within class, what does it do?

self() as function within class, what does it do? Question: Sorry for the poor title but I’m unsure how better to describe the question. So I recently watched Andrej Kaparthy’s build GPT video which is awesome and now trying to reconstruct the code myself I notices that he uses self() as a function and was …

Total answers: 1

How to improve pytroch model?

How to improve pytroch model? Question: Good evening, I have 4 classes with black and white images each class has 3000 images with a test of 600 images so how to improve this model and that’s my full code: data_transform = transforms.Compose([ transforms.Grayscale(num_output_channels=1), transforms.Resize(size=(150, 150)), transforms.ToTensor(), transforms.Normalize(mean=[0.5], std=[0.5]), ]) Loading Images using ImageFolder train_data = …

Total answers: 2