RuntimeError: Input type (torch.FloatTensor) and weight type (torch.cuda.FloatTensor) should be the same
Question:
This:
device = torch.device("cuda" if torch.cuda.is_available() else "cpu")
model.to(device)
for data in dataloader:
inputs, labels = data
outputs = model(inputs)
Gives the error:
RuntimeError: Input type (torch.FloatTensor) and weight type (torch.cuda.FloatTensor) should be the same
Answers:
You get this error because your model is on the GPU, but your data is on the CPU. So, you need to send your input tensors to the GPU.
inputs, labels = data # this is what you had
inputs, labels = inputs.cuda(), labels.cuda() # add this line
Or like this, to stay consistent with the rest of your code:
device = torch.device("cuda:0" if torch.cuda.is_available() else "cpu")
inputs, labels = inputs.to(device), labels.to(device)
The same error will be raised if your input tensors are on the GPU but your model weights aren’t. In this case, you need to send your model weights to the GPU.
model = MyModel()
if torch.cuda.is_available():
model.cuda()
First check cuda is available or not:
if torch.cuda.is_available():
device = 'cuda'
else:
device = 'cpu'
In case you want to load some model do this:
checkpoint = torch.load('./generator_release.pth', map_location=device)
G = Generator().to(device)
Now you probably get this error:
RuntimeError: Input type (torch.FloatTensor) and weight type (torch.cuda.FloatTensor) should be the same
It is needed to convert the type of input data from torch.tensor to torch.cuda.tensor by :
if torch.cuda.is_available():
data = data.cuda()
result = G(data)
and then convert the result from torch.cuda.tensor to torch.tensor:
if torch.cuda.is_available():
result = result.cpu()
As already mentioned in the previous answers, the issue can be that your model is trained on the GPU, but it’s tested on the CPU. If that’s the case then you need to port your model’s weights and the data from the GPU to the CPU like this:
device = args.device # "cuda" / "cpu"
if "cuda" in device and not torch.cuda.is_available():
device = "cpu"
data = data.to(device)
model.to(device)
NOTE: Here we still check if the configuration arguments are set to GPU or CPU, so that this piece of code can be used for both training (on the GPU) and testing (on the CPU).
The new API is to use .to() method.
The advantage is obvious and important.
Your device may tomorrow be something other than "cuda":
- cpu
- cuda
- mkldnn
- opengl
- opencl
- ideep
- hip
- msnpu
- xla
So try to avoid model.cuda()
It is not wrong to check for the device
dev = torch.device("cuda") if torch.cuda.is_available() else torch.device("cpu")
or to hardcode it:
dev=torch.device("cuda")
same as:
dev="cuda"
In general you can use this code:
model.to(dev)
data = data.to(dev)
* when you get this error::RuntimeError: Input type
(torch.FloatTensor) and weight type (torch.cuda.FloatTensor should
be the same
# Move tensors to GPU is CUDA is available
# Check if CUDA is available
train_on_gpu = torch.cuda.is_available()
If train_on_gpu:
print("CUDA is available! Training on GPU...")
else:
print("CUDA is not available. Training on CPU...")
-------------------
# Move tensors to GPU is CUDA is available
if train_on_gpu:
model.cuda()
I have same problem,My CNN model:
class CNN(nn.Module):
def __init__(self):
super(CNN,self).__init__()
self.device = torch.device(device)
self.dummy_param = nn.Parameter(torch.empty(0))
l1 = nn.Conv2d(3, 64, kernel_size=(3, 3), stride=(1, 1), padding= (1,1)).to(device)
l2 = nn.Conv2d(64, 128, kernel_size=(3, 3), stride=(1, 1), padding=(1,1)).to(device)
l3 = nn.Conv2d(128, 256, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1)).to(device)
l4 = nn.Conv2d(256, 512, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1)).to(device)
l5 = nn.Conv2d(512, 512, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1)).to(device)
self.layers = [l1,l2,l3,l4,l5]
self.layers = [l1,l2]
def forward(self,x):
features = []
for l in self.layers:
x = l(x)
features.append(x)
return features
I put for Conv2d.to(device) its work for me.
When loading a model, both weights and inputs have to be in the same device, we can do this by using the .to(device) as pointed by others.
However it might be the case that also the datatype of the saved weights and the input tensors are different. If this is the case then we must also change the datatype of both model weights and inputs:
model = torch.load(PATH).type(torch.FloatTensor).to(device)
input = input.type(torch.FloatTensor).to(device)
x = x.to(device, dtype=torch.float32)
y = y.to(device, dtype=torch.float32)
Works, perfectly fine…
Notice that (from pytorch documentation):
If the self Tensor already has the correct torch.dtype and torch.device, then self is returned. Otherwise, the returned tensor is a copy of self with the desired torch.dtype and torch.device.
That is, you might need to do:
model = model.to("cuda")
data = data.to("cuda")
Instead of just:
model.to("cuda")
data.to("cuda")
With the first approach you’ll be in the safe side.
This:
device = torch.device("cuda" if torch.cuda.is_available() else "cpu")
model.to(device)
for data in dataloader:
inputs, labels = data
outputs = model(inputs)
Gives the error:
RuntimeError: Input type (torch.FloatTensor) and weight type (torch.cuda.FloatTensor) should be the same
You get this error because your model is on the GPU, but your data is on the CPU. So, you need to send your input tensors to the GPU.
inputs, labels = data # this is what you had
inputs, labels = inputs.cuda(), labels.cuda() # add this line
Or like this, to stay consistent with the rest of your code:
device = torch.device("cuda:0" if torch.cuda.is_available() else "cpu")
inputs, labels = inputs.to(device), labels.to(device)
The same error will be raised if your input tensors are on the GPU but your model weights aren’t. In this case, you need to send your model weights to the GPU.
model = MyModel()
if torch.cuda.is_available():
model.cuda()
First check cuda is available or not:
if torch.cuda.is_available():
device = 'cuda'
else:
device = 'cpu'
In case you want to load some model do this:
checkpoint = torch.load('./generator_release.pth', map_location=device)
G = Generator().to(device)
Now you probably get this error:
RuntimeError: Input type (torch.FloatTensor) and weight type (torch.cuda.FloatTensor) should be the same
It is needed to convert the type of input data from torch.tensor to torch.cuda.tensor by :
if torch.cuda.is_available():
data = data.cuda()
result = G(data)
and then convert the result from torch.cuda.tensor to torch.tensor:
if torch.cuda.is_available():
result = result.cpu()
As already mentioned in the previous answers, the issue can be that your model is trained on the GPU, but it’s tested on the CPU. If that’s the case then you need to port your model’s weights and the data from the GPU to the CPU like this:
device = args.device # "cuda" / "cpu"
if "cuda" in device and not torch.cuda.is_available():
device = "cpu"
data = data.to(device)
model.to(device)
NOTE: Here we still check if the configuration arguments are set to GPU or CPU, so that this piece of code can be used for both training (on the GPU) and testing (on the CPU).
The new API is to use .to() method.
The advantage is obvious and important.
Your device may tomorrow be something other than "cuda":
- cpu
- cuda
- mkldnn
- opengl
- opencl
- ideep
- hip
- msnpu
- xla
So try to avoid model.cuda()
It is not wrong to check for the device
dev = torch.device("cuda") if torch.cuda.is_available() else torch.device("cpu")
or to hardcode it:
dev=torch.device("cuda")
same as:
dev="cuda"
In general you can use this code:
model.to(dev)
data = data.to(dev)
* when you get this error::RuntimeError: Input type
(torch.FloatTensor) and weight type (torch.cuda.FloatTensor should
be the same
# Move tensors to GPU is CUDA is available
# Check if CUDA is available
train_on_gpu = torch.cuda.is_available()
If train_on_gpu:
print("CUDA is available! Training on GPU...")
else:
print("CUDA is not available. Training on CPU...")
-------------------
# Move tensors to GPU is CUDA is available
if train_on_gpu:
model.cuda()
I have same problem,My CNN model:
class CNN(nn.Module):
def __init__(self):
super(CNN,self).__init__()
self.device = torch.device(device)
self.dummy_param = nn.Parameter(torch.empty(0))
l1 = nn.Conv2d(3, 64, kernel_size=(3, 3), stride=(1, 1), padding= (1,1)).to(device)
l2 = nn.Conv2d(64, 128, kernel_size=(3, 3), stride=(1, 1), padding=(1,1)).to(device)
l3 = nn.Conv2d(128, 256, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1)).to(device)
l4 = nn.Conv2d(256, 512, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1)).to(device)
l5 = nn.Conv2d(512, 512, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1)).to(device)
self.layers = [l1,l2,l3,l4,l5]
self.layers = [l1,l2]
def forward(self,x):
features = []
for l in self.layers:
x = l(x)
features.append(x)
return features
I put for Conv2d.to(device) its work for me.
When loading a model, both weights and inputs have to be in the same device, we can do this by using the .to(device) as pointed by others.
However it might be the case that also the datatype of the saved weights and the input tensors are different. If this is the case then we must also change the datatype of both model weights and inputs:
model = torch.load(PATH).type(torch.FloatTensor).to(device)
input = input.type(torch.FloatTensor).to(device)
x = x.to(device, dtype=torch.float32)
y = y.to(device, dtype=torch.float32)
Works, perfectly fine…
Notice that (from pytorch documentation):
If the self Tensor already has the correct torch.dtype and torch.device, then self is returned. Otherwise, the returned tensor is a copy of self with the desired torch.dtype and torch.device.
That is, you might need to do:
model = model.to("cuda")
data = data.to("cuda")
Instead of just:
model.to("cuda")
data.to("cuda")
With the first approach you’ll be in the safe side.