Is .data still useful in pytorch?
Question:
I’m new to pytorch. I read much pytorch code which heavily uses tensor’s .data
member. But I search .data
in the official document and Google, finding little. I guess .data
contains the data in the tensor, but I don’t know when we need it and when not?
Answers:
.data
was an attribute of Variable
(object representing Tensor
with history tracking e.g. for automatic update), not Tensor
. Actually, .data
was giving access to the Variable
‘s underlying Tensor
.
However, since PyTorch version 0.4.0
, Variable
and Tensor
have been merged (into an updated Tensor
structure), so .data
disappeared along the previous Variable
object (well Variable
is still there for backward-compatibility, but is deprecated).
Paragraph from Release Notes for version 0.4.0
(I recommend reading the whole section about Variable
/Tensor
updates):
What about .data
?
.data
was the primary way to get the underlying Tensor
from a
Variable
. After this merge, calling y = x.data
still has similar
semantics. So y
will be a Tensor
that shares the same data with
x
, is unrelated with the computation history of x
, and has
requires_grad=False
.
However, .data
can be unsafe in some cases. Any changes on x.data
wouldn’t be tracked by autograd
, and the computed gradients would be
incorrect if x
is needed in a backward pass. A safer alternative is
to use x.detach()
, which also returns a Tensor
that shares data
with requires_grad=False
, but will have its in-place changes
reported by autograd
if x
is needed in backward.
Aside from @benjaminplanche ‘s answer, I’d use it to manually change values of parameters.
For instance, I have the following model:
model = nn.Sequential(nn.Linear(10, 1))
and for some reason, I’d like to manually update the values of its parameters. Then, I can do:
for param in model.parameters():
param.data = 10 * param.data # multiply the parameter values by 10.
Note that if we remove .data
behind param
, the parameter values won’t be updated.
Such use can be found in BYOL (Bootstrap your own latent) and this Github webpage for BYOL pytorch implementation.
From PyTorch 0.4.0 and at least version 1.7.1 – you can take torch.Tensor content via this attributed ".data".
Assume you have point=torch.Tensor(size=(1,2,)); point.requires_grad_(True);
In that case:
point.data
will be a Tensor that shares the same data with point
.
You can check it with: point.data_ptr(); point.data.data_ptr();
-
it’s is unrelated to the computation history of point.data
, has requires_grad=False
even point
has requires_grad=True
-
Any changes on initial_point.data wouldn’t be tracked by autograd
Documentation:
I’m new to pytorch. I read much pytorch code which heavily uses tensor’s .data
member. But I search .data
in the official document and Google, finding little. I guess .data
contains the data in the tensor, but I don’t know when we need it and when not?
.data
was an attribute of Variable
(object representing Tensor
with history tracking e.g. for automatic update), not Tensor
. Actually, .data
was giving access to the Variable
‘s underlying Tensor
.
However, since PyTorch version 0.4.0
, Variable
and Tensor
have been merged (into an updated Tensor
structure), so .data
disappeared along the previous Variable
object (well Variable
is still there for backward-compatibility, but is deprecated).
Paragraph from Release Notes for version 0.4.0
(I recommend reading the whole section about Variable
/Tensor
updates):
What about
.data
?
.data
was the primary way to get the underlyingTensor
from a
Variable
. After this merge, callingy = x.data
still has similar
semantics. Soy
will be aTensor
that shares the same data with
x
, is unrelated with the computation history ofx
, and has
requires_grad=False
.However,
.data
can be unsafe in some cases. Any changes onx.data
wouldn’t be tracked byautograd
, and the computed gradients would be
incorrect ifx
is needed in a backward pass. A safer alternative is
to usex.detach()
, which also returns aTensor
that shares data
withrequires_grad=False
, but will have its in-place changes
reported byautograd
ifx
is needed in backward.
Aside from @benjaminplanche ‘s answer, I’d use it to manually change values of parameters.
For instance, I have the following model:
model = nn.Sequential(nn.Linear(10, 1))
and for some reason, I’d like to manually update the values of its parameters. Then, I can do:
for param in model.parameters():
param.data = 10 * param.data # multiply the parameter values by 10.
Note that if we remove .data
behind param
, the parameter values won’t be updated.
Such use can be found in BYOL (Bootstrap your own latent) and this Github webpage for BYOL pytorch implementation.
From PyTorch 0.4.0 and at least version 1.7.1 – you can take torch.Tensor content via this attributed ".data".
Assume you have point=torch.Tensor(size=(1,2,)); point.requires_grad_(True);
In that case:
point.data
will be a Tensor that shares the same data withpoint
.
You can check it with: point.data_ptr(); point.data.data_ptr();
-
it’s is unrelated to the computation history of
point.data
, hasrequires_grad=False
evenpoint
hasrequires_grad=True
-
Any changes on initial_point.data wouldn’t be tracked by autograd
Documentation: