How to Deep Copy Tensors in PyTorch
Description
PyTorch tensors, like other objects, can be deep-copied using copy.deepcopy().
>>> import torch
>>> import copy
>>> a = torch.ones(2,2)
>>> b = a
>>> c = copy.deepcopy(a)
>>> a += 1
>>> a
tensor([[2., 2.],
[2., 2.]])
>>> b
tensor([[2., 2.],
[2., 2.]])
>>> c
tensor([[1., 1.],
[1., 1.]])
However, this is only possible for tensors that are explicitly defined by the user. For instance, attempting to deep-copy the output of a deep learning model using copy.deepcopy() results in the following error.
RuntimeError: Only Tensors created explicitly by the user (graph leaves) support the deepcopy protocol at the moment
PyTorch supports deep copying of tensors via .clone()1, allowing deep copies regardless of whether the tensor is user-defined or obtained as an output.
>>> a = torch.ones(2,2)
>>> b = a.clone()
>>> a += 1
>>> a
tensor([[2., 2.],
[2., 2.]])
>>> b
tensor([[1., 1.],
[1., 1.]])