📂Machine Learning

How to Pad PyTorch Tensors

Code ¹

torch.nn.functional.pad(input, pad, mode='constant', value=0.0)

• input: The tensor to pad

• pad: Where to pad

• mode: Method of padding

• value: Value for padding

Description

• pad

  When using a tensor of dimension $n$ as an input, you can pass up to $2n-$pairs of arguments.

  $$ ((n-1)_{\text{low}}, (n-1)_{\text{up}}, (n-2)_{\text{low}}, (n-2)_{\text{up}}, \dots, 1_{\text{low}}, 1_{\text{up}}, 0_{\text{low}}, 0_{\text{up}}) $$

  $i_{\text{low}}$ indicates how many values to pad before the $i$th dimension’s lower index, i.e., before the $0$th value.

  $i_{\text{up}}$ indicates how many values to pad after the $i$th dimension’s higher index, i.e., after the last value.

  >>> A = torch.ones(2,3)
  >>> A
  tensor([[1., 1., 1.],
          [1., 1., 1.]])
  
  >>> PA = F.pad(A, (1,2,3,4), "constant", 0)
  tensor([[0., 0., 0., 0., 0., 0.],
          [0., 0., 0., 0., 0., 0.],
          [0., 0., 0., 0., 0., 0.],
          [0., 1., 1., 1., 0., 0.],
          [0., 1., 1., 1., 0., 0.],
          [0., 0., 0., 0., 0., 0.],
          [0., 0., 0., 0., 0., 0.],
          [0., 0., 0., 0., 0., 0.],
          [0., 0., 0., 0., 0., 0.]])
  
  >>> PA.shape
  torch.Size([9, 6])
  

• mode

  constant maps all specified areas to a fixed constant.

  >>> F.pad(A, (2,3), "constant", 3)
  tensor([[3., 3., 1., 1., 1., 3., 3., 3.],
          [3., 3., 1., 1., 1., 3., 3., 3.]])
  

  reflect pads with symmetrical copies based on the zeroth and last values of each dimension. For some reason, it only accepts tensors with three or more dimensions and the padding size is fixed to $2(n-2)-$ pairs. Thus, the zeroth and first dimensions are not padded.

  >>> B = torch.tensor([[[1,2,3,4,5.]],[[6,7,8,9,10]]]
  tensor([[[ 1.,  2.,  3.,  4.,  5.]],
          [[ 6.,  7.,  8.,  9., 10.]]])
  >>> F.pad(B, (2,3), 'reflect')
  tensor([[[ 3.,  2.,  1.,  2.,  3.,  4.,  5.,  4.,  3.,  2.]],
          [[ 8.,  7.,  6.,  7.,  8.,  9., 10.,  9.,  8.,  7.]]])
  

  

  replicate directly copies the zeroth and last values for padding, and circular pads such that the values of each dimension are circular.