Tensor Operations

arisdelaCruz1413618857's avatar
arisdelaCruz1413618857's avatar
arisdelaCruz1413618857
Published Sep 4, 2024
Contribute to Docs

In PyTorch, tensor operations are fundamentals for performing various tensor computations. Tensors are multi-dimensional arrays that can be manipulated using a wide range of operations.

Fundamental Tensor Operations

Here are the fundamental operations that can be performed on tensors:

  • .expand(): Expands the tensor along specified dimensions, creating a larger tensor with repeated values.
  • .permute(): Reorders the dimensions of the tensor according to a specified order.
  • .tolist(): Converts the tensor to a Python list or nested list.
  • .narrow(): Returns a tensor that is a narrowed view of the original tensor based on specified dimensions.
  • .where(): Returns a new tensor by applying a condition to the original tensor.

Arithmetic Operations

PyTorch provides a set of arithmetic operations that can be performed on tensors. These operations include:

  • +: Addition
  • -: Subtraction
  • *: Multiplication
  • /: Division

Element-wise Operations

Element-wise operations are operations that are applied to each element of a tensor individually. Some of these operations are as follows:

  • torch.pow(): Computes the power of each element in the tensor, raising them to the specified exponent.
  • torch.sqrt(): Calculates the square root of each element in the tensor.
  • torch.abs(): Returns the absolute value of each element in the tensor.

Reduction Operations

Reduction operations compute a single result from multiple tensor elements. These operations include:

  • .sum(): Calculates the sum of all elements.
  • .mean(): Computes the mean of all elements.
  • .max(): Finds the maximum value among all elements.
  • .min(): Finds the minimum value among all elements.

Advanced Operations

Advanced tensor operations include the following:

  • Matrix Multiplication: Performed using the torch.mm() method or the @ operator.
  • Transposition: Flips the dimensions of a tensor. For 2D tensors, it exchanges rows and columns. Achieved using torch.t().
  • Reshaping: Changes the shape of a tensor while preserving its data. This can be done using torch.reshape() or .view().
  • Concatenation: Joins two or more tensors along a specified dimension. This can be performed using torch.cat().

Tensor Operations

.adjoint()
Computes the adjoint of a 2D complex-valued tensor in PyTorch.
.conj()
Computes the complex conjugate of each element in a given tensor.
.gather()
Retrieves specific elements from a tensor along a defined axis based on indices.
.hsplit()
Splits a tensor horizontally into multiple sub-tensors.
.hstack()
Concatenates two or more tensors along the horizontal axis (column-wise).
.index_copy_()
Copies values in-place into specified indices of a given tensor along the specified dimension.
.index_reduce_()
Reduces a tensor along a specified dimension using indices to map input elements to positions in the output tensor, applying reduction operations such as sum, product, or mean.
.index_select()
Extracts specific elements from a tensor along a specified dimension based on indices and returns a new tensor.
.masked_select()
Selects elements from a tensor, based on a boolean mask, and returns them as a 1D tensor.
.mm()
Calculates the matrix product of two given tensors.
.movedim()
Returns a tensor with the dimensions moved from the positions specified in source to the positions specified in destination.
.narrow()
Returns a narrow subsection of a tensor along a specified dimension.
.narrow_copy()
Creates a new tensor containing a narrowed subsection of data from the input tensor along a specified dimension.
.nonzero()
Returns a tensor containing the indices of non-zero elements in the input tensor.
.permute()
Returns a view of the given tensor with its dimensions permuted or rearranged according to a specific order.
.row_stack()
Stacks or arranges a sequence of tensors vertically (row-wise).
.scatter()
Writes values from a source into specific locations of a tensor along a specified dimension, based on indices.
.select()
Selects a specific slice along the given dimension in a tensor.
.split()
Splits a tensor into chunks of specified sizes along a given dimension.
.squeeze()
Removes dimensions of size 1 from a tensor.
.stack()
Stacks the given tensors along a new specified dimension.
.swapaxes()
Swaps two specified axes (dimensions) of a tensor, effectively rearranging its shape.
.t()
Returns the transpose of a given 2D tensor.
.take()
Returns a 1D tensor containing elements from input at the specified indices.
index_add()
Adds values to specific indices of a tensor along a specified dimension.
Specifying Data Types
Determines how tensors are stored and processed, impacting precision, memory usage, and computation speed.

All contributors

Contribute to Docs

Learn PyTorch on Codecademy