Tensor Operations

arisdelaCruz1413618857

Published Sep 4, 2024

Contribute to Docs

In PyTorch, tensor operations are fundamentals for performing various tensor computations. Tensors are multi-dimensional arrays that can be manipulated using a wide range of operations.

Fundamental Tensor Operations

Here are the fundamental operations that can be performed on tensors:

.expand(): Expands the tensor along specified dimensions, creating a larger tensor with repeated values.
.permute(): Reorders the dimensions of the tensor according to a specified order.
.tolist(): Converts the tensor to a Python list or nested list.
.narrow(): Returns a tensor that is a narrowed view of the original tensor based on specified dimensions.
.where(): Returns a new tensor by applying a condition to the original tensor.

Arithmetic Operations

PyTorch provides a set of arithmetic operations that can be performed on tensors. These operations include:

+: Addition
-: Subtraction
*: Multiplication
/: Division

Element-wise Operations

Element-wise operations are operations that are applied to each element of a tensor individually. Some of these operations are as follows:

torch.pow(): Computes the power of each element in the tensor, raising them to the specified exponent.
torch.sqrt(): Calculates the square root of each element in the tensor.
torch.abs(): Returns the absolute value of each element in the tensor.

Reduction Operations

Reduction operations compute a single result from multiple tensor elements. These operations include:

.sum(): Calculates the sum of all elements.
.mean(): Computes the mean of all elements.
.max(): Finds the maximum value among all elements.
.min(): Finds the minimum value among all elements.

Advanced Operations

Advanced tensor operations include the following:

Matrix Multiplication: Performed using the torch.mm() method or the @ operator.
Transposition: Flips the dimensions of a tensor. For 2D tensors, it exchanges rows and columns. Achieved using torch.t().
Reshaping: Changes the shape of a tensor while preserving its data. This can be done using torch.reshape() or .view().
Concatenation: Joins two or more tensors along a specified dimension. This can be performed using torch.cat().

Tensor Operations

.abs(): Computes the absolute value of each element in a PyTorch tensor, returning the magnitude for complex numbers.
.acos(): Computes the inverse cosine (arccos) of each element in the input tensor.
.acosh(): Computes the inverse hyperbolic cosine of elements in a PyTorch tensor.
.add(): Performs element-wise addition of two tensors, with optional scaling. Supports broadcasting for tensors of different shapes.
.addcdiv(): Performs element-wise division of tensors followed by scalar multiplication and addition to another tensor.
.addcmul(): Performs element-wise multiplication of two tensors and adds a scaled result to a third tensor.
.adjoint(): Computes the adjoint of a 2D complex-valued tensor in PyTorch.
.angle(): Computes the element-wise angle of complex tensors in PyTorch.
.asin(): Computes the inverse sine (arcsine) of each element in a PyTorch tensor.
.asinh(): Returns the inverse hyperbolic sine of each element in an input tensor.
.as_strided(): Creates a view of a tensor with a specified shape and strides.
.as_tensor(): Converts input data into a PyTorch tensor, sharing memory when possible.
.atan(): Computes the inverse tangent of each element in the input tensor.
.atan2(): Computes the arctangent of the quotient of two tensors, element-wise.
.atanh(): Calculates the inverse hyperbolic tangent for each value in a tensor.
.bernoulli(): Generates a tensor with binary values (0s and 1s) based on the probabilities provided in an input tensor.
.bitwise_and(): Performs the element-wise bitwise AND operation on tensors in PyTorch.
.bitwise_not(): Performs element-wise bitwise NOT operation on the input tensor, flipping each bit (0 to 1 and 1 to 0). Applicable to integer and boolean tensors.
.conj(): Computes the complex conjugate of each element in a given tensor.
.diagonal_scatter(): Inserts all values from the source tensor into the input tensor at specified diagonal indices.
.from_numpy(): Converts a NumPy array into a PyTorch tensor, sharing memory between both and changes to one affect the other.
.gather(): Retrieves specific elements from a tensor along a defined axis based on indices.
.hsplit(): Splits a tensor horizontally into multiple sub-tensors.
.hstack(): Concatenates two or more tensors along the horizontal axis (column-wise).
.index_copy_(): Copies values in-place into specified indices of a given tensor along the specified dimension.
.index_reduce_(): Reduces a tensor along a specified dimension using indices to map input elements to positions in the output tensor, applying reduction operations such as sum, product, or mean.
.index_select(): Extracts specific elements from a tensor along a specified dimension based on indices and returns a new tensor.
.is_nonzero(): Checks if a scalar tensor contains a non-zero element, returning a boolean indicating whether the element is non-zero.
.is_tensor(): Checks if the given object is a PyTorch tensor.
.masked_select(): Selects elements from a tensor, based on a boolean mask, and returns them as a 1D tensor.
.mm(): Calculates the matrix product of two given tensors.
.movedim(): Returns a tensor with the dimensions moved from the positions specified in source to the positions specified in destination.
.multinomial(): Samples elements from a given probability distribution.
.narrow(): Returns a narrow subsection of a tensor along a specified dimension.
.narrow_copy(): Creates a new tensor containing a narrowed subsection of data from the input tensor along a specified dimension.
.nonzero(): Returns a tensor containing the indices of non-zero elements in the input tensor.
.normal(): Generates a tensor of random numbers from a normal (Gaussian) distribution.
.numel(): Returns the total number of elements in a tensor.
.permute(): Returns a view of the given tensor with its dimensions permuted or rearranged according to a specific order.
.poisson(): Generates a tensor where each element is drawn from a Poisson distribution with the corresponding rate given in the input tensor.
.randint(): Returns a tensor filled with random integers generated uniformly between specified bounds.
.randperm(): Generates a random permutation of integers from 0 to n-1.
.row_stack(): Stacks or arranges a sequence of tensors vertically (row-wise).
.scatter(): Writes values from a source into specific locations of a tensor along a specified dimension, based on indices.
.scatter_add_(): Accumulates values into a tensor along specified dimensions using indices.
.scatter_reduce(): Reduces values from the source tensor using the given reduction method and scatters the result into the input tensor at the specified indices.
.select(): Selects a specific slice along the given dimension in a tensor.
.select_scatter(): Inserts all values from the source tensor into the input tensor at the given indices.
.slice_scatter(): Inserts all values from the source tensor into the input tensor at the given dimension and returns a new tensor.
.split(): Splits a tensor into chunks of specified sizes along a given dimension.
.squeeze(): Removes dimensions of size 1 from a tensor.
.stack(): Stacks the given tensors along a new specified dimension.
.swapaxes(): Swaps two specified axes (dimensions) of a tensor, effectively rearranging its shape.
.swapdims(): Swaps two specified dimensions of a tensor, effectively transposing those dimensions.
.t(): Returns the transpose of a given 2D tensor.
.take(): Returns a 1D tensor containing elements from input at the specified indices.
.take_along_dim(): Select elements from a tensor along a specified dimension using indices.
.tensor_split(): Splits a tensor into multiple sub-tensors along a specified dimension, based on either specified indices or the number of equal parts.
.tile(): Creates a new tensor by repeating the elements of an existing tensor along specified dimensions.
.transpose(): Reorders dimensions of a tensor by swapping two specified dimensions.
.unbind(): Removes a dimension and returns a tuple of slices along the given dimension.
.unravel_index(): Converts flat indices into coordinate tuples based on the shape of a tensor, enabling multi-dimensional indexing.
.unsqueeze(): Adds a singleton dimension (dimension of size 1) to a tensor at the specified position, altering its shape without changing the data.
.vsplit(): Divides a tensor into multiple sub-tensors vertically.
.vstack(): Stacks a sequence of tensors vertically (row-wise).
.where(): Returns values from one tensor if a condition is true; otherwise, it takes values from another tensor.
index_add(): Adds values to specific indices of a tensor along a specified dimension.
Moving Tensors Across Devices(CPU/GPU): Ways to transfer tensors between CPU and GPU to optimize performance by executing operations on the most suitable hardware while maintaining data integrity.
Specifying Data Types: Determines how tensors are stored and processed, impacting precision, memory usage, and computation speed.

All contributors

arisdelaCruz1413618857

Contribute to Docs

Learn more about how to get involved.
Edit this page on GitHub to fix an error or make an improvement.
Submit feedback to let us know how we can improve Docs.

Learn PyTorch on Codecademy