PyTorch Tensor Operations

arisdelaCruz1413618857's avatar
arisdelaCruz1413618857's avatar
arisdelaCruz1413618857
Published Sep 4, 2024
Contribute to Docs

In PyTorch, tensor operations are fundamentals for performing various tensor computations. Tensors are multi-dimensional arrays that can be manipulated using a wide range of operations.

  • Free course

    Intro to PyTorch and Neural Networks

    Learn how to use PyTorch to build, train, and test artificial neural networks in this course.
    • Intermediate.
      3 hours
  • Course

    PyTorch for Classification

    Build AI classification models with PyTorch using binary and multi-label techniques.
    • With Certificate
    • Beginner Friendly.
      3 hours

Fundamental Tensor Operations

Here are the fundamental operations that can be performed on tensors:

  • .expand(): Expands the tensor along specified dimensions, creating a larger tensor with repeated values.
  • .permute(): Reorders the dimensions of the tensor according to a specified order.
  • .tolist(): Converts the tensor to a Python list or nested list.
  • .narrow(): Returns a tensor that is a narrowed view of the original tensor based on specified dimensions.
  • .where(): Returns a new tensor by applying a condition to the original tensor.

Arithmetic Operations

PyTorch provides a set of arithmetic operations that can be performed on tensors. These operations include:

  • +: Addition
  • -: Subtraction
  • *: Multiplication
  • /: Division

Element-wise Operations

Element-wise operations are operations that are applied to each element of a tensor individually. Some of these operations are as follows:

  • torch.pow(): Computes the power of each element in the tensor, raising them to the specified exponent.
  • torch.sqrt(): Calculates the square root of each element in the tensor.
  • torch.abs(): Returns the absolute value of each element in the tensor.

Reduction Operations

Reduction operations compute a single result from multiple tensor elements. These operations include:

  • .sum(): Calculates the sum of all elements.
  • .mean(): Computes the mean of all elements.
  • .max(): Finds the maximum value among all elements.
  • .min(): Finds the minimum value among all elements.

Advanced Operations

Advanced tensor operations include the following:

  • Matrix Multiplication: Performed using the torch.mm() method or the @ operator.
  • Transposition: Flips the dimensions of a tensor. For 2D tensors, it exchanges rows and columns. Achieved using torch.t().
  • Reshaping: Changes the shape of a tensor while preserving its data. This can be done using torch.reshape() or .view().
  • Concatenation: Joins two or more tensors along a specified dimension. This can be performed using torch.cat().

Tensor Operations

.abs()
Computes the absolute value of each element in a PyTorch tensor, returning the magnitude for complex numbers.
.acos()
Computes the inverse cosine (arccos) of each element in the input tensor.
.acosh()
Computes the inverse hyperbolic cosine of elements in a PyTorch tensor.
.add()
Performs element-wise addition of two tensors, with optional scaling. Supports broadcasting for tensors of different shapes.
.addcdiv()
Performs element-wise division of tensors followed by scalar multiplication and addition to another tensor.
.addcmul()
Performs element-wise multiplication of two tensors and adds a scaled result to a third tensor.
.adjoint()
Computes the adjoint of a 2D complex-valued tensor in PyTorch.
.angle()
Computes the element-wise angle of complex tensors in PyTorch.
.asin()
Computes the inverse sine (arcsine) of each element in a PyTorch tensor.
.asinh()
Returns the inverse hyperbolic sine of each element in an input tensor.
.as_strided()
Creates a view of a tensor with a specified shape and strides.
.as_tensor()
Converts input data into a PyTorch tensor, sharing memory when possible.
.atan()
Computes the inverse tangent of each element in the input tensor.
.atan2()
Computes the arctangent of the quotient of two tensors, element-wise.
.atanh()
Calculates the inverse hyperbolic tangent for each value in a tensor.
.bernoulli()
Generates a tensor with binary values (0s and 1s) based on the probabilities provided in an input tensor.
.bitwise_and()
Performs the element-wise bitwise AND operation on tensors in PyTorch.
.bitwise_left_shift()
Shifts each element of the input tensor to the left by a specified number of bits.
.bitwise_not()
Performs element-wise bitwise NOT operation on the input tensor, flipping each bit (0 to 1 and 1 to 0). Applicable to integer and boolean tensors.
.bitwise_or()
Computes the element-wise bitwise OR between corresponding bits of two integer tensors.
.bitwise_xor()
Computes the bitwise XOR value of the inputs.
.ceil()
Returns a new tensor with the smallest integer greater than or equal to each element of the input tensor.
.clamp()
Limits each element of a tensor to a specified range.
.conj()
Computes the complex conjugate of each element in a given tensor.
.conj_physical()
Computes the complex conjugate of each element in a tensor.
.copysign()
Returns a tensor with the magnitude of the first input and the sign of the second input, element-wise.
.cos()
Computes the cosine of each element in the input tensor.
.cosh()
Returns the hyperbolic cosine of the given input.
.deg2rad()
Converts angles from degrees to radians.
.diagonal_scatter()
Inserts all values from the source tensor into the input tensor at specified diagonal indices.
.digamma()
Computes the digamma function, which is the logarithmic derivative of the gamma function.
.div()
Performs element-wise division of input tensors or a tensor by a scalar.
.erfc()
Computes the complementary error function of input.
.erfinv()
Computes the inverse error function of input.
.exp()
Returns a new tensor with the exponential of each element in the input tensor.
.exp2()
Returns a new tensor with the base-2 exponential of each element in the input tensor.
.expand()
Creates a new view of a tensor by expanding its singleton dimensions to a larger size without allocating new memory.
.expm1()
Calculates the exponential of each element in a tensor and then subtracts 1.
.float_power()
Raises each element of a tensor to the power of the given exponent with the result always returned in floating point.
.floor()
Returns a new tensor with each element rounded down to the nearest integer.
.floor_divide()
Computes element-wise division of tensors (or tensor and scalar) and applies floor rounding.
.fmod()
Returns the element-wise remainder of division, keeping the same sign as the dividend.
.frexp()
Decomposes input into mantissa and exponent tensors.
.from_numpy()
Converts a NumPy array into a PyTorch tensor, sharing memory between both and changes to one affect the other.
.gather()
Retrieves specific elements from a tensor along a defined axis based on indices.
.hsplit()
Splits a tensor horizontally into multiple sub-tensors.
.hstack()
Concatenates two or more tensors along the horizontal axis (column-wise).
.hypot()
Calculates the hypotenuse of a right triangle given the lengths of its two legs.
.i0()
Computes the modified Bessel function of the first kind of order zero.
.igamma()
Computes the lower incomplete gamma function for tensor inputs.
.igammac()
Computes the regularized upper incomplete gamma function.
.imag
Returns the imaginary part of each element in a complex tensor.
.index_copy_()
Copies values in-place into specified indices of a given tensor along the specified dimension.
.index_reduce_()
Reduces a tensor along a specified dimension using indices to map input elements to positions in the output tensor, applying reduction operations such as sum, product, or mean.
.index_select()
Extracts specific elements from a tensor along a specified dimension based on indices and returns a new tensor.
.is_nonzero()
Checks if a scalar tensor contains a non-zero element, returning a boolean indicating whether the element is non-zero.
.is_tensor()
Checks if the given object is a PyTorch tensor.
.ldexp()
Returns a new tensor obtained by multiplying each element of the input tensor by a power of two.
.lerp()
Returns a tensor containing the linear interpolation of two tensors, controlled by a scalar or tensor weight.
.lgamma()
Computes the natural log of the absolute gamma function for each element in a tensor.
.log()
Returns a new tensor with the natural logarithm of each element in the input tensor.
.log10()
Returns a tensor containing the base-10 logarithm of each element in the input tensor.
.log1p()
Computes the natural logarithm of one plus the input tensor element-wise.
.log2()
Computes the base-2 logarithm of each element in the input tensor and returns a new tensor with the results.
.logaddexp()
Computes the element-wise logarithm of the sum of exponentials of two input tensors.
.logaddexp2()
Computes the logarithm of the sum of exponentials of two tensors in base 2.
.logical_not()
Performs element-wise logical NOT on boolean tensors, returning a tensor where each element is the boolean negation of the input.
.logical_or()
Computes the element-wise logical OR between two tensors.
.logit()
Returns the logit of each element in the input tensor.
.masked_select()
Selects elements from a tensor, based on a boolean mask, and returns them as a 1D tensor.
.mm()
Calculates the matrix product of two given tensors.
.movedim()
Returns a tensor with the dimensions moved from the positions specified in source to the positions specified in destination.
.mul()
Performs element-wise multiplication of two tensors or multiplies a tensor by a scalar.
.multinomial()
Samples elements from a given probability distribution.
.mvlgamma()
Computes the multivariate log-gamma function for tensor inputs.
.nan_to_num()
Replaces NaN, positive infinity, and negative infinity values in a tensor with specified finite values.
.narrow()
Returns a narrow subsection of a tensor along a specified dimension.
.narrow_copy()
Creates a new tensor containing a narrowed subsection of data from the input tensor along a specified dimension.
.neg()
Returns a new tensor with the negative of each element in the input tensor.
.nextafter()
Returns the next floating-point value after each element of input in the direction of the corresponding element of other.
.nonzero()
Returns a tensor containing the indices of non-zero elements in the input tensor.
.normal()
Generates a tensor of random numbers from a normal (Gaussian) distribution.
.numel()
Returns the total number of elements in a tensor.
.permute()
Returns a view of the given tensor with its dimensions permuted or rearranged according to a specific order.
.poisson()
Generates a tensor where each element is drawn from a Poisson distribution with the corresponding rate given in the input tensor.
.polygamma()
Computes the n-th derivative of the digamma function for each element in the input tensor.
.positive()
Returns a copy of the input tensor with the same values. Implements the unary plus operation.
.pow()
Raises each element of the input tensor to the specified power and returns a new tensor with the computed values.
.rad2deg()
Converts the elements of a tensor from radians to degrees.
.randint()
Returns a tensor filled with random integers generated uniformly between specified bounds.
.randperm()
Generates a random permutation of integers from 0 to n-1.
.real()
Returns the real part of each element in a complex tensor in PyTorch.
.reciprocal()
Computes the element-wise reciprocal of each element in the input tensor.
.reshape_as()
Returns a tensor with the same shape as the input tensor.
.row_stack()
Stacks or arranges a sequence of tensors vertically (row-wise).
.scatter()
Writes values from a source into specific locations of a tensor along a specified dimension, based on indices.
.scatter_add_()
Accumulates values into a tensor along specified dimensions using indices.
.scatter_reduce()
Reduces values from the source tensor using the given reduction method and scatters the result into the input tensor at the specified indices.
.select()
Selects a specific slice along the given dimension in a tensor.
.select_scatter()
Inserts all values from the source tensor into the input tensor at the given indices.
.sgn()
Computes the sign of each element in the input tensor, returning a tensor with the same shape.
.slice_scatter()
Inserts all values from the source tensor into the input tensor at the given dimension and returns a new tensor.
.split()
Splits a tensor into chunks of specified sizes along a given dimension.
.squeeze()
Removes dimensions of size 1 from a tensor.
.stack()
Stacks the given tensors along a new specified dimension.
.swapaxes()
Swaps two specified axes (dimensions) of a tensor, effectively rearranging its shape.
.swapdims()
Swaps two specified dimensions of a tensor, effectively transposing those dimensions.
.t()
Returns the transpose of a given 2D tensor.
.take()
Returns a 1D tensor containing elements from input at the specified indices.
.take_along_dim()
Select elements from a tensor along a specified dimension using indices.
.tensor_split()
Splits a tensor into multiple sub-tensors along a specified dimension, based on either specified indices or the number of equal parts.
.tile()
Creates a new tensor by repeating the elements of an existing tensor along specified dimensions.
.transpose()
Reorders dimensions of a tensor by swapping two specified dimensions.
.unbind()
Removes a dimension and returns a tuple of slices along the given dimension.
.unravel_index()
Converts flat indices into coordinate tuples based on the shape of a tensor, enabling multi-dimensional indexing.
.unsqueeze()
Adds a singleton dimension (dimension of size 1) to a tensor at the specified position, altering its shape without changing the data.
.vsplit()
Divides a tensor into multiple sub-tensors vertically.
.vstack()
Stacks a sequence of tensors vertically (row-wise).
.where()
Returns values from one tensor if a condition is true; otherwise, it takes values from another tensor.
erf()
Computes the error function element-wise for each element in the input tensor
frac()
Returns the fractional part of each element in a PyTorch tensor.
gradient()
Numerically estimates the gradient of a multi-dimensional function represented by a PyTorch tensor.
index_add()
Adds values to specific indices of a tensor along a specified dimension.
Moving Tensors Across Devices(CPU/GPU)
Ways to transfer tensors between CPU and GPU to optimize performance by executing operations on the most suitable hardware while maintaining data integrity.
Specifying Data Types
Determines how tensors are stored and processed, impacting precision, memory usage, and computation speed.

All contributors

Contribute to Docs

Learn PyTorch on Codecademy

  • Free course

    Intro to PyTorch and Neural Networks

    Learn how to use PyTorch to build, train, and test artificial neural networks in this course.
    • Intermediate.
      3 hours
  • Course

    PyTorch for Classification

    Build AI classification models with PyTorch using binary and multi-label techniques.
    • With Certificate
    • Beginner Friendly.
      3 hours