TensorFlow - Basics

TensorFlow - Understanding Tensors

At the heart of TensorFlow is the “tensor.” Tensors are multi-dimensional arrays with a uniform data type. They are the fundamental data structures used in TensorFlow. You can think of tensors as generalizations of vectors and matrices to potentially higher dimensions.

Tensor Attributes

Tensors are characterized by three key attributes:

Rank (or Order)

The rank of a tensor describes its number of dimensions. It’s sometimes referred to as order or degree.

• A scalar (a single number) is a rank-0 tensor.
• A vector (a 1D array) is a rank-1 tensor.
• A matrix (a 2D array) is a rank-2 tensor.
• And so on for rank-3 (e.g., a cube of numbers), rank-4, etc.

Shape

The shape of a tensor is a tuple of integers describing the size of each dimension. For example:

• A scalar has an empty shape ().
• A vector of length 5 has shape (5,).
• A matrix with 3 rows and 4 columns has shape (3, 4).
• A rank-3 tensor might have shape (2, 3, 4).

Data Type (dtype)

The data type (dtype) defines the type of data stored in the tensor (e.g., tf.float32, tf.int32, tf.string). All elements within a single tensor must have the same data type.

Creating tensors often involves starting with Python lists or NumPy arrays and then converting them into tf.Tensor objects using tf.constant() or tf.Variable().

Creating Tensors of Various Ranks

Let’s look at how to create tensors of different ranks using TensorFlow 2.x, often starting from NumPy arrays for familiarity.

Rank-0 Tensor (Scalar)

import tensorflow as tf
import numpy as np

# Creating a scalar (rank-0 tensor)
scalar_np = np.array(10)
scalar_tf = tf.constant(scalar_np)

print("NumPy scalar:", scalar_np)
print("TensorFlow scalar (tensor):")
print(scalar_tf)
print(f"Rank: {tf.rank(scalar_tf).numpy()}, Shape: {scalar_tf.shape}, Dtype: {scalar_tf.dtype}")

Output will show the scalar value and its attributes (rank 0, empty shape).

Rank-1 Tensor (Vector)

# Creating a vector (rank-1 tensor)
vector_np = np.array([1.3, 1.0, 4.0, 23.99])
vector_tf = tf.constant(vector_np, dtype=tf.float32)

print("\nNumPy vector:", vector_np)
print("TensorFlow vector (tensor):")
print(vector_tf)
print(f"Rank: {tf.rank(vector_tf).numpy()}, Shape: {vector_tf.shape}, Dtype: {vector_tf.dtype}")

# Accessing elements (similar to Python lists/NumPy arrays)
print("Element at index 0:", vector_tf[0].numpy())
print("Element at index 2:", vector_tf[2].numpy())

Output will show the vector, its attributes (rank 1, shape e.g., (4,)), and examples of element access.

Rank-2 Tensor (Matrix)

# Creating a matrix (rank-2 tensor)
matrix_np = np.array([(1, 2, 3, 4), (5, 6, 7, 8), (9, 10, 11, 12)])
matrix_tf = tf.constant(matrix_np, dtype=tf.int32)

print("\nNumPy matrix:\n", matrix_np)
print("TensorFlow matrix (tensor):")
print(matrix_tf)
print(f"Rank: {tf.rank(matrix_tf).numpy()}, Shape: {matrix_tf.shape}, Dtype: {matrix_tf.dtype}")

# Accessing elements (e.g., element in 2nd row, 3rd column - 0-indexed)
print("Element at [1, 2]:", matrix_tf[1, 2].numpy()) # Accesses value 7

Output will display the matrix and its attributes (e.g., rank 2, shape (3, 4)).

Basic Tensor Operations

TensorFlow provides a rich set of operations for manipulating tensors. In TensorFlow 2.x, these operations execute eagerly, meaning results are computed immediately (unlike TF1.x’s graph execution model with sessions).

# Define two matrices as TensorFlow constants
matrix1_tf = tf.constant([[2, 2, 2], [2, 2, 2], [2, 2, 2]], dtype=tf.int32)
matrix2_tf = tf.constant([[1, 1, 1], [1, 1, 1], [1, 1, 1]], dtype=tf.int32)

print("\nMatrix 1:")
print(matrix1_tf)
print("Matrix 2:")
print(matrix2_tf)

# Element-wise addition
matrix_sum = tf.add(matrix1_tf, matrix2_tf)
print("\nMatrix Sum (matrix1 + matrix2):")
print(matrix_sum)

# Matrix multiplication
matrix_product = tf.matmul(matrix1_tf, matrix2_tf)
print("\nMatrix Product (matrix1 * matrix2):")
print(matrix_product)

# Determinant of a matrix (must be float or complex type)
matrix_float_tf = tf.constant([[2.0, 7.0, 2.0],
                               [1.0, 4.0, 2.0],
                               [9.0, 0.0, 2.0]], dtype=tf.float32)
print("\nFloat Matrix for Determinant:")
print(matrix_float_tf)

matrix_determinant = tf.linalg.det(matrix_float_tf)
print("\nDeterminant of Float Matrix:")
print(matrix_determinant.numpy()) # .numpy() to get the scalar value

Output:

The code will print the defined matrices, their sum, their product, and the determinant of the float matrix. In TensorFlow 2.x, these operations are performed directly without needing a tf.Session().run(...) call, which was characteristic of TensorFlow 1.x.

Explanation of Changes

Compared to TensorFlow 1.x:

• Eager Execution: Operations compute immediately. No tf.Session is needed to evaluate tensors.
• .numpy(): To get the Python/NumPy value from a TensorFlow tensor, you use the .numpy() method.
• API Updates: Some functions might have new names or reside in different modules (e.g., tf.matrix_determinant became tf.linalg.det).
• Python 3 Syntax: Modern Python syntax like print() functions is used.

Understanding tensors and basic operations is foundational for working with any deep learning framework, including TensorFlow. For more advanced tensor manipulations and a comprehensive list of operations, refer to the official TensorFlow documentation on Tensors: https://www.tensorflow.org/guide/tensor