NumPy - Broadcasting

NumPy - Broadcasting

Broadcasting describes how NumPy handles arithmetic operations between arrays of different shapes. Typically, operations are performed element-wise, requiring arrays to have the exact same shape.

Example 1: Arrays with the Same Shape

Element-wise multiplication between two arrays of identical shape.

import numpy as np

a = np.array([1, 2, 3, 4])
b = np.array([10, 20, 30, 40])
c = a * b # Element-wise multiplication

print(f"Array a: {a}")
print(f"Array b: {b}")
print(f"Result c = a * b: {c}")

Output:

Array a: [1 2 3 4]
Array b: [10 20 30 40]
Result c = a * b: [ 10  40  90 160]

However, NumPy can also perform operations on arrays with compatible but different shapes through a powerful mechanism called broadcasting. If certain rules are met, the smaller array is conceptually ‘broadcast’ (stretched or tiled) to match the shape of the larger array without actually copying data, making operations memory-efficient.

Broadcasting Rules:

Two arrays are compatible for broadcasting if, for each dimension (starting from the trailing dimension):

• Their dimension sizes are equal, OR
• One of the dimension sizes is 1.

If the arrays have different numbers of dimensions, the shape of the array with fewer dimensions is conceptually padded with ones on its left side until the numbers of dimensions match.

During the operation, dimensions of size 1 are ‘stretched’ or ‘tiled’ to match the corresponding dimension size of the other array. The resulting array has a shape where each dimension size is the maximum of the input array sizes in that dimension.

Common Scenario: Operating between an array and a scalar (a single number). The scalar is broadcast to match the array’s shape.

import numpy as np
arr = np.array([1, 2, 3])
scalar = 10
result = arr + scalar # Scalar is broadcast
print(result) # Output: [11 12 13]

Example 2: Broadcasting a 1D Array to a 2D Array

Demonstrates adding a 1D array to each row of a 2D array.

import numpy as np

a = np.array([
    [0.0, 0.0, 0.0],
    [10.0, 10.0, 10.0],
    [20.0, 20.0, 20.0],
    [30.0, 30.0, 30.0]
]) # Shape (4, 3)

b = np.array([1.0, 2.0, 3.0]) # Shape (3,)

print('First array (a, shape {}):'.format(a.shape))
print(a)
print('\n')

print('Second array (b, shape {}):'.format(b.shape))
print(b)
print('\n')

# Add a and b. Broadcasting occurs here.
# b's shape (3,) is compatible with a's trailing dimension (3).
# b is conceptually stretched along the first dimension to match a's shape (4, 3).
c = a + b

print('Result of a + b (shape {}):'.format(c.shape))
print(c)

Output:

First array (a, shape (4, 3)):
[[ 0.  0.  0.]
 [10. 10. 10.]
 [20. 20. 20.]
 [30. 30. 30.]]


Second array (b, shape (3,)):
[1. 2. 3.]


Result of a + b (shape (4, 3)):
[[ 1.  2.  3.]
 [11. 12. 13.]
 [21. 22. 23.]
 [31. 32. 33.]]

Visualizing the Broadcast:

To perform a + b where a has shape (4, 3) and b has shape (3,):

1. NumPy compares shapes from right to left.
2. Trailing dimensions: a has 3, b has 3. They match.
3. Next dimension: a has 4, b has no dimension here. NumPy conceptually adds a dimension of size 1 to b, making its shape (1, 3).
4. Comparing again: (4, 3) and (1, 3).
5. Trailing dimensions (3 vs 3): Match.
6. First dimensions (4 vs 1): One is 1. Compatible.
7. Result shape is the maximum in each dimension: (max(4,1), max(3,3)) = (4, 3).
8. During the operation, the dimension of size 1 in b (now conceptually (1, 3)) is stretched along the first axis (rows) to match a’s size of 4. It effectively acts like [[1., 2., 3.], [1., 2., 3.], [1., 2., 3.], [1., 2., 3.]] for the addition, but without creating this temporary array in memory.

Broadcasting is a powerful feature for writing concise and efficient code, but be mindful of the rules to avoid unexpected shape mismatches or unintended operations.