AI with Python – Data Preparation

AI with Python – Data Preparation

Machine learning algorithms, whether supervised or unsupervised, require data to be in a specific, structured format to learn effectively. Raw data collected from the real world is often messy, inconsistent, and unsuitable for direct input into these algorithms. Data preparation, also known as data preprocessing, is the crucial process of cleaning, transforming, and organizing raw data into a high-quality dataset ready for machine learning.

This chapter focuses on essential data preparation techniques for machine learning algorithms using Python.

The Importance of Data Preprocessing

Real-world data often suffers from issues like missing values, noise, inconsistencies, and irrelevant information. Data preprocessing addresses these issues to improve the quality of the data, which in turn enhances the performance, efficiency, and reliability of machine learning models. “Garbage in, garbage out” is a common adage in machine learning, highlighting the necessity of good data.

General Data Preprocessing Steps in Python

Here’s a typical workflow for preprocessing data in Python:

Step 1: Import Necessary Packages

The primary libraries for data manipulation and preprocessing in Python are NumPy and Scikit-learn.

import numpy as np
from sklearn import preprocessing # For most preprocessing utilities
from sklearn.impute import SimpleImputer # For handling missing values

• NumPy: A fundamental package for numerical computation, providing support for multi-dimensional arrays and matrices.
• Scikit-learn (sklearn): A comprehensive machine learning library that offers a wide range of tools for data preprocessing, model building, and evaluation. The sklearn.preprocessing module contains many common utility functions and transformer classes.

Step 2: Load and Inspect Your Data

Load your dataset (e.g., from a CSV file using Pandas, or a NumPy array). Then, inspect its structure, data types, and identify potential issues like missing values or outliers.

For demonstration, let’s define some sample data:

# Sample data with some potential issues (e.g., varying scales)
raw_feature_data = np.array([
    [2.1, -1.9, 5.5],
    [-1.5, 2.4, 3.5],
    [0.5, -7.9, 5.6],
    [5.9, 2.3, -5.8]
])

# Sample data with missing values (NaN)
data_with_missing = np.array([
    [1.0, 2.0, np.nan],
    [4.0, np.nan, 6.0],
    [7.0, 8.0, 9.0],
    [np.nan, 11.0, 12.0]
])

# Sample categorical labels
categorical_labels = ['red', 'black', 'red', 'green', 'black', 'yellow', 'white']

Step 3: Apply Preprocessing Techniques

Based on the data inspection, apply appropriate preprocessing techniques. These are detailed in the following sections.

Common Data Preprocessing Techniques

1. Handling Missing Data

Missing data is a common problem. Strategies include:

• Deletion: Removing rows (listwise deletion) or columns with missing values. This is simple but can lead to loss of valuable data if many values are missing.
• Imputation: Filling missing values with substitutes. Common methods include using the mean, median (for numerical data), or mode (for categorical data) of the respective column.

Scikit-learn’s SimpleImputer can be used for this:

# Impute missing values using the mean strategy
mean_imputer = SimpleImputer(missing_values=np.nan, strategy='mean')
imputed_data_mean = mean_imputer.fit_transform(data_with_missing)
print("Data imputed with mean:\n", imputed_data_mean)

# Impute missing values using the median strategy
median_imputer = SimpleImputer(missing_values=np.nan, strategy='median')
imputed_data_median = median_imputer.fit_transform(data_with_missing)
print("\nData imputed with median:\n", imputed_data_median)

Output for mean imputation (actual values depend on non-missing data):

Data imputed with mean:
[[ 1.          2.          9.        ]
 [ 4.          7.          6.        ]
 [ 7.          8.          9.        ]
 [ 4.         11.         12.        ]]
(Column 0 NaN replaced by (1+4+7)/3 = 4. Column 1 NaN by (2+8+11)/3 = 7. Column 2 NaN by (6+9+12)/3 = 9.)

2. Feature Scaling

Many machine learning algorithms perform better when input features are on a similar scale. This prevents features with larger values from dominating the learning process.

a) Standardization (Z-score Normalization)

Transforms data to have a mean of 0 and a standard deviation of 1. It’s useful when algorithms assume data is centered around zero, like SVMs or logistic regression with regularization.

print("\nOriginal data (raw_feature_data):\n", raw_feature_data)
print("Original Mean = ", raw_feature_data.mean(axis=0))
print("Original Std deviation = ", raw_feature_data.std(axis=0))

standardized_data = preprocessing.scale(raw_feature_data)
print("\nStandardized data:\n", standardized_data)
print("Standardized Mean = ", standardized_data.mean(axis=0))
print("Standardized Std deviation = ", standardized_data.std(axis=0))

Output for raw_feature_data:

Original data (raw_feature_data):
[[ 2.1 -1.9  5.5]
 [-1.5  2.4  3.5]
 [ 0.5 -7.9  5.6]
 [ 5.9  2.3 -5.8]]
Original Mean =  [ 1.75  -1.275  2.2  ]
Original Std deviation =  [2.71431391 4.20022321 4.69414529]

Standardized data:
[[ 0.12895966  0.60115576  0.70299017]
 [-1.19731725  0.87494731  0.27693006]
 [-0.46055594 -1.57727448  0.72427818]
 [ 1.52891352  0.09911408 -1.70419841]]
Standardized Mean =  [1.11022302e-16 0.00000000e+00 0.00000000e+00]
Standardized Std deviation =  [1. 1. 1.]

b) Min-Max Scaling (Normalization)

Scales data to a fixed range, usually 0 to 1. It’s useful when algorithms require data in a bounded interval, or when the distribution is not Gaussian.

minmax_scaler = preprocessing.MinMaxScaler(feature_range=(0, 1))
data_scaled_minmax = minmax_scaler.fit_transform(raw_feature_data)
print("\nMin-Max scaled data (to [0,1]):\n", data_scaled_minmax)

Output for raw_feature_data (corrected from original tutorial’s example):

Min-Max scaled data (to [0,1]):
[[0.48648649 0.58252427 0.99122807]
 [0.         1.         0.81578947]
 [0.27027027 0.         1.        ]
 [1.         0.99029126 0.        ]]

3. Encoding Categorical Data

Machine learning algorithms typically require numerical input. Categorical features (text labels) must be converted into numbers.

a) Label Encoding

Assigns a unique integer to each category. Suitable for ordinal categorical features (where order matters) or for target variables.

# Using 'categorical_labels' defined earlier: ['red', 'black', ...]
label_encoder = preprocessing.LabelEncoder()
label_encoder.fit(categorical_labels)
print("\nOriginal Labels:", categorical_labels)

encoded_values = label_encoder.transform(categorical_labels)
print("Encoded Values:", encoded_values)

# We can also inverse transform to get original labels
decoded_values = label_encoder.inverse_transform(encoded_values)
print("Decoded Labels:", list(decoded_values))

Output:

Original Labels: ['red', 'black', 'red', 'green', 'black', 'yellow', 'white']
Encoded Values: [2 0 2 1 0 4 3]
Decoded Labels: ['red', 'black', 'red', 'green', 'black', 'yellow', 'white']
(Note: The integer mapping depends on the alphabetical order of unique labels found during fit: black=0, green=1, red=2, white=3, yellow=4)

b) One-Hot Encoding

Creates new binary (0 or 1) columns for each category. Prevents the model from assuming an ordinal relationship between categories (which Label Encoding might imply). Preferred for nominal categorical features.

# Reshape categorical_labels for OneHotEncoder (it expects 2D array)
reshaped_labels = np.array(categorical_labels).reshape(-1, 1)

onehot_encoder = preprocessing.OneHotEncoder(sparse_output=False) # sparse_output=False for dense array
onehot_encoded_values = onehot_encoder.fit_transform(reshaped_labels)
print("\nOne-Hot Encoded Values (first 3 rows shown):")
print(onehot_encoded_values[:3])
print("Feature names from OneHotEncoder:", onehot_encoder.get_feature_names_out(['color']))

Output (first 3 rows correspond to ‘red’, ‘black’, ‘red’):

One-Hot Encoded Values (first 3 rows shown):
[[0. 0. 1. 0. 0.]
 [1. 0. 0. 0. 0.]
 [0. 0. 1. 0. 0.]]
Feature names from OneHotEncoder: ['color_black' 'color_green' 'color_red' 'color_white' 'color_yellow']
(Each row has a 1 in the column corresponding to its color.)

Pandas get_dummies() function is another convenient way to perform one-hot encoding on DataFrame columns.

4. Binarization

Converts numerical features into binary (0 or 1) based on a threshold. Can be useful for creating presence/absence features.

# Using 'raw_feature_data'
binarizer = preprocessing.Binarizer(threshold=0.5)
data_binarized = binarizer.transform(raw_feature_data)
print("\nBinarized data (threshold=0.5):\n", data_binarized)

Output: Values > 0.5 become 1, others become 0.

Binarized data (threshold=0.5):
[[1. 0. 1.]
 [0. 1. 1.]
 [0. 0. 1.]
 [1. 1. 0.]]

5. Normalization (Vector Norm)

This type of normalization scales individual samples (rows) to have unit norm (length). It’s often used in text classification or clustering when the magnitude of samples doesn’t matter, only their direction/orientation.

L1 Normalization (Least Absolute Deviations): Modifies values so that the sum of absolute values in each row is 1.

data_normalized_l1 = preprocessing.normalize(raw_feature_data, norm='l1')
print("\nL1 Normalized data:\n", data_normalized_l1)

Output (each row sums to 1, considering absolute values):

L1 Normalized data:
[[ 0.22105263 -0.2         0.57894737]
 [-0.2027027   0.32432432  0.47297297]
 [ 0.03571429 -0.56428571  0.4       ]
 [ 0.42142857  0.16428571 -0.41428571]]

L2 Normalization (Least Squares): Modifies values so that the sum of squares in each row is 1 (each row becomes a unit vector).

data_normalized_l2 = preprocessing.normalize(raw_feature_data, norm='l2')
print("\nL2 Normalized data:\n", data_normalized_l2)

Output (sum of squares of each row is 1):

L2 Normalized data:
[[ 0.33946114 -0.30713151  0.88906489]
 [-0.33325106  0.53320169  0.7775858 ]
 [ 0.05156558 -0.81473612  0.57753446]
 [ 0.68706914  0.26784051 -0.6754239 ]]

Labeled vs. Unlabeled Data

Unlabeled data consists of samples without any corresponding target or class information. Examples include raw text, images, audio recordings. It’s often abundant and easy to collect.

Labeled data, on the other hand, has each sample augmented with a meaningful tag, label, or class. For example, an image labeled as ‘cat’ or ‘dog’, an email labeled as ‘spam’ or ‘not spam’. Labeling data usually requires human expertise and can be time-consuming and expensive.

Supervised learning algorithms require labeled data for training. Unsupervised learning algorithms work with unlabeled data to find patterns or structures. Semi-supervised learning is an approach that combines a small amount of labeled data with a large amount of unlabeled data to build better models.

Effective data preparation is a cornerstone of successful machine learning projects. The techniques chosen depend heavily on the nature of the data and the requirements of the algorithm being used.