Preparing Data

Machine Learning with Python - Preparing Data

Introduction

Data is the fuel for Machine Learning algorithms. However, raw data collected from the real world is often messy, inconsistent, and incomplete. Feeding such data directly into ML models typically leads to poor performance or errors. Therefore, Data Preparation (also known as data preprocessing or data wrangling) is a critical and often time-consuming step in the ML workflow.

Data preparation involves transforming raw data into a clean, well-structured format suitable for ML algorithms. This ensures the data has the correct scale, format, and meaningful features for the specific problem.

Why is Data Preprocessing Necessary?

Preprocessing is essential because:

• Inconsistent Scales: Features often have different units and ranges (e.g., age in years, income in dollars). Many algorithms (like SVMs, k-NN, gradient descent-based methods) perform poorly or converge slowly if features are on vastly different scales.
• Algorithm Requirements: Some algorithms have specific data format expectations (e.g., numerical inputs only, normalized data).
• Missing Values: Real-world data frequently contains missing entries, which most algorithms cannot handle directly.
• Categorical Data: Algorithms typically require numerical input, so categorical features (like ‘color’ or ‘country’) need encoding.
• Irrelevant or Redundant Features: Some features might not contribute useful information or might be highly correlated with others, potentially degrading model performance.
• Outliers: Extreme values can disproportionately influence some models.
• Data Quality: Addresses errors, inconsistencies, and noise in the data.

Common Data Preprocessing Techniques

Let’s explore key preprocessing techniques using Python’s Scikit-learn library.

1. Handling Missing Data

A crucial first step. Common strategies include:

• Deletion: Removing rows (samples) or columns (features) with missing values. Simple, but can lead to significant data loss.
• Imputation: Filling missing values with estimated replacements. Common methods:
• • Mean/Median/Mode Imputation: Replace missing numerical values with the feature’s mean or median, and categorical values with the mode. Use sklearn.impute.SimpleImputer.
• • More Advanced Methods: Regression imputation, k-NN imputation, or using models to predict missing values can be more accurate but are more complex.

(Code examples for imputation are omitted here for brevity but are essential in practice. Refer to Scikit-learn documentation for SimpleImputer.)

2. Feature Scaling and Standardization

Adjusting the scale of features is vital for many algorithms.

a) Scaling (Normalization to [0, 1])

Rescales features to a fixed range, typically [0, 1] or [-1, 1]. Useful when the data distribution is not necessarily Gaussian or when algorithms expect inputs within a bounded range.

Use sklearn.preprocessing.MinMaxScaler.

Example (Scaling)

import pandas as pd
import numpy as np
from sklearn.preprocessing import MinMaxScaler

# Load data (replace path)
path = 'pima-indians-diabetes.csv'
names = ['preg', 'plas', 'pres', 'skin', 'test', 'mass', 'pedi', 'age', 'class']
dataframe = pd.read_csv(path, names=names)

# Separate features (X) and target (y) - good practice
array = dataframe.values
X = array[:, :-1] # Features
y = array[:, -1]  # Target

# Initialize the scaler
scaler = MinMaxScaler(feature_range=(0, 1))

# Fit the scaler to the features and transform the data
# IMPORTANT: Fit ONLY on training data in a real scenario, then transform train and test sets
X_scaled = scaler.fit_transform(X)

# Summarize scaled data
np.set_printoptions(precision=2) # Set print precision
print("Scaled data (first 5 rows):\n", X_scaled[0:5])

b) Standardization (Z-score Normalization)

Transforms features to have zero mean (μ=0) and unit variance (σ=1). It centers the data around zero. This is often preferred for algorithms that assume a Gaussian distribution or are sensitive to feature variance (like PCA, Logistic Regression, SVMs).

Use sklearn.preprocessing.StandardScaler.

Example (Standardization)

import pandas as pd
import numpy as np
from sklearn.preprocessing import StandardScaler

# Load data (replace path)
path = 'pima-indians-diabetes.csv'
names = ['preg', 'plas', 'pres', 'skin', 'test', 'mass', 'pedi', 'age', 'class']
dataframe = pd.read_csv(path, names=names)

# Separate features (X) and target (y)
array = dataframe.values
X = array[:, :-1]
y = array[:, -1]

# Initialize the scaler
scaler = StandardScaler()

# Fit the scaler to the features and transform the data
# IMPORTANT: Fit ONLY on training data in a real scenario
X_standardized = scaler.fit_transform(X)

# Summarize standardized data
np.set_printoptions(precision=2)
print("Standardized data (first 5 rows):\n", X_standardized[0:5])

Note: Other scalers like RobustScaler exist, which are less sensitive to outliers.

3. Normalization (Sample-wise)

Normalization rescales each sample (row) independently to have unit norm (length 1). This is different from feature scaling which operates column-wise. It’s often used in text classification or clustering when the magnitude of samples doesn’t matter as much as their direction or relative proportions.

Types: L1 (sum of absolute values is 1) and L2 (sum of squares is 1, Euclidean norm).

Use sklearn.preprocessing.Normalizer.

Example (L2 Normalization)

import pandas as pd
import numpy as np
from sklearn.preprocessing import Normalizer

# Load data (replace path)
path = 'pima-indians-diabetes.csv'
names = ['preg', 'plas', 'pres', 'skin', 'test', 'mass', 'pedi', 'age', 'class']
dataframe = pd.read_csv(path, names=names)

# Separate features (X)
array = dataframe.values
X = array[:, :-1]

# Initialize the normalizer (L2 norm by default)
normalizer = Normalizer(norm='l2') # Or norm='l1'

# Fit (not strictly necessary for Normalizer) and transform
X_normalized = normalizer.fit_transform(X)

# Summarize normalized data
np.set_printoptions(precision=2)
print("L2 Normalized data (first 5 rows):\n", X_normalized[0:5])

4. Binarization

Converts numerical features into binary (0 or 1) values based on a specified threshold. Useful for creating binary flags or when dealing with probabilities you want to make crisp.

Use sklearn.preprocessing.Binarizer.

Example (Binarization)

import pandas as pd
import numpy as np
from sklearn.preprocessing import Binarizer

# Load data (replace path)
path = 'pima-indians-diabetes.csv'
names = ['preg', 'plas', 'pres', 'skin', 'test', 'mass', 'pedi', 'age', 'class']
dataframe = pd.read_csv(path, names=names)

# Separate features (X)
array = dataframe.values
X = array[:, :-1]

# Initialize the binarizer with a threshold (e.g., 0.0, adjust as needed)
binarizer = Binarizer(threshold=0.0)

# Fit (not necessary) and transform
X_binarized = binarizer.fit_transform(X)

# Summarize binarized data
np.set_printoptions(precision=2)
print("Binarized data (threshold=0.0, first 5 rows):\n", X_binarized[0:5])

5. Encoding Categorical Features

ML algorithms require numerical input. Categorical features (text labels) must be converted.

a) Label Encoding

Assigns a unique integer to each category. Example: [‘red’, ‘green’, ‘blue’] -> [0, 1, 2].

Use Case: Primarily suitable for encoding the target variable (y) in classification tasks.

Caution: Avoid using Label Encoding for input features (X) if there’s no inherent order among categories, as it introduces an artificial ordinal relationship (e.g., implies green is ‘greater’ than red).

Use sklearn.preprocessing.LabelEncoder.

Example (Label Encoding - Suitable for Target Variable)

import numpy as np
from sklearn.preprocessing import LabelEncoder

# Sample target variable labels
labels = np.array(['cat', 'dog', 'cat', 'fish', 'dog'])
print("Original labels:", labels)

# Initialize encoder
encoder = LabelEncoder()

# Fit and transform
encoded_labels = encoder.fit_transform(labels)
print("Encoded labels:", encoded_labels)

# See the mapping
print("Mapping (Classes to Encoded Values):", dict(zip(encoder.classes_, encoder.transform(encoder.classes_))))

# Inverse transform (get original labels back)
decoded_labels = encoder.inverse_transform(encoded_labels)
print("Decoded labels:", decoded_labels)

b) One-Hot Encoding

Creates new binary (0 or 1) features for each category. Example: ‘color’ feature with [‘red’, ‘green’, ‘blue’] becomes three features: ‘color_red’, ‘color_green’, ‘color_blue’. An instance that was ‘red’ becomes [1, 0, 0].

Use Case: The standard method for encoding nominal categorical input features (X) (where categories have no order).

Use sklearn.preprocessing.OneHotEncoder (more flexible) or pandas.get_dummies (often simpler for DataFrames).

Example (One-Hot Encoding - Suitable for Input Features)

import pandas as pd
from sklearn.preprocessing import OneHotEncoder

# Sample DataFrame with a categorical feature
data = pd.DataFrame({'color': ['red', 'green', 'blue', 'red'], 'size': ['M', 'L', 'S', 'M']})
print("Original DataFrame:\n", data)

# Using pandas.get_dummies (simple)
data_encoded_pd = pd.get_dummies(data, columns=['color', 'size'], drop_first=False) # drop_first=True avoids multicollinearity if needed
print("\nEncoded using pandas.get_dummies:\n", data_encoded_pd)

# Using sklearn.preprocessing.OneHotEncoder (more control, integrates with Pipelines)
encoder_ohe = OneHotEncoder(sparse_output=False) # sparse=False returns a dense array

# Fit and transform (select categorical columns first)
categorical_cols = ['color', 'size']
encoded_array = encoder_ohe.fit_transform(data[categorical_cols])

# Get feature names for the new columns
feature_names = encoder_ohe.get_feature_names_out(categorical_cols)

# Create a new DataFrame with encoded features
data_encoded_sklearn = pd.DataFrame(encoded_array, columns=feature_names, index=data.index)
print("\nEncoded using sklearn.OneHotEncoder:\n", data_encoded_sklearn)

# You would typically concatenate this back with numerical features if any

Conclusion

Data preparation is a foundational skill in machine learning. The specific techniques used depend on the data and the chosen algorithm. Applying these steps thoughtfully ensures that your model receives data in the most effective format, leading to better performance and more reliable results. Remember to apply preprocessing steps consistently to both training and testing data, ideally using Scikit-learn Pipelines to prevent data leakage.

Further Resources:

• Scikit-learn Preprocessing Guide: https://scikit-learn.org/stable/modules/preprocessing.html
• Scikit-learn Imputation Guide: https://scikit-learn.org/stable/modules/impute.html