Implementations

Python Deep Learning - Implementation Example: Customer Churn Prediction

In this implementation example, we’ll build a simple Deep Neural Network (DNN) using Python to predict customer attrition (churn) for a bank. We want to identify customers who are likely to stop using the bank’s services based on their past behavior and characteristics.

We will use the Keras API within TensorFlow 2.x, a popular and user-friendly framework for deep learning. The dataset (Churn_Modelling.csv) is relatively small (10,000 rows, 14 columns) and suitable for a basic classification task.

Assumption: You have a Python environment set up (e.g., Anaconda) with necessary libraries installed. If not, you typically install TensorFlow (which includes Keras) using pip:

# Ensure TensorFlow is installed (includes Keras)
# pip install tensorflow pandas numpy scikit-learn matplotlib

Step 1: Load and Inspect Data

First, we import necessary libraries and load the dataset using pandas.

# Importing the libraries
import numpy as np
import pandas as pd
import tensorflow as tf
from sklearn.model_selection import train_test_split
from sklearn.preprocessing import StandardScaler, OneHotEncoder
from sklearn.compose import ColumnTransformer
from sklearn.metrics import confusion_matrix, accuracy_score

# Load the dataset
dataset = pd.read_csv('Churn_Modelling.csv')

# Display first few rows and info
print(dataset.head())
print(dataset.info())

# Define features (X) and target (y)
# We drop RowNumber, CustomerId, Surname as they are irrelevant identifiers
X = dataset.iloc[:, 3:13] # Columns from CreditScore to EstimatedSalary
y = dataset.iloc[:, 13]  # 'Exited' column

Step 2: Data Preprocessing

Neural networks require numerical input. We need to handle categorical features (‘Geography’, ‘Gender’) and scale numerical features.

# Identify categorical and numerical features
categorical_features = ['Geography', 'Gender']
numerical_features = ['CreditScore', 'Age', 'Tenure', 'Balance', 'NumOfProducts', 'HasCrCard', 'IsActiveMember', 'EstimatedSalary']

# Create preprocessing pipelines for numerical and categorical features
preprocessor = ColumnTransformer(
    transformers=[
        ('num', StandardScaler(), numerical_features),
        ('cat', OneHotEncoder(handle_unknown='ignore'), categorical_features)
    ],
    remainder='passthrough' # Keep other columns (if any), though we selected all relevant ones
)

# Apply preprocessing
X = preprocessor.fit_transform(X)

# Check the shape of processed features
print(f"Shape of processed X: {X.shape}") # Shape will change due to OneHotEncoding

# Display some processed data (optional)
# Note: Output is a NumPy array, column names are lost in transformation
print("\nSample processed features:")
print(X[:2])

Explanation: StandardScaler scales numerical features to have zero mean and unit variance. OneHotEncoder converts categorical features into numerical format by creating binary columns for each category. ColumnTransformer applies these transformations to the correct columns.

Step 3: Split Data into Training and Test Sets

We split the data to train the model on one part and evaluate its performance on unseen data.

# Splitting the dataset into the Training set and Test set (80% train, 20% test)
X_train, X_test, y_train, y_test = train_test_split(X, y, test_size = 0.2, random_state = 42) # random_state for reproducibility

print(f"\nTraining set shape: X={X_train.shape}, y={y_train.shape}")
print(f"Test set shape: X={X_test.shape}, y={y_test.shape}")

Step 4: Build the Neural Network Model

We define a sequential DNN using TensorFlow/Keras.

# Initialize the ANN
model = tf.keras.models.Sequential()

# Get the number of input features after preprocessing
input_dim = X_train.shape[1]

# Add the input layer and the first hidden layer
# - units: number of neurons in the layer (hyperparameter)
# - activation: activation function (ReLU is common for hidden layers)
# - kernel_initializer: method to initialize weights ('he_uniform' is often good with ReLU)
# - input_shape: required for the first layer to know the input dimension
model.add(tf.keras.layers.Dense(units=6, activation='relu', kernel_initializer='he_uniform', input_shape=(input_dim,)))

# Add the second hidden layer
model.add(tf.keras.layers.Dense(units=6, activation='relu', kernel_initializer='he_uniform'))

# Add the output layer
# - units=1: for binary classification
# - activation='sigmoid': outputs a probability between 0 and 1
# - kernel_initializer='glorot_uniform': often suitable for sigmoid/output layers
model.add(tf.keras.layers.Dense(units=1, activation='sigmoid', kernel_initializer='glorot_uniform'))

# Display model summary
model.summary()

Choosing the number of layers and units per layer is often experimental. We start with a simple 2-hidden-layer network with 6 units each.

Step 5: Compile the Model

Before training, we configure the learning process.

# Compile the ANN
# - optimizer: algorithm to update weights (Adam is a popular choice)
# - loss: function to measure error ('binary_crossentropy' for binary classification)
# - metrics: performance measure to evaluate during training/testing
model.compile(optimizer='adam', loss='binary_crossentropy', metrics=['accuracy'])

Step 6: Train the Model

We fit the model to the training data.

# Train the ANN on the Training set
# - batch_size: number of samples per gradient update
# - epochs: number of times to iterate over the entire training dataset
history = model.fit(X_train, y_train,
                    batch_size = 32, # Common batch size
                    epochs = 50,    # Number of training cycles
                    validation_split=0.1, # Use 10% of training data for validation during training
                    verbose=1) # Show progress

batch_size and epochs are hyperparameters. 32 is a common batch size, and 50-100 epochs might be sufficient for this small dataset, but monitoring validation loss is key (EarlyStopping callback could be added).

Step 7: Make Predictions and Evaluate the Model

We use the trained model to predict on the test set and evaluate its performance.

# Predicting the Test set results
y_pred_proba = model.predict(X_test)
y_pred = (y_pred_proba > 0.5) # Convert probabilities to binary predictions (0 or 1)

# Print some predictions vs actual values
print("\nSample Predictions vs Actual:")
print(np.concatenate((y_pred[:10].reshape(-1,1), y_test[:10].values.reshape(-1,1)), axis=1))

# Making the Confusion Matrix
cm = confusion_matrix(y_test, y_pred)
acc = accuracy_score(y_test, y_pred)

print("\nConfusion Matrix:")
print(cm)
print(f"\nAccuracy on Test Set: {acc:.4f}")

The confusion matrix shows True Negatives (TN), False Positives (FP), False Negatives (FN), and True Positives (TP). Accuracy = (TN + TP) / Total.

Step 8: Predicting a Single New Observation (Example)

Let’s predict if a hypothetical new customer will churn.

# Example: Predict for a single customer
# Data: Geography: Spain, Credit Score: 500, Gender: Female, Age: 40, Tenure: 3, Balance: 50000,
#       NumOfProducts: 2, HasCrCard: Yes (1), IsActiveMember: Yes (1), EstimatedSalary: 40000

# Create DataFrame for the single observation to use the preprocessor
new_customer_data = pd.DataFrame({
    'CreditScore': [500],
    'Geography': ['Spain'],
    'Gender': ['Female'],
    'Age': [40],
    'Tenure': [3],
    'Balance': [50000],
    'NumOfProducts': [2],
    'HasCrCard': [1],
    'IsActiveMember': [1],
    'EstimatedSalary': [40000]
})

# Important: Use the SAME preprocessor fitted on the training data
new_customer_processed = preprocessor.transform(new_customer_data)

# Make prediction
new_prediction_proba = model.predict(new_customer_processed)
new_prediction = (new_prediction_proba > 0.5)

print(f"\nPrediction for new customer (Probability): {new_prediction_proba[0][0]:.4f}")
print(f"Prediction for new customer (Churn?): {'Yes' if new_prediction[0][0] else 'No'}")

This completes a basic implementation workflow for a deep learning classification task. Further improvements could involve hyperparameter tuning, adding regularization (like Dropout), using callbacks (like EarlyStopping), or exploring more complex architectures.

Understanding Forward Propagation (Manual Example)

To better understand what happens inside the network during prediction (forward propagation), let’s manually calculate the output for a very simple network with NumPy. This is purely for illustration; frameworks like TensorFlow handle this automatically and efficiently.

Consider a network with 2 input neurons, one hidden layer with 2 neurons (using ReLU activation), and one output neuron (no activation for simplicity here, or maybe linear).

Input -> Hidden Layer (ReLU) -> Output Layer (Linear)

import numpy as np

def relu(input_val):
    # Rectified Linear Unit Activation
    return np.maximum(0, input_val)

# Sample input data (e.g., 2 features for one customer)
input_data = np.array([2, 3])

# Sample weights (randomly initialized or pre-defined for example)
weights = {
    'node_0': np.array([1.0, -0.5]), # Weights for first hidden neuron
    'node_1': np.array([-1.5, 2.0]), # Weights for second hidden neuron
    'output': np.array([0.5, -1.0])  # Weights connecting hidden layer to output
}

# Biases (often used, simplified here by omitting them or assuming they are part of weights)

# Calculate hidden layer inputs
node_0_input = (input_data * weights['node_0']).sum()
node_1_input = (input_data * weights['node_1']).sum()

# Calculate hidden layer outputs (apply ReLU)
node_0_output = relu(node_0_input)
node_1_output = relu(node_1_input)

hidden_layer_outputs = np.array([node_0_output, node_1_output])
print(f"Hidden Layer Outputs: {hidden_layer_outputs}")

# Calculate final output layer input
output_layer_input = (hidden_layer_outputs * weights['output']).sum()

# Final output (assuming linear activation for this example)
model_output = output_layer_input

print(f"Manual Forward Pass Output: {model_output}")

This manual calculation shows the flow: weighted sums followed by activation functions, layer by layer. Real networks involve matrices for handling batches of data and more complex operations, all optimized by the underlying framework.