AI with Python – Neural Networks

Neural Networks, inspired by the human brain’s structure, are powerful computational models in AI. They excel at tasks like pattern recognition, classification, regression, and generative modeling. Their ability to learn complex, non-linear relationships from data makes them highly versatile.

What are Artificial Neural Networks (ANNs)?

An Artificial Neural Network (ANN) is a system of interconnected processing units called neurons or nodes, organized in layers. Each connection between neurons has an associated weight, which is adjusted during the learning process. Neurons receive input signals, process them using an activation function, and produce an output signal that is passed to other neurons.

Key components of an ANN include:

Neurons (Nodes): Basic computational units.
Layers: Neurons are typically organized into an input layer, one or more hidden layers, and an output layer.
Weights and Biases: Parameters that the network learns. Weights modulate the strength of signals between neurons, and biases shift the activation function’s output.
Activation Function: Introduces non-linearity, allowing the network to learn complex patterns (e.g., Sigmoid, ReLU, Tanh).

For further conceptual details, you can refer to introductory materials on ANNs and Deep Learning. Modern deep learning often involves networks with many hidden layers.

Modern Tools: Scikit-learn, TensorFlow, and PyTorch

While older libraries like NeuroLab existed, contemporary neural network development in Python primarily uses Scikit-learn for simpler models, and TensorFlow (with Keras API) or PyTorch for more complex and custom architectures.

For the examples in this introductory chapter, we will use Scikit-learn’s MLPClassifier (Multi-layer Perceptron Classifier) and MLPRegressor, which are suitable for basic neural network tasks. You can install Scikit-learn via pip:

pip install scikit-learn matplotlib numpy

Building Neural Networks with Scikit-learn

Perceptron: The Simplest Neural Network

A Perceptron is the simplest type of ANN, consisting of a single neuron with an activation function (typically a step function). It’s a linear classifier. Scikit-learn provides sklearn.linear_model.Perceptron.

Example: A Perceptron for a simple binary classification task (e.g., AND gate).

import numpy as np
from sklearn.linear_model import Perceptron
import matplotlib.pyplot as plt

# Input data for AND gate
X = np.array([[0, 0], [0, 1], [1, 0], [1, 1]])
# Target output for AND gate
y = np.array([0, 0, 0, 1])

# Create and train the Perceptron model
# max_iter: maximum number of passes over the training data (epochs)
# tol: stopping criterion (training stops if loss > previous_loss - tol)
# eta0: learning rate
perceptron_model = Perceptron(max_iter=100, tol=1e-3, eta0=0.1, random_state=42)
perceptron_model.fit(X, y)

# Test the model
print("Perceptron predictions for AND gate inputs:")
for i, sample in enumerate(X):
    prediction = perceptron_model.predict([sample])
    print(f"Input: {sample}, Actual: {y[i]}, Predicted: {prediction[0]}")

# Plotting decision boundary (for 2D data)
# (This is more complex and often skipped for basic perceptron examples)
# print(f"Learned weights: {perceptron_model.coef_}, Intercept: {perceptron_model.intercept_}")

Perceptrons can only solve linearly separable problems. For more complex tasks, multi-layer networks are needed.

Multi-Layer Perceptron (MLP) for Classification

An MLP consists of an input layer, one or more hidden layers, and an output layer. This allows for learning non-linear decision boundaries.

Let’s create a simple dataset and classify it using MLPClassifier.

from sklearn.neural_network import MLPClassifier
from sklearn.model_selection import train_test_split
from sklearn.preprocessing import StandardScaler
from sklearn.datasets import make_classification

# Generate a synthetic dataset for classification
X_clf, y_clf = make_classification(n_samples=200, n_features=2, n_informative=2,
                                   n_redundant=0, random_state=42, n_clusters_per_class=1)

# Scale features for better performance
scaler = StandardScaler()
X_clf_scaled = scaler.fit_transform(X_clf)

# Split data
X_train, X_test, y_train, y_test = train_test_split(X_clf_scaled, y_clf, test_size=0.3, random_state=42)

# Create MLPClassifier model
# hidden_layer_sizes: tuple, e.g., (50,) means one hidden layer with 50 neurons
# activation: 'relu' is common
# solver: 'adam' is a popular optimizer
# max_iter: number of epochs
mlp_clf = MLPClassifier(hidden_layer_sizes=(50,), activation='relu', solver='adam',
                        max_iter=500, random_state=42, early_stopping=True, tol=1e-4)

# Train the model
mlp_clf.fit(X_train, y_train)

# Evaluate the model
accuracy = mlp_clf.score(X_test, y_test)
print(f"MLP Classifier Accuracy: {accuracy:.4f}")

# Plot loss curve
plt.figure()
plt.plot(mlp_clf.loss_curve_)
plt.title('MLP Classifier Training Loss Curve')
plt.xlabel('Epochs')
plt.ylabel('Loss')
plt.grid(True)
plt.show()

[Image description: A line graph showing the training loss of the MLP classifier decreasing over epochs. The x-axis is ‘Epochs’ and the y-axis is ‘Loss’. A downward trend indicates the model is learning.]

This example demonstrates a basic classification task. The loss_curve_ attribute shows how the error decreased during training.

Multi-Layer Perceptron (MLP) for Regression

MLPs can also be used for regression tasks, predicting continuous values. Scikit-learn provides MLPRegressor.

Let’s generate data based on a non-linear function y = 2x^2 - x + 3 + noise and try to fit an MLPRegressor.

from sklearn.neural_network import MLPRegressor
import numpy as np
import matplotlib.pyplot as plt

# Generate synthetic data for regression
np.random.seed(42)
num_points = 100
X_reg = np.linspace(-5, 5, num_points).reshape(-1, 1)
y_reg = (2 * X_reg**2 - X_reg + 3 + np.random.randn(num_points, 1) * 5).ravel()

# Scale features (important for NNs)
scaler_X_reg = StandardScaler()
X_reg_scaled = scaler_X_reg.fit_transform(X_reg)

# Split data
X_reg_train, X_reg_test, y_reg_train, y_reg_test = train_test_split(X_reg_scaled, y_reg, test_size=0.2, random_state=42)

# Create MLPRegressor model
# Two hidden layers: 64 neurons, then 32 neurons
mlp_reg = MLPRegressor(hidden_layer_sizes=(64, 32), activation='relu', solver='adam',
                       max_iter=1000, random_state=42, early_stopping=True, tol=1e-4)

# Train the model
mlp_reg.fit(X_reg_train, y_reg_train)

# Make predictions
y_reg_pred = mlp_reg.predict(X_reg_test)

# Evaluate (e.g., R-squared score)
r2 = mlp_reg.score(X_reg_test, y_reg_test)
print(f"MLP Regressor R-squared: {r2:.4f}")

# Plot actual vs. predicted values
# To plot the regression line smoothly, predict on a dense range of X values
X_dense = np.linspace(X_reg_scaled.min(), X_reg_scaled.max(), 200).reshape(-1, 1)
y_dense_pred = mlp_reg.predict(X_dense)

plt.figure(figsize=(10, 6))
plt.scatter(X_reg_test, y_reg_test, color='blue', label='Actual Data', alpha=0.6)
# Need to sort X_dense for correct line plotting if using original (unscaled) X values for x-axis
# For simplicity, plotting against scaled X_dense
plt.plot(X_dense, y_dense_pred, color='red', linewidth=2, label='MLP Regression Fit')
plt.title('MLP Regressor: Actual vs. Predicted')
plt.xlabel('Scaled Feature X')
plt.ylabel('Target y')
plt.legend()
plt.grid(True)
plt.show()

[Image description: A scatter plot showing actual data points in blue. A red line, representing the MLP regressor’s fit, attempts to follow the non-linear trend of these data points. The x-axis is ‘Scaled Feature X’ and y-axis is ‘Target y’.]

These examples provide a basic introduction to using neural networks with Scikit-learn. For more advanced applications, exploring TensorFlow/Keras or PyTorch is recommended, as they offer greater flexibility in defining custom architectures, layers, and training loops.