Codecademy Logo

Introduction to Explainable AI

Understanding XAI

Explainable AI (XAI) refers to methods that clarify how AI models make decisions. It enhances our ability to understand, interpret, and trust machine learning systems. XAI is crucial in scenarios requiring accountability and transparency.

Black-Box Models

Black-box models, like deep neural networks, can process data in ways that humans find complex and non-transparent. These models offer powerful predictions but make it difficult to interpret their decision-making process. They are widely used for complex tasks where explanation isn’t the priority.

White-Box Modeling

White-box models such as linear regression and decision trees are characterized by their transparency. These models have decision-making processes that are easy to understand and interpret, making them useful in scenarios where explanations of the model’s logic are crucial.

Model-Agnostic Methods

Model-agnostic explainability methods offer insights into any model’s decision-making process without requiring an understanding of its internal workings. Notable examples include Permutation Importance, which gauges the influence of individual features, Partial Dependence Plots (PDP) for visualizing relationships, and SHAP for feature impact estimation.

Python Permutation Importance

Permutation Importance is a technique to evaluate feature significance by observing model performance decline after feature value shuffling. This helps in understanding which features are crucial for model predictions.

from sklearn.inspection import permutation_importance


# Calculate Permutation Importance
perm_importance = permutation_importance(model, X, y, n_repeats=10, random_state=0)


# Output Results
feature_names = X.columns
for i in perm_importance.importances_mean.argsort()[::-1]:
    if perm_importance.importances_mean[i] - 2 * perm_importance.importances_std[i] > 0:
        print(f"{feature_names[i]}: {perm_importance.importances_mean[i]:.3f}")

Python PDP Visualization

Partial Dependence Plots (PDPs) are crucial for understanding the influence of a single feature on model predictions, by averaging out other variables. They help in recognizing overall trends and patterns across data, thus boosting model interpretability.

import matplotlib.pyplot as plt
from sklearn.inspection import plot_partial_dependence
from sklearn.ensemble import RandomForestClassifier


# Fit the model
model = RandomForestClassifier()
model.fit(X_train, y_train)


# Generate and plot PDP
features = [0]  # Index or column name for the feature of interest
plot_partial_dependence(model, X_train, features)
plt.title('Partial Dependence Plot')
plt.show()

Python ICE Plots

Individual Conditional Expectation (ICE) plots show variations of how data instances react to changes in a feature. They expand on Partial Dependence Plots by highlighting differences across data points, providing a detailed look at predictive model behaviors linked to specific features.

from sklearn.ensemble import RandomForestRegressor
from sklearn.inspection import partial_dependence, plot_partial_dependence
import matplotlib.pyplot as plt


# Example Data
# Assume X and y are your dataset features and target
model = RandomForestRegressor().fit(X, y)


# Create ICE plot
fig, ax = plt.subplots()  
plot_partial_dependence(model, X, [0], kind='both', ax=ax)
plt.title('ICE Plot Example')
plt.xlabel('Feature 0')
plt.ylabel('Partial Dependence')
plt.show()

Model Explainability Concepts

Model-specific explainability methods help us understand how a model makes predictions by examining its internal components. For instance, you can look at regression coefficients for linear models or feature importance in tree-based models to gain insights into their behavior.

Python Coefficients

In Python, linear regression coefficients show how much the dependent variable changes when an independent variable increases by one unit, keeping others constant. The intercept indicates the predicted outcome when all variables are zero.

from sklearn.linear_model import LinearRegression
import numpy as np


# Sample data
X = np.array([[0], [1], [2], [3]])
y = np.array([4, 6, 8, 10])


# Create the model
model = LinearRegression()
model.fit(X, y)


# Display coefficients
coefficient = model.coef_[0]
intercept = model.intercept_


print(f"Coefficient: {coefficient}")
print(f"Intercept: {intercept}")

Python Logistic Regression

In logistic regression, coefficients indicate the change in log-odds for a one-unit feature increase, keeping others constant. By exponentiating these coefficients, you obtain the odds ratio, reflecting how odds of the target class change with a feature increase.

from sklearn.linear_model import LogisticRegression
from sklearn.datasets import load_iris
import numpy as np


# Load data
iris = load_iris()
X = iris.data
y = (iris.target == 0).astype(int)  # Binary classification: class 0 or not


# Initialize and fit logistic regression model
log_reg = LogisticRegression()
log_reg.fit(X, y)


# Coefficient and odds ratio
coef = log_reg.coef_[0]
odds_ratio = np.exp(coef)


print('Coefficients:', coef)
print('Odds Ratios:', odds_ratio)

Feature Importance in Python

Feature importance in tree-based models is derived from the reduction in impurity caused by each feature at splits. In regression trees, this impurity is judged by Mean Squared Error (MSE), whereas in classification trees it uses Gini impurity or entropy. A feature’s importance increases with frequent and impactful splits.

from sklearn.tree import DecisionTreeClassifier
from sklearn.datasets import load_iris
from sklearn.model_selection import train_test_split


# Load dataset
iris = load_iris()
X, y = iris.data, iris.target


# Split data
X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.3, random_state=42)


# Fit a Decision Tree model
clf = DecisionTreeClassifier(random_state=42)
clf.fit(X_train, y_train)


# Display feature importances
importances = clf.feature_importances_


# Output the feature importances
for feature, importance in zip(iris.feature_names, importances):
    print(f"{feature}: {importance:.3f}")
Next

Learn more on Codecademy