AI for Good

Machine learning has enormous potential to address pressing social challenges when applied thoughtfully and responsibly. From improving healthcare outcomes to advancing climate science to enhancing education, ML systems can amplify human capability in ways that benefit society broadly.

Measuring Social Impact

The measurement of social impact presents unique challenges. Unlike commercial metrics that are easily quantifiable, social outcomes may be difficult to measure, may manifest over long time horizons, and may involve competing stakeholder interests. Defining appropriate metrics and evaluation frameworks is a critical first step that requires community engagement.

Healthcare AI shows enormous promise across medical imaging, drug discovery, clinical decision support, and genomics. Deep learning models can detect diseases in medical images with accuracy comparable to or exceeding that of specialist physicians, but deploying these systems in clinical practice requires navigating regulatory, ethical, and trust challenges.

Medical Image Classification with Transfer Learning

Transfer learning from ImageNet-pretrained models is the standard approach for medical image classification, because labeled medical datasets are typically small (hundreds to low thousands of images). Fine-tuning a pretrained ResNet or EfficientNet on domain-specific data achieves strong performance with limited data. The following pipeline demonstrates a complete medical image classification workflow.

import torch
import torch.nn as nn
import torch.optim as optim
from torch.utils.data import DataLoader
from torchvision import transforms, datasets, models
from sklearn.metrics import (
    classification_report, roc_auc_score
)
import numpy as np

class="tok-comment"># --- Data preparation with medical imaging augmentations ---
train_transforms = transforms.Compose([
    transforms.Resize((class="tok-number">224, class="tok-number">224)),
    transforms.RandomHorizontalFlip(),
    transforms.RandomRotation(class="tok-number">15),
    transforms.ColorJitter(
        brightness=class="tok-number">0.2, contrast=class="tok-number">0.2
    ),
    transforms.ToTensor(),
    transforms.Normalize(
        mean=[class="tok-number">0.485, class="tok-number">0.456, class="tok-number">0.406],
        std=[class="tok-number">0.229, class="tok-number">0.224, class="tok-number">0.225],
    ),
])

val_transforms = transforms.Compose([
    transforms.Resize((class="tok-number">224, class="tok-number">224)),
    transforms.ToTensor(),
    transforms.Normalize(
        mean=[class="tok-number">0.485, class="tok-number">0.456, class="tok-number">0.406],
        std=[class="tok-number">0.229, class="tok-number">0.224, class="tok-number">0.225],
    ),
])

class="tok-comment"># Load dataset (e.g., chest X-ray pneumonia detection)
train_dataset = datasets.ImageFolder(
    class="tok-string">"data/chest_xray/train", transform=train_transforms
)
val_dataset = datasets.ImageFolder(
    class="tok-string">"data/chest_xray/val", transform=val_transforms
)

train_loader = DataLoader(
    train_dataset, batch_size=class="tok-number">32, shuffle=True,
    num_workers=class="tok-number">4, pin_memory=True,
)
val_loader = DataLoader(
    val_dataset, batch_size=class="tok-number">32, shuffle=False,
    num_workers=class="tok-number">4, pin_memory=True,
)

class="tok-comment"># --- Transfer learning: fine-tune a pretrained ResNet-class="tok-number">50 ---
model = models.resnet50(weights=models.ResNet50_Weights.DEFAULT)

class="tok-comment"># Freeze early layers (feature extractor)
for param in model.parameters():
    param.requires_grad = False

class="tok-comment"># Replace the classification head for binary classification
num_features = model.fc.in_features
model.fc = nn.Sequential(
    nn.Linear(num_features, class="tok-number">256),
    nn.ReLU(),
    nn.Dropout(class="tok-number">0.3),
    nn.Linear(class="tok-number">256, class="tok-number">1),  class="tok-comment"># Binary: pneumonia vs. normal
)

device = torch.device(class="tok-string">"cuda" if torch.cuda.is_available() else class="tok-string">"cpu")
model = model.to(device)

criterion = nn.BCEWithLogitsLoss()
optimizer = optim.Adam(model.fc.parameters(), lr=class="tok-number">1e-4)
scheduler = optim.lr_scheduler.ReduceLROnPlateau(
    optimizer, patience=class="tok-number">3, factor=class="tok-number">0.5
)

class="tok-comment"># --- Training loop with validation ---
best_auc = class="tok-number">0.0
for epoch in range(class="tok-number">20):
    model.train()
    running_loss = class="tok-number">0.0
    for images, labels in train_loader:
        images = images.to(device)
        labels = labels.float().to(device)
        optimizer.zero_grad()
        outputs = model(images).squeeze(class="tok-number">1)
        loss = criterion(outputs, labels)
        loss.backward()
        optimizer.step()
        running_loss += loss.item()

    class="tok-comment"># Validation
    model.eval()
    all_labels, all_probs = [], []
    with torch.no_grad():
        for images, labels in val_loader:
            images = images.to(device)
            outputs = model(images).squeeze(class="tok-number">1)
            probs = torch.sigmoid(outputs).cpu().numpy()
            all_probs.extend(probs)
            all_labels.extend(labels.numpy())

    val_auc = roc_auc_score(all_labels, all_probs)
    scheduler.step(class="tok-number">1 - val_auc)

    if val_auc > best_auc:
        best_auc = val_auc
        torch.save(model.state_dict(), class="tok-string">"best_model.pth")

    print(
        class="tok-string">f"Epoch {epoch+class="tok-number">1}: loss={running_loss/len(train_loader):.4f}, "
        class="tok-string">f"val_AUC={val_auc:.4f}"
    )

class="tok-comment"># --- Final evaluation ---
model.load_state_dict(torch.load(class="tok-string">"best_model.pth"))
model.eval()
all_labels, all_preds, all_probs = [], [], []
with torch.no_grad():
    for images, labels in val_loader:
        images = images.to(device)
        outputs = model(images).squeeze(class="tok-number">1)
        probs = torch.sigmoid(outputs).cpu().numpy()
        preds = (probs >= class="tok-number">0.5).astype(int)
        all_probs.extend(probs)
        all_preds.extend(preds)
        all_labels.extend(labels.numpy())

print(class="tok-string">"\n" + classification_report(
    all_labels, all_preds,
    target_names=[class="tok-string">"Normal", class="tok-string">"Pneumonia"],
))
print(class="tok-string">f"AUC-ROC: {roc_auc_score(all_labels, all_probs):.4f}")

Evaluation Metrics for Medical AI

Medical AI systems require careful evaluation beyond simple accuracy. In healthcare, the relative cost of false positives (unnecessary follow-up) versus false negatives (missed disease) varies dramatically by application. The following metrics are essential for evaluating medical ML systems.

Natural language processing has been transformed by large language models. The shift from task-specific models to general-purpose LLMs has fundamentally changed how NLP systems are designed, trained, and deployed. Prompting a foundation model can now accomplish tasks that previously required months of custom model development.

Production NLP Pipeline Architecture

Recommender systems drive engagement on major platforms and are among the most commercially impactful ML applications. These systems predict user preferences from interaction history, user features, and item features, powering personalized experiences at scale.

Collaborative Filtering with Matrix Factorization

Matrix factorization decomposes the sparse user-item interaction matrix R into two dense matrices: a user matrix U and an item matrix V. Each user and item is represented by a latent factor vector, and the predicted rating is the dot product of these vectors. The following implementation demonstrates this approach.

import numpy as np
from typing import Optional


class MatrixFactorizationCF:
    class="tok-string">class="tok-string">""class="tok-string">"Collaborative filtering via matrix factorization.

    Decomposes the user-item interaction matrix R into
    low-rank matrices U (users) and V (items) such that
    R ≈ U @ V.T. Trained with SGD on observed entries.
    "class="tok-string">""

    def __init__(
        self,
        n_users: int,
        n_items: int,
        n_factors: int = class="tok-number">50,
        lr: float = class="tok-number">0.01,
        reg: float = class="tok-number">0.02,
    ):
        self.n_factors = n_factors
        self.lr = lr
        self.reg = reg
        class="tok-comment"># Initialize latent factors with small random values
        self.user_factors = np.random.normal(
            class="tok-number">0, class="tok-number">0.1, (n_users, n_factors)
        )
        self.item_factors = np.random.normal(
            class="tok-number">0, class="tok-number">0.1, (n_items, n_factors)
        )
        class="tok-comment"># Bias terms capture user/item-specific tendencies
        self.user_bias = np.zeros(n_users)
        self.item_bias = np.zeros(n_items)
        self.global_bias = class="tok-number">0.0

    def fit(
        self,
        ratings: list[tuple[int, int, float]],
        n_epochs: int = class="tok-number">20,
        val_ratings: Optional[list[tuple[int, int, float]]] = None,
    ):
        class="tok-string">class="tok-string">""class="tok-string">"Train on (user_id, item_id, rating) tuples.

        Uses stochastic gradient descent with L2
        regularization on both factors and biases.
        "class="tok-string">""
        all_ratings = [r for _, _, r in ratings]
        self.global_bias = np.mean(all_ratings)

        for epoch in range(n_epochs):
            np.random.shuffle(ratings)
            total_loss = class="tok-number">0.0

            for user, item, rating in ratings:
                pred = self.predict(user, item)
                error = rating - pred

                class="tok-comment"># Update biases
                self.user_bias[user] += self.lr * (
                    error - self.reg * self.user_bias[user]
                )
                self.item_bias[item] += self.lr * (
                    error - self.reg * self.item_bias[item]
                )

                class="tok-comment"># Update latent factors
                u = self.user_factors[user].copy()
                self.user_factors[user] += self.lr * (
                    error * self.item_factors[item]
                    - self.reg * u
                )
                self.item_factors[item] += self.lr * (
                    error * u
                    - self.reg * self.item_factors[item]
                )

                total_loss += error ** class="tok-number">2

            rmse = np.sqrt(total_loss / len(ratings))
            msg = class="tok-string">f"Epoch {epoch+class="tok-number">1}: train RMSE={rmse:.4f}"
            if val_ratings:
                val_rmse = self.evaluate(val_ratings)
                msg += class="tok-string">f", val RMSE={val_rmse:.4f}"
            print(msg)

    def predict(self, user: int, item: int) -> float:
        class="tok-string">class="tok-string">""class="tok-string">"Predict rating for a user-item pair."class="tok-string">""
        return (
            self.global_bias
            + self.user_bias[user]
            + self.item_bias[item]
            + self.user_factors[user]
            @ self.item_factors[item]
        )

    def recommend(
        self, user: int, n: int = class="tok-number">10,
        exclude: Optional[set[int]] = None,
    ) -> list[tuple[int, float]]:
        class="tok-string">class="tok-string">""class="tok-string">"Return top-n item recommendations for a user."class="tok-string">""
        scores = (
            self.global_bias
            + self.user_bias[user]
            + self.item_bias
            + self.item_factors @ self.user_factors[user]
        )
        if exclude:
            for item_id in exclude:
                scores[item_id] = -np.inf
        top_items = np.argsort(scores)[::-class="tok-number">1][:n]
        return [(int(i), float(scores[i])) for i in top_items]

    def evaluate(
        self, test_ratings: list[tuple[int, int, float]]
    ) -> float:
        class="tok-string">class="tok-string">""class="tok-string">"Compute RMSE on test set."class="tok-string">""
        errors = [
            (r - self.predict(u, i)) ** class="tok-number">2
            for u, i, r in test_ratings
        ]
        return np.sqrt(np.mean(errors))


class="tok-comment"># Example usage
mf = MatrixFactorizationCF(
    n_users=class="tok-number">1000, n_items=class="tok-number">5000, n_factors=class="tok-number">50
)
mf.fit(train_ratings, n_epochs=class="tok-number">20, val_ratings=val_ratings)
top_recs = mf.recommend(user=class="tok-number">42, n=class="tok-number">10, exclude=seen_items)
print(class="tok-string">"Top recommendations:", top_recs)

Ranking Metrics for Recommender Systems

Recommender systems are evaluated with ranking metrics that account for the position of relevant items in the recommendation list. A relevant item at position 1 is far more valuable than the same item at position 100.

Modern Recommendation Architecture

Production recommender systems at scale use a multi-stage architecture to handle catalogs of millions of items efficiently. The retrieval stage uses approximate nearest neighbor search to generate a broad candidate set (hundreds from millions). The ranking stage applies a more complex model to score and reorder candidates. The post-ranking stage applies business rules, diversity constraints, and freshness boosts.

1. Candidate retrieval (100-1000 from millions): Two-tower models, ANN search, collaborative signals
2. Scoring/ranking (tens from hundreds): Deep ranking models with rich features (user, item, context)
3. Re-ranking (final order): Diversity, freshness, business rules, ad insertion, deduplication
4. Serving: Real-time feature lookup, low-latency inference, A/B testing framework

Autonomous vehicles (AVs) represent one of the most technically demanding AI applications, requiring real-time perception, prediction, planning, and control in safety-critical environments. The ML stack for AVs integrates computer vision, sensor fusion, behavioral prediction, and motion planning into a system that must operate reliably across diverse driving conditions at speeds where milliseconds matter.

Perception and Sensor Fusion

The perception system processes data from cameras, LiDAR, radar, and ultrasonic sensors to build a comprehensive understanding of the vehicle's environment. Each sensor modality contributes unique information: cameras provide rich semantic information and color, LiDAR gives precise 3D geometry, radar works reliably in adverse weather, and ultrasonics handle close-range detection.

ML Applications Comparison: Healthcare vs Finance vs AVs

Different domains impose fundamentally different requirements on ML systems. Healthcare demands clinical validation and regulatory approval. Finance demands auditability and real-time performance. Autonomous vehicles demand real-time safety-critical decision-making. Understanding these differences is essential for engineers moving between domains.

The Long-Tail Safety Challenge

The safety challenge in autonomous driving is defined by the long tail of rare events. Standard performance on common driving scenarios is relatively straightforward, but safely handling unusual situations -- construction zones, emergency vehicles, unusual road geometry, adverse weather, aggressive drivers -- requires massive datasets and sophisticated testing. A self-driving system must handle millions of unique scenarios, many of which occur too rarely to be well-represented in training data.

Constraint-driven innovation -- the strictest deployment constraints drive the most creative systems engineering solutions.Deeper InsightDomains like medical AI, autonomous vehicles, and aerospace face the tightest constraints: real-time latency, regulatory compliance, explainability mandates, and near-zero error tolerance. Rather than limiting progress, these constraints force engineers to develop creative solutions -- simulation-at-scale testing, sensor fusion architectures, formal verification methods, and human-in-the-loop designs -- that eventually benefit the entire field. The most robust and well-engineered ML systems almost always emerge from the most constrained domains.Think of it like...Like how Formula 1 racing constraints (weight limits, fuel restrictions, safety regulations) have driven innovations in materials science, aerodynamics, and engine efficiency that eventually make it into consumer vehicles. Click to collapse

Robotics Beyond Driving

The same perception, planning, and control techniques that power autonomous vehicles are being applied across robotics domains. Warehouse robots (Amazon, Ocado), surgical robots (Intuitive Surgical's da Vinci), agricultural robots (harvesting, weeding), and household robots (cleaning, delivery) all benefit from advances in ML-driven perception and manipulation.

• Warehouse automation: Robots handle over 75% of Amazon fulfillment center operations, using vision for picking and placing diverse objects
• Surgical robotics: ML assists with tissue identification, surgical planning, and semi-autonomous suturing
• Agricultural robotics: Computer vision enables selective harvesting, precision weeding without herbicides, and crop health monitoring
• Manipulation: Foundation models for robotics (RT-2, Octo) transfer knowledge across tasks, reducing the need for task-specific training data
• Humanoid robots: Companies like Figure, Tesla (Optimus), and Boston Dynamics combine locomotion control with manipulation and language understanding

ML systems contribute to climate action through improved weather and climate modeling, energy system optimization, environmental monitoring, and materials discovery for clean energy technologies. These applications leverage ML's ability to model complex physical systems and extract patterns from large observational datasets.

ML-Powered Weather Forecasting

Weather forecasting has been revolutionized by ML approaches. GraphCast and Pangu-Weather demonstrate that ML models can match or exceed traditional numerical weather prediction (NWP) models at a fraction of the computational cost. These models are trained on decades of reanalysis data from the European Centre for Medium-Range Weather Forecasts (ECMWF) and produce global forecasts in seconds rather than hours.

Energy System Optimization

Energy system optimization uses ML to balance supply and demand in power grids with increasing renewable energy penetration. ML forecasts renewable generation, predicts demand patterns, and optimizes battery storage dispatch. These applications directly reduce carbon emissions by enabling higher renewable energy utilization.

Environmental Monitoring

• Deforestation tracking: ML classifies satellite imagery to detect forest loss within days, enabling rapid response
• Biodiversity monitoring: Acoustic sensors + ML identify species from sound recordings across vast areas
• Illegal mining detection: Satellite imagery analysis spots unauthorized mining operations in protected areas
• Ocean health: ML analyzes sea surface temperature, chlorophyll, and current data for marine ecosystem monitoring
• Air quality prediction: ML forecasts pollution levels to issue health warnings and guide policy
• Carbon accounting: ML estimates emissions from satellite observations of power plants, agriculture, and land use

ML-powered educational tools can personalize learning experiences, provide intelligent tutoring, automate assessment, and make education more accessible to underserved populations. Adaptive learning systems adjust content difficulty and sequencing based on individual student performance, providing each learner with an optimized path through the material.

NLP in Education

Natural language processing enables powerful educational applications that scale high-quality instruction to millions of learners. Large language models can serve as interactive tutoring systems that explain concepts, answer questions, and guide students through problem-solving with a Socratic approach.

Accessibility Applications

Accessibility applications use ML to break down barriers for people with disabilities, enabling more equitable access to information and technology. These tools benefit not only their primary audience but often improve usability for everyone.

• Speech recognition: Voice-controlled interfaces for users with motor impairments (accuracy now exceeds 95% for major languages)
• Image description: Automatic alt-text generation for visually impaired users (Microsoft Seeing AI, Apple VoiceOver)
• Sign language recognition: Real-time translation between sign language and text/speech
• Real-time captioning: Live speech-to-text for deaf and hard-of-hearing users
• Text simplification: Adapting complex text for users with cognitive disabilities
• Eye tracking: Gaze-based computer control for users with severe motor limitations

Humanitarian AI: Disaster Response and Food Security

Humanitarian applications of ML address urgent needs in disaster response, poverty alleviation, and food security. Speed is critical: after a natural disaster, ML systems analyzing satellite imagery can produce building damage maps within hours, guiding relief efforts to the most affected areas before ground teams can reach them.