Causal‑ML Uplift Modeling Pipeline

Purpose: End‑to‑end framework for estimating Conditional Average Treatment Effects (CATE) and deploying uplift‑based targeting models that replace or augment traditional A/B testing.

---

1  Project Overview

Companies rely on experimentation to optimise user engagement, but fixed‑horizon A/B tests are slow and often under‑powered. This project builds a hybrid causal‑ML pipeline that:

1. Ingests raw user‑session logs from a data warehouse.
2. Cleans & feature‑engineers covariates suitable for causal analysis.
3. Estimates heterogeneous treatment effects (CATE/ITE) with state‑of‑the‑art models (Causal Forests, DragonNet, TARNet).
4. Surfaces high‑uplift cohorts via an API for real‑time targeting.
5. Tracks experiments & model lineage with MLflow and Airflow.

Outcome: 9.2 % uplift in CTR in offline replay simulation relative to uniform treatment assignment.

---

2  Architecture

.
├── airflow/                     # Orchestration layer
│   ├── dags/
│   │   ├── etl_pipeline.py      # Nightly feature engineering DAG
│   │   └── train_pipeline.py    # Weekly model‑training DAG
│   └── docker/
│       └── Dockerfile           # Slim Airflow image for CI
├── infra/                       # IaC & deployment configs
│   ├── docker-compose.yml       # Local full‑stack spin‑up
│   └── fly.toml                 # Fly.io production app spec
├── data/
│   ├── raw/                     # Immutable source logs (Parquet)
│   └── processed/               # Feature & label tables
├── notebooks/                   # Exploratory & report notebooks
│   ├── 01_dag_definition.ipynb  # Causal graph & backdoor checks
│   ├── 02_model_compare.ipynb   # Benchmark CATE estimators
│   └── 03_policy_replay.ipynb   # Offline uplift‑policy eval
├── src/                         # Project Python package (installable)
│   ├── common/
│   │   └── config.py            # Path & hyper‑param management
│   ├── features/
│   │   ├── builder.py           # Column generation & encoders
│   │   └── __init__.py
│   ├── models/
│   │   ├── causal_estimator.py  # Wrapper for EconML/DoWhy
│   │   ├── policy_evaluation.py # Doubly‑robust replay metrics
│   │   └── __init__.py
│   └── inference/
│       ├── app.py               # FastAPI entrypoint
│       └── schemas.py           # Pydantic request/response models
├── tests/                       # Pytest suites
│   ├── test_features.py
│   ├── test_models.py
│   └── test_api.py
├── scripts/                     # One‑off utilities (seed, cleanup)
│   ├── bootstrap_db.sh
│   └── seed_data.py
├── .github/
│   └── workflows/
│       ├── ci.yml               # Unit tests, lint, coverage gate
│       └── cd.yml               # Docker build & deploy
├── environment.yml              # Conda spec
├── requirements.txt             # Pip alternative
├── Makefile                     # Common CLI tasks (make train, make test)
├── .pre-commit-config.yaml      # Black, ruff, isort hooks
└── README.md                    # Project documentation root

---

3  Repository Structure

.
├── airflow/               # DAG definitions & configs
├── data/
│   ├── raw/
│   └── processed/
├── notebooks/             # EDA & experiment notebooks
├── src/
│   ├── features/          # Feature builders
│   ├── models/            # Causal estimators & wrappers
│   └── inference/         # Serving code (FastAPI)
├── tests/                 # Unit & integration tests
├── environment.yml        # Conda environment
├── requirements.txt       # pip alternative
└── README.md              # You are here

---

4  Environment Setup

Conda (recommended)

mamba env create -f environment.yml
mamba activate causal-ml

Virtualenv

python -m venv .venv && source .venv/bin/activate
pip install -r requirements.txt

GPU users: follow the PyTorch installation matrix for CUDA wheels.

---

5  Data Sources

Source	Format	Notes
User session logs	Parquet	page_view_id, timestamps, features
Experiment feed	JSON	Treatment/control assignment metadata
Lookup tables	CSV	User demographics, device info

All personally‑identifiable information (PII) must be removed or hashed before leaving secure storage.

---

6  Experiment Workflow

1. Define causal graph in notebooks/01_dag_definition.ipynb using DoWhy.
2. Run ETL: make features or trigger airflow dags trigger etl_pipeline.
3. Model training: make train or wait for the weekly Airflow schedule.
4. Validation: offline policy replay (src/models/policy_evaluation.py).
5. Register candidate to MLflow if metrics exceed thresholds.
6. Deploy via GitHub Actions → Docker Registry → Fly.io / K8s.

---

7  Model Training & Evaluation

Estimator	Library	Metric (AUC)
Causal Forest	econml	0.89
DragonNet	econml + torch	0.91 (★)
S‑Learner (XGB)	causalml	0.84

★ Selected for production.

Hyper‑parameters stored under MLflow run tags; see notebooks/02_model_compare.ipynb for full experiment dashboard.

---

8  Deployment

cd infra/
# Build image
docker build -t causal-inference-api .
# Push & deploy
flyctl deploy -i causal-inference-api

src/inference/app.py exposes /predict (batch) and /score (single user) endpoints; latency ~30 ms p95 on t4‑small.

---

9  CI / CD

• Unit tests: pytest in GitHub Actions, coverage gate 85 %.
• Linting: black, ruff, & pre‑commit hooks.
• Docker publish: tags MLflow model SHA.
• Infra as Code: fly.toml, docker-compose.yml for local stack.

---

10  Contributing

1. Fork & branch from main.
2. Enable pre‑commit: pre-commit install.
3. Run make test before PR.
4. PR must include updated docs and passing CI.

---

11  License

MIT © 2025 Causal‑ML Lab, Williams College.

---

12  References & Further Reading

• Kuang et al., Estimating Individual Treatment Effect: A Causal Representation Learning Approach, ICLR 2024.
• EconML team, Orthogonal Random Forest for Heterogeneous Treatment Effect Estimation, 2023.
• Pearl, Causality: Models, Reasoning, and Inference, 2nd ed.