✨ SCOPE: Selective Context Preference Optimization ✨

Official training and evaluation code for selective trust under external signals

Xian Sun¹ · Wei Chow² · Yingshuo Wang³ · Junhao Liu⁴ · Wei Gao⁵ · Qing Wu⁶ · Lingdong Kong²

¹ Duke University   ·   ² National University of Singapore   ·   ³ UC Berkeley   ·   ⁴ UC Irvine   ·   ⁵ Northeastern University   ·   ⁶ Nanyang Technological University, Singapore

Quick Start · Training · Evaluation · Structure

---

📖 Overview

SCOPE teaches language models when to trust external signals. It mines problems that a base model answers correctly without added context but incorrectly under a plausible misleading signal, then reuses each preference pair across four matched conditions: clean, misleading, correct, and irrelevant.

The optimization remains standard full-completion sigmoid DPO. This repository provides the complete public pipeline for LoRA-DPO training, adapter merging, MIST inference, exact-match scoring, paired bootstrap confidence intervals, and release-data validation.

flowchart LR
    A["MIST-Train<br/>matched preference quartets"] --> B["SCOPE<br/>LoRA + DPO"]
    B --> C["Merged model"]
    C --> D["MIST-Bench<br/>4 matched conditions"]
    D --> E["Accuracy · Overall · SC2W"]

Loading

🚀 Quick Start

1. Install

git clone https://github.com/worldbench/SCOPE.git
cd SCOPE

conda create -n scope python=3.10 -y
conda activate scope
pip install -r requirements.txt

Training and evaluation require CUDA-capable GPUs. Multi-GPU evaluation is controlled by --tensor_parallel_size.

2. Verify the lightweight components

python -m unittest discover -s tests -v

3. Load the public datasets

from datasets import load_dataset

train = load_dataset("worldbench/MIST-Train", split="train")
benchmark = load_dataset("worldbench/MIST-Bench", split="test")

If the datasets are downloaded beside this repository, validate their schema and matched-condition structure:

python scripts/validate_data.py \
  --train_dir ../hf-train \
  --mist_file ../hf-mist/mist_1000.jsonl

🧠 Training

Set the base model once, then launch the paper recipe:

export BASE_MODEL="your-hugging-face-model"

accelerate launch scope/train.py \
  --model_name_or_path "$BASE_MODEL" \
  --dataset worldbench/MIST-Train \
  --output_dir outputs/scope

For a short formatting/data probe before a full run:

accelerate launch scope/train.py \
  --model_name_or_path "$BASE_MODEL" \
  --dataset worldbench/MIST-Train \
  --output_dir outputs/scope_probe \
  --max_items 8 \
  --selftest

Use --train_file ../hf-train/train.jsonl instead of --dataset for local data. Add --disable_thinking when the selected chat template should not enable thinking mode. Resume an interrupted run with --resume_from_checkpoint <CHECKPOINT>.

Default recipe

Setting	Value
Objective	Sigmoid DPO
DPO beta	0.1
Learning rate	5e-6
Schedule / warmup	Cosine / 10%
Steps	300
Maximum length	4096, keep-end truncation
Batch / accumulation	4 / 8
Precision	bf16
LoRA rank / alpha	64 / 128
LoRA targets	Attention and MLP projections
Seed	0

🔗 Merge the LoRA Adapter

python -m scope.merge_lora \
  --base "$BASE_MODEL" \
  --adapter outputs/scope \
  --output outputs/scope_merged

📊 Evaluation

Run MIST with vLLM:

python -m mist.evaluate \
  --model outputs/scope_merged \
  --dataset worldbench/MIST-Bench \
  --output_dir outputs/mist_scope

Use --data_file ../hf-mist/mist_1000.jsonl for a local benchmark. Before a full 1,000-item run, use --smoke_items 10 to exercise inference, parsing, scoring, and report generation end to end.

The evaluator writes:

File	Contents
run_config.json	Reproducibility settings and evaluated row/item counts
generations.jsonl	Raw model responses and token counts
scores.jsonl	Parsed predictions and row-level correctness
item_scores.jsonl	Four-condition item-level outcomes
metrics.json	Machine-readable metrics and confidence intervals
metrics.md	Ready-to-read result table

Reported metrics include accuracy for all four conditions, balanced Overall accuracy, and SC2W ↓: the fraction of clean-correct items that become wrong under misleading context. Confidence intervals use paired item-level bootstrap resampling.

🗂️ Repository Layout

README.md                    # Setup and end-to-end usage guide
requirements.txt            # Runtime dependencies
scope/
├── train.py              # Full-completion DPO + LoRA
└── merge_lora.py         # Merge an adapter into its base model
mist/
├── evaluate.py           # vLLM inference and report generation
├── answer_utils.py       # MC, numeric, and boolean parsing
└── metrics.py            # Accuracy, SC2W, and bootstrap CIs
scripts/
└── validate_data.py      # Public dataset integrity checks
tests/
├── test_answer_utils.py  # Parser and answer-matching tests
└── test_metrics.py       # Metric and bootstrap tests

📚 Citation

@article{scope2026,
  title   = {Learning When to Trust via Selective Context Preference Optimization},
  author  = {Sun, Xian and Chow, Wei and Wang, Yingshuo and Liu, Junhao and Gao, Wei and Wu, Qing and Kong, Lingdong},
  journal = {arXiv preprint arXiv:XXXX.XXXXX},
  year    = {2026}
}