Skip to content
Draft
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
112 commits
Select commit Hold shift + click to select a range
4ec270d
Update script to create the mock to be consistent with CosmoSIS
sachaguer Mar 3, 2026
273fc06
Update script to compute the two-point stats on mocks
sachaguer Mar 3, 2026
66000cc
Update script to obtain nz txt files
sachaguer Mar 3, 2026
5760bda
Update prior and values files
sachaguer Mar 3, 2026
13bb58e
Update gitignore
sachaguer Mar 3, 2026
9142cd8
Push ini files
sachaguer Mar 3, 2026
e6ba6ee
Update gitignore
sachaguer Mar 3, 2026
a6baa39
Update get_chi2_notebooks
sachaguer Mar 3, 2026
cdc1c72
Update of theory b modes script and cosmo_val
sachaguer Mar 3, 2026
45e1745
Update script to get harmonic space gausian sims
sachaguer Mar 3, 2026
dc6325b
Solve conflicts
sachaguer Mar 3, 2026
7674bef
Update cosmosis_fitting script
sachaguer Mar 3, 2026
1d8e615
Fix typo in the nstep computation
sachaguer Mar 4, 2026
8297930
Update the theory computation in the gaussian sims
sachaguer Mar 4, 2026
b80dca2
Fix bugs in cosmo_val after merge + add neutrinos to the theory compu…
sachaguer Mar 4, 2026
8839683
Add script to perform the postprocessing of polychord chains
sachaguer Mar 11, 2026
9218411
Add plotting scripts for unblinding
sachaguer Mar 12, 2026
3b78d16
Finish the script
sachaguer Mar 13, 2026
16ac551
Add updates of Lisa for the unblinding party
sachaguer Mar 16, 2026
36bf529
Update plotting notebooks and chain postprocessing.
sachaguer Mar 18, 2026
aee9b3e
stub: foundation — merge pending local code into develop
cailmdaley Jun 2, 2026
a367c10
Update paper plots scripts
sachaguer Jun 3, 2026
9f877a9
Update notebooks and file that has been modified
sachaguer Jun 3, 2026
8aa8296
Add the config files for the likelihood in harmonic space
sachaguer Jun 3, 2026
9f7a536
Remove the backup files
sachaguer Jun 3, 2026
c22f075
Merge branch 'stub/foundation-merge' into develop
sachaguer Jun 4, 2026
dd8fc68
Merge sachaguer/develop foundation into cleanup/restructuring
cailmdaley Jun 5, 2026
d14fc07
fix: add mnu to planck_defaults so bare get_cosmo() resolves
cailmdaley Jun 5, 2026
1a8b786
chore: gitignore generated apidoc stubs and uv.lock
cailmdaley Jun 5, 2026
fdd9008
chore: restore glass-mock cosmosis-config ignores dropped in the merge
cailmdaley Jun 5, 2026
e993b9c
test: add back-pressure guard #1 — imports/first-party resolution
cailmdaley Jun 5, 2026
9a85bc0
test: add restructuring back-pressure guards
cailmdaley Jun 5, 2026
2f33b90
test: reduce config path guard baseline
cailmdaley Jun 5, 2026
226c15c
test: make config path guard pass
cailmdaley Jun 6, 2026
4d0a4ca
chore: sync restructuring fibers
cailmdaley Jun 6, 2026
916ef73
test: remove dangling dashboard symlinks
cailmdaley Jun 6, 2026
a961cf2
test: enable bmodes workflow dry run
cailmdaley Jun 6, 2026
01ce1c6
test: clear legacy script import xfail
cailmdaley Jun 6, 2026
c6e0804
refactor: extract bmodes workflow common helpers
cailmdaley Jun 6, 2026
f33d8fb
refactor: extract bmodes twopoint rules
cailmdaley Jun 6, 2026
d114a66
chore: update bmodes split fiber
cailmdaley Jun 6, 2026
313000f
felt: sync fiber store (reorg + root fiber)
cailmdaley Jun 10, 2026
1b8b2f3
fix: resolve twopoint script paths relative to rule file
cailmdaley Jun 10, 2026
3b725d0
chore: drop dead specs.smk (included nowhere; in git history)
cailmdaley Jun 10, 2026
6e8fa59
refactor: slim bmodes Snakefile to pointers (constraints + PLANCK18 i…
cailmdaley Jun 10, 2026
091bba8
refactor: move 2D_bmodes_paper_workflow to papers/bmodes
cailmdaley Jun 10, 2026
aa0d906
refactor: extract generic workflow/ as module behind papers/bmodes
cailmdaley Jun 11, 2026
5a4b926
fix: resolve workflow module path through run-dir symlink (realpath)
cailmdaley Jun 11, 2026
c722cde
refactor: move catalog_paper_plot to papers/catalog
cailmdaley Jun 11, 2026
1ed7e66
refactor: move 2D_harmonic_space_cosmic_shear_plots to papers/harmonic
cailmdaley Jun 11, 2026
2e5c1ed
felt: close bmodes-workflow-split (split realized, net green)
cailmdaley Jun 11, 2026
e1bff38
refactor: promote notebooks/cosmo_val to top-level cosmo_val
cailmdaley Jun 11, 2026
321e419
chore: gitignore — one results/ home, drop dead hand-listed notebook …
cailmdaley Jun 11, 2026
733e469
feat: add tracked scratch/ per-person space
cailmdaley Jun 11, 2026
0c0c294
chore: delete defunct/ (quarantined Oct 2024; recoverable from git hi…
cailmdaley Jun 11, 2026
5ffc801
feat: nbstripout + large-file pre-commit hooks (bloat guard)
cailmdaley Jun 11, 2026
944f557
fix: test_cosmo_val cat_config path follows cosmo_val promotion
cailmdaley Jun 11, 2026
b6d5d4b
felt: phase-2 moves complete — outcomes refreshed (restructuring + gu…
cailmdaley Jun 11, 2026
e185ec0
felt: close back-pressure-suite (guard net realized, caught 2 regress…
cailmdaley Jun 11, 2026
1430155
fix(tests): tracked-file guards fall back to tree walk where .git is …
cailmdaley Jun 11, 2026
d500447
fix(tests): bmodes dry-run guard skips off-cluster, sets PYTHONUNBUFF…
cailmdaley Jun 11, 2026
e5092ff
fix(docs): exclude broken sphinxawesome-theme 6.0.3 wheel
cailmdaley Jun 11, 2026
f5b3083
Merge remote-tracking branch 'origin/develop' into cleanup/restructuring
cailmdaley Jun 11, 2026
87d6600
felt: ci-hermeticity finding (git-less image guards, broken theme whe…
cailmdaley Jun 11, 2026
e3b4779
felt: sync ci-hermeticity frontmatter (serialization round-trip)
cailmdaley Jun 13, 2026
757651d
Merge remote-tracking branch 'origin/develop' into cleanup/restructuring
cailmdaley Jun 13, 2026
6abf405
feat(workflow): decompose CosmologyValidation suite into Snakemake rules
cailmdaley Jun 13, 2026
a6fad45
feat(papers): add cosmo_val paper composing the validation workflow
cailmdaley Jun 13, 2026
51bedde
felt: cosmo-val-workflow — Snakemake decomposition of run_cosmo_val
cailmdaley Jun 13, 2026
c08cf29
test(cosmo_val): synthetic-catalog integration tests for compute seams
cailmdaley Jun 13, 2026
1232efb
feat: fold GLASS-mock generation core into sp_validation.glass_mock
cailmdaley Jun 13, 2026
ab7df6a
merge: integrate feature/cosmo-val-integration-tests into cleanup/res…
cailmdaley Jun 13, 2026
a1b49ad
merge: integrate feature/glass-mock-fold into cleanup/restructuring
cailmdaley Jun 13, 2026
7650742
merge: integrate feature/cosmo-val-workflow into cleanup/restructuring
cailmdaley Jun 13, 2026
57cb9b7
config(cosmo_val): include_pseudo_cl on by default
cailmdaley Jun 13, 2026
6804265
fix(cosmo_val): cv_cosebis_npz must mirror save_cosebis_results path
cailmdaley Jun 13, 2026
9e5c735
build(docker): add glass + cs_util>=0.2.1, install with [test,glass] …
cailmdaley Jun 13, 2026
7618fbc
merge: integrate feature/container-refresh into cleanup/restructuring
cailmdaley Jun 13, 2026
8d80635
test(b_modes): value-drift characterization for E/B estimators
cailmdaley Jun 13, 2026
dfba2e4
test(basic): value-drift characterization for metacal R-matrix + masks
cailmdaley Jun 13, 2026
3749cf6
test(calibration): value-drift characterization for shear m/c calibra…
cailmdaley Jun 13, 2026
a5d5a7e
test(cosmo_val): pure-EB redo — broader integration + value-drift pins
cailmdaley Jun 13, 2026
31d3a82
chore: strip output from all tracked notebooks (nbstripout)
cailmdaley Jun 13, 2026
58dc1d3
test(glass_mock): xfail the GLASS map test pending glass/cosmology AP…
cailmdaley Jun 13, 2026
ce1db1e
feat(workflow): config-drive inference.smk + resolve pseudo-Cℓ schism
cailmdaley Jun 13, 2026
8d8263f
test(cosmosis_fitting): pin CosmoSIS data-FITS assembly
cailmdaley Jun 13, 2026
d75cb2b
chore: untrack regenerable PDFs + glass_mock configs; add stray-outpu…
cailmdaley Jun 13, 2026
8a954af
felt: glass-cosmology-api-pin — glass/cosmology API mismatch in glass…
cailmdaley Jun 13, 2026
957b222
chore(workflow): use canonical code/sp_validation path in common.py
cailmdaley Jun 13, 2026
da6f18c
build(glass): pin glass==2025.1 + glass.ext.camb + cosmology
cailmdaley Jun 13, 2026
0375729
feat(scratch): composed ASTRA project for the UNIONS-3500 release
cailmdaley Jun 13, 2026
e46802b
felt: sync fiber frontmatter (serialization round-trip)
cailmdaley Jun 13, 2026
88f2134
felt(glass-cosmology-api-pin): resolved — pin verified under uv
cailmdaley Jun 13, 2026
665b307
felt: log pseudo-Cl + canonical-set decisions; file parallel reorg th…
claude Jun 16, 2026
cf6f937
docs(reorg): add structural READMEs for restructured top-level dirs
claude Jun 16, 2026
8f2d34b
docs(reorg): document repository layout in README and Sphinx docs
claude Jun 16, 2026
cfbce19
Delete exploratory notebooks and junk from notebooks/
claude Jun 16, 2026
74ff2d4
feat(glass_mock): fold mock post-processing helpers into the library
claude Jun 16, 2026
9b3eeea
refactor(glass_mock): move runner scripts to scripts/glass_mock, drop…
claude Jun 16, 2026
7bf17f3
docs(glass_mock): refresh references to the moved runner location
claude Jun 16, 2026
9fa8cf5
Move reusable notebooks/ scripts to scripts/examples/
claude Jun 16, 2026
7ef39a0
chore: gitignore Claude Code agent worktrees (.claude/worktrees/)
claude Jun 16, 2026
a8d7028
Update references for notebooks/ to scripts/examples/ move
claude Jun 16, 2026
109bb4b
Merge reorg/glass-mock-migration: fold glass_mock into src/ + scripts/
claude Jun 16, 2026
ca1481e
Merge reorg/notebooks-cleanup: delete/convert notebooks to scripts/ex…
claude Jun 16, 2026
82168eb
style: wrap long MOVE_MAP entry in dangling-ref guard (E501)
claude Jun 16, 2026
0881c7c
felt: close notebooks-cleanup, glass-mock-migration, restructuring-docs
claude Jun 16, 2026
853e396
felt: reopen notebooks-cleanup — revisiting blanket .ipynb deletion
claude Jun 16, 2026
17bb75f
docs: add 'Using the weak-lensing catalogues' tutorial page
claude Jun 16, 2026
e3147b5
Convert cosmo_val notebooks: lib-extract + scratch/guerrini
claude Jun 17, 2026
ac193fb
felt: close cosmo-val-notebooks fiber
claude Jun 17, 2026
88ab5e7
chore: remove stray TLDR tool artifacts and gitignore them
claude Jun 17, 2026
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
The table of contents is too big for display.
Diff view
Diff view
  •  
  •  
  •  
Original file line number Diff line number Diff line change
@@ -1,4 +1,5 @@
---
id: 01KTCHX01Z812TKPWGCZNWGHDR
name: 'cat_config: SP_v1.5.4 missing PSF file'
status: closed
tags:
Expand Down
5 changes: 4 additions & 1 deletion .felt/docker-build-uv-venv-base/docker-build-uv-venv-base.md
Original file line number Diff line number Diff line change
@@ -1,10 +1,13 @@
---
id: 01KTCHX02B8APEX03YEC97AW4E
name: 'Docker build: uv-venv base image'
status: closed
tags:
- docker
- ci
created-at: 2026-06-03T10:22:53.245547768+02:00
outcome: 'shapepipe:develop (moving tag) switched to a uv venv at /app/.venv and dropped libtool, breaking the sp_validation image two ways; fixed in Dockerfile by adding autotools + using uv pip. Recurring risk: the base tag moves.'
closed-at: 2026-06-10T17:13:38.572339006+02:00
outcome: 'sp_validation image fixed for the uv-venv shapepipe:develop base: autotools restored via apt for pymaster, uv pip install so packages land in /app/.venv. Standing risk documented: moving base tag can break again — diff base env assumptions before blaming our code.'
---

The `Dockerfile` builds `FROM ghcr.io/cosmostat/shapepipe:develop` — a **moving tag**.
Expand Down
5 changes: 4 additions & 1 deletion .felt/docs-deploy-modernized/docs-deploy-modernized.md
Original file line number Diff line number Diff line change
@@ -1,10 +1,13 @@
---
id: 01KTCHX02BFAABNBRKN0RNN0E3
name: 'Docs deploy modernized: gh-pages CI + Sphinx 8'
status: closed
tags:
- docs
- ci
created-at: 2026-06-04T00:47:23.328929872+02:00
outcome: Replaced the dead master/py3.8/conda cd-build.yml with a deploy-docs job in deploy-image.yml that builds inside the freshly-published container image and pushes API docs to gh-pages on develop. Modernized the docs stack and conf.py for Sphinx 8+/9.
closed-at: 2026-06-10T17:13:38.595662941+02:00
outcome: Dead cd-build.yml replaced by deploy-docs job building inside the freshly-published container image; sphinx/myst modernized; gh-pages deploy on develop. Live.
---

The old `cd-build.yml` was a relic of the pre-uv era: triggered on `master`, set up py3.8 via conda, ran `setup.py test`, and deployed Sphinx API docs to gh-pages. Fully dead since the move to `develop`/uv and the container CI. Removed it; the image build + tests now live in `deploy-image.yml` (see [[docker-build-uv-venv-base]]).
Expand Down
61 changes: 61 additions & 0 deletions .felt/docs-narrative-pages/docs-narrative-pages.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,61 @@
---
id: 01KTCHX02W2H6V4A8S86GDT112
name: 'Docs: real narrative pages, not template stubs'
status: closed
tags:
- constitution
- sp_validation
- docs
created-at: 2026-06-04T16:51:19.13286682+02:00
closed-at: 2026-06-04T21:48:33.143370641+02:00
outcome: 'SHIPPED & LIVE. PR #196 merged to develop (574f4bf); deploy-docs published to gh-pages and cosmostat.github.io/sp_validation now serves the real pages — verified live: landing page shows the writing-pass content (''Four main tasks make up a run''), zero template boilerplate, all four nav sections, about-page philosophizing gone. Delivered: four real narrative pages (writing-pass-polished), existing guides woven into a regrouped toctree, citing stub removed, a new deploy-docs.yml CI (build-on-PR + deploy-on-develop + downloadable docs-html artifact), and docstring RST fixes (78b4d17). Desired state realized. Interactive: session alive for Cail; fiber his to temper/close on the kanban.'
shuttle:
enabled: true
kind: oneshot
interactive: true
host: candide
project_dir: /automnt/n17data/cdaley/unions/pure_eb/code/sp_validation
agent: claude-opus
tempered: true
---

# Docs: real narrative pages, not template stubs

The `sp_validation` developer docs deploy to [cosmostat.github.io/sp_validation](https://cosmostat.github.io/sp_validation/) but read as blank. Every hand-written page is still the unmodified Python-package-template placeholder. The Sphinx *machinery* is fine — this is a **content** job, not a plumbing one.

## Desired State

The deployed docs introduce the package and orient a developer, instead of showing template boilerplate. Concretely, the four stub pages under `docs/source/` carry real, grounded prose:

- **`index.rst`** — a true landing page: what `sp_validation` is (validation of weak-lensing galaxy/star-shape catalogues produced by ShapePipe), the four tasks it performs (shear validation → post-processing → cosmology validation → cosmology inference), and how to navigate the docs. Strip the template `.. note::`, the Admonitions / Code-Blocks demo sections, everything that literally says "you should put some introductory information here."
- **`about.rst`** — the package in its CosmoStat / UNIONS context; the problem it solves; authors and contributors (from the README); contact.
- **`installation.rst`** — the *real* install story: container / Apptainer first (the recommended path — `apptainer build --sandbox sp_validation docker://ghcr.io/cosmostat/sp_validation:develop`), then dev install via `uv pip install -e '.[develop]'`. The package is **not on PyPI** — remove the template's `pip install sp_validation` and PyPI-URL instructions; don't ship a false claim.
- **`quickstart.rst`** — a real first-run walkthrough grounded in the actual entry points: the validation run, `notebooks/params.py` configuration, the `cosmo_inference/pipeline.sh` flags (`--pcf` / `--covmat` / `--inference` / `--mcmc_process`). Not the template's bare "import sp_validation" stub.

The existing *real* docs — `run_validation.md`, `post_processing.md`, `Leakage_object_Tutorial.md` — are woven into the `toc.rst` toctree coherently (they exist, but the narrative around them is missing), and the build no longer warns about stray `.md` files outside any toctree.

The build is **verified clean**, not assumed: build the docs the way CI does and inspect the rendered HTML. Either inside the container (`ghcr.io/cosmostat/sp_validation:develop`, which carries the stack autodoc imports) or via `uv pip install -e '.[docs]'`, run `sphinx-apidoc -t docs/_templates -feTMo docs/source src/sp_validation` then `sphinx-build -b html docs/source docs/_build`, and open / inspect `docs/_build/index.html`. No autodoc import failures leaving API pages empty; the landing page renders real content.

Work on a branch off `develop` (the prior docs PRs used `chore/…`) and open a PR — **don't merge**. Docs redeploy when Cail merges to `develop`; PR-per-change is this repo's grammar.

### Explicitly not

- **Not re-doing the CI / deploy / Sphinx stack** — that landed in [[docs-deploy-modernized]] and works. Don't go hunting for broken wiring: the apidoc, autodoc-with-source-links, theme, and gh-pages deploy all function (verified — the live nav is fully populated and API module pages carry documented members).
- **Not a theme / design overhaul** — keep the `sphinxawesome_theme`.
- **Don't invent science.** Every statement about what the package does is grounded in the README, `CLAUDE.md`, or the code. Where the right *framing* or emphasis needs Cail's taste (what to foreground, scientific nuance, how far the quickstart should walk), stage it and ask in the interactive phase rather than guessing.

## Context

**The reframe.** Cail's read was "we made a PR to hook up the docs but they're super blank — not hooked up correctly." Ground truth (verified 2026-06-04): the wiring is *fully functional*. CI builds inside the published container, `sphinx-apidoc` + autodoc generate populated API pages (e.g. `sp_validation.cosmology`, `.b_modes` render documented members with `[source]` links), the awesome theme renders, gh-pages deploys, and the sidebar nav lists every module and guide. What reads as "blank" is that `index`, `about`, `installation`, `quickstart` are still the **python-package-template placeholders** — so the landing page and Getting Started section have no real content. The job is to write that content, not to fix plumbing.

**Source material is already written** — most of this is porting / adapting, not inventing:

- `README.md` — strong prose overview, the four-task summary, authors / contributors, the container-install walkthrough.
- `CLAUDE.md` — architecture and per-module map (`b_modes`, `calibration`, `cat`, `cosmo_val`, `cosmology`, `rho_tau`, …), the `cosmo_inference/pipeline.sh` flags, container usage.
- The existing real docs (`run_validation.md`, `post_processing.md`, `Leakage_object_Tutorial.md`) — content to weave in, not duplicate.

**Build environment (candide).** This dispatches on candide in the repo (`/automnt/n17data/cdaley/unions/pure_eb/code/sp_validation`). The container is the reliable autodoc env; the `.[docs]` install also works. [[docs-deploy-modernized]] records the `conf.py` PEP-621 metadata traps already fixed, and notes the *remaining* warnings are exactly these content issues (markdown header levels; two stray `.md` not in any toctree).

**Prior art.** [[docs-deploy-modernized]] is the CI / stack PR Cail is thinking of — it hooked up the deploy. [[front-page-makeover]] cleaned the *README* badges (the GitHub front page), which is distinct from the Sphinx landing page.

**Interactive handoff.** Do a strong first pass solo — real `index` / `about` / `installation` / `quickstart` grounded in the material above, build locally, confirm the rendered landing page shows real content — then leave the session alive at a checkpoint for Cail to refine the scientific framing, emphasis, and quickstart narrative before the PR is finalized.
Loading
Loading