Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
24 commits
Select commit Hold shift + click to select a range
4283200
Replace uid_map with direct parameter references in aliases
AndrewSazonov Apr 2, 2026
eb1202f
Auto-enable constraints on create, add enable/disable API
AndrewSazonov Apr 2, 2026
0a0385c
Implement Project.load() from CIF directory
AndrewSazonov Apr 3, 2026
47e268e
Encode free flags via CIF uncertainty brackets
AndrewSazonov Apr 3, 2026
4db00cf
Update notebooks
AndrewSazonov Apr 3, 2026
0cbfc94
Update serialization process to apply constraints before saving project
AndrewSazonov Apr 3, 2026
87d3669
Move CIF loop truncation from persistence to display methods
AndrewSazonov Apr 3, 2026
9a11882
Add CIF round-trip integration tests for experiments and structures
AndrewSazonov Apr 3, 2026
ca8d736
Add new integration test
AndrewSazonov Apr 3, 2026
21941c9
Move analysis.cif into analysis/ directory
AndrewSazonov Apr 3, 2026
5dffac3
Add destination parameter to extract_data_paths_from_zip
AndrewSazonov Apr 3, 2026
16af6de
Add missing Returns sections to docstrings
AndrewSazonov Apr 3, 2026
4ddd38e
Add sequential fitting infrastructure with CSV output
AndrewSazonov Apr 3, 2026
de0d218
Unify plot_param_series to read from CSV with snapshot fallback
AndrewSazonov Apr 3, 2026
0a6bd3b
Add multiprocessing support to fit_sequential
AndrewSazonov Apr 3, 2026
69accc1
Write results.csv from existing single-fit mode
AndrewSazonov Apr 3, 2026
5a4e760
Add apply_params_from_csv for dataset replay
AndrewSazonov Apr 3, 2026
2d49d16
Prevent spawn re-import of __main__ in parallel fit_sequential
AndrewSazonov Apr 3, 2026
8b209df
Support negative indexing and force recalc in apply_params_from_csv
AndrewSazonov Apr 3, 2026
e88d18d
Add extract_project_from_zip helper function
AndrewSazonov Apr 3, 2026
dcaef62
Refactor extract_project_from_zip call to use zip_path variable
AndrewSazonov Apr 3, 2026
ed22516
Refactor notebook cell IDs and update project loading process
AndrewSazonov Apr 3, 2026
8a30ae1
Remove CSV writing from fit() to fix sequential crash recovery
AndrewSazonov Apr 3, 2026
592a2c5
Fix extract_project_from_zip to find project.cif from zip contents
AndrewSazonov Apr 4, 2026
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
5 changes: 4 additions & 1 deletion .github/copilot-instructions.md
Original file line number Diff line number Diff line change
Expand Up @@ -42,7 +42,8 @@
and UPPER_SNAKE_CASE for constants.
- Use `from __future__ import annotations` in every module.
- Type-annotate all public function signatures.
- Docstrings on all public classes and methods (numpy style).
- Docstrings on all public classes and methods (numpy style). These must
include sections Parameters, Returns and Raises, where applicable.
- Prefer flat over nested, explicit over clever.
- Write straightforward code; do not add defensive checks for unlikely
edge cases.
Expand Down Expand Up @@ -147,6 +148,8 @@
`docs/architecture/architecture.md`.
- After changes, run linting and formatting fixes with `pixi run fix`.
Do not check what was auto-fixed, just accept the fixes and move on.
Then, run linting and formatting checks with `pixi run check` and
address any remaining issues until the code is clean.
- After changes, run unit tests with `pixi run unit-tests`.
- After changes, run integration tests with
`pixi run integration-tests`.
Expand Down
13 changes: 9 additions & 4 deletions docs/architecture/architecture.md
Original file line number Diff line number Diff line change
Expand Up @@ -188,7 +188,7 @@ GuardedBase
└── GenericDescriptorBase # name, value (validated via AttributeSpec), description
├── GenericStringDescriptor # _value_type = DataTypes.STRING
└── GenericNumericDescriptor # _value_type = DataTypes.NUMERIC, + units
└── GenericParameter # + free, uncertainty, fit_min, fit_max, constrained, uid
└── GenericParameter # + free, uncertainty, fit_min, fit_max, constrained
```

CIF-bound concrete classes add a `CifHandler` for serialisation:
Expand Down Expand Up @@ -714,12 +714,13 @@ Projects are saved as a directory of CIF files:
```shell
project_dir/
├── project.cif # ProjectInfo
├── analysis.cif # Analysis settings
├── summary.cif # Summary report
├── structures/
│ └── lbco.cif # One file per structure
└── experiments/
└── hrpt.cif # One file per experiment
├── experiments/
│ └── hrpt.cif # One file per experiment
└── analysis/
└── analysis.cif # Analysis settings
```

### 7.3 Verbosity
Expand Down Expand Up @@ -919,6 +920,10 @@ project.experiments['xray_pdf'].peak_profile_type = 'gaussian-damped-sinc'
- `DatablockItem` = one CIF `data_` block, `DatablockCollection` = set
of blocks.
- `CategoryItem` = one CIF category, `CategoryCollection` = CIF loop.
- **Free-flag encoding**: A parameter's free/fixed status is encoded in
CIF via uncertainty brackets. `3.89` = fixed, `3.89(2)` = free with
esd, `3.89()` = free without esd. There is no separate list of free
parameters; the brackets are the single source of truth.

### 9.2 Immutability of Experiment Type

Expand Down
50 changes: 50 additions & 0 deletions docs/architecture/issues_closed.md
Original file line number Diff line number Diff line change
Expand Up @@ -4,6 +4,56 @@ Issues that have been fully resolved. Kept for historical reference.

---

## Implement `Project.load()`

**Resolution:** implemented `Project.load(dir_path)` as a classmethod
that reads `project.cif`, `structures/*.cif`, `experiments/*.cif`, and
`analysis/analysis.cif` (with fallback to `analysis.cif` at root for
backward compatibility). Reconstructs the full project state including
alias parameter references via `_resolve_alias_references()`.
Integration tests verify save → load → parameter comparison and save →
load → fit → χ² comparison. Also used by `fit_sequential` workers to
reconstruct projects from CIF strings.

---

## Eliminate Dummy `Experiments` Wrapper in Single-Fit Mode

**Resolution:** refactored `Fitter.fit()` and `_residual_function()` to
accept `experiments: list[ExperimentBase]` instead of requiring an
`Experiments` collection. `Analysis.fit()` passes
`experiments_list = [experiment]` in single-fit mode and
`list(experiments.values())` in joint-fit mode. Removed the
`object.__setattr__` hack that forced `_parent` on the dummy wrapper.

---

## Replace UID Map with Direct References and Auto-Apply Constraints

**Resolution:** eliminated `UidMapHandler` and random UID generation
from parameters entirely. Aliases now store a direct object reference to
the parameter (`Alias._param_ref`) instead of a random UID string.
`ConstraintsHandler.apply()` uses the direct reference — no map lookup.
For CIF serialisation, `Alias._param_unique_name` stores the parameter's
deterministic `unique_name`. `_minimizer_uid` now returns
`unique_name.replace('.', '__')` instead of a random string.

Also added `enable()`/`disable()` on `Constraints` with auto-enable on
`create()`, replacing the manual `apply_constraints()` call.
`Analysis._update_categories()` now always syncs handler state from the
current aliases and constraints when `constraints.enabled` is `True`,
eliminating stale-state bugs (former issue #4). `_set_value_constrained`
bypasses validation like `_set_value_from_minimizer` since constraints
run inside the minimiser loop. `Analysis.fit()` calls
`_update_categories()` before collecting free parameters so that
constrained parameters are correctly excluded.

API change: `aliases.create(label=..., param_uid=...uid)` →
`aliases.create(label=..., param=...)`. `apply_constraints()` removed;
`constraints.create()` auto-enables.

---

## Dirty-Flag Guard Was Disabled

**Resolution:** added `_set_value_from_minimizer()` on
Expand Down
95 changes: 15 additions & 80 deletions docs/architecture/issues_open.md
Original file line number Diff line number Diff line change
Expand Up @@ -10,25 +10,6 @@ needed.

---

## 1. 🔴 Implement `Project.load()`

**Type:** Completeness

`save()` serialises all components to CIF files but `load()` is a stub
that raises `NotImplementedError`. Users cannot round-trip a project.

**Why first:** this is the highest-severity gap. Without it the save
functionality is only half useful — CIF files are written but cannot be
read back. Tutorials that demonstrate save/load are blocked.

**Fix:** implement `load()` that reads CIF files from the project
directory and reconstructs structures, experiments, and analysis
settings.

**Depends on:** nothing (standalone).

---

## 2. 🟡 Restore Minimiser Variant Support

**Type:** Feature loss + Design limitation
Expand Down Expand Up @@ -83,31 +64,6 @@ exactly match `project.experiments.names`.

---

## 4. 🔴 Refresh Constraint State Before Automatic Updates and Fitting

**Type:** Correctness

`ConstraintsHandler` is only synchronised from `analysis.aliases` and
`analysis.constraints` when the user explicitly calls
`project.analysis.apply_constraints()`. The normal fit / serialisation
path calls `constraints_handler.apply()` directly, so newly added or
edited aliases and constraints can be ignored until that manual sync
step happens.

**Why high:** this produces silently incorrect results. A user can
define constraints, run a fit, and believe they were applied when the
active singleton still contains stale state from a previous run or no
state at all.

**Fix:** before any automatic constraint application, always refresh the
singleton from the current `Aliases` and `Constraints` collections. The
sync should happen inside `Analysis._update_categories()` or inside the
constraints category itself, not only in a user-facing helper method.

**Depends on:** nothing.

---

## 5. 🟡 Make `Analysis` a `DatablockItem`

**Type:** Consistency
Expand Down Expand Up @@ -150,24 +106,6 @@ effectively fixed after experiment creation.

---

## 7. 🟡 Eliminate Dummy `Experiments` Wrapper in Single-Fit Mode

**Type:** Fragility

Single-fit mode creates a throw-away `Experiments` collection per
experiment, manually forces `_parent` via `object.__setattr__`, and
passes it to `Fitter`. This bypasses `GuardedBase` parent tracking and
is fragile.

**Fix:** make `Fitter.fit()` accept a list of experiment objects (or a
single experiment) instead of requiring an `Experiments` collection. Or
add a `fit_single(experiment)` method.

**Depends on:** nothing, but simpler after issue 5 (Analysis refactor)
clarifies the fitting orchestration.

---

## 8. 🟡 Add Explicit `create()` Signatures on Collections

**Type:** API safety
Expand Down Expand Up @@ -339,21 +277,18 @@ re-derivable default.

## Summary

| # | Issue | Severity | Type |
| --- | ------------------------------------------ | -------- | ----------------------- |
| 1 | Implement `Project.load()` | 🔴 High | Completeness |
| 2 | Restore minimiser variants | 🟡 Med | Feature loss |
| 3 | Rebuild joint-fit weights | 🟡 Med | Fragility |
| 4 | Refresh constraint state before auto-apply | 🔴 High | Correctness |
| 5 | `Analysis` as `DatablockItem` | 🟡 Med | Consistency |
| 6 | Restrict `data_type` switching | 🔴 High | Correctness/Data safety |
| 7 | Eliminate dummy `Experiments` | 🟡 Med | Fragility |
| 8 | Explicit `create()` signatures | 🟡 Med | API safety |
| 9 | Future enum extensions | 🟢 Low | Design |
| 10 | Unify update orchestration | 🟢 Low | Maintainability |
| 11 | Document `_update` contract | 🟢 Low | Maintainability |
| 12 | CIF round-trip integration test | 🟢 Low | Quality |
| 13 | Suppress redundant dirty-flag sets | 🟢 Low | Performance |
| 14 | Finer-grained change tracking | 🟢 Low | Performance |
| 15 | Validate joint-fit weights | 🟡 Med | Correctness |
| 16 | Persist per-experiment `calculator_type` | 🟡 Med | Completeness |
| # | Issue | Severity | Type |
| --- | ---------------------------------------- | -------- | ----------------------- |
| 2 | Restore minimiser variants | 🟡 Med | Feature loss |
| 3 | Rebuild joint-fit weights | 🟡 Med | Fragility |
| 5 | `Analysis` as `DatablockItem` | 🟡 Med | Consistency |
| 6 | Restrict `data_type` switching | 🔴 High | Correctness/Data safety |
| 8 | Explicit `create()` signatures | 🟡 Med | API safety |
| 9 | Future enum extensions | 🟢 Low | Design |
| 10 | Unify update orchestration | 🟢 Low | Maintainability |
| 11 | Document `_update` contract | 🟢 Low | Maintainability |
| 12 | CIF round-trip integration test | 🟢 Low | Quality |
| 13 | Suppress redundant dirty-flag sets | 🟢 Low | Performance |
| 14 | Finer-grained change tracking | 🟢 Low | Performance |
| 15 | Validate joint-fit weights | 🟡 Med | Correctness |
| 16 | Persist per-experiment `calculator_type` | 🟡 Med | Completeness |
Loading
Loading