Skip to content

AGENTS.md

Latest commit

 

History

History
487 lines (376 loc) · 13.8 KB

AGENTS.md

File metadata and controls

487 lines (376 loc) · 13.8 KB

Autonomous Agent Operations Guide for eo-processor

This document defines how autonomous engineering agents should operate within the eo-processor repository. It covers:

  1. Core mission and constraints
  2. Repository awareness
  3. Workflow patterns
  4. How to add functionality (Rust + Python)
  5. Safety and quality gates
  6. Pre-commit / pre-push checklist (MANDATORY)
  7. Decision trees
  8. Failure handling & rollback
  9. Communication templates

1. Mission

Enhance and maintain a hybrid Rust + Python Earth Observation (EO) processing library that provides high-performance spectral index computations and related utilities.

Agents MUST:

  • Preserve performance and safety guarantees
  • Avoid introducing unnecessary complexity
  • Maintain consistency across both Rust and Python layers
  • Keep documentation and badges up to date

Agents MUST NOT:

  • Commit broken builds
  • Downgrade performance without justification
  • Introduce insecure or unsafe Rust code without explicit rationale
  • Push experimental code directly to main

2. Repository Awareness

Key technologies:

  • Rust core implemented with PyO3 (native extension)
  • Python packaging via maturin
  • Tests: Python (pytest), optional Rust tests (cargo test)
  • Tooling: tox, uv, ruff, mypy, cargo fmt, cargo clippy
  • Coverage: Python coverage XML → coverage-badge.svg (generated by script)
  • Documentation: README.md, QUICKSTART.md, function listings
  • Security posture: memory-safe, no network/file I/O inside core functions

Important files/directories:

  • src/ Rust source
  • python/eo_processor/ Python module exports and type hints
  • tests/ Python test suite
  • scripts/generate_coverage_badge.py Coverage badge generator
  • pyproject.toml Project metadata and dependencies
  • tox.ini Multi-env and coverage configuration
  • Makefile Helper targets

3. Standard Workflow Patterns

A. Add New Computational Function

  1. Define Rust implementation in src/lib.rs
  2. Expose via #[pyfunction] and register in #[pymodule]
  3. Add Python import/export in python/eo_processor/__init__.py
  4. Add stub in python/eo_processor/__init__.pyi
  5. Add tests in tests/test_<feature>.py
  6. Document usage in README.md (Function list + example)
  7. Run verification checklist (see Section 6)

B. Improve Performance

  1. Benchmark existing implementation vs candidate approach (timing with large arrays)
  2. Validate numerical correctness (same output within tolerance)
  3. Avoid unsafe blocks unless strictly necessary
  4. Add comparison note in PR description

C. Refactor

  1. Confirm test coverage touches refactored paths
  2. Preserve public API signatures
  3. If API changes: update docs, bump version appropriately (SemVer logic)

4. Adding Functionality (Rust + Python Integration)

Rust Function Template

#[pyfunction]
fn my_index<'py>(
    py: Python<'py>,
    a: PyReadonlyArray2<f64>,
    b: PyReadonlyArray2<f64>,
) -> PyResult<&'py PyArray2<f64>> {
    let a_arr = a.as_array();
    let b_arr = b.as_array();
    assert_eq!(a_arr.shape(), b_arr.shape(), "Input shapes must match");
    // Compute (a - b) / (a + b + EPSILON)
    let result = a_arr - &b_arr;
    let denom = a_arr + &b_arr + 1e-10;
    let out = &result / &denom;
    Ok(out.into_pyarray(py))
}

Register in Module

#[pymodule]
fn _core(_py: Python, m: &PyModule) -> PyResult<()> {
    // existing functions...
    m.add_function(wrap_pyfunction!(my_index, m)?)?;
    Ok(())
}

Python Export

from ._core import my_index

__all__ = [
    # previous exports...
    "my_index",
]

Type Stub

def my_index(a: NDArray[np.float64], b: NDArray[np.float64]) -> NDArray[np.float64]: ...

Test

def test_my_index():
    import numpy as np
    from eo_processor import my_index
    a = np.array([[0.6, 0.7], [0.8, 0.9]])
    b = np.array([[0.2, 0.3], [0.4, 0.5]])
    out = my_index(a, b)
    assert out.shape == a.shape
    assert np.isfinite(out).all()

Documentation Update

  • Add to “Available Functions” section
  • Provide usage snippet
  • Mention domain relevance if applicable

5. Quality and Safety Principles

  • Deterministic builds and tests
  • Avoid global state
  • Defensive shape checks for mismatched arrays
  • Prefer explicit over implicit conversions
  • Keep public APIs stable
  • Respect performance constraints (large arrays typical)
  • Maintain accessible README examples (runnable)

6. Pre-Commit / Pre-Push Checklist (MANDATORY)

Before ANY commit:

  1. Git status clean of unrelated changes
  2. File encodings UTF-8
  3. No secrets / tokens added
  4. Run: cargo fmt
  5. Run: cargo clippy --all-targets -- -D warnings
  6. Run: cargo test (if Rust tests exist) OR cargo check --lib
  7. Python formatting (if Python code touched):
    • ruff check python/eo_processor (fix if necessary: ruff fix ...)
    • Optional: black / isort if adopted (not yet mandated here)
  8. mypy python/eo_processor (ensure no new type errors)
  9. Run Python tests: tox -e py312 (or full tox)
  10. If coverage impacted:
    • tox -e coverage
    • Regenerate badge: python scripts/generate_coverage_badge.py coverage.xml coverage-badge.svg
    • Confirm SVG renders correctly (open file locally)
  11. Update README.md, stubs, and Sphinx docs (autosummary lists, architecture or functions pages) if public API or performance model changed; rebuild locally with make docs
  12. Increment version in pyproject.toml if semantic change:
    • Patch: bug fixes
    • Minor: new backward-compatible features
    • Major: breaking changes
  13. Re-run uv sync if dependencies changed
  14. Confirm no large binary artifacts staged (git diff --cached)
  15. Commit message format (see Section 8)
  16. Push only after all above pass

Failure to complete ANY step = BLOCKED commit.

7. Decision Trees

A. Should I Bump Version?

  • Added function (backward compatible): Minor
  • Fixed bug without API change: Patch
  • Changed function signature or behavior incompatibly: Major

B. Add vs Extend Existing Function?

  • If logic is a specialization (e.g., normalized difference variant): extend docs, not new function.
  • If formula/domain distinct (new spectral index): new function.

C. Optimize or Keep Simpler?

  • If speedup > 1.2x on 1000x1000 arrays with same correctness → optimize.
  • If complexity increases >2x lines without measurable benefit → reject.

8. Commit Message Conventions

Format:

<type>(scope): short summary

Body: Optional deeper explanation.
Refs: issues, benchmarks, design notes.

Types:
- feat: new functionality
- fix: bug fix
- perf: performance improvement
- docs: docs change only
- chore: maintenance / refactor without feature
- test: adding/improving tests
- build: tooling / packaging updates
- ci: workflow changes

Example:

feat(indices): add soil moisture index function

Implements SMI in Rust with 1D/2D variants. Benchmarked 1.35x faster vs NumPy baseline.
Ref: #42

9. Failure Handling & Rollback

If a step fails:

  • DO NOT force commit.
  • Investigate root cause (e.g., failing test, clippy warning).
  • Add temporary debug logs ONLY in local branch; remove before commit.
  • If performance regression found: revert change or open performance issue with benchmarks.

Rollback Procedure:

  1. git checkout main
  2. git pull origin main
  3. git branch -D <broken-branch> (only if safe)
  4. Recreate branch from fresh main

10. Coverage Badge Regeneration Rules

Regenerate when:

  • Python logic changed
  • Added/removed tests
  • Coverage threshold changed in tox.ini Steps:
  1. Run tox -e coverage
  2. Confirm coverage.xml exists
  3. Run badge script: 
    python scripts/generate_coverage_badge.py coverage.xml coverage-badge.svg
  4. Open badge file locally (ensure text aligned, no clipping)
  5. Stage updated coverage-badge.svg

11. Performance Benchmark Guidance

Minimal benchmarking template:

import numpy as np, time
from eo_processor import ndvi

a = np.random.rand(5000, 5000)
b = np.random.rand(5000, 5000)

t0 = time.time()
r1 = ndvi(a, b)
t1 = time.time() - t0

t0 = time.time()
r2 = (a - b) / (a + b)
t2 = time.time() - t0

print("Rust:", t1, "NumPy:", t2, "Speedup:", t2 / t1)

Acceptable performance claim: ≥1.2x speedup with same numerical result to 1e-9 tolerance.

12. Security Considerations

Agents MUST:

  • Avoid adding network calls or file I/O to core library
  • Avoid dynamic code execution (eval, exec)
  • Preserve numerical safety (avoid silent NaNs)
  • Use local constants (e.g., EPSILON) rather than magic numbers sprinkled

13. Documentation Consistency Rules

When adding features or modifying behavior, keep ALL documentation layers synchronized:

Mandatory updates:

  • README: function table, usage snippet, and (for indices) mathematical formula
  • QUICKSTART: add only if it materially improves newcomer onboarding (avoid bloat)
  • Sphinx docs (docs/):
    • docs/source/api/functions.rst autosummary list (add/remove function names)
    • docs/source/api/architecture.rst if parallel strategy, performance thresholds, or internal design rationale changes
    • Regenerate local HTML via make docs and verify new/changed pages render
  • Python wrapper docstring (in python/eo_processor/__init__.py)
  • Type stub (__init__.pyi) signature coherence with implementation
  • Inline Rust doc comments (///) including formula, assumptions (units, value ranges), and numeric stability notes (EPSILON usage, NaN handling)
  • CHANGELOG entry summarizing addition or change (if project policy requires)

If adding a new spectral or temporal index:

  • Provide canonical formula with symbols explained
  • Reference standard source (paper / NASA spec) if widely recognized
  • Note expected input scaling (e.g., surface reflectance 0–1 vs radiance)
  • Describe valid output range and typical interpretation bands

If performance refactor:

  • Update architecture page with:
    • What changed (e.g. fused loops, new parallel threshold)
    • Rationale (cache locality, reduced temporaries, etc.)
    • Benchmark summary (array shape, old/new timings, speedup)
  • Remove or adjust outdated statements about the old approach

Version / metadata alignment:

  • Ensure __version__ in Python package, pyproject.toml, and any version badge references match
  • If a new function is added (minor bump) or a breaking doc-codified behavior changes (major bump), update version before publishing docs

Read the Docs considerations:

  • If new doc build dependencies introduced, update [project.optional-dependencies].docs and (if required) docs/requirements.txt
  • If environment changes (e.g., need Rust for signature introspection), adjust .readthedocs.yaml accordingly

Validation checklist before commit (docs-related):

  1. Search for stale references to removed/renamed functions
  2. Open generated HTML (docs/build/html/index.html) and navigate to:
    • API functions page
    • Architecture page (if modified)
    • Any new function page
  3. Confirm autosummary generated a page for each new function
  4. Confirm internal cross-links (e.g., :doc:, :ref:) resolve (no broken links in Sphinx build output)
  5. Confirm README and Sphinx descriptions do not diverge materially

Prohibited:

  • Adding large narrative sections unrelated to EO processing scope
  • Leaving placeholder TODOs in published docs
  • Claiming unverified performance improvements

14. Checklist (Compact Form)

# Item Status
1 cargo fmt
2 cargo clippy (no warnings)
3 cargo test / check
4 ruff (no errors)
5 mypy (no new errors)
6 pytest (tox or single env)
7 Coverage (if needed)
8 Badge regenerated (if coverage changed)
9 README / stubs updated
10 Version bump (if required)
11 No secrets / large binaries
12 Commit message per convention
13 Benchmarks (if perf change)
14 Docs match public API

Agents should maintain a local markdown copy and tick boxes before committing.

15. Escalation Criteria

Open an issue (instead of direct fix) when:

  • Requires architectural change (new module structure)
  • Introduces dependency with non-trivial maintenance cost
  • Needs unsafe Rust for performance
  • Drops coverage below threshold in tox.ini

16. Communication Templates

Issue Template (internal):

Title: <short summary>

Context:
<why needed>

Proposal:
<changes>

Impact:
<performance / API / maintenance>

Risks:
<list>

Next Steps:
- [ ] Approve design
- [ ] Implement
- [ ] Test

PR Summary:

## Summary
<overview>

## Changes
- <list>

## Performance
<benchmark results>

## Tests
<added/updated test files>

## Docs
Updated: README, stubs

## Checklist
- [x] fmt
- [x] clippy
- [x] tests pass
- [x] coverage badge

17. Agent Behavior Guidelines

  • Work incrementally (small, testable chunks)
  • Never batch unrelated changes
  • Prefer explicit naming (e.g., normalized_difference_2d)
  • Keep function granularity consistent (1D vs 2D vs generic dispatcher)
  • Avoid dead code (remove unused experimental sections)
  • Treat warnings as errors

18. Dealing With Badge Layout Issues

If SVG renders incorrectly:

  1. Validate file diff (no truncation)
  2. Open standalone in browser
  3. Re-run generation using existing script
  4. Ensure coverage.xml is valid (not empty)
  5. If persistent: log issue + attach SVG + coverage.xml

19. Future Extension Ideas (For Agents to Propose)

  • Add more EO indices (EVI, SAVI, NBR)
  • Multi-band compositing utilities
  • Optional GPU path (evaluate feasibility)
  • Batch processing API for stacked arrays
  • Dask higher-level wrappers (automatic chunk optimization)
  • Rust-based windowed operations (moving statistics)

Final Reminder

DO NOT PUSH if any mandatory checklist item is incomplete. Quality > speed.

Agents must act as responsible maintainers ensuring stability, performance, and clarity.

End of autonomous operations guide.