diff --git a/.github/CODEOWNERS b/.github/CODEOWNERS new file mode 100644 index 0000000..615a0c2 --- /dev/null +++ b/.github/CODEOWNERS @@ -0,0 +1 @@ +* @gwaybio @d33bs diff --git a/assets/css/custom.css b/assets/css/custom.css index 7cba22d..def2687 100644 --- a/assets/css/custom.css +++ b/assets/css/custom.css @@ -48,6 +48,12 @@ html.dark .v2-callout { } html.dark .v2-callout p { color: #93c5fd !important; } html.dark .v2-callout strong { color: #bfdbfe !important; } +html.dark .roadmap-link-callout { + background: #052e16 !important; + border-color: #22c55e !important; +} +html.dark .roadmap-link-callout p { color: #86efac !important; } +html.dark .roadmap-link-callout a { color: #bbf7d0 !important; } html.dark .problem-card { background: #1f2937 !important; border-color: #374151 !important; @@ -138,6 +144,128 @@ html.dark .hero-btn-primary { background: #3b82f6 !important; } html.dark .hero-btn-outline { border-color: #374151 !important; color: #d1d5db !important; } html.dark .hero-secondary-link { color: #6b7280 !important; border-bottom-color: #374151 !important; } +/* Focused development initiative */ +.roadmap-doc { + font-size: 0.95rem; +} + +.roadmap-doc table { + display: block; + font-size: 0.86rem; + overflow-x: auto; + width: 100%; +} + +.roadmap-doc th { + white-space: nowrap; +} + +.roadmap-doc ul { + list-style: disc outside; + margin: 0.75rem 0 1rem; + padding-left: 1.25rem; +} + +.roadmap-doc li { + display: list-item; + padding-left: 0.15rem; +} + +.roadmap-doc li::marker { + color: #16a34a; +} + +.roadmap-doc strong { + color: #111827; +} + +html.dark .roadmap-doc strong { + color: #f3f4f6; +} + +.roadmap-figure { + margin: 2rem auto 2.25rem; + max-width: min(900px, 100%); + width: 100%; +} + +.roadmap-figure img { + border: 1px solid #e5e7eb; + border-radius: 8px; + display: block; + height: auto; + width: 100%; +} + +.roadmap-figure figcaption { + color: #6b7280; + font-size: 0.78rem; + line-height: 1.45; + margin-top: 0.65rem; + text-align: left; +} + +html.dark .roadmap-figure img { + border-color: #374151; +} + +html.dark .roadmap-figure figcaption { + color: #9ca3af; +} + +.focused-summary-grid { + display: grid; + gap: 1rem; +} + +.focused-summary-grid > div { + background: #ffffff; + border: 1px solid #e5e7eb; + border-radius: 8px; + padding: 1rem 1.1rem; +} + +.focused-summary-grid h3 { + font-size: 1rem; + font-weight: 750; + margin: 0.15rem 0 0.5rem; +} + +.focused-summary-grid p, +.focused-summary-grid li, +.roadmap-doc td { + color: #4b5563; + line-height: 1.55; +} + +.focused-kicker { + color: #16a34a !important; + font-size: 0.72rem !important; + font-weight: 800; + letter-spacing: 0.07em; + margin: 0 !important; + text-transform: uppercase; +} + +html.dark .focused-summary-grid > div { + background: #1f2937; + border-color: #374151; +} + +html.dark .focused-summary-grid h3 { + color: #f3f4f6; +} + +html.dark .focused-summary-grid p, +html.dark .focused-summary-grid li, +html.dark .roadmap-doc td { + color: #d1d5db; +} + +html.dark .focused-kicker { + color: #86efac !important; +} + /* Inline code: remove Congo's visible backtick pseudo-elements, style like standard markdown */ .prose :where(code):not(:where([class~="not-prose"],[class~="not-prose"] *))::before, .prose :where(code):not(:where([class~="not-prose"],[class~="not-prose"] *))::after { diff --git a/content/tools/_index.md b/content/tools/_index.md index 55fec00..e53c85b 100644 --- a/content/tools/_index.md +++ b/content/tools/_index.md @@ -5,4 +5,4 @@ description: Production-ready tools for image-based profiling bioinformatics. The Cytomining ecosystem provides a suite of production-ready tools for every stage of the image-based profiling workflow. -For upcoming capabilities and in-progress projects, see the [Roadmap](https://cytomining.github.io/experimental/), which includes our experimental tools under active development. +For upcoming capabilities and in-progress projects, see the [Roadmap](https://cytomining.github.io/experimental/), which includes our experimental tools under active development. For a focused plan across OME-Arrow, single-cell scale efficiency, and AI-ready workflows, see the [focused development initiative](/roadmap/#focused-development-initiative). diff --git a/layouts/experimental/list.html b/layouts/experimental/list.html index 0cf924d..e1e5bf4 100644 --- a/layouts/experimental/list.html +++ b/layouts/experimental/list.html @@ -8,9 +8,16 @@

Roadmap

forming the foundation of the Cytomining roadmap.

+
+

Focused development initiative

+

+ For the detailed implementation plan focused on OME-Arrow, single-cell scale efficiency, and AI-ready workflows, see the AI-ready infrastructure initiative below. +

+
+ {{/* Roadmap pitch */}} -
-

What gaps does our roadmap address?

+
+

What gaps do our experimental tools address?

The current Cytomining stack was designed around 2D single-cell data from CellProfiler. As the field moves toward 3D organoid imaging, single-cell profiling, larger-scale screens, and deep learning feature extraction, @@ -20,7 +27,7 @@

Roadmap

{{/* Tool list */}} -
+

Experimental tools

These tools are under active development in the WayScience organization. @@ -136,6 +143,295 @@

+

Focused development initiative

+

+ This initiative defines the next phase of Cytomining infrastructure work: unified multimodal data, efficient single-cell scale processing, and AI-ready image-based profiling. +

+

+ The work connects three streams: OME-Arrow for unified multimodal data representation, Polars and iceberg-bioimage for scalable single-cell data access, and typed APIs plus agent skills for AI-ready workflows with human-in-the-loop checkpoints. +

+ +
+ Future-state Cytomining AI-ready infrastructure roadmap showing assay and microscopy image capture flowing through OME-Arrow, CytoTable, iceberg-bioimage, Polars, and the downstream quality control, profiling, and analysis layer. +
Future-state Cytomining infrastructure: image capture and feature extraction flow into OME-Arrow representations, CytoTable harmonizes single-cell formats, iceberg-bioimage organizes scalable warehouses, Polars provides a shared dataframe engine, and downstream tools support QC, profiling, and analysis.
+
+ +

Why this work matters

+

+ Cytomining tools already support high-content microscopy workflows, but the surrounding data infrastructure still creates friction for large, single-cell image-based profiling projects. +

+
    +
  • Images, profiles, segmentation geometries, and QC annotations often live in separate files or stores.
    • +
  • Large single-cell datasets are difficult to query lazily across plates, projects, and public reference collections.
    • +
  • Workflow expertise is distributed across documentation, notebooks, and maintainer knowledge rather than encoded as typed APIs and reusable agent guidance.
    • +
+ +

Roadmap summary

+
+
+

Component 1

+

Unified multimodal data infrastructure

+

Integrate OME-Arrow across the Cytomining stack so microscopy images, segmentation geometries, morphology profiles, metadata, and QC annotations share a schema-enforced data model.

+
    +
  • Native OME-Arrow read/write support in CytoTable.
    • +
  • Pycytominer and coSMicQC operation on OME-Arrow datasets.
    • +
  • Documentation, tutorials, and OME community specification work.
    • +
+
+
+

Component 2

+

Single-cell scale efficiency

+

Make Cytomining workflows efficient at population scale through lazy queries, faster dataframe execution, versioned bioimage catalogs, and release-indexed benchmarks.

+
    +
  • Polars integration across Cytomining tools.
    • +
  • Benchmarking across AnnData, Zarr, CSV, SQLite, Parquet, DuckDB, Lance, and related approaches.
    • +
  • iceberg-bioimage support for FAIR, cloud-ready, version-controlled warehouses.
    • +
+
+
+

Component 3

+

AI-ready image-based profiling

+

Make Cytomining workflows executable by both researchers and code-writing agents through typed APIs, reusable workflow knowledge, and explicit human-in-the-loop decision points.

+
    +
  • A versioned Cytomining ecosystem skill.
    • +
  • Agent-readable APIs, docstrings, and schemas across tools.
    • +
  • AI-readiness benchmarks against expert-validated reference workflows.
    • +
+
+
+ +

Component 1 planned work: unified multimodal data infrastructure

+

+ This component removes a major source of manual reconciliation. Researchers should be able to trace a morphology measurement back to the source image and segmentation context without fragile joins across independent files. +

+ + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +
DeliverableImplementation notesTarget
OME-Arrow schema and CytoTable read/write supportAdd native OME-Arrow input and output paths in CytoTable. Verify numerical equivalence against public reference datasets such as JUMP-CP and RxRx19.Year 1
Pycytominer and coSMicQC integrationEnable Pycytominer to operate directly on OME-Arrow datasets and coSMicQC to write QC annotations back into the same store.Year 1
Documentation and tutorialsPublish OME-Arrow tutorials and revise Cytomining notebooks to show end-to-end workflows without format conversion.Year 1
OME community specificationPrepare a formal OME-Arrow specification and participate in OME community review through public issues or pull requests.Year 2
Project migrationMove OME-Arrow from WayScience to Cytomining once it meets ecosystem maturity expectations.Year 2
+

Success indicators

+
    +
  • Fewer workflow steps are required to connect source images with morphology profiles compared with a documented baseline workflow.
    • +
  • End-to-end pipelines run without manual conversion between image, profile, and QC stores.
    • +
  • At least three external groups adopt OME-Arrow in public workflows, publications, or repositories.
    • +
  • OME community discussion shows documented progress toward broader standard recognition.
    • +
+ +

Component 2 planned work: single-cell scale efficiency

+

+ This component lets researchers work with large-scale morphology data as queryable scientific infrastructure rather than local files that must be fully loaded and copied at each step. +

+ + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +
DeliverableImplementation notesTarget
Polars integrationIntegrate Polars across Cytomining tools for ingestion, QC, normalization, feature selection, and related table operations.Year 1
Format and engine benchmarkCompare speed, memory, and storage across AnnData, Zarr, CSV, SQLite, Parquet, DuckDB, Lance, and related approaches.Year 1
Release benchmark dashboardAdd automated benchmark runs for Cytomining releases and publish version-indexed results.Year 1
iceberg-bioimage v1.0Release cloud-agnostic object storage support with FAIR, version-controlled image cataloging.Year 2
Cytomining integration for iceberg-bioimageConnect iceberg-bioimage with CytoTable, coSMicQC, and Pycytominer through a reference pipeline notebook.Year 2
Project migrationMove iceberg-bioimage from WayScience to Cytomining once it reaches production maturity.Year 2
+

Success indicators

+
    +
  • Benchmarks show major speed and memory improvements over pandas-based workflows across core pipeline stages.
    • +
  • At least one population-level reference dataset can be queried lazily without loading the full dataset into memory.
    • +
  • Benchmark results are published for at least two major Cytomining releases.
    • +
  • Benchmark output can identify bottlenecks before releases.
    • +
+ +

Component 3 planned work: AI-ready image-based profiling

+

+ This component lowers the barrier to running image-based profiling while keeping expert biological judgment in the loop. AI systems can help execute workflows, but human review, choices, and decisions remain explicit and recorded. +

+ + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +
DeliverableImplementation notesTarget
Cytomining ecosystem skillPublish a versioned skill with tool-specific sections for pipeline orchestration, QC conventions, normalization strategies, and processing decisions.Year 1
Agent-readable APIsValidate function signatures, docstrings, and schemas across Cytomining tools using agentic coding systems.Year 1
Human-in-the-loop taxonomyDefine biological judgment points, starting with QC threshold selection, and log human decisions into provenance records.Year 1
AI-readiness benchmark suiteEvaluate agent-driven execution with and without the Cytomining skill against expert-validated reference outputs.Year 2
Ecosystem protocol manuscriptDocument the Cytomining ecosystem protocol and supporting technologies in a citable manuscript.Year 2
+

Success indicators

+
    +
  • Agent-driven workflows complete more accurately and faster when using the Cytomining skill than when using baseline documentation alone.
    • +
  • Human decisions at biological judgment points are logged and can be reviewed after workflow execution.
    • +
  • At least two external groups without prior Cytomining expertise complete a profiling workflow using the skill and typed APIs.
    • +
  • Benchmark reports track correctness, time to completion, and divergence between agent recommendations and human choices.
    • +
+ +

Cross-cutting implementation principles

+
    +
  • Composable tools: Each component should improve the shared ecosystem without making any single tool monolithic.
    • +
  • Public validation: Outputs should be checked against public reference datasets where possible.
    • +
  • Provenance by default: Data transformations, QC annotations, human decisions, and agent actions should be recorded.
    • +
  • Cloud-ready formats: Storage and query layers should work across local, HPC, and cloud object-storage environments.
    • +
  • Maturity before migration: New projects should move from WayScience to Cytomining only after they have stable releases, documentation, tests, and active maintenance.
    • +
+ +

Dependencies and sequencing

+ + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +
SequenceDependencyRationale
1Define OME-Arrow schema and CytoTable I/OThe shared representation must exist before downstream tools can rely on it.
2Integrate Pycytominer and coSMicQC with OME-ArrowProcessing and QC need to operate on the same data model to remove conversion steps.
3Add Polars and benchmark infrastructurePerformance changes should be measured before and after integration across tools.
4Release iceberg-bioimage and reference notebooksVersioned image catalogs should connect to the validated Cytomining data flow.
5Publish AI-ready APIs and ecosystem skillAgent guidance should encode the stabilized workflow, not a moving target.
6Run AI-readiness benchmarksBenchmarks should evaluate the integrated workflow against expert references.
+ +

Milestones

+ + + + + + + + + + + + + + + + + +
TimeframePlanned work
Year 1OME-Arrow schema and CytoTable I/O; Pycytominer and coSMicQC OME-Arrow support; Polars integration; release benchmark dashboard; first Cytomining ecosystem skill and API validation.
Year 2OME-Arrow specification and adoption package; iceberg-bioimage v1.0 and Cytomining integration; AI-readiness benchmark suite; ecosystem protocol manuscript; mature project migration into Cytomining where appropriate.
+ +

What done looks like

+
    +
  • Microscopy images, segmentation geometries, morphology profiles, metadata, and QC annotations can be co-located in an OME-Arrow-compatible representation.
    • +
  • Large single-cell datasets can be queried lazily and reproducibly across plates, screens, and releases.
    • +
  • CytoTable, coSMicQC, and Pycytominer expose typed, documented APIs suitable for both researchers and agentic workflows.
    • +
  • Human review at biological judgment points is preserved in provenance records.
    • +
  • Performance and AI-readiness can be compared across releases using public benchmark reports.
    • +
+ +
diff --git a/static/img/ai-ready-infrastructure-roadmap.png b/static/img/ai-ready-infrastructure-roadmap.png new file mode 100644 index 0000000..daa749a Binary files /dev/null and b/static/img/ai-ready-infrastructure-roadmap.png differ