Skip to content

feat: declarative YAML-driven HTTP tool engine for zero-code tool addition #2

@man4ish

Description

@man4ish

Summary

Replace per-tool Python handlers with a single generic HTTP executor engine,
enabling new REST-backed bioinformatics tools to be added by editing
tools.example.yaml only — no Python code required.

Motivation

Currently adding a new tool requires:

  • A new toolserver/tools/<tool>.py handler
  • Manual registration in toolserver/tools/__init__.py
  • A YAML entry in tools.example.yaml

With 100+ bioinformatics REST APIs planned (UniProt, NCBI, STRING, KEGG,
Reactome, GO, Europe PMC, ClinVar, gnomAD, etc.), this per-tool overhead
is unsustainable.

Proposed Changes

omnibioai-toolserver

  • Add toolserver/adapters/http_tool_executor.py
    Generic HTTP executor — resolves URL path params, query params,
    headers, GET/POST (form + JSON), response dot-path mapping, secret masking
  • Update toolserver/tools/__init__.py
    Auto-register all tools with an http: block from YAML via a single loop
    No per-tool Python files needed for HTTP tools

omnibioai-tes (no adapter changes needed)

  • Update configs/tools.example.yaml
    Add 100 declarative tool definitions across categories:
    Sequence, Structure, Pathways, Interactions, Variants, Expression,
    Ontology, Literature, Compounds, Disease, Gene Annotation
  • Update configs/servers.example.yaml
    Register ToolServer with auto-discovery (TES already calls /capabilities)

New YAML Tool Schema

- tool_id: europepmc_search
  display_name: Europe PMC Search
  http:
    method: GET
    url: "https://www.ebi.ac.uk/europepmc/webservices/rest/search"
    params:
      query: "{query}"
      format: json
    timeout: 20
  inputs:
    - name: query
      type: string
      required: true
  response_map:
    total_results: "hitCount"
    results: "resultList.result"

Tool Categories (100 tools total)

Category Count Examples
Sequence & Protein 10 UniProt, NCBI eFetch, Ensembl
Structure 4 PDB, AlphaFold
Pathways 9 KEGG, Reactome, WikiPathways
Interactions 5 STRING, BioGRID, IntAct
Variants & Disease 9 ClinVar, gnomAD, OMIM, GWAS
Ontology 7 GO, OLS, HPO, EFO, DO
Expression 5 GTEx, GEO, Expression Atlas, HPA
Compounds 8 PubChem, ChEMBL, DrugBank, HMDB
Gene Annotation 7 MyGene.info, HGNC, BioMart
Literature 5 PubMed, Europe PMC, Semantic Scholar
Disease & Phenotype 5 DisGeNET, Monarch, Orphanet
RNA & Epigenetics 4 Rfam, miRBase, ENCODE
Taxonomy 3 NCBI Taxonomy, GBIF
Tool Registry 3 bio.tools, FAIRsharing

Acceptance Criteria

  • GET /capabilities on ToolServer advertises all YAML-defined tools
  • New tool added to YAML is live after server restart, zero code changes
  • All 100 tools validated with real API calls
  • Secrets (secret: true) are masked in all logs
  • Existing enrichr_pathway tool continues to work unchanged

Metadata

Metadata

Assignees

Labels

No labels
No labels

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions