Skip to content

feat: add aggregate query support (GROUP BY + aggregate functions)#750

Merged
patroza merged 2 commits into
mainfrom
copilot/add-query-aggregation-support
May 13, 2026
Merged

feat: add aggregate query support (GROUP BY + aggregate functions)#750
patroza merged 2 commits into
mainfrom
copilot/add-query-aggregation-support

Conversation

Copy link
Copy Markdown
Contributor

Copilot AI commented May 13, 2026

Summary

Adds Q.aggregate(schema, aggregateMap) — a new query operator that performs GROUP BY + aggregate functions at the database level (Memory, SQL, Cosmos) instead of fetching all rows and grouping in memory.

Motivation

Replaces patterns like:

const orders = yield* orderRepo.query(Q.where(...), Q.project(...))
const byShipment = new Map()
for (const order of orders) {
  // manual group-by + counting
}

With a single DB-level aggregate query:

const result = yield* orderRepo.query(
  Q.where("shippingDate", "gte", today.toISOString()),
  Q.where("state._tag", "neq", "cancelled"),
  Q.aggregate(
    S.Struct({ cdcCity: CDCCity, shippingDate: S.DateFromString, picks: S.NonNegativeInt, ... }),
    {
      cdcCity: Q.agg.field("cdcAddress.city"),
      shippingDate: Q.agg.field("shippingDate"),
      picks: Q.agg.countWhen((q) => q.pipe(Q.where("state._tag", "in", ["initial", "valid", "picking"]))),
      packs: Q.agg.countWhen((q) => q.pipe(Q.where("state._tag", "eq", "picked"))),
      packed: Q.agg.countWhen((q) => q.pipe(Q.where("state._tag", "eq", "packed"))),
      done: Q.agg.countWhen((q) => q.pipe(Q.where("state._tag", "eq", "closed")))
    }
  )
)

New API

agg DSL namespace:

  • agg.field(path) — group-by field reference (doc path → output alias)
  • agg.count() — COUNT(*)
  • agg.countWhen(operation) — COUNT of rows matching filter
  • agg.sum(field) — SUM of a numeric field
  • agg.min(field) / agg.max(field) — MIN/MAX of a field

aggregate(schema, aggregateMap) — query operator using the above

Changes

File Change
src/Model/query/new-kid-interpreter.ts AggregateIrExpression type, AggregateIrItem, extend Result, handle "aggregate" mode
src/Model/query/dsl.ts AggregateExpression, AggregateMap, agg const, aggregate() function
src/Store/service.ts Extend FilterArgs.select with { key, path } and { key, aggregate } item types
src/Store/Memory.ts In-memory GROUP BY + aggregation in memFilter
src/Store/SQL/query.ts GROUP BY + COUNT(CASE WHEN ...) / SUM / MIN / MAX
src/Store/Cosmos/query.ts GROUP BY + SUM(IIF(...)) for conditional counts
src/Model/Repository/internal/internal.ts "aggregate" mode dispatch — bypass id/_etag augmentation, decode raw rows
test/sql-store.test.ts 4 new aggregate integration tests
test/cosmos-query.test.ts 3 new aggregate query builder tests
test/query.test.ts 3 new memFilter aggregate unit tests

Copilot AI and others added 2 commits May 13, 2026 09:58
- Add AggregateIrExpression type and AggregateIrItem to new-kid-interpreter.ts
- Add AggregateExpression, AggregateMap, agg namespace, aggregate DSL function to dsl.ts
- Extend FilterArgs.select with { key, path } (aliased field) and { key, aggregate } types
- Implement GROUP BY + aggregate in Memory store (memFilter)
- Implement GROUP BY + aggregate in SQL store (buildWhereSQLQuery)
- Implement GROUP BY + SUM(IIF) for Cosmos DB (buildWhereCosmosQuery3)
- Handle mode === "aggregate" in Repository.query (bypass id/_etag augmentation)
- Add tests for sql-store, cosmos-query, and query (memFilter) test suites

Agent-Logs-Url: https://github.com/effect-app/libs/sessions/4a46d705-8acf-4e09-b756-77ff419dca70

Co-authored-by: patroza <42661+patroza@users.noreply.github.com>
@patroza
Copy link
Copy Markdown
Member

patroza commented May 13, 2026

@copilot Is there some overlap now between especially agregrate and projectComputed?

Copy link
Copy Markdown
Contributor Author

Copilot AI commented May 13, 2026

@copilot Is there some overlap now between especially agregrate and projectComputed?

There is some conceptual overlap (both compute values in the query layer), but they solve different shapes: projectComputed is per-row enrichment (1 output row per entity), while aggregate is grouped/collapsed output (GROUP BY, 1 row per group). So projectComputed is for entity projections; aggregate is for reporting/rollups.

@patroza patroza marked this pull request as ready for review May 13, 2026 10:35
@patroza patroza merged commit 1cee480 into main May 13, 2026
5 checks passed
@patroza patroza deleted the copilot/add-query-aggregation-support branch May 13, 2026 10:37
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants