Skip to content

Add result-flow paging#25

Draft
carldebilly wants to merge 21 commits intomainfrom
dev/cdb/result-flow-paging
Draft

Add result-flow paging#25
carldebilly wants to merge 21 commits intomainfrom
dev/cdb/result-flow-paging

Conversation

@carldebilly
Copy link
Copy Markdown
Member

Summary

Draft PR for the new Result Flow paging experience.

This adds first-class paging support for commands that return large result sets. Handlers can now return one explicit ReplPage<T> or an IReplPageSource<T> that lets Repl fetch later pages on demand. Human users can continue through results in the same command run, while MCP and machine outputs get structured page metadata.

Functional Highlights

Page results

Handlers can return a page directly when they own the cursor contract:

app.Map("contacts", async (IReplPagingContext paging, ContactStore store, CancellationToken ct) =>
{
    var result = await store.SearchAsync(paging.Cursor, paging.SuggestedPageSize, ct);

    return paging.Page(
        result.Items,
        nextCursor: result.NextCursor,
        totalCount: result.TotalCount);
});

JSON output now uses an explicit page envelope:

{
  "$type": "page",
  "items": [
    { "id": 1, "name": "Alice" }
  ],
  "pageInfo": {
    "cursor": null,
    "nextCursor": "page-2",
    "totalCount": 42,
    "pageSize": 1,
    "hasMore": true
  }
}

Page sources

Handlers can return a lazy source when Repl should fetch future pages interactively:

app.Map("events", (EventStore store) =>
    ReplPageSource.FromOffset<EventRow>(
        (offset, take, ct) => store.QueryAsync(offset, take, ct),
        totalCount: store.TotalCount));

This improves terminal UX: the integrated pager can continue fetching data without asking users to rerun the command with --result:cursor.

Built-in helpers

This PR adds helpers for:

  • bounded in-memory lists via FromItems(...);
  • offset/limit stores via FromOffset(...);
  • replayable async streams via FromAsyncEnumerable(...);
  • state-based overloads for static lambdas;
  • client-side filters as a fallback when the source cannot filter server-side.

Example with state and a client-side filter:

app.Map("contacts", (ContactStore store, string search) =>
    ReplPageSource.FromOffset<ContactRow, ContactStore>(
        store,
        static (state, offset, take, ct) => state.QueryAsync(offset, take, ct),
        filter: (_, row) => row.Name.Contains(search, StringComparison.OrdinalIgnoreCase)));

Client-side filtering is documented as a fallback because it may fetch and discard rows. Server-side filtering remains preferred.

Result Flow Help

Paged handlers now show a Result Flow help section:

--result:page-size <n>
--result:cursor <value>
--result:all
--result:pager=auto|off|more|scroll|external

The section appears only for handlers that support paging: IReplPagingContext, ReplPage<T>, or IReplPageSource<T>.

Integrated Pager

The human pager now has two paths:

  • more fallback for limited terminals, with no cursor movement or ANSI redraw;
  • scroll viewport for ANSI terminals, inspired by less.

The scroll pager enters the alternate screen, keeps an internal line buffer, redraws a viewport explicitly, supports up/down/page navigation, and fetches additional page-source payloads as the user reaches the buffered end.

MCP Behavior

MCP tools expose reserved paging inputs:

  • _replCursor
  • _replPageSize

Paged results are returned as structured content with $type: "page". Text summaries stay short and avoid echoing raw cursor values:

Returned 1 item(s). Total: 42. Continue with _replCursor; cursor available in structured content.

Hardening included:

  • page detection uses $type: "page" instead of items + pageInfo heuristics;
  • raw cursor values are not interpolated into text summaries;
  • _replCursor is validated before becoming CLI tokens;
  • _replPageSize must be compact and numeric before normal clamping.

Documentation And Samples

Updated docs cover:

  • cursor basics;
  • page-source helpers;
  • offset, keyset, external token, nextLink, and snapshot cursor patterns;
  • async enumerable cancellation and replayability requirements;
  • client-side filtering caveats;
  • MCP and Spectre testing notes;
  • unknown totalCount scenarios;
  • why live/infinite feeds are a separate future use case.

Core Basics and Spectre samples now demonstrate page-source driven activity feeds.

Safety Notes

  • FromAsyncEnumerable requires a replayable, idempotent, deterministic factory.
  • FromOffset and FromAsyncEnumerable reject --result:all by default because they may be unbounded.
  • FromItems can honor --result:all because the source is already bounded.
  • The scroll pager buffer is session-local; bounded eviction/ring-buffer behavior can be added separately if needed.

carldebilly added 19 commits May 4, 2026 12:37
- Map Space/PageDown/F explicitly to page-forward; Enter maps to
  line-forward (same as DownArrow); unknown keys are no-ops instead
  of silently advancing the viewport
- Add regression tests: unrecognized key does not advance or fetch,
  Enter advances by one line only
- Document O(offset/page) cost on FromAsyncEnumerable overloads and
  point callers toward the Create overload for large/expensive sources
- Replace char-by-char line splitter with EnumerateLines + trailing-
  empty trim; simpler, handles all newline conventions natively
- Extract PagerState.Lines backing field so Reset does not expose a
  setter on the property
- Pre-allocate emptyRow in MarkdownOutputTransformer table renderer
  to avoid one allocation per null item
Comment thread samples/01-core-basics/ActivityFeed.cs Fixed
Comment thread samples/07-spectre/ActivityFeed.cs Fixed
Comment thread src/Repl.Tests/Given_ResultFlowPager.cs Fixed
Comment thread src/Repl.Tests/Given_ResultFlowPager.cs Fixed
Comment thread src/Repl.Tests/Given_ResultFlowPager.cs Fixed
Comment thread src/Repl.Tests/Given_ResultFlowPager.cs Fixed
Comment thread src/Repl.Tests/Given_ResultFlowPager.cs Fixed
Comment thread src/Repl.Tests/Given_ResultFlowPager.cs Fixed
Comment thread src/Repl.Tests/Given_ResultFlowPager.cs Fixed
Comment thread src/Repl.Tests/Given_ResultFlowPager.cs Fixed
Copy link
Copy Markdown

@devin-ai-integration devin-ai-integration Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

✅ Devin Review: No Issues Found

Devin Review analyzed this PR and found no potential bugs to report.

View in Devin Review to see 6 additional findings.

Open in Devin Review

Comment thread src/Repl.McpTests/Given_McpServerEndToEnd.cs Fixed
Comment thread src/Repl.Core/CoreReplApp.Execution.cs Fixed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant