Skip to content

chore: Update from upstream#8

Merged
Chriztiaan merged 427 commits into
mainfrom
upstream/update-main-4
Jun 11, 2026
Merged

chore: Update from upstream#8
Chriztiaan merged 427 commits into
mainfrom
upstream/update-main-4

Conversation

@Chriztiaan

Copy link
Copy Markdown

This updates the main branch from upstream. This only includes changes from upstream. We don't have any custom code in this fork's main branch. The diff is only for upstream code.

KyleAMathews and others added 30 commits December 16, 2025 17:06
* Don't pin @electric-sql/client version

* Add changeset

* update lock file

* Fix sherif linter errors for dependency version mismatches

Standardizes @electric-sql/client to use ^1.2.0 across react-db, solid-db, and vue-db packages, and updates @tanstack/query-db-collection to ^1.0.8 in todo examples.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>

---------

Co-authored-by: Claude Sonnet 4.5 <noreply@anthropic.com>
Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
This PR makes it possible to delete a row by key when using the write function passed to a collection's sync function.
…tack#942)

Builds on top of Electric's ts-client support for tagging rows and move out events: electric-sql/electric#3497

This PR extends tanstack DB such that it handles tagged rows and move out events. A tagged row is removed from the Electric collection when its tag set becomes empty. Note that rows only have tags when the shape they belong to has one or more subqueries.
…TanStack#1044)

* fix: emit delete events when subscribing with includeInitialState: false

When subscribing to a collection with includeInitialState: false,
delete events were being filtered out because the sentKeys set was
empty. This affected live queries with limit/offset where users
would subscribe to get future changes after already loading
initial data via preload() or values().

Changes:
- Add skipFiltering flag separate from loadedInitialState to allow
  filtering to be skipped while still allowing requestSnapshot to work
- Call markAllStateAsSeen() when includeInitialState is explicitly false
- Change internal subscriptions to not pass includeInitialState: false
  explicitly, so they can be distinguished from user subscriptions
- Add tests for optimistic delete behavior with limit

Fixes the issue where deleted items would not disappear from live
queries when using .limit() and subscribing with includeInitialState: false.

* debug: add extensive logging to track delete event flow

This is a DEBUG BUILD with [TanStack-DB-DEBUG] logs to help track down
why delete events may not be reaching subscribers when using limit/offset.

The debug logs cover:
- subscribeChanges: when subscriptions are created
- emitEvents: when events are emitted to subscriptions
- Subscription.emitEvents: when individual subscriptions receive events
- filterAndFlipChanges: when events are filtered or passed through
- recomputeOptimisticState: when optimistic state is recomputed and events emitted
- sendChangesToPipeline: when changes flow through the D2 pipeline
- applyChanges: when D2 pipeline outputs to the live query collection

To use: Filter browser console for "[TanStack-DB-DEBUG]"

Also includes the fix for includeInitialState: false not emitting deletes.

* ci: apply automated fixes

* debug: add more logging to track delete event flow in live queries

Add comprehensive debug logging to:
- createFilteredCallback in change-events.ts for whereExpression filtering
- sendChangesToInput for D2 pipeline input
- subscribeToOrderedChanges for orderBy/limit path
- splitUpdates for update event handling
- recomputeOptimisticState for pending sync key filtering

This additional logging helps track where delete events may be filtered
out when using live queries with limit/offset and where clauses.

* ci: apply automated fixes

* debug: add logging to graph scheduling and execution

Add debug logging to track:
- scheduleGraphRun: when graph run is scheduled
- executeGraphRun: when graph run executes or returns early
- maybeRunGraph: when graph actually runs, pending work status

This helps diagnose issues where deletes are sent to D2 pipeline
but never appear in the output (applyChanges not called).

* ci: apply automated fixes

* debug: add detailed logging to D2 reduce and topK operators

Add debug logging to track:
- ReduceOperator: input processing, key handling, and result output
- topK: consolidation, sorting, slicing, and result details

Also add two new test cases:
1. Test delete from different page (page 1 delete while viewing page 2)
   - Verifies items shift correctly when delete occurs on earlier page
2. Test delete beyond TopK window (no-op case)
   - Verifies deleting item outside window doesn't affect results

These tests and debug logs will help diagnose issues where deleted
items don't disappear from live queries when using limit/offset.

* ci: apply automated fixes

* debug: add more detailed logging to D2 graph and subscription

Add additional debug logging to help diagnose delete issues:

D2 graph (d2.ts):
- Log when run() starts and completes with step count
- Log pendingWork() results with operator IDs
- Log when operators have pending work in step()

Output operator (output.ts):
- Log when run is called with message count
- Log items in each message being processed

Subscription (subscription.ts):
- Log trackSentKeys with keys being added
- Show total sentKeys count

This should help diagnose scenarios where delete events are sent
to D2 but no applyChanges output is produced.

* ci: apply automated fixes

* debug: add operator type logging to trace D2 pipeline

Add operatorType property to Operator base class and log it when
operators run. This will help identify which operators are processing
the delete and where the data is being lost.

Also add detailed logging to LinearUnaryOperator.run() to show:
- Input message count
- Input/output item counts
- Sample of input and output items

This should reveal exactly which operator is dropping the delete.

* debug: add logging to TopKWithFractionalIndexOperator

This is the key operator for orderBy+limit queries. Add detailed logging to:
- run(): Show message count and index size
- processElement(): Show key, multiplicity changes, and action (INSERT/DELETE/NO_CHANGE)
- processElement result: Show moveIn/moveOut keys

This should reveal exactly why deletes aren't producing output changes
when the item exists in the TopK index.

* ci: apply automated fixes

* fix: filter duplicate inserts in subscription to prevent D2 multiplicity issues

When an item is inserted multiple times without a delete in between,
D2 multiplicity goes above 1. Then when a single delete arrives,
multiplicity goes from 2 to 1 (not 0), so TopK doesn't emit a DELETE event.

This fix:
1. Filters out duplicate inserts in filterAndFlipChanges when key already in sentKeys
2. Removes keys from sentKeys on delete in both filterAndFlipChanges and trackSentKeys
3. Updates test expectation to reflect correct behavior (2 events instead of 3)

Root cause: Multiple subscriptions or sync mechanisms could send duplicate
insert events for the same key, causing D2 to track multiplicity > 1.

* fix: add D2 input level deduplication to prevent multiplicity > 1

The previous fix in CollectionSubscription.filterAndFlipChanges was only
catching duplicates at the subscription level. But each live query has its
own CollectionSubscriber with its own D2 pipeline.

This fix adds a sentToD2Keys set in CollectionSubscriber to track which
keys have been sent to the D2 input, preventing duplicate inserts at the
D2 level regardless of which code path triggers them.

Also clears the tracking on truncate events.

* docs: add detailed changeset for delete fix

* ci: apply automated fixes

* chore: remove debug logging from D2 pipeline and subscription code

Remove all TanStack-DB-DEBUG console statements that were added during
investigation of the deleted items not disappearing from live queries bug.

The fix for duplicate D2 inserts is preserved - just removing the verbose
debug output now that the issue is resolved.

* debug: add logging to trace source of duplicate D2 inserts

Add targeted debug logging to understand where duplicate inserts originate:

1. recomputeOptimisticState: Track what events are generated and when
2. CollectionSubscription.filterAndFlipChanges: Trace filtering decisions
3. CollectionSubscriber.sendChangesToPipeline: Track D2-level deduplication

This will help determine if duplicates come from:
- Multiple calls to recomputeOptimisticState for the same key
- Overlap between initial snapshot and change events
- Multiple code paths feeding the D2 pipeline

* ci: apply automated fixes

* fix: prevent race condition in snapshot loading by adding keys to sentKeys before callback

The race condition occurred because snapshot methods (requestSnapshot, requestLimitedSnapshot)
added keys to sentKeys AFTER calling the callback, while filterAndFlipChanges added keys
BEFORE. If a change event arrived during callback execution, it would not see the keys
in sentKeys yet, allowing duplicate inserts.

Changes:
- Add keys to sentKeys BEFORE calling callback in requestSnapshot and requestLimitedSnapshot
- Remove redundant D2-level deduplication (sentToD2Keys) - subscription-level filtering is sufficient
- Remove debug logging added during investigation

* docs: update changeset to reflect race condition fix

* cleanup

* simplify changeset

---------

Co-authored-by: Claude <noreply@anthropic.com>
Co-authored-by: autofix-ci[bot] <114827586+autofix-ci[bot]@users.noreply.github.com>
Co-authored-by: Sam Willis <sam.willis@gmail.com>
Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
…ync mode (TanStack#1043)

* fix(db): re-request subsets after truncate for on-demand sync mode

When a must-refetch (409) occurs in on-demand sync mode, the collection
receives a truncate which clears all data and resets the loadSubset
deduplication state. However, subscriptions were not re-requesting
their previously loaded subsets, leaving the collection empty.

This fix adds a truncate event listener to CollectionSubscription that:
1. Resets pagination/snapshot tracking state (but NOT sentKeys)
2. Re-requests all previously loaded subsets

We intentionally keep sentKeys intact because the truncate event is
emitted BEFORE delete events are sent to subscribers. If we cleared
sentKeys, delete events would be filtered by filterAndFlipChanges.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

* feat: Buffer subscription changes during truncate

This change buffers subscription changes during a truncate event until all
loadSubset refetches complete. This prevents a flash of missing content
between deletes and new inserts.

Co-authored-by: sam.willis <sam.willis@gmail.com>

* ci: apply automated fixes

* changeset

* tweaks

* test

* address review

* ci: apply automated fixes

---------

Co-authored-by: Igor Barakaiev <ibarakaiev@gmail.com>
Co-authored-by: Claude Opus 4.5 <noreply@anthropic.com>
Co-authored-by: Cursor Agent <cursoragent@cursor.com>
Co-authored-by: autofix-ci[bot] <114827586+autofix-ci[bot]@users.noreply.github.com>
Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
…ies (TanStack#1054)

* add test

* fix

* changeset

* fix versions

* ci: apply automated fixes

---------

Co-authored-by: autofix-ci[bot] <114827586+autofix-ci[bot]@users.noreply.github.com>
Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
* fix(electric): preserve message buffer across batches for awaitMatch

The buffer was being cleared at the start of each new batch, which caused
messages to be lost when multiple batches arrived before awaitMatch was
called. This led to:
- awaitMatch timing out (~3-5s per attempt)
- Transaction rollbacks when the timeout threw an error

The fix removes the buffer clearing between batches. Messages are now
preserved until the buffer reaches MAX_BATCH_MESSAGES (1000), at which
point the oldest messages are dropped. This ensures awaitMatch can find
messages even when heartbeat batches or other sync activity arrives
before the API call completes.

Added test case for the specific race condition: multiple batches
(including heartbeats) arriving while onInsert's API call is in progress.

* chore: align query-db-collection versions in examples

Update todo examples to use ^1.0.8 to match other examples and fix
sherif version consistency check.

* chore: align example dependency versions

Update todo and paced-mutations-demo examples to use consistent versions:
- @tanstack/query-db-collection: ^1.0.11
- @tanstack/react-db: ^0.1.59

---------

Co-authored-by: Claude <noreply@anthropic.com>
Add changeset for PR TanStack#1029 (awaitMatch performance fix)

Co-authored-by: Claude <noreply@anthropic.com>
Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
…ing order on partial page (TanStack#970)

* test(react-db): add tests for deletion from partial page in infinite query

Add tests for deleting items from a page with fewer rows than pageSize
in useLiveInfiniteQuery. Tests both ascending and descending order to
verify the behavior difference.

* fix: correctly extract indexed value in requestLimitedSnapshot

The `biggestObservedValue` variable was incorrectly set to the full row object
instead of the indexed value (e.g., salary). This caused the BTree comparison
in `index.take()` to fail, resulting in the same data being loaded multiple
times. Each item would be inserted with a multiplicity > 1, and when deleted,
the multiplicity would decrement but not reach 0, so the item would remain
visible in the TopK.

This fix creates a value extractor from the orderBy expression and uses it to
extract the actual indexed value from each row before tracking it as the
"biggest observed value".

* chore: add changeset for DESC deletion fix

* Main branch merge conflict (TanStack#1066)

* fix(query-db-collection): use deep equality for object field comparison (TanStack#967)

* test: add test for object field update rollback issue

Add test that verifies object field updates with refetch: false
don't rollback to previous values after server response.

* fix: use deep equality for object field comparison in query observer

Replace shallow equality (===) with deep equality when comparing items
in the query observer callback. This fixes an issue where updating
object fields with refetch: false would cause rollback to previous
values every other update.

* chore: add changeset for object field update rollback fix

* ci: Version Packages (TanStack#961)

Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>

* docs: regenerate API documentation (TanStack#969)

Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>

* ci: run prettier autofix action (TanStack#972)

* ci: prettier auto-fix

* Sync prettier config with other TanStack projects

* Fix lockfile

* docs: correct local relative links (TanStack#973)

* fix(db-ivm): use row keys for stable ORDER BY tie-breaking (TanStack#957)

* fix(db-ivm): use row keys for stable ORDER BY tie-breaking

Replace hash-based object ID tie-breaking with direct key comparison
for deterministic ordering when ORDER BY values are equal.

- Use row key directly as tie-breaker (always string | number, unique per row)
- Remove globalObjectIdGenerator dependency
- Simplify TaggedValue from [K, V, Tag] to [K, T] tuple
- Clean up helper functions (tagValue, getKey, getVal, getTag)

This ensures stable, deterministic ordering across page reloads and
eliminates potential hash collisions.

* ci: apply automated fixes

---------

Co-authored-by: autofix-ci[bot] <114827586+autofix-ci[bot]@users.noreply.github.com>

* fix(db): ensure deterministic iteration order for collections and indexes (TanStack#958)

* fix(db): ensure deterministic iteration order for collections and indexes

- SortedMap: add key-based tie-breaking for deterministic ordering
- SortedMap: optimize to skip value comparison when no comparator provided
- BTreeIndex: sort keys within same indexed value for deterministic order
- BTreeIndex: add fast paths for empty/single-key sets
- CollectionStateManager: always use SortedMap for deterministic iteration
- Extract compareKeys utility to utils/comparison.ts
- Add comprehensive tests for deterministic ordering behavior

* ci: apply automated fixes

---------

Co-authored-by: autofix-ci[bot] <114827586+autofix-ci[bot]@users.noreply.github.com>

* fix(db) loadSubset when orderby has multiple columns (TanStack#926)

* fix(db) loadSubset when orderby has multiple columns

failing test for multiple orderby and loadsubset

push down multiple orderby predicates to load subset

split order by cursor predicate build into two, inprecise wider band for local lading, precise for the sync loadSubset

new e2e tests for composite orderby and pagination

changeset

when doing gt/lt comparisons to a bool cast to string

fix: use non-boolean columns in multi-column orderBy e2e tests

Electric/PostgreSQL doesn't support comparison operators (<, >, <=, >=)
on boolean types. Changed tests to use age (number) and name (string)
columns instead of isActive (boolean) to avoid this limitation.

The core multi-column orderBy functionality still works correctly -
this is just a test adjustment to work within Electric's SQL parser
constraints.

* ci: apply automated fixes

---------

Co-authored-by: autofix-ci[bot] <114827586+autofix-ci[bot]@users.noreply.github.com>

* ci: sync changes from other projects (TanStack#978)

* ci: fix vitest/lint, fix package.json

* ci: apply automated fixes

* Remove ts-expect-error

---------

Co-authored-by: autofix-ci[bot] <114827586+autofix-ci[bot]@users.noreply.github.com>

* ci: more changes from other projects (TanStack#980)

* Revert husky removal (TanStack#981)

* revert: restore husky pre-commit hook removed in TanStack#980

This restores the husky pre-commit hook configuration that was removed
in PR TanStack#980. The hook runs lint-staged on staged .ts and .tsx files.

* chore: update pnpm-lock.yaml and fix husky pre-commit format

Update lockfile with husky and lint-staged dependencies.
Update pre-commit hook to modern husky v9 format.

---------

Co-authored-by: Claude <noreply@anthropic.com>

* Restore only publishing docs on release (TanStack#982)

* ci: revert doc publishing workflow changes from TanStack#980

Restore the doc publishing workflow that was removed in PR TanStack#980:
- Restore docs-sync.yml for daily auto-generated docs
- Restore doc generation steps in release.yml after package publish
- Restore NPM_TOKEN environment variable

* ci: restore doc generation on release only

Restore doc generation steps in release.yml that run after package
publish. Remove the daily docs-sync.yml workflow - docs should only
be regenerated when we release.

---------

Co-authored-by: Claude <noreply@anthropic.com>

* ci: sync package versions (TanStack#984)

* chore(deps): update dependency @angular/common to v19.2.16 [security] (TanStack#919)

Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>

* chore(deps): update all non-major dependencies (TanStack#986)

Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>

* Fix build example site auth secret error (TanStack#988)

fix(ci): add BETTER_AUTH_SECRET for projects example build

Copy .env.example to .env during CI build to provide required
BETTER_AUTH_SECRET that better-auth now enforces.

Co-authored-by: Claude <noreply@anthropic.com>

* Unit tests for data equality comparison (TanStack#992)

* Add @tanstack/config dev dependency

* Tests for eq comparison of Date objects

* ci: apply automated fixes

---------

Co-authored-by: autofix-ci[bot] <114827586+autofix-ci[bot]@users.noreply.github.com>

* Handle invalid collection getKey return values (TanStack#1008)

* Add runtime validation for collection getKey return values

Throws InvalidKeyError when getKey returns values other than string or
number (e.g., null, objects, booleans). The validation is optimized to
only require 1 typeof check on the happy path (string keys).

- Add InvalidKeyError class to errors.ts
- Update generateGlobalKey to validate key type before generating key
- Add tests for invalid key types (null, object, boolean)
- Add tests confirming valid keys (string, number, empty string, zero)

* ci: apply automated fixes

---------

Co-authored-by: Claude <noreply@anthropic.com>
Co-authored-by: autofix-ci[bot] <114827586+autofix-ci[bot]@users.noreply.github.com>

* Remove doc regeneration from autofix action (TanStack#1009)

ci: remove doc regeneration from autofix workflow

Docs are regenerated as part of releases, not on every PR/push.

Co-authored-by: Claude <noreply@anthropic.com>

* feat(db,electric,query): separate cursor expressions for flexible pagination (TanStack#960)

* feat(db,electric,query): separate cursor expressions from where clause in loadSubset

- Add CursorExpressions type with whereFrom, whereCurrent, and lastKey
- LoadSubsetOptions.where no longer includes cursor - passed separately via cursor property
- Add offset to LoadSubsetOptions for offset-based pagination support
- Electric sync layer makes two parallel requestSnapshot calls when cursor present
- Query collection serialization includes offset for query key generation

This allows sync layers to choose between cursor-based or offset-based pagination,
and Electric can efficiently handle tie-breaking with targeted requests.

test(react-db): update useLiveInfiniteQuery test mock to handle cursor expressions

The test mock's loadSubset handler now handles the new cursor property
in LoadSubsetOptions by combining whereCurrent (ties) and whereFrom (next page)
data, deduplicating by id, and re-sorting.

fix(electric): make cursor requestSnapshot calls sequential

Changed parallel requestSnapshot calls to sequential to avoid potential
issues with concurrent snapshot requests that may cause timeouts in CI.

fix(electric): combine cursor expressions into single requestSnapshot

Instead of making two separate requestSnapshot calls (one for whereFrom,
one for whereCurrent), combine them using OR into a single request.
This avoids potential issues with multiple sequential snapshot requests
that were causing timeouts in CI.

The combined expression (whereFrom OR whereCurrent) matches the original
behavior where cursor was combined with the where clause.

wip

working?

update changeset

fix query test

* update docs

* ci: apply automated fixes

* fixups

---------

Co-authored-by: autofix-ci[bot] <114827586+autofix-ci[bot]@users.noreply.github.com>

* ci: Version Packages (TanStack#974)

Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>

* Query collection select writeupsert error (TanStack#1023)

* test(query-db-collection): add failing tests for select + writeInsert/writeUpsert bug

Add tests that reproduce the bug where using writeInsert or writeUpsert
with a collection that has a select option causes an error:
"select() must return an array of objects. Got: undefined"

The bug occurs because performWriteOperations sets the query cache with
a raw array, but the select function expects the wrapped response format.

Related issue: https://github.com/TanStack/db/issues/xyz

* fix(query-db-collection): fix select option breaking writeInsert/writeUpsert

When using the `select` option to extract items from a wrapped API response
(e.g., `{ data: [...], meta: {...} }`), calling `writeInsert()` or
`writeUpsert()` would corrupt the query cache by setting it to a raw array.

This caused the `select` function to receive the wrong data format and
return `undefined`, triggering the error:
"select() must return an array of objects. Got: undefined"

The fix adds a `hasSelect` flag to the SyncContext and skips the
`setQueryData` call when `select` is configured. This is the correct
behavior because:
1. The collection's synced store is already updated
2. The query cache stores the wrapped response format, not the raw items
3. Overwriting the cache with raw items would break the select function

* fix(query-db-collection): fix select option breaking writeInsert/writeUpsert

When using the `select` option to extract items from a wrapped API response
(e.g., `{ data: [...], meta: {...} }`), calling `writeInsert()` or
`writeUpsert()` would corrupt the query cache by setting it to a raw array.

This caused the `select` function to receive the wrong data format and
return `undefined`, triggering the error:
"select() must return an array of objects. Got: undefined"

The fix adds an `updateCacheData` function to the SyncContext that properly
handles cache updates for both cases:
- Without `select`: sets the cache directly with the raw array
- With `select`: uses setQueryData with an updater function to preserve
  the wrapper structure while updating the items array inside it

Also added a comprehensive test that verifies the wrapped response format
(including metadata) is preserved after write operations.

* ci: apply automated fixes

* refactor(query-db-collection): lift updateCacheData out of inline context

Move the updateCacheData function from being inline in the writeContext
object to a standalone function for better readability and maintainability.

* fix(query-db-collection): improve updateCacheData with select-based property detection

Address review feedback:
1. Use select(oldData) to identify the correct array property by reference
   equality instead of "first array property wins" heuristic
2. Fallback to common property names (data, items, results) before scanning
3. Return oldData unchanged instead of raw array when property can't be found
   to avoid breaking select
4. Make updateCacheData optional in SyncContext to avoid breaking changes
5. Add changeset for release

* ci: apply automated fixes

* fix: resolve TypeScript type errors for queryKey

Handle both static and function-based queryKey types properly:
- Get base query key before passing to setQueryData
- Keep writeContext.queryKey as Array<unknown> for SyncContext compatibility

* chore: fix version inconsistencies in example apps

Update example apps to use consistent versions of:
- @tanstack/query-db-collection: ^1.0.7
- @tanstack/react-db: ^0.1.56

Fixes sherif multiple-dependency-versions check.

---------

Co-authored-by: Cursor Agent <cursoragent@cursor.com>
Co-authored-by: autofix-ci[bot] <114827586+autofix-ci[bot]@users.noreply.github.com>

* Fix serializing null/undefined when generating subset queries (TanStack#951)

* Fix invalid Electric proxy queries with missing params for null values

When comparison operators (eq, gt, lt, etc.) were used with null/undefined
values, the SQL compiler would generate placeholders ($1, $2) in the WHERE
clause but skip adding the params to the dictionary because serialize()
returns empty string for null/undefined.

This resulted in invalid queries being sent to Electric like:
  subset__where="name" = $1
  subset__params={}

The fix:
- For eq(col, null): Transform to "col IS NULL" syntax
- For other comparisons (gt, lt, gte, lte, like, ilike): Throw a clear
  error since null comparisons don't make semantic sense in SQL

Added comprehensive tests for the sql-compiler including null handling.

* chore: add changeset for Electric null params fix

* fix: use type assertions in sql-compiler tests for phantom type compatibility

* fix: handle edge cases for null comparisons in sql-compiler

Address reviewer feedback:
- eq(null, null) now throws (both args null would cause missing params)
- eq(null, literal) now throws (comparing literal to null is nonsensical)
- Only allow ref and func types as non-null arg in eq(..., null)
- Update changeset to explicitly mention undefined behavior
- Add tests for edge cases and OR + null equality

* test: add e2e tests for eq(col, null) transformation to IS NULL

* test: improve e2e tests for eq(col, null) with longer timeout and baseline comparison

* fix: handle eq(col, null) in local evaluator to match SQL IS NULL semantics

When eq() is called with a literal null/undefined value, the local
JavaScript evaluator now treats it as an IS NULL check instead of
using 3-valued logic (which would always return UNKNOWN).

This matches the SQL compiler's transformation of eq(col, null) to
IS NULL, ensuring consistent behavior between local query evaluation
and remote SQL queries.

- eq(col, null) now returns true if col is null/undefined, false otherwise
- eq(null, col) is also handled symmetrically
- eq(null, null) returns true (both are null)
- 3-valued logic still applies for column-to-column comparisons

This fixes e2e test failures where eq(col, null) queries returned 0
results because all rows were being excluded by the UNKNOWN result.

* docs: explain eq(col, null) handling and 3-valued logic reasoning

Add design document explaining the middle-ground approach for handling
eq(col, null) in the context of PR TanStack#765's 3-valued logic implementation.

Key points:
- Literal null values in eq() are transformed to isNull semantics
- 3-valued logic still applies for column-to-column comparisons
- This maintains SQL/JS consistency and handles dynamic values gracefully

* fix: throw error for eq(col, null) - use isNull() instead

Per Kevin's feedback on the 3-valued logic design (PR TanStack#765), eq(col, null)
should throw an error rather than being transformed to IS NULL. This is
consistent with how other comparison operators (gt, lt, etc.) handle null.

Changes:
- Revert JS evaluator change that transformed eq(col, null) to isNull semantics
- Update SQL compiler to throw error for eq(col, null) instead of IS NULL
- Update all related unit tests to expect errors
- Remove e2e tests for eq(col, null) (now throws error)
- Update documentation to explain the correct approach

Users should:
- Use isNull(col) or isUndefined(col) to check for null values
- Handle dynamic null values explicitly in their code
- Use non-nullable columns for cursor-based pagination

The original bug (invalid SQL with missing params) is fixed by throwing
a clear error that guides users to the correct approach.

* fix: update changeset and remove design doc per review feedback

- Update changeset to reflect that eq(col, null) throws error (not transforms to IS NULL)
- Remove docs/design/eq-null-handling.md - was only for review, not meant to be merged

* ci: apply automated fixes

---------

Co-authored-by: Claude <noreply@anthropic.com>
Co-authored-by: autofix-ci[bot] <114827586+autofix-ci[bot]@users.noreply.github.com>

* Fix awaitMatch helper on collection inserts and export isChangeMessage (TanStack#1000)

fix(electric-db-collection): fix awaitMatch race condition on inserts and export isChangeMessage

Two issues fixed:

1. **isChangeMessage not exported**: The `isChangeMessage` and `isControlMessage` utilities
   were exported from electric.ts but not re-exported from the package's index.ts, making
   them unavailable to users despite documentation stating otherwise.

2. **awaitMatch race condition on inserts**: When `awaitMatch` was called after the Electric
   messages had already been processed (including up-to-date), it would timeout because:
   - The message buffer (`currentBatchMessages`) was cleared on up-to-date
   - Immediate matches found in the buffer still waited for another up-to-date to resolve

   Fixed by:
   - Moving buffer clearing to the START of batch processing (preserves messages until next batch)
   - Adding `batchCommitted` flag to track when a batch is committed
   - For immediate matches: resolve immediately only if batch is committed (consistent with awaitTxId)
   - For immediate matches during batch processing: wait for up-to-date (maintains commit semantics)
   - Set `batchCommitted` BEFORE `resolveMatchedPendingMatches()` to avoid timing window
   - Set `batchCommitted` on snapshot-end in on-demand mode (matching "ready" semantics)

Fixes issue reported on Discord where inserts would timeout while updates worked.

Co-authored-by: Claude <noreply@anthropic.com>

* ci: Version Packages (TanStack#1026)

Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>

* docs: regenerate API documentation (TanStack#1027)

Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>

* Don't pin @electric-sql/client version (TanStack#1031)

* Don't pin @electric-sql/client version

* Add changeset

* update lock file

* Fix sherif linter errors for dependency version mismatches

Standardizes @electric-sql/client to use ^1.2.0 across react-db, solid-db, and vue-db packages, and updates @tanstack/query-db-collection to ^1.0.8 in todo examples.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>

---------

Co-authored-by: Claude Sonnet 4.5 <noreply@anthropic.com>

* ci: Version Packages (TanStack#1033)

Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>

* Handle subset end message in Electric collection (TanStack#1004)

Adds support for the subset-end message introduced in electric-sql/electric#3582

* Delete a row by its key (TanStack#1003)

This PR makes it possible to delete a row by key when using the write function passed to a collection's sync function.

* Tagged rows and support for move outs in Electric DB collection (TanStack#942)

Builds on top of Electric's ts-client support for tagging rows and move out events: electric-sql/electric#3497

This PR extends tanstack DB such that it handles tagged rows and move out events. A tagged row is removed from the Electric collection when its tag set becomes empty. Note that rows only have tags when the shape they belong to has one or more subqueries.

* Fix: deleted items not disappearing from live queries with `.limit()` (TanStack#1044)

* fix: emit delete events when subscribing with includeInitialState: false

When subscribing to a collection with includeInitialState: false,
delete events were being filtered out because the sentKeys set was
empty. This affected live queries with limit/offset where users
would subscribe to get future changes after already loading
initial data via preload() or values().

Changes:
- Add skipFiltering flag separate from loadedInitialState to allow
  filtering to be skipped while still allowing requestSnapshot to work
- Call markAllStateAsSeen() when includeInitialState is explicitly false
- Change internal subscriptions to not pass includeInitialState: false
  explicitly, so they can be distinguished from user subscriptions
- Add tests for optimistic delete behavior with limit

Fixes the issue where deleted items would not disappear from live
queries when using .limit() and subscribing with includeInitialState: false.

* debug: add extensive logging to track delete event flow

This is a DEBUG BUILD with [TanStack-DB-DEBUG] logs to help track down
why delete events may not be reaching subscribers when using limit/offset.

The debug logs cover:
- subscribeChanges: when subscriptions are created
- emitEvents: when events are emitted to subscriptions
- Subscription.emitEvents: when individual subscriptions receive events
- filterAndFlipChanges: when events are filtered or passed through
- recomputeOptimisticState: when optimistic state is recomputed and events emitted
- sendChangesToPipeline: when changes flow through the D2 pipeline
- applyChanges: when D2 pipeline outputs to the live query collection

To use: Filter browser console for "[TanStack-DB-DEBUG]"

Also includes the fix for includeInitialState: false not emitting deletes.

* ci: apply automated fixes

* debug: add more logging to track delete event flow in live queries

Add comprehensive debug logging to:
- createFilteredCallback in change-events.ts for whereExpression filtering
- sendChangesToInput for D2 pipeline input
- subscribeToOrderedChanges for orderBy/limit path
- splitUpdates for update event handling
- recomputeOptimisticState for pending sync key filtering

This additional logging helps track where delete events may be filtered
out when using live queries with limit/offset and where clauses.

* ci: apply automated fixes

* debug: add logging to graph scheduling and execution

Add debug logging to track:
- scheduleGraphRun: when graph run is scheduled
- executeGraphRun: when graph run executes or returns early
- maybeRunGraph: when graph actually runs, pending work status

This helps diagnose issues where deletes are sent to D2 pipeline
but never appear in the output (applyChanges not called).

* ci: apply automated fixes

* debug: add detailed logging to D2 reduce and topK operators

Add debug logging to track:
- ReduceOperator: input processing, key handling, and result output
- topK: consolidation, sorting, slicing, and result details

Also add two new test cases:
1. Test delete from different page (page 1 delete while viewing page 2)
   - Verifies items shift correctly when delete occurs on earlier page
2. Test delete beyond TopK window (no-op case)
   - Verifies deleting item outside window doesn't affect results

These tests and debug logs will help diagnose issues where deleted
items don't disappear from live queries when using limit/offset.

* ci: apply automated fixes

* debug: add more detailed logging to D2 graph and subscription

Add additional debug logging to help diagnose delete issues:

D2 graph (d2.ts):
- Log when run() starts and completes with step count
- Log pendingWork() results with operator IDs
- Log when operators have pending work in step()

Output operator (output.ts):
- Log when run is called with message count
- Log items in each message being processed

Subscription (subscription.ts):
- Log trackSentKeys with keys being added
- Show total sentKeys count

This should help diagnose scenarios where delete events are sent
to D2 but no applyChanges output is produced.

* ci: apply automated fixes

* debug: add operator type logging to trace D2 pipeline

Add operatorType property to Operator base class and log it when
operators run. This will help identify which operators are processing
the delete and where the data is being lost.

Also add detailed logging to LinearUnaryOperator.run() to show:
- Input message count
- Input/output item counts
- Sample of input and output items

This should reveal exactly which operator is dropping the delete.

* debug: add logging to TopKWithFractionalIndexOperator

This is the key operator for orderBy+limit queries. Add detailed logging to:
- run(): Show message count and index size
- processElement(): Show key, multiplicity changes, and action (INSERT/DELETE/NO_CHANGE)
- processElement result: Show moveIn/moveOut keys

This should reveal exactly why deletes aren't producing output changes
when the item exists in the TopK index.

* ci: apply automated fixes

* fix: filter duplicate inserts in subscription to prevent D2 multiplicity issues

When an item is inserted multiple times without a delete in between,
D2 multiplicity goes above 1. Then when a single delete arrives,
multiplicity goes from 2 to 1 (not 0), so TopK doesn't emit a DELETE event.

This fix:
1. Filters out duplicate inserts in filterAndFlipChanges when key already in sentKeys
2. Removes keys from sentKeys on delete in both filterAndFlipChanges and trackSentKeys
3. Updates test expectation to reflect correct behavior (2 events instead of 3)

Root cause: Multiple subscriptions or sync mechanisms could send duplicate
insert events for the same key, causing D2 to track multiplicity > 1.

* fix: add D2 input level deduplication to prevent multiplicity > 1

The previous fix in CollectionSubscription.filterAndFlipChanges was only
catching duplicates at the subscription level. But each live query has its
own CollectionSubscriber with its own D2 pipeline.

This fix adds a sentToD2Keys set in CollectionSubscriber to track which
keys have been sent to the D2 input, preventing duplicate inserts at the
D2 level regardless of which code path triggers them.

Also clears the tracking on truncate events.

* docs: add detailed changeset for delete fix

* ci: apply automated fixes

* chore: remove debug logging from D2 pipeline and subscription code

Remove all TanStack-DB-DEBUG console statements that were added during
investigation of the deleted items not disappearing from live queries bug.

The fix for duplicate D2 inserts is preserved - just removing the verbose
debug output now that the issue is resolved.

* debug: add logging to trace source of duplicate D2 inserts

Add targeted debug logging to understand where duplicate inserts originate:

1. recomputeOptimisticState: Track what events are generated and when
2. CollectionSubscription.filterAndFlipChanges: Trace filtering decisions
3. CollectionSubscriber.sendChangesToPipeline: Track D2-level deduplication

This will help determine if duplicates come from:
- Multiple calls to recomputeOptimisticState for the same key
- Overlap between initial snapshot and change events
- Multiple code paths feeding the D2 pipeline

* ci: apply automated fixes

* fix: prevent race condition in snapshot loading by adding keys to sentKeys before callback

The race condition occurred because snapshot methods (requestSnapshot, requestLimitedSnapshot)
added keys to sentKeys AFTER calling the callback, while filterAndFlipChanges added keys
BEFORE. If a change event arrived during callback execution, it would not see the keys
in sentKeys yet, allowing duplicate inserts.

Changes:
- Add keys to sentKeys BEFORE calling callback in requestSnapshot and requestLimitedSnapshot
- Remove redundant D2-level deduplication (sentToD2Keys) - subscription-level filtering is sufficient
- Remove debug logging added during investigation

* docs: update changeset to reflect race condition fix

* cleanup

* simplify changeset

---------

Co-authored-by: Claude <noreply@anthropic.com>
Co-authored-by: autofix-ci[bot] <114827586+autofix-ci[bot]@users.noreply.github.com>
Co-authored-by: Sam Willis <sam.willis@gmail.com>

* ci: Version Packages (TanStack#1036)

Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>

* fix(db): re-request and buffer subsets after truncate for on-demand sync mode (TanStack#1043)

* fix(db): re-request subsets after truncate for on-demand sync mode

When a must-refetch (409) occurs in on-demand sync mode, the collection
receives a truncate which clears all data and resets the loadSubset
deduplication state. However, subscriptions were not re-requesting
their previously loaded subsets, leaving the collection empty.

This fix adds a truncate event listener to CollectionSubscription that:
1. Resets pagination/snapshot tracking state (but NOT sentKeys)
2. Re-requests all previously loaded subsets

We intentionally keep sentKeys intact because the truncate event is
emitted BEFORE delete events are sent to subscribers. If we cleared
sentKeys, delete events would be filtered by filterAndFlipChanges.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

* feat: Buffer subscription changes during truncate

This change buffers subscription changes during a truncate event until all
loadSubset refetches complete. This prevents a flash of missing content
between deletes and new inserts.

Co-authored-by: sam.willis <sam.willis@gmail.com>

* ci: apply automated fixes

* changeset

* tweaks

* test

* address review

* ci: apply automated fixes

---------

Co-authored-by: Igor Barakaiev <ibarakaiev@gmail.com>
Co-authored-by: Claude Opus 4.5 <noreply@anthropic.com>
Co-authored-by: Cursor Agent <cursoragent@cursor.com>
Co-authored-by: autofix-ci[bot] <114827586+autofix-ci[bot]@users.noreply.github.com>

* ci: Version Packages (TanStack#1050)

Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>

* fix: prevent duplicate inserts from reaching D2 pipeline in live queries (TanStack#1054)

* add test

* fix

* changeset

* fix versions

* ci: apply automated fixes

---------

Co-authored-by: autofix-ci[bot] <114827586+autofix-ci[bot]@users.noreply.github.com>

* docs: regenerate API documentation (TanStack#1051)

Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>

* ci: Version Packages (TanStack#1055)

Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>

* Fix slow onInsert awaitMatch performance issue (TanStack#1029)

* fix(electric): preserve message buffer across batches for awaitMatch

The buffer was being cleared at the start of each new batch, which caused
messages to be lost when multiple batches arrived before awaitMatch was
called. This led to:
- awaitMatch timing out (~3-5s per attempt)
- Transaction rollbacks when the timeout threw an error

The fix removes the buffer clearing between batches. Messages are now
preserved until the buffer reaches MAX_BATCH_MESSAGES (1000), at which
point the oldest messages are dropped. This ensures awaitMatch can find
messages even when heartbeat batches or other sync activity arrives
before the API call completes.

Added test case for the specific race condition: multiple batches
(including heartbeats) arriving while onInsert's API call is in progress.

* chore: align query-db-collection versions in examples

Update todo examples to use ^1.0.8 to match other examples and fix
sherif version consistency check.

* chore: align example dependency versions

Update todo and paced-mutations-demo examples to use consistent versions:
- @tanstack/query-db-collection: ^1.0.11
- @tanstack/react-db: ^0.1.59

---------

Co-authored-by: Claude <noreply@anthropic.com>

* Add missing changeset for PR 1029 (TanStack#1062)

Add changeset for PR TanStack#1029 (awaitMatch performance fix)

Co-authored-by: Claude <noreply@anthropic.com>

* ci: Version Packages (TanStack#1063)

Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>

* ci: apply automated fixes

---------

Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
Co-authored-by: Lachlan Collins <1667261+lachlancollins@users.noreply.github.com>
Co-authored-by: autofix-ci[bot] <114827586+autofix-ci[bot]@users.noreply.github.com>
Co-authored-by: Kyle Mathews <mathews.kyle@gmail.com>
Co-authored-by: Claude <noreply@anthropic.com>
Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>
Co-authored-by: Kevin <kevin@electric-sql.com>
Co-authored-by: Cursor Agent <cursoragent@cursor.com>
Co-authored-by: Igor Barakaiev <ibarakaiev@gmail.com>

---------

Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
Co-authored-by: Lachlan Collins <1667261+lachlancollins@users.noreply.github.com>
Co-authored-by: autofix-ci[bot] <114827586+autofix-ci[bot]@users.noreply.github.com>
Co-authored-by: Kyle Mathews <mathews.kyle@gmail.com>
Co-authored-by: Claude <noreply@anthropic.com>
Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>
Co-authored-by: Kevin <kevin@electric-sql.com>
Co-authored-by: Cursor Agent <cursoragent@cursor.com>
Co-authored-by: Igor Barakaiev <ibarakaiev@gmail.com>
Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
…k#1073)

* auto bump example deps on publish

* update pnpm lock
* chore(deps): update all non-major dependencies

* chore(deps): sync @trpc/server version with @trpc/client

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

* update lock file

---------

Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>
Co-authored-by: Kyle Mathews <mathews.kyle@gmail.com>
Co-authored-by: Claude Opus 4.5 <noreply@anthropic.com>
…ent values (TanStack#1070)

* fix: handle overlapping subset queries returning same row with different values

When multiple subset queries return the same row (e.g., different WHERE
clauses that both match the same record), the server sends `insert`
operations for each response. If the row's data changed between requests
(e.g., timestamp field updated), this caused a DuplicateKeySyncError
because TanStack DB's sync layer throws when inserting an existing key
with a different value.

This fix tracks synced keys in the Electric adapter and converts
subsequent `insert` operations to `update` for keys that have already
been synced. The tracked keys are cleared on truncate/must-refetch to
stay in sync with the collection state.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

* add tests

* ci: apply automated fixes

---------

Co-authored-by: Claude Opus 4.5 <noreply@anthropic.com>
Co-authored-by: Sam Willis <sam.willis@gmail.com>
Co-authored-by: autofix-ci[bot] <114827586+autofix-ci[bot]@users.noreply.github.com>
…efetch in progressive mode (TanStack#1069)

* fix(electric-db-collection): prevent orphan transactions after must-refetch in progressive mode

When a `must-refetch` message is received in progressive mode, it starts a
transaction with `begin()` and calls `truncate()`. This resets `hasReceivedUpToDate`
to `false`, causing `isBufferingInitialSync()` to return `true`.

The bug: subsequent messages after must-refetch were being buffered instead of
written to the existing transaction. When `up-to-date` was received, the atomic
swap code would create a NEW transaction, leaving the first transaction (from
must-refetch) uncommitted forever. This "orphan transaction" caused the collection
to become corrupted with undefined values.

The fix: Add `&& !transactionStarted` checks to 5 places so that when a transaction
is already started (from must-refetch), messages are written directly to it instead
of being buffered for atomic swap.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

* add tests

---------

Co-authored-by: Claude Opus 4.5 <noreply@anthropic.com>
Co-authored-by: Sam Willis <sam.willis@gmail.com>
docs: fix where() examples to use eq() instead of JavaScript ===

The where() callback in useLiveQuery requires query builder functions
like eq() instead of JavaScript comparison operators. Using === causes
QueryCompilationError: Unknown expression type: undefined.

Fixes examples in:
- docs/collections/local-only-collection.md (2 examples)
- docs/collections/local-storage-collection.md (1 example)

Co-authored-by: Claude <noreply@anthropic.com>
Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>
…ng mock stubs (TanStack#1037)

* Remove misleading schema arguments (TanStack#1002) and backfill missing mock stubs.

* ci: apply automated fixes

---------

Co-authored-by: autofix-ci[bot] <114827586+autofix-ci[bot]@users.noreply.github.com>
* feat: [PowerSync] Add metadata tracking

* cleanup test

* pnpm format

* update docs

* update docs with origin
…TanStack#1001)

* feat(svelte-db): add findOne() support

* ci: apply automated fixes

* change cheset to patch

---------

Co-authored-by: autofix-ci[bot] <114827586+autofix-ci[bot]@users.noreply.github.com>
Co-authored-by: Sam Willis <sam.willis@gmail.com>
* fix(db): add validation for where() expressions to catch JavaScript operator usage

When users accidentally use JavaScript's comparison operators (===, !==, <, >, etc.)
in where() callbacks instead of query builder functions (eq, gt, etc.), the query
builder now throws a helpful InvalidWhereExpressionError with clear guidance.

Previously, this mistake would result in a confusing "Unknown expression type: undefined"
error at query compilation time. Now users get immediate feedback with an example of
the correct syntax.

Example of the mistake this catches:
  ❌ .where(({ user }) => user.id === 'abc')
  ✅ .where(({ user }) => eq(user.id, 'abc'))

* chore: add changeset for where() expression validation

* refactor: extract expression validation into helper function

Extract duplicate valueType determination logic from where() and having()
methods into a shared getValueTypeName() helper function.

Addresses review feedback from kevin-dp on PR TanStack#1082.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

---------

Co-authored-by: Claude <noreply@anthropic.com>
…played twice on page load (TanStack#1046)

fix(offline-transactions): prevent double replay of transactions on initialization

Fix a race condition in WebLocksLeader where transactions could be
replayed twice on page load. The issue occurred because:

1. requestLeadership() returned true immediately when lock was available
2. But notifyLeadershipChange(true) was called asynchronously when the
   fire-and-forget navigator.locks.request() actually acquired the lock
3. This caused loadAndReplayTransactions() to be called twice - once
   manually and once via the callback

The fix has two parts:
1. Set WebLocksLeader.isLeaderState = true synchronously when we know
   the lock is available, so the async notifyLeadershipChange(true)
   doesn't trigger the callback again
2. Set OfflineExecutor.isLeaderState based on requestLeadership() return
   value, since we can no longer rely on the callback to set it

Reported by Toba on Discord.

Co-authored-by: Claude <noreply@anthropic.com>
…es (TanStack#1040)

ci: add helpful error message when sherif check fails

Shows a clear fix command when dependency version mismatches are detected,
making it easier for contributors to resolve the issue locally.

Co-authored-by: Claude <noreply@anthropic.com>
…nStack#1007)

* fix(query-db-collection): sync=on-demand tanstack-query lifecycle

* ci: apply automated fixes

---------

Co-authored-by: autofix-ci[bot] <114827586+autofix-ci[bot]@users.noreply.github.com>
github-actions Bot and others added 27 commits April 7, 2026 08:42
Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
* Docs for includes (TanStack#1317)

* Document includes subqueries in live queries guide

Add an Includes section to docs/guides/live-queries.md covering:
- Basic includes with correlation conditions
- Additional filters including parent-referencing WHERE clauses
- Ordering and limiting per parent
- toArray() for plain array results
- Aggregates per parent
- Nested includes

Also add packages/db/INCLUDES.md with architectural documentation
and update the V2 roadmap to reflect implemented features.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* Improve includes docs: use concrete examples instead of generic "parent/child" terminology

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* Document how to use includes with React via subcomponents with useLiveQuery

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* Add React test for includes: child collection subscription via useLiveQuery

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>

* docs: add includes sections to framework SKILL.md files

Add hierarchical data (includes) documentation to all framework skills
(React, Solid, Vue, Svelte, Angular) and fix inaccurate toArray scalar
select constraint in db-core/live-queries skill.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* ci: apply automated fixes

* chore: add changeset for includes SKILL.md updates

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
Co-authored-by: Kyle Mathews <mathews.kyle@gmail.com>
Co-authored-by: autofix-ci[bot] <114827586+autofix-ci[bot]@users.noreply.github.com>
…anStack#1461)

* fix(ci): reuse existing open docs PR instead of creating duplicates

The release workflow now checks for an existing open "regenerate API
documentation" PR before creating a new one. If found, it pushes new
docs changes to that PR's branch instead of spawning a duplicate.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* fix(ci): address review findings in docs PR reuse logic

- Use git checkout -B to reset branch to current HEAD (avoids dirty
  working tree conflict and keeps freshly generated docs)
- Use --force-with-lease for push since branch history is rewritten
- Guard git commit with diff --cached --quiet for no-change case
- Handle gh pr list failure gracefully with fallback to new PR
- Narrow PR search with --author and in:title filters
- Use separate CREATE_PR variable instead of dual-purpose state

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
…Stack#1471)

* Unit tests to check that includes are loaded lazily on-demand

* Lazy load included rows in on-demand mode when index is available

* ci: apply automated fixes

* Fix type issue

* changeset

* fix: pass child where clauses to loadSubset in includes (TanStack#1472)

* Unit tests to check that where clauses of included collections are passed

* Unit tests to reproduce the problem with where clauses on included collections not being passed

* Pass child where clauses to loadSubset

* ci: apply automated fixes

* changeset for child where clauses fix

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

---------

Co-authored-by: autofix-ci[bot] <114827586+autofix-ci[bot]@users.noreply.github.com>
Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

---------

Co-authored-by: autofix-ci[bot] <114827586+autofix-ci[bot]@users.noreply.github.com>
Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
…anStack#1479)

Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>
…ETE for on-demand PowerSync collections (TanStack#1470)

* Fixed bug where on-demand collections with the `id` column in their where clause would never be added to the PowerSync upload queue.

* Added minor comment.
* ci: add zizmor workflow

* ci: fix zizmor workflow findings

* Update .github/workflows/zizmor.yml

* Update .github/workflows/review-pr-claude.yml

* Update .github/workflows/reproduce-and-fix-issue-claude.yml

* Update .github/workflows/claude.yml

* Update .github/workflows/pr.yml

* Update .github/workflows/pr.yml

* Update .github/workflows/e2e-tests.yml

* Update .github/workflows/e2e-tests.yml

* Update .github/workflows/e2e-tests.yml

* ci: fix compressed size action ref comments
* fix(pnpm): update pnpm to v11

* fix pnpm/action-setup

---------

Co-authored-by: Lachlan Collins <1667261+lachlancollins@users.noreply.github.com>
Avoid routing the long-running Wrangler dev process through pnpm exec so the Cloudflare Durable Object e2e lane stays stable after the pnpm 11 upgrade.
docs(db): clarify mutate callback is synchronous
* docs: add caseWhen operator plan

Co-authored-by: Cursor <cursoragent@cursor.com>

* docs: decouple caseWhen plan from multi-source from

Co-authored-by: Cursor <cursoragent@cursor.com>

* feat(db): add caseWhen query operator

Co-authored-by: Cursor <cursoragent@cursor.com>

* feat(db): support guarded includes in caseWhen

Co-authored-by: Cursor <cursoragent@cursor.com>

* test(db): cover caseWhen plan gaps

Co-authored-by: Cursor <cursoragent@cursor.com>

* fix(db): complete caseWhen plan coverage

Co-authored-by: Cursor <cursoragent@cursor.com>

* test(db): assert caseWhen literal inference

Co-authored-by: Cursor <cursoragent@cursor.com>

* test(db): update caseWhen type assertions

Co-authored-by: Cursor <cursoragent@cursor.com>

* test(db): cover caseWhen collection includes

Co-authored-by: Cursor <cursoragent@cursor.com>

* test(db): cover caseWhen expression gaps

Co-authored-by: Cursor <cursoragent@cursor.com>

* docs(db): move caseWhen plan out of branch

Co-authored-by: Cursor <cursoragent@cursor.com>

* ci: apply automated fixes

* fix(db): address caseWhen review feedback

Thread conditional include destinations through caseWhen branches and tighten aggregate handling for grouped conditional projections.

Co-authored-by: Cursor <cursoragent@cursor.com>

* ci: apply automated fixes

* refactor(db): share caseWhen condition truthiness

Use one compiler helper for scalar and projection caseWhen branch truthiness so every evaluation path stays consistent.

Co-authored-by: Cursor <cursoragent@cursor.com>

* refactor(db): limit caseWhen overloads

Keep precise caseWhen typings to five branches to reduce API surface while preserving runtime behavior.

Co-authored-by: Cursor <cursoragent@cursor.com>

* fix(db): harden conditional select compilation

Preserve include nodes during aggregate extraction and handle null projection branches without misclassifying conditional selects.

Co-authored-by: Cursor <cursoragent@cursor.com>

* fix(db): resolve grouped caseWhen refs

Rewrite grouped refs inside aggregate-wrapped conditional projections so branch conditions and values evaluate against grouped keys.

Co-authored-by: Cursor <cursoragent@cursor.com>

* refactor(db): add broad caseWhen overload fallback

Allow longer caseWhen calls without reintroducing verbose precise overloads past five branches.

Co-authored-by: Cursor <cursoragent@cursor.com>

* ci: apply automated fixes

* fix(db): tighten caseWhen expression detection

Avoid treating projection objects with type fields as IR expressions when selecting conditional projections.

Co-authored-by: Cursor <cursoragent@cursor.com>

---------

Co-authored-by: Cursor <cursoragent@cursor.com>
Co-authored-by: autofix-ci[bot] <114827586+autofix-ci[bot]@users.noreply.github.com>
* docs: add caseWhen operator plan

Co-authored-by: Cursor <cursoragent@cursor.com>

* docs: decouple caseWhen plan from multi-source from

Co-authored-by: Cursor <cursoragent@cursor.com>

* feat(db): support multi-source from queries

Add union-style from handling across the query builder, compiler, optimizer, live traversal, and tests so independent sources can feed one live collection.

Co-authored-by: Cursor <cursoragent@cursor.com>

* fix(db): close multi-source from edge cases

Tighten union join typing, avoid inactive guarded include evaluation, and document multi-source ordering defaults.

Co-authored-by: Cursor <cursoragent@cursor.com>

* test(db): cover multi-source subquery branches

Add runtime and type coverage for union branches that are joined and projected subqueries, since that is the main pre-filtering pattern.

Co-authored-by: Cursor <cursoragent@cursor.com>

* fix(db): lazy-load union subquery joins

Resolve multi-source subquery join targets per branch so safe coalesce projections can load indexed subsets while computed or limited branches deopt to eager loading.

Co-authored-by: Cursor <cursoragent@cursor.com>

* fix(db): resolve lazy targets for subquery includes

Share union-aware lazy target resolution between joins and includes so correlated child collections can be built from subquery or multi-source sources without crashing.

Co-authored-by: Cursor <cursoragent@cursor.com>

* fix(db): propagate nested source include updates

Carry nested include routing through QueryRef sources and preserve keyed incremental updates so child live queries receive streaming materialization changes.

Co-authored-by: Cursor <cursoragent@cursor.com>

* fix(db): address multi-source review feedback

Tighten branch key encoding, join pushdown nullability, lazy target resolution, and conditional/grouped projection semantics based on review feedback.

Co-authored-by: Cursor <cursoragent@cursor.com>

* fix(db): align multi-source branch with caseWhen head

Restore nested update cloning after rebasing onto the latest caseWhen branch and update conditional include expectations.

Co-authored-by: Cursor <cursoragent@cursor.com>

* ci: apply automated fixes

* test(db): update subquery lazy alias expectation

Assert subquery-backed join lazy loading marks the concrete subscribed alias rather than the outer QueryRef alias.

Co-authored-by: Cursor <cursoragent@cursor.com>

* refactor(db): expose source unions as unionAll

Co-authored-by: Cursor <cursoragent@cursor.com>

* feat(db): support query branch unionAll

Co-authored-by: Cursor <cursoragent@cursor.com>

* fix(db): harden query branch unionAll

Co-authored-by: Cursor <cursoragent@cursor.com>

* fix(db): clarify query branch unionAll boundary

Co-authored-by: Cursor <cursoragent@cursor.com>

* fix(db): materialize unionAll includes before fnSelect

Co-authored-by: Cursor <cursoragent@cursor.com>

* chore(db): remove planning docs from branch

Co-authored-by: Cursor <cursoragent@cursor.com>

* ci: apply automated fixes

* ci: apply automated fixes (attempt 2/3)

* fix(db): address unionAll review feedback

Co-authored-by: Cursor <cursoragent@cursor.com>

* ci: apply automated fixes

* fix(db): address unionAll review comments

Guard join compilation against unionAll subqueries and show context-specific examples for invalid join sources.

Co-authored-by: Cursor <cursoragent@cursor.com>

* fix(db): address unionAll source review

Co-authored-by: Cursor <cursoragent@cursor.com>

* fix(db): harden unionAll review edges

Co-authored-by: Cursor <cursoragent@cursor.com>

* ci: apply automated fixes

---------

Co-authored-by: Cursor <cursoragent@cursor.com>
Co-authored-by: autofix-ci[bot] <114827586+autofix-ci[bot]@users.noreply.github.com>
Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
…1547)

* fix(db): drop stale direct optimistic rows after txid sync

* ci: run e2e tests on node 22

* refactor(db): remove dead DIRECT_TRANSACTION_METADATA_KEY code

The only consumer was in state.ts, which was removed in the prior commit.
Clean up the orphaned constant, import, and metadata usage in mutations.ts.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* chore: add changeset for stale optimistic row fix

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* docs: forbid rewriting published branch history

* fix(db): retain only direct optimistic rows until sync

---------

Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
…endpoint behavior (TanStack#1544)

* docs(db): clarify caseWhen, coalesce, manual transactions, and multi-endpoint behavior

Address documentation feedback: add caseWhen operator docs, clarify coalesce
handles null/undefined, document manual transaction handler bypass semantics,
and warn about multi-endpoint Query Collection pitfalls. Fix categorization
of utility functions and reference index ordering.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* chore: add changeset for docs clarifications

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* docs(db): mention unionAll as option for multi-endpoint collections

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* docs(db): replace redundant manual transaction example with explanatory prose

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
* feat: add materialize() helper for includes subqueries

materialize() resolves to Array<T> for multi-row subqueries and
T | undefined for findOne() subqueries, so callers don't have to
unwrap a singleton array when the child query returns at most one row.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* ci: apply automated fixes

---------

Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Co-authored-by: autofix-ci[bot] <114827586+autofix-ci[bot]@users.noreply.github.com>
…y recovery (TanStack#1572)

* feat(node-db-sqlite-persistence): prune applied_tx by default

createNodeSQLitePersistence left appliedTxPruneMaxRows/appliedTxPruneMaxAgeSeconds unset, so the applied_tx log grew without bound for every consumer. Default them to 1,000 rows and a 24h age backstop (both overridable; pass 0 to disable), exported as DEFAULT_APPLIED_TX_PRUNE_MAX_ROWS / DEFAULT_APPLIED_TX_PRUNE_MAX_AGE_SECONDS.

* fix: make sqlite replay pruning recovery-aware

* ci: apply automated fixes

* docs: clarify sqlite pruning scope

* feat: default sqlite applied_tx pruning across wrappers

---------

Co-authored-by: Kyle Mathews <mathews.kyle@gmail.com>
Co-authored-by: autofix-ci[bot] <114827586+autofix-ci[bot]@users.noreply.github.com>
Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
@Chriztiaan Chriztiaan requested a review from simolus3 June 11, 2026 07:24
@Chriztiaan Chriztiaan merged commit 082ce15 into main Jun 11, 2026
6 of 7 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.