Skip to content

oracledb_cdc: buffer out of order LOB_WRITE events until corresponding INSERT/UPDATE events are received#4458

Open
josephwoodward wants to merge 4 commits into
mainfrom
jw/lobwriteorder
Open

oracledb_cdc: buffer out of order LOB_WRITE events until corresponding INSERT/UPDATE events are received#4458
josephwoodward wants to merge 4 commits into
mainfrom
jw/lobwriteorder

Conversation

@josephwoodward

@josephwoodward josephwoodward commented May 27, 2026

Copy link
Copy Markdown
Contributor

This change fixes an issue where lob writes can be out of order (though only occasionally appeared to fail in the CI pipeline). Now we buffer the lob writes until we see their corresponding insert or update.

Following the fix I've run the CI job numerous times and it appears to fix the issue.

image

Comment thread internal/impl/oracledb/logminer/logminer.go Outdated
@claude

claude Bot commented May 27, 2026

Copy link
Copy Markdown

Commits

  1. oracledb_cdc: lob write order — message after the colon is a noun phrase, not imperative mood. Policy requires imperative (e.g., oracledb_cdc: fix lob write ordering or similar) — see the PR title oracledb_cdc: lob write race condition for a more descriptive phrasing.

Review
Single blocker: a missing ctx argument in the new drainPendingLOBWrites helper makes the package fail to compile.

  1. internal/impl/oracledb/logminer/logminer.go:779lm.inferLOBLocator(event) is missing the required ctx context.Context argument; drainPendingLOBWrites needs to accept and forward ctx from its caller at line 291.

@claude

claude Bot commented May 27, 2026

Copy link
Copy Markdown

Commits

  1. oracledb_cdc: lob write order — the message is not in imperative mood. The commit policy requires system: message where message uses imperative mood (e.g. "add", "fix", "buffer", "handle"). "lob write order" is a noun phrase that doesn't describe the action being performed. Consider rewording, e.g. oracledb_cdc: buffer lob writes that arrive before insert or oracledb_cdc: handle out-of-order lob writes.

Review
LGTM

Comment thread internal/impl/oracledb/logminer/logminer.go Outdated
@claude

claude Bot commented May 27, 2026

Copy link
Copy Markdown

Commits

  1. The oracledb_cdc: address merge conflict (cbc4a55) commit is a fixup that resolves the merge from main — it should be squashed into either the merge commit or the main oracledb_cdc: lob write order commit rather than standing on its own. Per the commit policy, each commit should be a self-contained logical change; merge-conflict resolutions are not.

Review

The change buffers LOB_WRITE events that arrive before their matching INSERT/UPDATE (BASICFILE DISABLE STORAGE IN ROW ordering) and drains them once the DML is processed. Cleanup on commit/rollback is consistent with the existing lobStates lifecycle, and the new unit test exercises the out-of-order arrival path.

  1. drainPendingLOBWrites returns a fatal error on ParseLobWrite failure, while the equivalent non-buffered OpLobWrite handler logs a warning and continues. See inline comment on logminer.go#L797.

Comment thread internal/impl/oracledb/logminer/logminer.go Outdated
Comment thread internal/impl/oracledb/logminer/logminer.go Outdated
Comment thread internal/impl/oracledb/logminer/logminer.go Outdated
@claude

claude Bot commented May 27, 2026

Copy link
Copy Markdown

Commits

  1. The first commit headline oracledb_cdc: lob write order is missing an imperative verb (e.g. fix, handle, order-as-verb is ambiguous here). The commit policy requires imperative mood like otlp: add authz support. The second commit (oracledb_cdc: address lob race condition) is fine.

Review
Targeted fix to buffer LOB_WRITE events that arrive before their matching INSERT/UPDATE under BASICFILE DISABLE STORAGE IN ROW ordering, with cleanup on commit/rollback and a unit test covering the out-of-order case.

  1. Two new fmt.Errorf sites wrap the underlying parse error with %v instead of %w, breaking the error chain — see inline comments on logminer.go#L425 and logminer.go#L796. The rest of the file consistently uses %w for wrapped errors.

@claude

claude Bot commented May 27, 2026

Copy link
Copy Markdown

Commits

  1. The oracledb_cdc: lob write order headline is vague — it lacks a verb and doesn't convey what the change does. Per the project convention used in nearby commits (e.g. oracledb_cdc: support cache resources for internal transaction buffer, oracledb_cdc: exclude LogMiner meta events for unmonitored tables when LOB is enabled), the message should be imperative and self-descriptive, e.g. oracledb_cdc: buffer LOB_WRITE events arriving before INSERT/UPDATE.

Review
The change buffers LOB_WRITE redo events that arrive before their matching INSERT/UPDATE (BASICFILE DISABLE STORAGE IN ROW ordering) and drains the buffer once the row event is processed. Cleanup is wired into both commit and rollback paths, and a focused unit test (logminer_test.go#L235-L267) covers the new ordering.

LGTM

@claude

claude Bot commented May 27, 2026

Copy link
Copy Markdown

Commits
LGTM

Review
Adds a buffer for LOB_WRITE events that arrive before their matching INSERT (BASICFILE DISABLE STORAGE IN ROW ordering), with drain on INSERT/UPDATE and cleanup on commit/rollback. The inferLOBLocator candidate-selection change correctly distinguishes out-of-line LOBs (NULL/absent from Data) from inline LOBs ([]byte{} placeholders handled by SELECT_LOB_LOCATOR). Sorted iteration via slices.Sorted makes column assignment deterministic.

LGTM

@josephwoodward josephwoodward changed the title oracledb_cdc: lob write race condition oracledb_cdc: fix LOB_WRITE race condition May 28, 2026
@josephwoodward josephwoodward marked this pull request as ready for review May 28, 2026 11:02
@claude

claude Bot commented May 28, 2026

Copy link
Copy Markdown

Commits

  1. Commit f61452d has a typo in the subject: determinsiticdeterministic.
  2. The 4 commits (lob write order, address lob race condition, ensure we wrap error, make order determinsitic) all incrementally fix the same logical change. Commits 3 and 4 in particular look like fixups (a single error-wrap line and a single-line slices.Sorted change) that should be squashed into the first two for cleaner history.

Review
The fix buffers out-of-order LOB_WRITE events into a new pendingLOBWrites map and drains them once the matching INSERT/UPDATE is added to the transaction cache, with cleanup on commit/rollback. The inferLOBLocator candidate-selection rewrite (treat columns absent or nil as candidates; []byte{} placeholders are not) plus sorted-key iteration for determinism look correct, and the new TestLOBWriteBeforeInsert exercises the buffer→drain path end-to-end.

LGTM

@josephwoodward josephwoodward changed the title oracledb_cdc: fix LOB_WRITE race condition oracledb_cdc: fix LOB_WRITE events appearing before INSERT/UPDATES May 28, 2026
@josephwoodward josephwoodward changed the title oracledb_cdc: fix LOB_WRITE events appearing before INSERT/UPDATES oracledb_cdc: fix LOB_WRITE events appearing before INSERT/UPDATE LOB events May 28, 2026
@josephwoodward josephwoodward changed the title oracledb_cdc: fix LOB_WRITE events appearing before INSERT/UPDATE LOB events oracledb_cdc: buffer out of order LOB_WRITE events until corresponding INSERT/UPDATE events are received May 28, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant