docs(data-warehouse): Document xmin sync method for PostgreSQL sources#17804
Open
inkeep[bot] wants to merge 2 commits into
Open
docs(data-warehouse): Document xmin sync method for PostgreSQL sources#17804inkeep[bot] wants to merge 2 commits into
inkeep[bot] wants to merge 2 commits into
Conversation
Adds documentation for the new xmin sync method introduced in PostHog/posthog#64721. Includes a new subsection under Sync methods describing xmin's purpose, requirements (PG13+, primary key, heap tables/matviews only), and limitations (no delete tracking). Also updates the schema filter list to include xmin.
Contributor
Deploy preview
|
Contributor
|
Vale prose linter → found 19 errors, 44 warnings, 2 suggestions in your markdown Full report → Copy the linter results into an LLM to batch-fix issues. Linter being weird? Update the rules!
|
| Line | Severity | Message | Rule |
|---|---|---|---|
| 18:162 | warning | Capitalize 'Product Analytics' for PostHog's product. Use 'product analytics' for the general industry concept. | PostHogBase.ProductNames |
| 40:5 | warning | Capitalize 'Data Warehouse' for PostHog's product. Use 'data warehouse' for the general industry concept. | PostHogBase.ProductNames |
| 44:58 | warning | 'Airbyte' is a possible misspelling. | PostHogBase.Spelling |
| 44:67 | warning | Use 'FiveTran' instead of 'Fivetran'. | Vale.Terms |
| 63:160 | warning | 'autoincrementing' is a possible misspelling. | PostHogBase.Spelling |
| 71:99 | warning | Capitalize 'Data Warehouse' for PostHog's product. Use 'data warehouse' for the general industry concept. | PostHogBase.ProductNames |
| 73:17 | warning | 'nullable' is a possible misspelling. | PostHogBase.Spelling |
| 73:160 | warning | 'nullable' is a possible misspelling. | PostHogBase.Spelling |
| 83:69 | warning | 'nullable' is a possible misspelling. | PostHogBase.Spelling |
| 93:42 | warning | 'nullable' is a possible misspelling. | PostHogBase.Spelling |
| 99:5 | warning | 'xmin' is a possible misspelling. | PostHogBase.Spelling |
| 101:1 | warning | 'xmin' is a possible misspelling. | PostHogBase.Spelling |
| 109:1 | warning | 'xmin' is a possible misspelling. | PostHogBase.Spelling |
| 133:1 | error | Hi, Andy here... use an en dash ( – ) with spaces. On Mac, holding down the Option and hyphen key will give you an en dash. | PostHogBase.EnDash |
| 134:1 | error | Hi, Andy here... use an en dash ( – ) with spaces. On Mac, holding down the Option and hyphen key will give you an en dash. | PostHogBase.EnDash |
| 134:46 | warning | 'xmin' is a possible misspelling. | PostHogBase.Spelling |
| 135:1 | error | Hi, Andy here... use an en dash ( – ) with spaces. On Mac, holding down the Option and hyphen key will give you an en dash. | PostHogBase.EnDash |
| 145:1 | error | Hi, Andy here... use an en dash ( – ) with spaces. On Mac, holding down the Option and hyphen key will give you an en dash. | PostHogBase.EnDash |
| 145:177 | warning | 'resync' is a possible misspelling. | PostHogBase.Spelling |
| 146:1 | error | Hi, Andy here... use an en dash ( – ) with spaces. On Mac, holding down the Option and hyphen key will give you an en dash. | PostHogBase.EnDash |
| 147:1 | error | Hi, Andy here... use an en dash ( – ) with spaces. On Mac, holding down the Option and hyphen key will give you an en dash. | PostHogBase.EnDash |
| 148:1 | error | Hi, Andy here... use an en dash ( – ) with spaces. On Mac, holding down the Option and hyphen key will give you an en dash. | PostHogBase.EnDash |
| 148:23 | warning | 'resync' is a possible misspelling. | PostHogBase.Spelling |
| 149:1 | error | Hi, Andy here... use an en dash ( – ) with spaces. On Mac, holding down the Option and hyphen key will give you an en dash. | PostHogBase.EnDash |
| 161:1 | error | Hi, Andy here... use an en dash ( – ) with spaces. On Mac, holding down the Option and hyphen key will give you an en dash. | PostHogBase.EnDash |
| 163:1 | error | Hi, Andy here... use an en dash ( – ) with spaces. On Mac, holding down the Option and hyphen key will give you an en dash. | PostHogBase.EnDash |
| 165:1 | error | Hi, Andy here... use an en dash ( – ) with spaces. On Mac, holding down the Option and hyphen key will give you an en dash. | PostHogBase.EnDash |
| 165:318 | warning | 'OAuth' is a possible misspelling. | PostHogBase.Spelling |
contents/docs/cdp/sources/postgres.mdx — 8 errors, 27 warnings, 2 suggestions
| Line | Severity | Message | Rule |
|---|---|---|---|
| 27:37 | suggestion | Address the reader directly. Use 'you' instead of 'the user'. | PostHogDocs.DirectAddress |
| 35:37 | error | Hi, Andy here... use an en dash ( – ) with spaces. On Mac, holding down the Option and hyphen key will give you an en dash. | PostHogBase.EnDash |
| 39:5 | warning | Capitalize 'Data Warehouse' for PostHog's product. Use 'data warehouse' for the general industry concept. | PostHogBase.ProductNames |
| 50:4 | warning | 'Uncheck' is a possible misspelling. | PostHogBase.Spelling |
| 63:1 | error | Hi, Andy here... use an en dash ( – ) with spaces. On Mac, holding down the Option and hyphen key will give you an en dash. | PostHogBase.EnDash |
| 64:1 | error | Hi, Andy here... use an en dash ( – ) with spaces. On Mac, holding down the Option and hyphen key will give you an en dash. | PostHogBase.EnDash |
| 64:10 | warning | 'resync' is a possible misspelling. | PostHogBase.Spelling |
| 64:41 | warning | 'resync' is a possible misspelling. | PostHogBase.Spelling |
| 80:63 | warning | 'xmin' is a possible misspelling. | PostHogBase.Spelling |
| 91:5 | warning | 'xmin' is a possible misspelling. | PostHogBase.Spelling |
| 91:147 | warning | 'xmin' is a possible misspelling. | PostHogBase.Spelling |
| 97:1 | warning | 'xmin' is a possible misspelling. | PostHogBase.Spelling |
| 103:224 | warning | 'xmin' is a possible misspelling. | PostHogBase.Spelling |
| 103:246 | warning | 'unindexed' is a possible misspelling. | PostHogBase.Spelling |
| 113:4 | warning | 'How PostHog CDC works' heading should be in sentence case, and product names should be capitalized. | PostHogBase.SentenceCase |
| 124:95 | warning | Capitalize 'Data Warehouse' for PostHog's product. Use 'data warehouse' for the general industry concept. | PostHogBase.ProductNames |
| 147:5 | warning | 'CDC history table only' heading should be in sentence case, and product names should be capitalized. | PostHogBase.SentenceCase |
| 164:4 | warning | 'CDC requirements' heading should be in sentence case, and product names should be capitalized. | PostHogBase.SentenceCase |
| 270:86 | suggestion | Address the reader directly. Use 'you' instead of 'The user'. | PostHogDocs.DirectAddress |
| 272:196 | warning | 'resync' is a possible misspelling. | PostHogBase.Spelling |
| 311:4 | warning | 'Managing CDC after source creation' heading should be in sentence case, and product names should be capitalized. | PostHogBase.SentenceCase |
| 315:5 | warning | 'Enabling CDC on an existing source' heading should be in sentence case, and product names should be capitalized. | PostHogBase.SentenceCase |
| 323:1 | error | Hi, Andy here... use an en dash ( – ) with spaces. On Mac, holding down the Option and hyphen key will give you an en dash. | PostHogBase.EnDash |
| 334:5 | warning | 'Disabling CDC' heading should be in sentence case, and product names should be capitalized. | PostHogBase.SentenceCase |
| 340:34 | warning | Capitalize 'Workflows' for PostHog's product. Use 'workflows' for the general industry concept. | PostHogBase.ProductNames |
| 347:128 | error | Hi, Andy here... use an en dash ( – ) with spaces. On Mac, holding down the Option and hyphen key will give you an en dash. | PostHogBase.EnDash |
| 347:205 | warning | 'resync' is a possible misspelling. | PostHogBase.Spelling |
| 351:5 | warning | 'Updating CDC settings' heading should be in sentence case, and product names should be capitalized. | PostHogBase.SentenceCase |
| 355:1 | error | Hi, Andy here... use an en dash ( – ) with spaces. On Mac, holding down the Option and hyphen key will give you an en dash. | PostHogBase.EnDash |
| 356:1 | error | Hi, Andy here... use an en dash ( – ) with spaces. On Mac, holding down the Option and hyphen key will give you an en dash. | PostHogBase.EnDash |
| 357:1 | error | Hi, Andy here... use an en dash ( – ) with spaces. On Mac, holding down the Option and hyphen key will give you an en dash. | PostHogBase.EnDash |
| 361:4 | warning | 'WAL lag protection' heading should be in sentence case, and product names should be capitalized. | PostHogBase.SentenceCase |
| 370:320 | warning | 'resync' is a possible misspelling. | PostHogBase.Spelling |
| 393:108 | warning | 'resync' is a possible misspelling. | PostHogBase.Spelling |
| 395:217 | warning | 'resync' is a possible misspelling. | PostHogBase.Spelling |
| 420:5 | warning | 'Source is errored because of WAL lag' heading should be in sentence case, and product names should be capitalized. | PostHogBase.SentenceCase |
| 422:165 | warning | 'resync' is a possible misspelling. | PostHogBase.Spelling |
Contributor
Bundle reportTotal JS (gzip)6.21 MiB (no change) Eager graph (static-import closure per entrypoint)
Largest modules in the
|
| Module | Size |
|---|---|
css ./node_modules/.pnpm/css-loader@5.2.7_webpack@5.101.3/node_modules/css-loader/dist/cjs.js??ruleSet[1].rules[8].oneOf[1].use[1]!./node_modules/.pnpm/postcss-loader@4.3.0_postcss@8.5.6_webpack@5.101.3/node_modules/postcss-loader/dist/cjs.js??ruleSet[1].rules[8].oneOf[1].use[2]!./src/styles/global.css |
709.9 KiB |
./src/components/Stickers/Stickers.tsx |
696.4 KiB |
./.cache/caches/gatsby-plugin-mdx/mdx-scopes-dir/31a094f140f119e73085d847ae81b99b.js + 2 modules |
510.2 KiB |
./node_modules/.pnpm/@radix-ui+react-icons@1.3.2_react@18.3.1/node_modules/@radix-ui/react-icons/dist/react-icons.esm.js |
481.4 KiB |
./node_modules/.pnpm/@codemirror+view@6.38.2/node_modules/@codemirror/view/dist/index.js |
458.1 KiB |
./node_modules/.pnpm/rehype-raw@7.0.0/node_modules/rehype-raw/lib/index.js + 29 modules |
395.1 KiB |
./node_modules/.pnpm/@posthog+icons@0.36.6_react-dom@18.3.1_react@18.3.1__react@18.3.1/node_modules/@posthog/icons/dist/posthog-icons.cjs.js |
364.8 KiB |
./node_modules/.pnpm/@posthog+icons@0.36.6_react-dom@18.3.1_react@18.3.1__react@18.3.1/node_modules/@posthog/icons/dist/posthog-icons.es.js |
354.8 KiB |
./src/hooks/useCustomers.tsx + 54 modules |
353.9 KiB |
./node_modules/.pnpm/react-markdown@8.0.7_@types+react@16.14.66_react@18.3.1/node_modules/react-markdown/lib/react-markdown.js + 88 modules |
351.4 KiB |
./node_modules/.pnpm/cloudinary-core@2.14.0_lodash@4.17.21/node_modules/cloudinary-core/cloudinary-core.js |
281.9 KiB |
./node_modules/.pnpm/@codesandbox+sandpack-react@2.20.0_react-dom@18.3.1_react@18.3.1__react@18.3.1/node_modules/@codesandbox/sandpack-react/dist/index.mjs |
266.6 KiB |
./src/components/ProductComparisonTable/index.tsx + 114 modules |
264.0 KiB |
./node_modules/.pnpm/d3@7.9.0/node_modules/d3/src/index.js + 208 modules |
247.4 KiB |
./src/components/Pricing/PricingSlider/Slider.tsx + 87 modules |
239.9 KiB |
Eager-graph budgets are report-only until a baseline is established. Sizes are gzip of public/**/*.js; eager size is webpack module source bytes.
Contributor
|
Hey @danielcarletti! This docs PR was generated for you by an agent. You're responsible for reviewing and merging it into production.
|
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Changes
This PR documents the new xmin sync method for PostgreSQL sources, introduced in PostHog/posthog#64721.
What is xmin replication?
xmin replication uses PostgreSQL's hidden
xminsystem column (transaction ID) as an incremental cursor. It provides a cursorless incremental sync option for tables that don't have a goodupdated_ator other incremental field, without requiring the complexity of CDC/logical replication.Files updated
contents/docs/cdp/sources/postgres.mdx_ph_xmincolumn that's added to synced tablescontents/docs/cdp/sources/index.mdxKey details from the implementation PR
pg_snapshot_xmin(pg_current_snapshot()))This PR was created by the Inkeep Content Writer agent, which is maintained by the Docs and Wizard team. Please reach out on Slack for help if needed.