|
| 1 | += Configure Change Data Capture (CDC) with the {product} |
| 2 | +:navtitle: Change Data Capture quickstart |
| 3 | + |
| 4 | +Change Data Capture (CDC) for {astra_db} automatically captures changes in real time, de-duplicates the changes, and then streams the clean set of changed data into xref:astra-streaming:ROOT:index.adoc[{astra_stream}] where it can be processed by client applications or sent to downstream systems. |
| 5 | + |
| 6 | +This page provides an end-to-end example that shows you how to use the {product} to enable CDC for an {astra-db} database. |
| 7 | + |
| 8 | +== Create a database |
| 9 | + |
| 10 | +Create an {astra_db} database: |
| 11 | + |
| 12 | +[source,shell] |
| 13 | +---- |
| 14 | +astra db create cdc_demo_db -r us-east1 -k cdc_demo_keyspace |
| 15 | +---- |
| 16 | + |
| 17 | +.Result |
| 18 | +[%collapsible] |
| 19 | +==== |
| 20 | +[source,console] |
| 21 | +---- |
| 22 | +Database 'cdc_demo_db' has been created with id 'ce5e563c-3248-4064-b03c-5892433a1347'. It is now active after waiting 422 seconds. |
| 23 | +---- |
| 24 | +==== |
| 25 | + |
| 26 | +== Create a streaming tenant |
| 27 | + |
| 28 | +Create an {astra_stream} tenant in the same region as your database: |
| 29 | + |
| 30 | +[source,shell] |
| 31 | +---- |
| 32 | +astra streaming create cdc-demo-tenant -c gcp -r useast1 |
| 33 | +---- |
| 34 | + |
| 35 | +.Result |
| 36 | +[%collapsible] |
| 37 | +==== |
| 38 | +[source,console] |
| 39 | +---- |
| 40 | +https://api.astra.datastax.com/v2/streaming/tenants/cdc-demo-tenant |
| 41 | +[OK] Tenant 'cdc-demo-tenant' has being created. |
| 42 | +---- |
| 43 | +==== |
| 44 | + |
| 45 | +== Create a table in your database |
| 46 | + |
| 47 | +Use `xref:commands:astra-db-cqlsh-exec.adoc[]` to create a table with a primary key column in your database: |
| 48 | + |
| 49 | +[source,shell] |
| 50 | +---- |
| 51 | +astra db cqlsh exec cdc_demo_db \ |
| 52 | + "CREATE TABLE IF NOT EXISTS cdc_demo_keyspace.cdc_demo_table ( \ |
| 53 | + key text PRIMARY KEY, \ |
| 54 | + c1 text \ |
| 55 | + );" |
| 56 | +---- |
| 57 | + |
| 58 | +.Result |
| 59 | +[%collapsible] |
| 60 | +==== |
| 61 | +[source,console] |
| 62 | +---- |
| 63 | +[INFO] Cqlsh is starting, please wait for connection establishment... |
| 64 | +---- |
| 65 | +==== |
| 66 | + |
| 67 | +Confirm table creation: |
| 68 | ++ |
| 69 | +[source,shell] |
| 70 | +---- |
| 71 | +astra db cqlsh exec cdc_demo_db \ |
| 72 | + "SELECT * FROM cdc_demo_keyspace.cdc_demo_table" |
| 73 | +---- |
| 74 | + |
| 75 | +.Result |
| 76 | +[%collapsible] |
| 77 | +==== |
| 78 | +[source,console] |
| 79 | +---- |
| 80 | +[INFO] Cqlsh is starting, please wait for connection establishment... |
| 81 | +
|
| 82 | + key | c1 |
| 83 | +-----+---- |
| 84 | +
|
| 85 | +(0 rows) |
| 86 | +---- |
| 87 | +==== |
| 88 | + |
| 89 | +== Create a CDC connection |
| 90 | + |
| 91 | +Use `xref:commands:astra-db-create-cdc.adoc[]` to create a CDC connection between your database table and streaming tenant: |
| 92 | + |
| 93 | +[source,shell] |
| 94 | +---- |
| 95 | +astra db create-cdc cdc_demo_db \ |
| 96 | + -k cdc_demo_keyspace \ |
| 97 | + --table cdc_demo_table \ |
| 98 | + --tenant cdc-demo-tenant |
| 99 | +---- |
| 100 | ++ |
| 101 | +//// |
| 102 | +// TODO: The command is not working as expected. Must investigate and figure out why it is reporting the following error: [ERROR] INVALID_ARGUMENT: Error Code=422(422) Invalid information provided to create DB: 422 Unprocessable Entity: databaseId, keyspace, tableName, and orgId are mandatory fields |
| 103 | +.Result |
| 104 | +[%collapsible] |
| 105 | +==== |
| 106 | +[source,console] |
| 107 | +---- |
| 108 | +
|
| 109 | +---- |
| 110 | +==== |
| 111 | +//// |
| 112 | + |
| 113 | +Use `xref:commands:astra-db-list-cdcs.adoc[]` to confirm CDC details for the database and tenant: |
| 114 | + |
| 115 | +[source,shell] |
| 116 | +---- |
| 117 | +astra db list-cdcs cdc_demo_db |
| 118 | +---- |
| 119 | + |
| 120 | +.Result |
| 121 | +[%collapsible] |
| 122 | +==== |
| 123 | +[source,console] |
| 124 | +---- |
| 125 | ++-----------------------+-------------------+----------------+----------------+--------------------+----------------+----------------+ |
| 126 | +| id | keyspace | table | tenant | cluster | namespace | Status | |
| 127 | ++-----------------------+-------------------+----------------+----------------+--------------------+----------------+----------------+ |
| 128 | +| 57a3024f-cdcdemotable | cdc_demo_keyspace | cdc_demo_table | cdc-demo-tenant| pulsar-aws-useast1 | astracdc | Running | |
| 129 | ++-----------------------+-------------------+----------------+----------------+--------------------+----------------+----------------+ |
| 130 | +---- |
| 131 | +==== |
| 132 | + |
| 133 | +[source,shell] |
| 134 | +---- |
| 135 | +astra streaming list-cdc cdc-demo-tenant |
| 136 | +---- |
| 137 | + |
| 138 | +.Result |
| 139 | +[%collapsible] |
| 140 | +==== |
| 141 | +[source,console] |
| 142 | +---- |
| 143 | ++--------------------+----------------+----------------+-------------------+----------------+----------------+ |
| 144 | +| cluster | namespace | database | keyspace | table | status | |
| 145 | ++--------------------+----------------+----------------+-------------------+----------------+----------------+ |
| 146 | +| pulsar-aws-useast1 | astracdc | cdc_demo_db | cdc_demo_keyspace | cdc_demo_table | running | |
| 147 | ++--------------------+----------------+----------------+-------------------+----------------+----------------+ |
| 148 | +---- |
| 149 | +==== |
| 150 | + |
| 151 | +== Connect a sink |
| 152 | + |
| 153 | +After you enable CDC on your {db-serverless} database, you're ready to xref:astra-streaming:developing:astream-cdc.adoc#connect-a-sink[connect a sink]. |
0 commit comments