From 1d308df3927b659a0a258f3f965fca80224cdddf Mon Sep 17 00:00:00 2001 From: prajwal-pai77 Date: Wed, 8 Jul 2026 14:36:30 +0530 Subject: [PATCH 1/2] update looker studio data api connector --- .../looker-studio/dataapi/readme.md | 217 ++++++------------ 1 file changed, 75 insertions(+), 142 deletions(-) diff --git a/tutorial/markdown/connectors/looker-studio/dataapi/readme.md b/tutorial/markdown/connectors/looker-studio/dataapi/readme.md index be3ef47..44e3471 100644 --- a/tutorial/markdown/connectors/looker-studio/dataapi/readme.md +++ b/tutorial/markdown/connectors/looker-studio/dataapi/readme.md @@ -25,208 +25,141 @@ length: 20 Mins -## Overview +## Introduction -Use this connector to build Looker Studio reports directly on Couchbase via the Data API. You can: +In this tutorial, you'll connect [Google Looker Studio](https://lookerstudio.google.com/) to Couchbase using the Couchbase Data API connector and build reports directly on live data in your cluster — with no ETL and no intermediate storage. You'll authenticate to your cluster, choose the data to visualize (either by selecting a `bucket.scope.collection` or by writing a custom [SQL++](https://docs.couchbase.com/server/current/n1ql/n1ql-language-reference/index.html) query), and let the connector infer your schema automatically so fields are ready to drop into charts and tables within minutes. The connector reaches your cluster through the [Couchbase Data API](https://docs.couchbase.com/server/current/rest-api/rest-intro.html) and runs your queries via the Query Service — all from within Looker Studio. -- Query by selecting a specific `bucket.scope.collection`. -- Or run a custom SQL++ query. +What you'll build: a live Looker Studio data source connected to a Couchbase collection, with fields inferred and ready to use in charts and tables. -Behind the scenes, the connector authenticates with Basic Auth and talks to the Data API endpoints for caller identity checks and to the Query Service for SQL++ execution. Schema is inferred automatically from sampled data to make fields available in Looker Studio. +## How to run this tutorial -## Getting Access +This tutorial runs entirely in the Looker Studio web UI — there is nothing to install locally and no code to write. The only thing you need beforehand is access to a Couchbase cluster with the Data API enabled, covered in [Before you start](#before-you-start). -To use this connector, you need access to Couchbase: +## Before you start -### Couchbase Capella (Recommended) -- **Sign up**: Visit [Couchbase Capella](https://cloud.couchbase.com/) to create your account -- **Free Trial**: Start with a free trial to test the connector -- **Query Service**: All Capella clusters include the Query Service needed for this connector -- **Pricing**: Capella is billed based on compute and storage usage. See [Capella Pricing](https://www.couchbase.com/pricing/capella/) for details +### Have a Google account -### Self-Managed Couchbase -- Install Couchbase Server with Query Service enabled -- Ensure your deployment has a publicly accessible endpoint for Looker Studio -- Configure appropriate user credentials with query permissions +You need a Google account to sign in to [Looker Studio](https://lookerstudio.google.com/). -## Installation +### Create and Deploy Your Free Tier Operational cluster on Capella -To add the Couchbase Data API connector to your Looker Studio reports: +To get started with Couchbase Capella, create an account and use it to deploy a forever free tier operational cluster. This account provides you with an environment where you can explore and learn about Capella with no time constraint. -1. **Open Looker Studio**: Go to [Looker Studio](https://lookerstudio.google.com/) -2. **Create or Open Report**: Start a new report or open an existing one -3. **Add Data Source**: Click "Add data" or the "+" button -4. **Find Connector**: Search for "Couchbase Data API" in the connector gallery -5. **Authorize**: Grant necessary permissions when prompted -6. **Configure**: Follow the authentication and configuration steps below +To know more, please follow the [instructions](https://docs.couchbase.com/cloud/get-started/create-account.html). -## Prerequisites +#### Capella Configuration -- A Couchbase Capella cluster or a self-managed cluster with the Query Service reachable from Looker Studio. -- A database user with permissions to read the target collections and run queries. -- Network access from Looker Studio to your cluster host. +- Create the [database credentials](https://docs.couchbase.com/cloud/clusters/manage-database-users.html) to access the travel-sample bucket (Read and Write) used in the connector. The user also needs permission to query the system catalogs (`system:buckets`, `system:all_scopes`, `system:keyspaces`) and run `INFER` — these are used for collection discovery and schema inference. +- [Allow access](https://docs.couchbase.com/cloud/clusters/allow-ip-address.html) to the cluster by adding `0.0.0.0/0` (allow all) under Settings → Networking → Allowed IP Addresses. Looker Studio runs on Google's servers with dynamic IP addresses, so a fixed range cannot be allowlisted. +- [Enable the Data API](https://docs.couchbase.com/cloud/data-api-guide/data-api-start.html#enable-the-data-api) on your cluster: go to Connect → Data API and click Enable Data API. Once enabled, copy your Data API endpoint — you'll need it during authentication. +- Load the travel-sample bucket: go to Data Tools → Import and import travel-sample. This is used in the [Build your first report](#build-your-first-report) section. -## Authentication +## Add the connector in Looker Studio + +Once the prerequisites above are in place: -When you add the data source in Looker Studio, you will be prompted for: +1. Open Looker Studio — go to [Looker Studio](https://lookerstudio.google.com/) and sign in. +2. Create or open a report — start a new report or open an existing one. +3. Add a data source — click Add data (or the + button). +4. Find the connector — search for "Couchbase Data API" in the connector gallery and select it. -- Path: The cluster host (optionally with port). Examples: - - Capella: `cb..cloud.couchbase.com` - - Self-managed: `my.host:18095` (specify a non-443 port explicitly) -- Username and Password: Database credentials. +Next, you'll authenticate and configure the data source. -The connector validates credentials against the Data API (`/v1/callerIdentity`). If validation fails, verify host, port, credentials, and network access. +## Authentication -## Configuration +When prompted, enter your credentials. Note that Looker Studio labels the endpoint field "Path" — this is your Couchbase Data API endpoint. -After authentication, choose a configuration mode: +- Path — Your Data API endpoint. Copy it from Connect → Data API in the Capella UI. +- Username / Password — Your Couchbase database credentials. -- Configuration Mode: `Query by Collection` or `Use Custom Query`. +Refer to the [Couchbase Data API documentation](https://docs.couchbase.com/cloud/data-api-guide/data-api-start.html) for details on locating your endpoint. -### Mode: Query by Collection +## Build your first report -- Couchbase Collection: Pick a `bucket > scope > collection` from the dropdown. The connector discovers collections for you. -- Maximum Rows: Optional limit for returned rows (default 100). +After authenticating, you are taken to the configuration screen. Choose a mode and connect to your data. -What runs: +### Mode: Query by Collection -- Data: `SELECT RAW collection FROM \`bucket\`.\`scope\`.\`collection\` LIMIT ` -- Schema: `INFER \`bucket\`.\`scope\`.\`collection\` WITH {"sample_size": 100, "num_sample_values": 3, "similarity_metric": 0.6}` +Select a `bucket > scope > collection` from the dropdown — the connector discovers them automatically. Use this mode for quick exploration of a single collection. -### Mode: Use Custom Query +For this example, select travel-sample → inventory → airline and leave Maximum Rows at the default (100), then click Connect and Add to Report. -- Custom SQL++ Query: Paste any valid SQL++ statement. Include a `LIMIT` for performance. + -What runs: +Once added, click Insert → Bar chart. In the Data panel on the right, set Dimension to `country` and leave Metric as `Record Count`. Looker Studio renders a bar chart of airlines grouped by country. -- Schema inference first attempts to run `INFER` on your query (a `LIMIT 100` is added if absent): `INFER () WITH {"sample_size": 10000, "num_sample_values": 2, "similarity_metric": 0.1}` -- If that fails, it runs your query with `LIMIT 1` and infers the schema from one sample document. + -## Schema and Field Types +Every time Looker Studio refreshes, it re-queries the collection live — no cache, no ETL. + +### Mode: Use Custom Query -- Fields are inferred from sampled data. Types map to Looker Studio as: - - NUMBER → metric - - BOOLEAN → dimension - - STRING (default for text, objects, arrays) → dimension -- Nested fields use dot notation (for example, `address.city`). Arrays and objects not expanded become stringified values. -- If the collection has no documents or your query returns no rows, schema inference will fail. +Use this mode to write your own [SQL++](https://docs.couchbase.com/server/current/n1ql/n1ql-language-reference/index.html) query. Paste any valid statement — include a `LIMIT` for performance. This gives you full control over filtering, joining, and aggregating before the data reaches Looker Studio. -> **⚠️ Schema Inference Limitations**: Field types are inferred from sampled data and may not capture all variations in your dataset. Common issues include: -> - **Mixed data types**: Fields containing both numbers and text will be typed as STRING -> - **Incomplete sampling**: Fields present only in unsampled documents may not be detected -> - **Array complexity**: Arrays of objects become stringified JSON rather than individual fields -> - **Nested object depth**: Very deep object hierarchies may not be fully expanded -> - **Empty or null values**: Fields with only null values may not be detected or may be typed incorrectly +For example, the same airline count pre-aggregated on the cluster: -## Data Retrieval +```sql +SELECT country, COUNT(*) AS airline_count +FROM `travel-sample`.`inventory`.`airline` +GROUP BY country +ORDER BY airline_count DESC +LIMIT 20 +``` -- Only the fields requested by Looker Studio are returned. Nested values are extracted using dot paths where possible. -- Row limits: - - Collection mode: `Maximum Rows` controls the `LIMIT` (default 100). - - Custom query mode: You control `LIMIT` inside your query. +Pre-aggregating in SQL++ is more efficient for large collections than pulling all rows into Looker Studio. ## Tips and Best Practices -- **Prefer `Query by Collection` for quick starts and simpler schemas**: Collection mode provides more predictable schema inference than custom queries. -- **Always add a `LIMIT` when exploring with custom queries**: Use `LIMIT 100-1000` for initial testing to ensure fast schema inference and data retrieval. -- **Ensure your user has at least query and read access** on the target collections and system catalogs for metadata discovery. -- **For consistent schema inference**: Structure your data with consistent field types across documents. Avoid mixing numbers and strings in the same field. -- **Handle complex nested data**: Consider flattening deeply nested objects in your SQL++ queries for better Looker Studio compatibility. -- **Test schema inference separately**: Use small LIMIT clauses first to verify schema detection before processing large datasets. +- Prefer `Query by Collection` for quick starts and simpler schemas: Collection mode provides more predictable schema inference than custom queries. +- Always add a `LIMIT` when exploring with custom queries: Use `LIMIT 100-1000` for initial testing to ensure fast schema inference and data retrieval. +- Ensure your user has at least query and read access on the target collections and system catalogs for metadata discovery. +- For consistent schema inference: Structure your data with consistent field types across documents. Avoid mixing numbers and strings in the same field. +- Handle complex nested data: Consider flattening deeply nested objects in your SQL++ queries for better Looker Studio compatibility. +- Test schema inference separately: Use small LIMIT clauses first to verify schema detection before processing large datasets. ## Troubleshooting ### Authentication and Connection Issues -- **Authentication error**: Check host/port, credentials, and that the Data API is reachable from Looker Studio. -- **Timeout or network errors**: Verify network connectivity and firewall settings between Looker Studio and your Couchbase cluster. +- Authentication error: Check host/port, credentials, and that the Data API is reachable from Looker Studio. +- Timeout or network errors: Verify network connectivity and firewall settings between Looker Studio and your Couchbase cluster. ### Schema Inference Problems -- **Empty schema or no fields detected**: +- Empty schema or no fields detected: - Ensure the collection contains documents and is not empty - For custom queries, verify the statement returns results and add appropriate `LIMIT` clauses - Check that your user has permissions to read the collection and execute queries - -- **INFER statement failures**: + +- INFER statement failures: - The connector first attempts `INFER collection` or `INFER (customQuery)` with sampling options - If INFER fails, it falls back to executing your query with `LIMIT 1` and inferring from a single document - INFER may fail on very large collections or complex queries - the fallback usually resolves this - -- **Fields appear as STRING when they should be NUMBER**: + +- Fields appear as STRING when they should be NUMBER: - Your data has mixed types (some documents have numbers, others have strings) in the same field - The connector defaults to STRING for safety when types are inconsistent - Consider data cleanup or use SQL++ functions to cast types consistently - -- **Missing fields that exist in your data**: + +- Missing fields that exist in your data: - Schema inference is sample-based - fields present only in unsampled documents may not be detected - Try increasing the collection size or adjusting your query to ensure representative sampling - For custom queries, ensure your query includes all the fields you want to expose - -- **Nested fields not working correctly**: + +- Nested fields not working correctly: - Very deep object hierarchies may not be fully expanded by the INFER process - Arrays of objects become stringified JSON instead of individual fields - Consider flattening complex structures in your SQL++ query for better field detection - -- **"No properties in any INFER flavors" error**: + +- "No properties in any INFER flavors" error: - The INFER statement succeeded but found no recognizable field structures - This typically happens with collections containing only primitive values or very inconsistent document structures - Try a custom query that shapes the data into a more consistent structure -### Query and Data Issues -- **Query errors from the service**: Review the error text surfaced in Looker Studio; fix syntax, permissions, or keyspace names. -- **Permission errors during schema inference**: Ensure your user can execute INFER statements and read from system catalogs. -- **Performance issues**: Add appropriate `LIMIT` clauses and avoid very complex JOINs for better connector performance. - -## Next Steps - -- Create charts and tables in Looker Studio from the exposed fields. -- Iterate on custom SQL++ queries to shape the dataset for your dashboards. - -## Support - -For assistance with the Couchbase Data API Looker Studio connector: - -### Technical Support -- **Documentation**: Complete guides at [Couchbase Developer Portal](https://developer.couchbase.com/) -- **Community Forum**: Get help from the community at [Couchbase Forums](https://forums.couchbase.com/) -- **Developer Community**: Connect with [Couchbase Developer Community](https://www.couchbase.com/developers/community/) -- **Discord**: Join real-time discussions on [Couchbase Discord](https://discord.gg/sQ5qbPZuTh) - -### Issue Reporting -- **Connector Issues**: Report bugs or feature requests through [Couchbase Support](https://support.couchbase.com/hc/en-us) -- **General Contact**: For other inquiries, use [Couchbase Contact](https://www.couchbase.com/contact/) -- **Response Time**: We aim to respond to connector issues within 2-3 business days -- **Known Limitations**: Check the troubleshooting section above for common issues - -### Resources -- **Data API Documentation**: [Official Data API Docs](https://docs.couchbase.com/server/current/rest-api/rest-intro.html) -- **Query Service Documentation**: [SQL++ Query Documentation](https://docs.couchbase.com/server/current/n1ql/n1ql-language-reference/index.html) -- **Looker Studio Help**: [Google Looker Studio Support](https://support.google.com/looker-studio/) -- **Sample Data**: Use travel-sample bucket for testing and examples - -## Privacy Policy - -**Connector Data Usage**: This connector accesses your Couchbase database credentials and query results solely for data integration with Looker Studio. No data is stored permanently by the connector, and all processing occurs within Google's Looker Studio environment. - -**Official Privacy Policy**: For complete information about how Couchbase handles your data, please review our official [Couchbase Privacy Policy](https://www.couchbase.com/privacy-policy/). - -**Google Looker Studio**: This connector operates within Google Looker Studio, which has its own data handling policies. Please review [Google's Privacy Policy](https://policies.google.com/privacy) for their data practices. - -## Terms of Service - -**Connector Usage**: Use this connector only with Couchbase instances you own or have permission to access. Maintain security of your credentials and comply with your organization's data governance policies. - -**Official Terms**: Your use of Couchbase products and services is governed by our official [Couchbase Terms of Use](https://www.couchbase.com/terms-of-use/). For enterprise customers, additional terms may apply under your specific [Couchbase License Agreement](https://www.couchbase.com/legal/agreements/). - -**Google Looker Studio**: This connector operates within Google Looker Studio and is subject to [Google's Terms of Service](https://policies.google.com/terms). - -## Legal - -**Copyright**: © 2024 Couchbase, Inc. All rights reserved. - -**Legal Information**: For comprehensive legal information, licensing terms, and trademark policies, visit [Couchbase Legal](https://www.couchbase.com/legal/) and our [Trademark Policy](https://www.couchbase.com/trademark-policy/). - -**Trademarks**: Couchbase and the Couchbase logo are trademarks of Couchbase, Inc. Google Looker Studio is a trademark of Google LLC. +### Query and Data Issues +- Query errors from the service: Review the error text surfaced in Looker Studio; fix syntax, permissions, or keyspace names. +- Permission errors during schema inference: Ensure your user can execute INFER statements and read from system catalogs. +- Performance issues: Add appropriate `LIMIT` clauses and avoid very complex JOINs for better connector performance. -**Contact**: For legal inquiries, contact [legal@couchbase.com](mailto:legal@couchbase.com). +## Conclusion +You've connected Google Looker Studio to Couchbase using the Data API connector and built a live report on travel-sample data. For the connector source code and to report issues, see the [Couchbase Data API Looker Studio connector on GitHub](https://github.com/Couchbase-Ecosystem/couchbase-gcp-lookerstudio-connector). From 606f414c40822ad03d534a3e4ee5fa1bcb7ce37d Mon Sep 17 00:00:00 2001 From: prajwal-pai77 Date: Thu, 16 Jul 2026 13:45:18 +0530 Subject: [PATCH 2/2] address comments --- .../looker-studio/dataapi/readme.md | 26 ++++++++++++++++--- 1 file changed, 23 insertions(+), 3 deletions(-) diff --git a/tutorial/markdown/connectors/looker-studio/dataapi/readme.md b/tutorial/markdown/connectors/looker-studio/dataapi/readme.md index 44e3471..994f3ea 100644 --- a/tutorial/markdown/connectors/looker-studio/dataapi/readme.md +++ b/tutorial/markdown/connectors/looker-studio/dataapi/readme.md @@ -49,7 +49,7 @@ To know more, please follow the [instructions](https://docs.couchbase.com/cloud/ #### Capella Configuration -- Create the [database credentials](https://docs.couchbase.com/cloud/clusters/manage-database-users.html) to access the travel-sample bucket (Read and Write) used in the connector. The user also needs permission to query the system catalogs (`system:buckets`, `system:all_scopes`, `system:keyspaces`) and run `INFER` — these are used for collection discovery and schema inference. +- Create the [database credentials](https://docs.couchbase.com/cloud/clusters/manage-database-users.html) with Read access to the travel-sample bucket used in the connector. The user also needs permission to query the system catalogs (`system:buckets`, `system:all_scopes`, `system:keyspaces`) and run `INFER` — these are used for collection discovery and schema inference. - [Allow access](https://docs.couchbase.com/cloud/clusters/allow-ip-address.html) to the cluster by adding `0.0.0.0/0` (allow all) under Settings → Networking → Allowed IP Addresses. Looker Studio runs on Google's servers with dynamic IP addresses, so a fixed range cannot be allowlisted. - [Enable the Data API](https://docs.couchbase.com/cloud/data-api-guide/data-api-start.html#enable-the-data-api) on your cluster: go to Connect → Data API and click Enable Data API. Once enabled, copy your Data API endpoint — you'll need it during authentication. - Load the travel-sample bucket: go to Data Tools → Import and import travel-sample. This is used in the [Build your first report](#build-your-first-report) section. @@ -90,7 +90,7 @@ Once added, click Insert → Bar chart. In the Data panel on the right, set Dime -Every time Looker Studio refreshes, it re-queries the collection live — no cache, no ETL. +The connector adds no intermediate storage or ETL — when it queries the source, it reads live from your cluster. Note, however, that Looker Studio caches query results on its own end (its "data freshness" cache), so charts may serve cached data between refreshes rather than hitting the cluster on every view. Community connectors typically use Looker Studio's default freshness (up to 12 hours), which is generally not user-adjustable. You can force a live re-query at any time with **Refresh data** in the report toolbar. ### Mode: Use Custom Query @@ -161,5 +161,25 @@ Pre-aggregating in SQL++ is more efficient for large collections than pulling al ## Conclusion -You've connected Google Looker Studio to Couchbase using the Data API connector and built a live report on travel-sample data. For the connector source code and to report issues, see the [Couchbase Data API Looker Studio connector on GitHub](https://github.com/Couchbase-Ecosystem/couchbase-gcp-lookerstudio-connector). +You've connected Google Looker Studio to Couchbase using the Data API connector and built a live report on travel-sample data. + +## Next Steps + +Now that you have a live connection, you can: + +- Build additional charts and dashboards on other collections in your cluster. +- Switch to `Use Custom Query` mode to filter, join, or pre-aggregate data with SQL++ before it reaches Looker Studio. + +### Support + +The connector is open source. For the source code, or to report a bug or request a feature, see the [Couchbase Data API Looker Studio connector on GitHub](https://github.com/Couchbase-Ecosystem/couchbase-gcp-lookerstudio-connector) and open an [issue](https://github.com/Couchbase-Ecosystem/couchbase-gcp-lookerstudio-connector/issues). + +### References + +- [Couchbase Data API guide](https://docs.couchbase.com/cloud/data-api-guide/data-api-start.html) +- [Couchbase Data API reference](https://docs.couchbase.com/cloud/data-api-guide/data-api-intro.html) +- [SQL++ (N1QL) language reference](https://docs.couchbase.com/server/current/n1ql/n1ql-language-reference/index.html) +- [Couchbase Capella documentation](https://docs.couchbase.com/cloud/get-started/intro.html) +- [Google Looker Studio Help Center](https://support.google.com/looker-studio) +- [Looker Studio community connectors](https://developers.google.com/looker-studio/connector)