Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Original file line number Diff line number Diff line change
@@ -1,10 +1,10 @@
---
# frontmatter
path: "/tutorial-couchbase-capella-autovectorization-workflows-with-structured-data-and-langchain"
title: Auto-Vectorization of Strucutured Data with Couchbase Capella AI Services
title: Auto-Vectorization of Structured Data with Couchbase Capella AI Data Plane
short_title: Auto-Vectorization with Couchbase and Semantic Search using LangChain
description:
- Learn how to use Couchbase Capella's AI Services auto-vectorization feature to automatically convert your structured data into vector embeddings.
- Learn how to use Couchbase Capella's AI Data Plane auto-vectorization feature to automatically convert your structured data into vector embeddings.
- To learn about the auto-vectorization of unstuctured data read the following [tutorial](tutorial-couchbase-autovectorization-workdlows-with-unstructured-data-and-langchain).
- This tutorial demonstrates how to set up automated embedding generation workflows and perform semantic search using LangChain.
content_type: tutorial
Expand Down Expand Up @@ -57,7 +57,7 @@ Now, before we actually create embeddings for the documents, we need to deploy a
> **⚠️ IMPORTANT:** The model **must** be deployed in the **same region** as your database cluster for workflows to function properly. Failing to match regions will prevent the workflow from working and may require cluster redeployment.

## Selecting the Model
1. To select the model, you first need to navigate to the "<B>AI Services</B>" tab, then select "<B>Models</B>" and click on "<B>Deploy New Model</B>".
1. To select the model, you first need to navigate to the "<B>AI Data Plane</B>" tab, then select "<B>Models</B>" and click on "<B>Deploy New Model</B>".

<img src="https://github.com/couchbase-examples/vector-search-cookbook/blob/main/autovec-structured/img/importing_model.png?raw=true" width="950px" height="500px" style="padding: 5px; border-radius: 10px 20px 30px 40px; border: 2px solid #555;">

Expand All @@ -67,7 +67,7 @@ Now, before we actually create embeddings for the documents, we need to deploy a

## Access Control to the Model

1. After deploying the model, go to the "<B>Models</B>" tab in the <B>AI Services</B> and click on "<B>Setup Access</B>".
1. After deploying the model, go to the "<B>Models</B>" tab in the <B>AI Data Plane</B> and click on "<B>Setup Access</B>".

<img src="https://github.com/couchbase-examples/vector-search-cookbook/blob/main/autovec-structured/img/model_setup_access.png?raw=true" width="1100px" height="400px" style="padding: 5px; border-radius: 10px 20px 30px 40px; border: 2px solid #555;">

Expand All @@ -84,7 +84,7 @@ Now, before we actually create embeddings for the documents, we need to deploy a

Now, we are at the step that will help us create the embeddings/vectors. To proceed with the vectorization process, please follow the steps below. For more details, refer to the [data processing documentation](https://docs.couchbase.com/ai/build/vectorization-service/data-processing.html).

1. For deploying the autovectorization, you need to go to the <B>`AI Services`</B> tab, then click on <B>`Workflows`</B>, and then click on <B>`Create New Workflow`</B>.
1. For deploying the autovectorization, you need to go to the <B>`AI Data Plane`</B> tab, then click on <B>`Workflows`</B>, and then click on <B>`Create New Workflow`</B>.

<img src="https://github.com/couchbase-examples/vector-search-cookbook/blob/main/autovec-structured/img/workflow.png?raw=true" width="1000px" height="500px" style="padding: 5px; border-radius: 10px 20px 30px 40px; border: 2px solid #555;">

Expand Down
Original file line number Diff line number Diff line change
@@ -1,10 +1,10 @@
---
# frontmatter
path: "/tutorial-couchbase-autovectorization-workflows-with-unstructured-data-and-langchain"
title: Auto-Vectorization on Unstructured Data Stored in S3 Buckets Using Couchbase Capella AI Services
title: Auto-Vectorization on Unstructured Data Stored in S3 Buckets Using Couchbase Capella AI Data Plane
short_title: Auto-Vectorization on Unstructured Data Stored in S3 Buckets
description:
- Learn how to use Couchbase Capella's AI Services Auto-Vectorization feature to automatically process unstructured data from S3 buckets.
- Learn how to use Couchbase Capella's AI Data Plane Auto-Vectorization feature to automatically process unstructured data from S3 buckets.
- Configure workflows to chunk and vectorize documents (PDFs, images, etc.) and import them into Capella collections.
- Perform semantic vector search using LangChain and the generated embeddings.
content_type: tutorial
Expand Down Expand Up @@ -39,7 +39,7 @@ When running Couchbase using [Capella](https://cloud.couchbase.com/sign-in), the
# Deploying the Model
Now, before we actually create embeddings for the documents, we need to deploy a model that will create the embeddings for us. Make sure the model is deployed in the same region as that of database for workflows to work. To know more about model services click [here](https://docs.couchbase.com/ai/build/model-service/deploy-embed-model.html).
## Selecting the Model
1. To select the model, you first need to navigate to the "<B>AI Services</B>" tab, then select "<B>Models</B>" and click on "<B>Deploy New Model</B>".
1. To select the model, you first need to navigate to the "<B>AI Data Plane</B>" tab, then select "<B>Models</B>" and click on "<B>Deploy New Model</B>".

<img src="https://github.com/couchbase-examples/vector-search-cookbook/blob/main/autovec_unstructured/img/importing_model.png?raw=true" width="950px" height="500px" style="padding: 5px; border-radius: 10px 20px 30px 40px; border: 2px solid #555;">

Expand All @@ -49,7 +49,7 @@ Now, before we actually create embeddings for the documents, we need to deploy a

## Access Control to the Model

1. After deploying the model, go to the "<B>Models</B>" tab in the <B>AI Services</B> and click on "<B>Setup Access</B>".
1. After deploying the model, go to the "<B>Models</B>" tab in the <B>AI Data Plane</B> and click on "<B>Setup Access</B>".

<img src="https://github.com/couchbase-examples/vector-search-cookbook/blob/main/autovec_unstructured/img/model_setup_access.png?raw=true" width="1100px" height="400px" style="padding: 5px; border-radius: 10px 20px 30px 40px; border: 2px solid #555;">

Expand All @@ -64,7 +64,7 @@ Now, before we actually create embeddings for the documents, we need to deploy a
# Data upload from S3 bucket to Couchbase (with chunking and vectorization)

In order to import unstructured data from the S3 bucket, you need to create a workflow that connects to your S3 bucket and chunks your unstructured data before importing it into the collections. To do so, please follow the steps mentioned below:
1) Let's start by creating a new workflow. This can be done by clicking on the <B>`AI Services`</B> tab, then click on <B>`Workflows`</B>, and then click on <B>`Create New Workflow`</B>.
1) Let's start by creating a new workflow. This can be done by clicking on the <B>`AI Data Plane`</B> tab, then click on <B>`Workflows`</B>, and then click on <B>`Create New Workflow`</B>.

<img src="https://github.com/couchbase-examples/vector-search-cookbook/blob/main/autovec_unstructured/img/workflow.png?raw=true" width="1000px" height="500px" style="padding: 5px; border-radius: 10px 20px 30px 40px; border: 2px solid #555;">

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -34,7 +34,7 @@ This notebook demonstrates how to build a RAG system using:
- The [BBC News dataset](https://huggingface.co/datasets/RealTimeData/bbc_news_alltime) containing news articles
- Couchbase Capella as the vector store
- LlamaIndex framework for the RAG pipeline
- Capella AI Services for embeddings and text generation
- Capella AI Data Plane for embeddings and text generation

Semantic search goes beyond simple keyword matching by understanding the context and meaning behind the words in a query, making it an essential tool for applications that require intelligent information retrieval. This tutorial will equip you with the knowledge to create a fully functional RAG system using Capella Model Services and LlamaIndex.

Expand Down Expand Up @@ -437,9 +437,9 @@ try:

# Configure LlamaIndex to use this LLM
Settings.llm = llm
logging.info("Successfully created the LLM in Capella AI Services")
logging.info("Successfully created the LLM in Capella AI Data Plane")
except Exception as e:
raise ValueError(f"Error creating LLM in Capella AI Services: {str(e)}")
raise ValueError(f"Error creating LLM in Capella AI Data Plane: {str(e)}")
```

# Creating the Vector Store Index
Expand Down Expand Up @@ -492,9 +492,9 @@ except RecursionError as e:
raise RuntimeError(f"Error performing semantic search: {e}")
```

## Caching in Capella AI Services
## Caching in Capella AI Data Plane

To optimize performance and reduce costs, Capella AI services employ two caching mechanisms:
To optimize performance and reduce costs, Capella AI Data Plane employs two caching mechanisms:

1. Semantic Cache

Expand Down Expand Up @@ -560,6 +560,6 @@ The key components of our RAG system include:

1. **Couchbase Capella** as the vector database for storing and retrieving document embeddings
2. **LlamaIndex** as the framework for connecting our data to the LLM
3. **Capella AI Services** for generating embeddings and LLM responses
3. **Capella AI Data Plane** for generating embeddings and LLM responses

This approach allows us to enhance the capabilities of large language models by grounding their responses in specific, up-to-date information from our knowledge base.
Loading