Adds explanation on when to use dense or sparse embeddings#6727
Adds explanation on when to use dense or sparse embeddings#6727kosabogi wants to merge 1 commit into
Conversation
Elastic Docs AI PR menuCheck the box to run an AI review for this pull request.
Powered by GitHub Agentic Workflows and docs-actions. For more information, reach out to the docs team. |
🔍 Preview links for changed docs |
Vale Linting ResultsSummary: 1 suggestion found 💡 Suggestions (1)
The Vale linter checks documentation changes against the Elastic Docs style guide. To use Vale locally or report issues, refer to Elastic style guide for Vale. |
seanhandley
left a comment
seanhandley
left a comment
There was a problem hiding this comment.
Looks almost ready @kosabogi - just a couple of thoughts.
| - [Natural language Q&A](/explore-analyze/machine-learning/nlp/ml-nlp-text-emb-vector-search-example.md): Match questions like "How do I reset my password?" to FAQ entries, product documentation, or policy pages. | ||
| - [Recommendations and similarity](knn.md): Find related articles, products, or media. For example, you can surface articles like the current one or visually similar product images. | ||
|
|
||
| Dense embeddings are a good choice when you need multilingual retrieval or a specific third-party embedding model you have already evaluated on your data. |
There was a problem hiding this comment.
I'd remove this sentence. Not all dense embeddings models are multilingual. We support Jina Embeddings v3, which is English only, for example.
Also, the need to use a model that's already been evaluated for a usecase could apply to a sparse embedding model too - it's a question of previous technical decisions and the need to accommodate them.
Could maybe say something like
Dense embeddings are ideal when you care more about the semantic meaning of search terms than exact keyword matches - they excel at retrieving relevant results based on synonyms and paraphrasing of the original query to return results that reflect the user's intensions.
|
|
||
| Common use cases include: | ||
|
|
||
| - [Retrieval augmented generation (RAG)](../rag.md): Retrieve document passages that answer a user's question, even when the question and the source text use different words. |
There was a problem hiding this comment.
Could be an opportunity here to mention Context Engineering more prominently than RAG?
RAG is still a useful concept but I think we're positioning ourselves in the market as a broader solution for context engineering as a whole.
There was a problem hiding this comment.
++ there's some good definitions in this Anthropic blog post: https://www.anthropic.com/engineering/effective-context-engineering-for-ai-agents
this could be a good spot to link to agent builder for OOTB toolkit
Per the glossary definition:
"Agent Builder combines LLM reasoning with skills, tools, and best practices for context engineering and retrieval, so responses are accurately and efficiently grounded in your data."
3 participants
Summary
This PR adds explanation on when to use dense and sparse vectors to the Tutorial: Dense and sparse workflows using ingest pipelines page.
Related issue: https://github.com/elastic/docs-content-internal/issues/854
Generative AI disclosure
- Yes
- No
Claude in Cursor