-
Notifications
You must be signed in to change notification settings - Fork 35
feat(datafabric): standalone ontology tool grounded on OWL + R2RML #911
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Open
sankalp-uipath
wants to merge
31
commits into
main
Choose a base branch
from
feat/datafabric-ontology-fetch-tool
base: main
Could not load branches
Branch not found: {{ refName }}
Loading
Could not load tags
Nothing to show
Loading
Are you sure you want to change the base?
Some commits from the old base branch may be removed from the timeline,
and old review comments may become outdated.
Open
Changes from all commits
Commits
Show all changes
31 commits
Select commit
Hold shift + click to select a range
c6e73eb
feat(datafabric): add fetch_ontology tool to DF inner SQL agent
sankalp-uipath b67e170
Merge branch 'main' into feat/datafabric-ontology-fetch-tool
sankalp-uipath da19087
feat(datafabric): resolve ontology from agent.json binding (name + fo…
sankalp-uipath 4c22b8f
refactor(datafabric): fetch ontology via SDK EntitiesService.get_onto…
sankalp-uipath 68f7cbf
feat(datafabric): support multiple ontologies per context (ontologySet)
sankalp-uipath ab77d65
Merge remote-tracking branch 'origin/main' into feat/datafabric-ontol…
sankalp-uipath 40acdec
fix(datafabric): end loop on any successful SQL; drop env-var ontolog…
sankalp-uipath 7a5bb69
test(datafabric): cover ontology fetch tool, subgraph routing, and fa…
sankalp-uipath 04f79c5
fix(datafabric): return only terminal tool msgs on END; drop ToolMess…
sankalp-uipath 0ed6210
perf(datafabric): fetch configured ontologies concurrently (asyncio.g…
sankalp-uipath e9c4cfb
feat(datafabric): resolve ontologies via ontology_refs
sankalp-uipath be5ef26
Merge branch 'main' into feat/datafabric-ontology-fetch-tool
sankalp-uipath 1fd7a30
chore: consume uipath dev build (#1728) to unblock CI
sankalp-uipath a871a0a
chore: revert temp dev-build pin; fix datafabric test mypy
sankalp-uipath dfdd3d6
Merge branch 'main' into feat/datafabric-ontology-fetch-tool
sankalp-uipath a07adb9
Merge branch 'main' into feat/datafabric-ontology-fetch-tool
sankalp-uipath 54db78f
refactor(datafabric): resolve ontologies from nested ontologySet
sankalp-uipath 941f3ff
refactor(datafabric): gather ontologies from datafabricontology context
sankalp-uipath 86e5912
feat(datafabric): gate fetch_ontology behind DataFabricOntologyEnable…
sankalp-uipath 826f036
test(datafabric): drop ontology referenceKey fixture
sankalp-uipath e57d1b0
refactor(datafabric): gate ontology flag at every entry; share flag c…
sankalp-uipath 2f41f40
Merge remote-tracking branch 'origin/main' into feat/datafabric-ontol…
sankalp-uipath 7fab6d5
refactor(datafabric): address review nits (split bind test, single-st…
sankalp-uipath a35807b
refactor(datafabric): inject ontology into system prompt, drop fetch_…
sankalp-uipath 4edb26a
feat(datafabric): standalone ontology tool (R2RML-driven, flag-gated)
sankalp-uipath eebdfc2
refactor(datafabric): call get_ontology_bundle_async (SDK rename)
sankalp-uipath 0f99ac0
Revert "refactor(datafabric): call get_ontology_bundle_async (SDK ren…
sankalp-uipath ee0532c
build(datafabric): require uipath>=2.12.5 / uipath-platform>=0.1.91
sankalp-uipath 72dc9cb
refactor(datafabric): keep entity prompt builder untouched; ontology …
sankalp-uipath 28d284d
Merge remote-tracking branch 'origin/main' into feat/datafabric-ontol…
sankalp-uipath 89cb44e
refactor(datafabric): drop unused logger from ontology_fetcher
sankalp-uipath File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -1,9 +1,10 @@ | ||
| """Data Fabric tool module for entity-based SQL queries.""" | ||
| """Data Fabric tool module for entity-based and ontology-based SQL queries.""" | ||
|
|
||
| from .datafabric_tool import ( | ||
| create_datafabric_query_tool, | ||
| ) | ||
| from .datafabric_ontology_tool import create_datafabric_ontology_tool | ||
| from .datafabric_tool import DATAFABRIC_ONTOLOGY_FF, create_datafabric_query_tool | ||
|
|
||
| __all__ = [ | ||
| "DATAFABRIC_ONTOLOGY_FF", | ||
| "create_datafabric_ontology_tool", | ||
| "create_datafabric_query_tool", | ||
| ] |
145 changes: 145 additions & 0 deletions
145
src/uipath_langchain/agent/tools/datafabric_tool/datafabric_ontology_prompt_builder.py
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,145 @@ | ||
| """Inner system-prompt builder for the Data Fabric **ontology** tool. | ||
|
|
||
| Separate from the entity tool's ``datafabric_prompt_builder`` so the two prompts | ||
| can evolve independently. This builder assembles an ontology-grounded prompt: | ||
| the OWL (authoritative semantic schema) and the R2RML (ontology→table/column | ||
| mapping) come first, then the SQL strategy/constraints and the entity schema | ||
| tables. | ||
|
|
||
| It reuses only ``build_sql_context`` (entity-context + strategy prompt | ||
| construction) from ``datafabric_prompt_builder``; the top-level layout, the | ||
| ontology-specific framing, and the entity-schema rendering are all owned here so | ||
| the shared entity-tool builder is not modified by this feature. | ||
| """ | ||
|
|
||
| from uipath.platform.entities import Entity | ||
|
|
||
| from .datafabric_prompt_builder import SQLContext, build_sql_context | ||
|
|
||
|
|
||
| def _render_entity_schema_sections(ctx: SQLContext) -> list[str]: | ||
| """Render the entity-schema tables + query patterns as prompt lines.""" | ||
| lines: list[str] = ["## All available Data Fabric Entities", ""] | ||
|
|
||
| for entity_ctx in ctx.entity_contexts: | ||
| entity = entity_ctx.entity_schema | ||
| lines.append( | ||
| f"### Entity: {entity.display_name} (SQL table: `{entity.entity_name}`)" | ||
| ) | ||
| if entity.description: | ||
| lines.append(f"_{entity.description}_") | ||
| lines.append("") | ||
| lines.append("| Field | Type | Description |") | ||
| lines.append("|-------|------|-------------|") | ||
| for field in entity.fields: | ||
| desc = (field.description or "").replace("|", r"\|").replace("\n", " ") | ||
| lines.append(f"| {field.name} | {field.display_type} | {desc} |") | ||
|
|
||
| lines.append("") | ||
|
|
||
| lines.append(f"**Query Patterns for {entity.entity_name}:**") | ||
| lines.append("") | ||
| lines.append("| User Intent | SQL Pattern |") | ||
| lines.append("|-------------|-------------|") | ||
| for p in entity_ctx.query_patterns: | ||
| lines.append(f"| '{p.intent}' | `{p.sql}` |") | ||
| lines.append("") | ||
|
|
||
| return lines | ||
|
sankalp-uipath marked this conversation as resolved.
|
||
|
|
||
|
|
||
| def format_ontology_context( | ||
| ctx: SQLContext, | ||
| ontology_text: str = "", | ||
| r2rml_text: str = "", | ||
| ) -> str: | ||
| """Format a SQLContext + ontology artifacts as the ontology-tool prompt.""" | ||
| lines: list[str] = [] | ||
|
|
||
| if ctx.base_system_prompt: | ||
| lines.append("## Agent Instructions") | ||
| lines.append("") | ||
| lines.append(ctx.base_system_prompt) | ||
| lines.append("") | ||
|
|
||
| if ontology_text: | ||
| lines.append( | ||
| "## Available Ontology (authoritative semantic schema)\n\n" | ||
| "The ontology below is the authoritative source for the exact column " | ||
| "names, value formats (date formats, codes, zero-padding), allowed " | ||
| "values, and the relationships between entities — richer and more " | ||
| "reliable than the field list further down, which omits value formats " | ||
| "and semantics. Base your column names, filter values, and joins on " | ||
| "it; when it and the entity tables disagree, the ontology wins.\n\n" | ||
| f"{ontology_text}" | ||
| ) | ||
| lines.append("") | ||
|
|
||
| if r2rml_text: | ||
| lines.append( | ||
| "## Ontology→Table Mapping (R2RML)\n\n" | ||
| "The R2RML below maps the ontology to the physical tables and columns " | ||
| "you must use in SQL: `rr:tableName` is the SQL table, `rr:column` is " | ||
| "the SQL column for each ontology predicate, and `rr:joinCondition` " | ||
| "(child/parent) gives the exact columns to join on. Use it to turn " | ||
| "ontology terms into precise SQL identifiers and joins.\n\n" | ||
| f"{r2rml_text}" | ||
| ) | ||
| lines.append("") | ||
|
|
||
| if ctx.sql_expert_system_prompt: | ||
| lines.append("## SQL Query Generation Guidelines") | ||
| lines.append("") | ||
| lines.append(ctx.sql_expert_system_prompt) | ||
| lines.append("") | ||
|
|
||
| if ctx.constraints: | ||
| lines.append("## SQL Constraints") | ||
| lines.append("") | ||
| lines.append(ctx.constraints) | ||
| lines.append("") | ||
|
|
||
| if ctx.resource_description: | ||
| lines.append("## Ontology description") | ||
| lines.append("") | ||
| lines.append(ctx.resource_description) | ||
| lines.append("") | ||
|
|
||
| lines.extend(_render_entity_schema_sections(ctx)) | ||
|
|
||
| return "\n".join(lines) | ||
|
|
||
|
|
||
| def build( | ||
| entities: list[Entity], | ||
| resource_description: str = "", | ||
| base_system_prompt: str = "", | ||
| ontology_text: str = "", | ||
| r2rml_text: str = "", | ||
| prompt_version: str | None = None, | ||
| ) -> str: | ||
| """Build the ontology-tool inner system prompt. | ||
|
|
||
| Args: | ||
| entities: Resolved Data Fabric entities (from the R2RML allow-list). | ||
| resource_description: Optional description of the ontology context. | ||
| base_system_prompt: Optional system prompt from the outer agent. | ||
| ontology_text: The fetched ontology OWL content (authoritative schema). | ||
| r2rml_text: The fetched ontology R2RML mapping (ontology→table/column). | ||
| prompt_version: Optional SQL-strategy prompt version key. | ||
|
|
||
| Returns: | ||
| Formatted prompt string for the inner LLM system message. | ||
| """ | ||
| if not entities: | ||
| return "" | ||
|
|
||
| ctx = build_sql_context( | ||
| entities, | ||
| resource_description, | ||
| base_system_prompt, | ||
| prompt_version=prompt_version, | ||
| ) | ||
| return format_ontology_context( | ||
| ctx, ontology_text=ontology_text, r2rml_text=r2rml_text | ||
| ) | ||
Oops, something went wrong.
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Uh oh!
There was an error while loading. Please reload this page.