Skip to content

Auto-load prior DSG into Neo4j on chatdsg startup - #2

Draft
harelb wants to merge 3 commits into
masterfrom
fix/chatdsg-auto-load-dsg
Draft

Auto-load prior DSG into Neo4j on chatdsg startup#2
harelb wants to merge 3 commits into
masterfrom
fix/chatdsg-auto-load-dsg

Conversation

@harelb

@harelb harelb commented Jul 30, 2026

Copy link
Copy Markdown

Fixes MIT-SPARK/Awesome-DCIST-T4#337.

Problem

  1. The load path was broken. chatdsg.py called load_dsg_to_db() with the old five-argument signature (object_labelspace, room_labelspace, uri, creds, graph), but heracles.utils.load_dsg_to_db() takes (neo4j_uri, neo4j_creds, scene_graph) and reads labelspaces from the graph metadata. Passing --scene-graph raised TypeError.
  2. Nothing ever triggered a load. The dcist launch components invoke ./chatdsg.py with no arguments and the load was gated on if args.scene_graph:, so the prior scene graph was never loaded.
  3. The example was coupled to the embedding stack. chatdsg.py read ADT4_HERACLES_IP/ADT4_HERACLES_PORT and agent_config.yaml built its Neo4j URI from the same pair, so the example could not be configured without ADT4-specific variables. heracles/docker/docker-compose.yaml had to set them alongside the HERACLES_* ones it already provides.

Changes

  • Fixed the load_dsg_to_db() call and removed --object-labelspace / --room-labelspace. Labelspaces come from the graph; extract_labelspaces_from_dsg() warns when one carries none rather than silently loading unlabeled objects and rooms.
  • Configuration is HERACLES_* only. --db_ip/--db_port are replaced by --neo4j-uri, defaulting to $HERACLES_NEO4J_URI — the name heracles' own load_scene_graph.py uses and that docker-compose.yaml already sets. Credentials come from HERACLES_NEO4J_USERNAME/PASSWORD. The cypher tool's uri is $HERACLES_NEO4J_URI.
  • The scene graph path comes only from --scene-graph; the deployment supplies it. nargs="?" is gone, since a valueless flag behaved identically to omitting the argument.
  • Added --no-dsg-load, because loading calls initialize_db(), which clears the database — so a load on every startup wipes Neo4j. There is an existing feature/chatdsg_optional_dsg_reset branch suggesting the same need.
  • logging.basicConfig() so startup messages print before the Textual app takes the terminal.
  • Missing URI or credentials skips the load with a warning instead of producing a half-initialized startup.

Companion change

MIT-SPARK/Awesome-DCIST-T4 supplies the path and translates its variables into the HERACLES_* names, which is that repo's job:

  • launch_components/heracles_prior_dsg.yaml runs ./chatdsg.py --scene-graph $ADT4_PRIOR_MAP/hydra/backend/dsg.json
  • base_launch/base_launch.yaml exports HERACLES_NEO4J_URI and HERACLES_AGENTS_PATH (it already mapped the credentials and OpenAI key)

Both must land together: with the new launch config against old chatdsg.py, --scene-graph hits the TypeError.

Testing

Against a live Neo4j, with example_dsg.json staged as a prior map.

  • With every ADT4_* variable unset and only the HERACLES_* values the launch env now exports: DSG loads (65 objects / 96 2D places / 5 rooms) and LlmAgent constructs successfully.
  • Cypher queries confirm labels come from the graph's embedded labelspace, not the YAML files — objects resolve to seating, storage, sign, decor; rooms to hallway, lounge.
  • $HERACLES_NEO4J_URI expansion in agent_config.yaml verified through HeraclesDsgInterface (neo4j://127.0.0.1:7687).
  • Guard paths: missing DSG file, missing URI, and missing credentials each warn and skip without touching the database. --no-dsg-load skips.
  • pre-commit run --all-files clean.

unit-tests.yml has not been run locally.

Note

One ADT4 reference remains in examples/chatdsg/agent_config.yaml: planner_topic: /${ADT4_ROBOT_NAME}/omniplanner_node/..., expanded by the shell inside os.system() in pddl_calling_tool.py. It names a topic on an omniplanner node that only exists in the larger stack, so it is arguably deployment config rather than a leak — but it can be renamed to a HERACLES_* variable for consistency if preferred.

chatdsg.py called load_dsg_to_db() with the old five-argument signature
(object labelspace, room labelspace, uri, creds, graph), but heracles now
takes (uri, creds, graph) and reads the labelspaces out of the graph
metadata. Passing --scene-graph therefore raised TypeError, and the tmux
launch components invoke ./chatdsg.py with no arguments at all, so the
prior scene graph was never loaded.

- Default --scene-graph to $ADT4_PRIOR_MAP/hydra/backend/dsg.json, which
  is the layout the launch system already forwards to the chatdsg pane.
- Fix the load_dsg_to_db() call and drop the --object-labelspace and
  --room-labelspace flags; labelspaces now come from the graph itself.
  Warn via extract_labelspaces_from_dsg() when a graph carries none,
  rather than silently loading unlabeled objects and rooms.
- Add --no-dsg-load, since loading calls initialize_db() and so clears
  the database on every startup.
- Fall back to ADT4_NEO4J_* credentials when HERACLES_NEO4J_* is unset,
  for running the example outside tmuxp.
- Configure logging so the startup messages print before the Textual app
  takes over the terminal.

Fixes MIT-SPARK/Awesome-DCIST-T4#337
Comment thread examples/chatdsg/chatdsg.py Outdated
logger = logging.getLogger(__name__)

# Location of the backend DSG inside a saved map directory (see $ADT4_PRIOR_MAP).
PRIOR_DSG_RELPATH = os.path.join("hydra", "backend", "dsg.json")

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We should have the dcist launch config from adt4 repo pass this in via the commadn line rather than getting the path here.

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done in 70bf92a. Removed PRIOR_DSG_RELPATH and default_scene_graph_path(); the path now comes only from --scene-graph, and nargs="?" is gone since a valueless flag was equivalent to omitting it.

The launch side is MIT-SPARK/Awesome-DCIST-T4launch_components/heracles_prior_dsg.yaml now runs:

./chatdsg.py --scene-graph $ADT4_PRIOR_MAP/hydra/backend/dsg.json

regenerated via scripts/generate_configs.sh. Verified with that exact invocation: 65 objects / 96 2D places / 5 rooms loaded, labels from the graph's embedded labelspace. If $ADT4_PRIOR_MAP is unset the pane passes /hydra/backend/dsg.json, which hits the missing-file guard and warns instead of failing.

harelb added 2 commits July 30, 2026 13:55
Address review: the example should not know the deployment's map layout.
Drop the $ADT4_PRIOR_MAP/hydra/backend/dsg.json default and let the dcist
launch config pass the path in explicitly.

--scene-graph also loses nargs="?", which only allowed a valueless flag
that behaved identically to omitting the argument.
heracles_agents needs to run standalone; ADT4_* variables belong to the
larger stack that embeds it. chatdsg read ADT4_HERACLES_IP/PORT, and the
example agent config built its Neo4j URI from the same pair.

- Replace --db_ip/--db_port with --neo4j-uri, defaulting to
  $HERACLES_NEO4J_URI, which is what heracles' own load_scene_graph.py
  example already uses and what docker-compose already sets.
- Read credentials only from HERACLES_NEO4J_USERNAME/PASSWORD. The
  ADT4_NEO4J_* fallback added earlier in this branch was wrong: the
  agent's HeraclesDsgInterface requires the HERACLES_* names, so falling
  back would load the DSG and then fail on config validation.
- Point the cypher tool's uri at $HERACLES_NEO4J_URI.

Skip the load with a warning when no URI is configured, so a missing URI
cannot produce a half-initialized startup.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Heracles Agents Automatic Loading of DSG to Neo4j

1 participant