Version-1 changes for Maple Chatbot pr_2198_bot - #2218
Conversation
|
The latest updates on your projects. Learn more about Vercel for GitHub.
|
There was a problem hiding this comment.
Code in llm/ not relevant to this PR, so I recommend not modifying the requirements here
nesanders
left a comment
nesanders
left a comment
There was a problem hiding this comment.
This is initial high level feedback.
| @@ -0,0 +1,104 @@ | |||
| import { useState, useRef, useEffect } from "react" | |||
There was a problem hiding this comment.
A big open question is if we should be authoring our own chat widget or adopting a framework like copilotkit instead.
| │ functions/src/llm/agent.ts │ | ||
| │ │ | ||
| │ createReactAgent(@langchain/langgraph/prebuilt) │ | ||
| │ Model: OpenAI gpt-4o-mini • temperature: 0 │ |
There was a problem hiding this comment.
As we discussed, we'd prefer to align on using gemini models in our google account rather than these older openai models used in the old llm/ code
|
|
||
| // Logged-in users: persistent monthly token budget, tracked in the | ||
| // top-level `llmUsage` collection (see usage.ts). | ||
| loggedInMonthlyTokenBudget: 50_000 |
There was a problem hiding this comment.
These default budgets might be too small to be useful for either logged in or anonymous users, but we can modify later
| --- | ||
|
|
||
| ## Backfilling Existing Documents | ||
|
|
There was a problem hiding this comment.
Is there anything we actually need to backfill? I think we already have the relevant embeddings for bills, questions, and testimony in the index.
| "rimraf": "^3.0.2", | ||
| "ts-jest": "^29.2.5", | ||
| "typescript": "4.5.5" | ||
| "typescript": "^5.5.4" |
There was a problem hiding this comment.
I don't think we will update typescript version as part of this PR, though w may do so later for other reasons
| "@google-cloud/pubsub": "^3.0.1", | ||
| "@langchain/core": "^0.3.0", | ||
| "@langchain/langgraph": "^0.2.0", | ||
| "@langchain/openai": "^0.3.0", |
There was a problem hiding this comment.
As noted elsewhere, we will use google instead of openai
| { | ||
| name: "search_bills", | ||
| description: | ||
| "Semantic search over Massachusetts legislative bills (title and full text). Use this to find bills related to a topic, policy area, or question.", |
There was a problem hiding this comment.
In the MCP server code in this repo, we have a function called something like 'search_policies' that explicitly searches across both bills and questions. We should reuse that code if possible (to eliminate redundancy) and generally follow the same pattern. We should expose a bill-only search tool, a question-only search tool, and a combined search called 'policy'.
| { | ||
| name: "search_testimony", | ||
| description: | ||
| "Semantic search over public testimony submitted on bills. Optionally scope to a specific bill by ID. Use this to find what people have said about a bill or issue.", |
There was a problem hiding this comment.
Same comment here - we want the agent to be able to search for testimony on only bills, testimony on only questions, or both. Right now, in the MCP server code, we do that through 3 separate tools and we should generally follow the same pattern here.
| import { httpsCallable } from "firebase/functions" | ||
| import { functions } from "components/firebase" | ||
| import { useAuth } from "components/auth" | ||
| import styles from "./ChatWidget.module.css" |
There was a problem hiding this comment.
i think we need to add a mock frontend pag that just exposes this widget on a placeholder page for initial testing purposes.
| import { vectorSearchTools } from "./vectorSearchTools" | ||
| import { LLM_CONFIG } from "./config" | ||
|
|
||
| const SYSTEM_PROMPT = `You are a helpful assistant for the MAPLE platform, answering questions about Massachusetts legislation, testimony, and ballot questions. |
There was a problem hiding this comment.
Good start! We can probably elaborate on this with input from the design team.
2 participants
Summary
Maple Chatbot where a User can ask Questions about Bill/Policy issue question answering chat
ReAct AI agent with tool calling and reasoning loop capability
Access to firebase vector index search
Add a short summary of the changes, and a reference to the original issue using
#and the issue number, like #1Checklist
firestore.indexes.json(Please do not only create indexes through the Firebase Web UI, even though the error messages may reccommend it - indexes created this way may be obliterated by subsequent deploys)Screenshots
Add some screenshots highlighting your changes.
Known issues
If you've run against limitations or caveats, include them here. Include follow-up issues as well.
Steps to test/reproduce
For each feature or bug fix, create a step by step list for how a reviewer can test it out. E.g.: