GitHub filter#38
Merged
Merged
Conversation
Signed-off-by: JaredforReal <w13431838023@gmail.com>
Signed-off-by: JaredforReal <w13431838023@gmail.com>
There was a problem hiding this comment.
Pull request overview
This PR adds first-class “filter” support across Loom’s GitHub sources (and exposes similar filter fields for RSS/arXiv) by extending source configuration, wiring new config fields into the daemon, and applying the filters during GitHub polling.
Changes:
- Web UI: separates source fields into “Config” vs “Filters” sections and adds filter inputs for GitHub/RSS/arXiv.
- Daemon/GitHub adaptor: introduces
include_labels,keywords, andauthorsfilters (with backward compatibility forlabels_filter) and applies them during polling. - CLI: adds flags for GitHub label/author filters and RSS
title_filter, and updates duplicate-source identity to include filters.
Reviewed changes
Copilot reviewed 4 out of 4 changed files in this pull request and generated 8 comments.
| File | Description |
|---|---|
| loom/webui/frontend/src/pages/config/ConfigFormEditor.tsx | Adds schema-driven “Filters” section and new filter fields; changes source card expansion behavior. |
| loom/daemon.py | Passes new GitHub filter fields into GitHubSourceConfig. |
| loom/cli/main.py | Adds CLI flags for new filters and includes them in duplicate-source detection and display. |
| loom/adaptor/github.py | Implements filtering by labels/keywords/authors and adds a server-side label query optimization for a single label. |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
| i === config.sources.length - 1 && | ||
| Object.keys(s).filter((k) => !SOURCE_COMMON_FIELDS.has(k) && s[k] !== undefined && s[k] !== "").length === 0 | ||
| } | ||
| defaultExpanded={true} |
Comment on lines
286
to
+309
| default=None, | ||
| help="arXiv categories, comma-separated (e.g. cs.AI,cs.CL)", | ||
| ) | ||
| source_add.add_argument( | ||
| "--keywords", | ||
| type=str, | ||
| required=False, | ||
| default=None, | ||
| help="Title keywords, comma-separated (e.g. LLM,agent)", | ||
| ) | ||
| source_add.add_argument( | ||
| "--max-results", | ||
| type=int, | ||
| required=False, | ||
| default=50, | ||
| help="Max papers per poll for arxiv (default 50)", | ||
| ) | ||
| source_add.add_argument( | ||
| "--title-filter", | ||
| type=str, | ||
| required=False, | ||
| default=None, | ||
| help="RSS: only include entries whose title matches any keyword (comma-separated)", | ||
| ) |
Comment on lines
667
to
+686
| def _source_dup_key(src: dict[str, Any]) -> tuple: | ||
| """Return an identity tuple used to detect duplicate sources.""" | ||
| kind = src.get("kind") | ||
| if kind == "github": | ||
| return ("github", src.get("owner"), src.get("repo")) | ||
| return ( | ||
| "github", | ||
| src.get("owner"), | ||
| src.get("repo"), | ||
| tuple(sorted(src.get("include_labels", []))), | ||
| tuple(sorted(src.get("keywords", []))), | ||
| tuple(sorted(src.get("authors", []))), | ||
| ) | ||
| if kind == "gmail": | ||
| return ("gmail", str(Path(src.get("client_secrets", "")).expanduser())) | ||
| if kind == "rss": | ||
| return ("rss", src.get("url")) | ||
| return ( | ||
| "rss", | ||
| src.get("url"), | ||
| tuple(sorted(src.get("title_filter", []))), | ||
| ) |
Comment on lines
168
to
169
| events=src.get("events", ["issues", "pull_requests"]), | ||
| labels_filter=src.get("labels_filter"), |
| # Filter by authors (case-insensitive OR) | ||
| if config.authors: | ||
| login = (item.get("user") or {}).get("login", "") | ||
| if login.lower() not in {a.lower() for a in config.authors}: |
Comment on lines
184
to
248
| @@ -219,10 +229,22 @@ async def _poll_source(self, key: str, config: GitHubSourceConfig) -> None: | |||
| if not is_pr and "issues" not in config.events: | |||
| continue | |||
|
|
|||
| # Filter by labels if configured | |||
| if config.labels_filter: | |||
| # Filter by labels if configured (client-side OR match) | |||
| if config.include_labels: | |||
| item_labels = {lbl["name"] for lbl in item.get("labels", [])} | |||
| if not any(lbl in item_labels for lbl in config.labels_filter): | |||
| if not any(lbl in item_labels for lbl in config.include_labels): | |||
| continue | |||
|
|
|||
| # Filter by keywords (title + body, case-insensitive OR) | |||
| if config.keywords: | |||
| text = f"{item.get('title', '')} {item.get('body', '')}".lower() | |||
| if not any(kw.lower() in text for kw in config.keywords): | |||
| continue | |||
|
|
|||
| # Filter by authors (case-insensitive OR) | |||
| if config.authors: | |||
| login = (item.get("user") or {}).get("login", "") | |||
| if login.lower() not in {a.lower() for a in config.authors}: | |||
| continue | |||
|
|
||
| # Filter by keywords (title + body, case-insensitive OR) | ||
| if config.keywords: | ||
| text = f"{item.get('title', '')} {item.get('body', '')}".lower() |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
No description provided.