Skip to content

DRAFT: BED-8357 Add enterprise + multi-org collection support#6

Draft
d3vzer0 wants to merge 68 commits into
mainfrom
feature/gh-enterprise2
Draft

DRAFT: BED-8357 Add enterprise + multi-org collection support#6
d3vzer0 wants to merge 68 commits into
mainfrom
feature/gh-enterprise2

Conversation

@d3vzer0
Copy link
Copy Markdown
Collaborator

@d3vzer0 d3vzer0 commented May 14, 2026 •
edited
Loading

Add GitHub Enterprise ingestion with multi-organization support. Additionally, adds new nodes/edges for Github Workflows.

Changes:

  • Add enterprise resource collector for enterprise orgs, members, teams etc.
  • Add org_login/environment id for lookups to support multi-org collection via enterprise credentials
  • Adds a parser for Github workflows with matching against existing repo/org/environment variables and secrets.
  • Additionally fixes an issue when the GraphQL helper returns no pagination metadata while this is required by the Pydantic model
  • Simplifies doc-generation descriptions for node properties via docstrings instead of dataclass field metadata

d3vzer0 added 30 commits May 4, 2026 12:30
@d3vzer0
Added enterprise resources
ee75652
@d3vzer0
Fixed conditions for collecting external identities and runners
d3d4c02
@d3vzer0
Generic try/except is already handled by openhound for resources and …
e7e2d7b 
…transformers
@d3vzer0
Added enterprise nodes and edges
621b4f2
@d3vzer0
Split org/enterprise resource collection
82577c9
@d3vzer0
Restructured enterprise + es org collection to use dlt resources and …
ff95e73 
…transformers
@d3vzer0
Added BaseUser model and removed _flatten_members method in favor of …
0f4abc5 
…in-resource/transformer parsing of users
@d3vzer0
enterprise_teams should be a transformer, not a resource
d100305
@d3vzer0
Improved and added enterprise_role_users + enterprise_role_teams
76f2b68
@d3vzer0
Improved and added enterprise_external_identities + enterprise_saml_p…
dbafa76 
…rovider
@d3vzer0
Iterate org-specific collection for each org_name in ctx
eb58821
@d3vzer0
Refer to org_name instead of ctx.org_name
d588d94
@d3vzer0
Add org-specific lookup logic
6333a6c
@d3vzer0
Updated org collection to use new per-org clients
e65ef57
@d3vzer0
Updated auth to get per-install token
62001b3
@d3vzer0
Else raise error if enterprise_name org org_name are not specified
43d61a3
@d3vzer0
Remove unused imports
02acef4
@d3vzer0
Split enterprise models
fc06796
@d3vzer0
Split enterprise models
5ae2fc4
@d3vzer0
Replaced dataclass field descriptions with attribute descriptions
2bb0c7e
@d3vzer0
Fixed enterprise lookups/edge generation
712ae10
@d3vzer0
Fixed EnterpriseRoleUser slug/id
263507f
@d3vzer0
Split role and EnterpriseAdmin
34b1670
@d3vzer0
Updated justfile for running collect/preproc/convert
5aecdde
@d3vzer0
Updated lock/deps
7462e25
@d3vzer0
Updated schema with missing nodes
3ea773f
@d3vzer0
Add GH_WorkflowJob and GH_WorkflowStep nodes and related edges
c625e0c
@d3vzer0
Add GH_WorkflowJob and GH_WorkflowStep models
5e51741
@d3vzer0
Dont allow extra data
37716c2
@d3vzer0
Collect workflow content + generate jobs/steps
53aab47
@d3vzer0 d3vzer0 changed the title DRAFT: Add enterprise support Add enterprise + multi-org collection support May 15, 2026
@d3vzer0 d3vzer0 changed the title Add enterprise + multi-org collection support DRAFT: Add enterprise + multi-org collection support May 18, 2026
@d3vzer0 d3vzer0 marked this pull request as draft May 18, 2026 20:59
d3vzer0 added 23 commits May 18, 2026 23:24
@d3vzer0
Secret scanning alert should have an edge with the repo as GH_Contains
edb5dc7
@d3vzer0
Add environment and workflow lookup to check for missing environments…
2312f22 
…/workflows
@d3vzer0
WIP: Auth rewrite
5f30ab3
@d3vzer0
WIP: Auth rewrite
962b4e9
@d3vzer0
WIP: Auth rewrite
b19864e
@d3vzer0
Remove dummy resource generator
3143f72
@d3vzer0
Additional logging for auth module
1c5de34
@d3vzer0
WIP: Auth rewrite
27f7635
@d3vzer0
Add workflows to lookup
d6172e0
@d3vzer0
Add workflows to lookup
ba1cc18
@d3vzer0
Decode workflow content
114bd28
@d3vzer0
Check if workflows/environments exist via lookup
03840b4
@d3vzer0
Add joserfc as dependency
d9d6116
@d3vzer0
Fixed source context bug
684a095
@d3vzer0
Add a token lock
488377c
@d3vzer0
Remove headers from init
c825565
@d3vzer0
Return RunnerGroup as Pydantic model
b7c58a9
@d3vzer0
Transform fix for the role_can_create_branch table
9c6d91d
@d3vzer0
Handle GraphQL rate limits, these can also return a 200 status code
014d5bf
@d3vzer0
Add runner edges to public/private/internal repos based on lookup
d33286a
@d3vzer0
Cache shared resources
5e61cde
@d3vzer0
Remove runner_group_repo_cache
61fb090
@d3vzer0
Split api handlers from the source.py file
0b1a44b
@juggernot325 juggernot325 changed the title DRAFT: Add enterprise + multi-org collection support DRAFT: BED-8357 Add enterprise + multi-org collection support May 22, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

Copilot code review Copilot Copilot left review comments

At least 1 approving review is required to merge this pull request.

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

2 participants

@d3vzer0