Add enterprise + multi-org collection support#6
Open
d3vzer0 wants to merge 42 commits into
Open
Conversation
…in-resource/transformer parsing of users
There was a problem hiding this comment.
Pull request overview
This PR introduces first-pass GitHub Enterprise ingestion alongside multi-organization support, and expands Actions workflow modeling to include job/step nodes and additional edges.
Changes:
- Add an
enterpriseresource pipeline (GraphQL + REST) to collect enterprise orgs, members, teams, roles, admins, and SAML external identities. - Refactor many GitHub models to carry
org_loginand resolveenvironmentidvia lookup to support multi-org enterprise collection. - Add workflow YAML parsing to emit
WorkflowJobandWorkflowStepnodes plus dependency/secret/variable edges; extend node/edge kinds and extension schema.
Reviewed changes
Copilot reviewed 65 out of 66 changed files in this pull request and generated 4 comments.
Show a summary per file
| File | Description |
|---|---|
| src/openhound_github/resources/enterprise.py | New enterprise collection resource/transformer pipeline (enterprise, orgs, members, teams, roles, admins, SAML). |
| src/openhound_github/resources/init.py | Export enterprise resources for use by the source wiring. |
| src/openhound_github/models/workflow.py | Adds workflow YAML parsing and row generation for jobs/steps; updates workflow properties for org scoping. |
| src/openhound_github/models/workflow_job.py | New asset model for workflow job nodes and related edges (deps/env/calls/secrets/vars). |
| src/openhound_github/models/workflow_step.py | New asset model for workflow step nodes and secret/variable reference edges. |
| src/openhound_github/models/user.py | Adds org_login/org lookup support and updates property defaults to be nullable. |
| src/openhound_github/models/team.py | Adds org_login/org lookup support; adjusts schema nullability and DLT config. |
| src/openhound_github/models/team_role.py | Adds org_login/org lookup support; updates properties to nullable fields. |
| src/openhound_github/models/team_member.py | Adds org_login/org lookup support for team member edges. |
| src/openhound_github/models/secret_scanning_alert.py | Adds org_login and org lookup fallback for environment scoping. |
| src/openhound_github/models/scim_user.py | Adds org_login to SCIM resources for org scoping. |
| src/openhound_github/models/saml_provider.py | Updates SAML provider properties and adds org_login scoping fields. |
| src/openhound_github/models/runner.py | Adds org_login scoping and uses org lookup for IDs/edges across runner assets. |
| src/openhound_github/models/repository.py | Adds org_login and org lookup for repository environment scoping; adjusts property nullability. |
| src/openhound_github/models/repository_variable.py | Adds org_login scoping and org lookup for repo variables. |
| src/openhound_github/models/repository_secret.py | Adds org_login scoping and org lookup for repo secrets. |
| src/openhound_github/models/repository_role.py | Adds org_login scoping and org lookup for repo roles and assignments. |
| src/openhound_github/models/repo_role_assignment.py | Adds org_login and org lookup for derived role assignment edges. |
| src/openhound_github/models/personal_access_token.py | Adds org_login scoping and org lookup for PAT IDs/edges; adds DLT config. |
| src/openhound_github/models/personal_access_token_request.py | Adds org_login scoping and org lookup for PAT request IDs/edges. |
| src/openhound_github/models/personal_access_token_access.py | Adds org_login scoping for PAT→repo access edge generation. |
| src/openhound_github/models/org.py | Updates org properties to nullable fields and adds DLT config. |
| src/openhound_github/models/org_variable.py | Adds org_login scoping and org lookup; updates node IDs to be org-specific. |
| src/openhound_github/models/org_secret.py | Adds org_login scoping and org lookup; updates repo visibility lookups to be org-specific. |
| src/openhound_github/models/org_role.py | Updates org role environment scoping to use org_login/org lookup. |
| src/openhound_github/models/org_role_team.py | Adds explicit org_node_id/org_login to build stable org-role IDs. |
| src/openhound_github/models/org_role_member.py | Adds explicit org_node_id/org_login to build stable org-role IDs. |
| src/openhound_github/models/external_identity.py | Adds org_login scoping and org-specific IdP lookup fallback. |
| src/openhound_github/models/environment.py | Adds org_login scoping and org lookup for environment assets. |
| src/openhound_github/models/environment_branch_policy.py | Adds org_login scoping and org lookup for environment branch policy edges. |
| src/openhound_github/models/env_variable.py | Adds org_login scoping and org lookup for environment variables. |
| src/openhound_github/models/env_secret.py | Adds org_login scoping and org lookup for environment secrets. |
| src/openhound_github/models/enterprise.py | New enterprise node asset model. |
| src/openhound_github/models/enterprise_user.py | New enterprise member user asset model and enterprise membership edge. |
| src/openhound_github/models/enterprise_team.py | New enterprise team asset model plus enterprise containment edge. |
| src/openhound_github/models/enterprise_team_role.py | New enterprise team role (GH_TeamRole) model mapping to enterprise teams. |
| src/openhound_github/models/enterprise_team_organization.py | New edge-only asset mapping enterprise teams to orgs and projected org teams. |
| src/openhound_github/models/enterprise_team_member.py | New edge-only asset assigning users to enterprise team roles. |
| src/openhound_github/models/enterprise_saml_provider.py | New enterprise SAML IdP asset model and enterprise→IdP edge. |
| src/openhound_github/models/enterprise_role.py | New enterprise role asset model plus enterprise containment edge. |
| src/openhound_github/models/enterprise_role_user.py | New edge-only asset assigning users to enterprise roles. |
| src/openhound_github/models/enterprise_role_team.py | New edge-only asset assigning enterprise teams to enterprise roles. |
| src/openhound_github/models/enterprise_organization.py | New “stub” org model discovered via enterprise, with enterprise containment edge. |
| src/openhound_github/models/enterprise_member.py | New GraphQL member models and helper to flatten enterprise membership shapes. |
| src/openhound_github/models/enterprise_managed_user.py | New enterprise managed user asset model and mapping edge to backing GH user. |
| src/openhound_github/models/enterprise_helpers.py | Helpers for stable enterprise team/role synthetic node IDs. |
| src/openhound_github/models/enterprise_external_identity.py | New enterprise external identity asset model and mapping edges (incl. foreign IdP). |
| src/openhound_github/models/enterprise_admin.py | Alias asset for enterprise admins as a specialized enterprise role assignment. |
| src/openhound_github/models/branch.py | Adds org_login scoping and org lookup for branch assets. |
| src/openhound_github/models/branch_protection_rule.py | Adds org_login scoping and org lookup for branch protection rules. |
| src/openhound_github/models/app_installation.py | Adds org_login scoping and org-specific lookup for app/installations & edges. |
| src/openhound_github/models/actions_permission.py | Makes selected_actions_url nullable and adds org_login scoping. |
| src/openhound_github/models/init.py | Exposes new enterprise/workflow job/step models and helpers. |
| src/openhound_github/main.py | Registers enterprise in preprocessing mapping. |
| src/openhound_github/lookup.py | Adds org-scoped lookup helpers (org id by login, repo ids by org, IdP by org, etc.). |
| src/openhound_github/kinds/nodes.py | Adds enterprise/workflow job/workflow step node kinds. |
| src/openhound_github/kinds/edges.py | Adds new edge kinds for enterprise membership and workflow job/step relationships. |
| src/openhound_github/helpers.py | Hardens GraphQL cursor pagination with explicit error handling and validation. |
| src/openhound_github/graphql.py | Adds enterprise GraphQL queries; updates SAML queries to support pagination/fields. |
| src/openhound_github/auth.py | Refactors GitHub App auth to support multiple installations and per-org tokens. |
| pyproject.toml | Pins openhound dev dependency version. |
| justfile | Updates CLI commands and defaults to use openhound entrypoints. |
| extension/schema.json | Adds enterprise + runner node kinds and workflow job/step edge kinds to the extension schema. |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
There was a problem hiding this comment.
Pull request overview
Copilot reviewed 65 out of 66 changed files in this pull request and generated 9 comments.
Comments suppressed due to low confidence (2)
src/openhound_github/models/workflow_job.py:246
self.org_node_idisstr | None, but this code unconditionally calls.upper()when building org-variable property matchers. If lookup returnsNone, this will raise at runtime. Guard onself.org_node_idbefore calling.upper()(and skip emitting the edge if it’s missing).
src/openhound_github/models/workflow_step.py:237self.org_node_idisstr | None, but this code unconditionally calls.upper()when building org-variable property matchers. If lookup returnsNone, this will raise. Guard onself.org_node_idbefore calling.upper()(and skip emitting the org-variable edge when it’s missing).
Comment on lines
+199
to
+203
| if installation_id is None: | ||
| org_login = org_name or self.org_name | ||
| if org_login: | ||
| installation_id = self.installation_id_for_org(org_login) | ||
| else: |
d3vzer0
changed the title
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Add GitHub Enterprise ingestion with multi-organization support. Additionally, adds new nodes/edges for Github Workflows.
Changes: