feat: committees implementation [CM-1066] by mbani01 · Pull Request #3995 · linuxfoundation/crowd.dev

mbani01 · 2026-04-02T13:58:00Z

This pull request introduces support for "committees" as a new platform throughout the data pipeline, including database, integrations, and attribute configuration. The main changes add new activity types for committee membership, implement the data extraction and transformation logic for committee events, and update the type system and configuration to recognize "committees" as a first-class source.

Support for "committees" platform and data pipeline:

Added two new activity types, added-to-committee and removed-from-committee, to the activityTypes table for tracking committee membership events.
Introduced a new source query and transformer for the "committees" platform in the Snowflake connectors, enabling extraction and transformation of committee membership data. [1] [2]
Registered the committees source and transformer in the integrations index, and defined its data source name. [1] [2] [3]

Type system and configuration updates:

Added "committees" to the PlatformType, OrganizationSource, and OrganizationAttributeSource enums, and included it in the organization attribute source priority list, ensuring proper handling and prioritization. [1] [2] [3] [4]
Defined new types and scoring grid for committee activities in the integrations library, and re-exported them for usage across the codebase. [1] [2]

Note

Medium Risk
Adds a new data source and transforms/exports committee membership events, plus new enum values and activity types; main risk is incorrect Snowflake query/transform mapping causing missing/duplicated activities or misattributed org identities.

Overview
Adds Committees as a first-class platform across the pipeline.

Introduces two new activity types (added-to-committee, removed-from-committee) via a DB migration and a new Committees scoring grid/types exported from @crowd/integrations.

Implements a new Snowflake connector source (buildSourceQuery) that joins Salesforce committee membership data with committee/project/segment, user, and account tables (with non-prod project limiting, dedup, and incremental export logic), and a transformer that emits membership add/remove activities with member identities and optional org domain identities. Updates platform/organization enums and org attribute source priority to include committees, and registers the new data source in the snowflake connectors platform registry.

^{Reviewed by Cursor Bugbot for commit f110c43. Bugbot is set up for automated code reviews on this repo. Configure here.}

Signed-off-by: Mouad BANI <mouad-mb@outlook.com>

github-actions · 2026-04-02T13:58:17Z

⚠️ Jira Issue Key Missing

Your PR title doesn't contain a Jira issue key. Consider adding it for better traceability.

Example:

feat: add user authentication (CM-123)
feat: add user authentication (IN-123)

Projects:

CM: Community Data Platform
IN: Insights

Please add a Jira issue key to your PR title.

github-actions · 2026-04-02T13:58:32Z

⚠️ Jira Issue Key Missing

Your PR title doesn't contain a Jira issue key. Consider adding it for better traceability.

Example:

feat: add user authentication (CM-123)
feat: add user authentication (IN-123)

Projects:

CM: Community Data Platform
IN: Insights

Please add a Jira issue key to your PR title.

github-actions · 2026-04-02T13:59:54Z

⚠️ Jira Issue Key Missing

Your PR title doesn't contain a Jira issue key. Consider adding it for better traceability.

Example:

feat: add user authentication (CM-123)
feat: add user authentication (IN-123)

Projects:

CM: Community Data Platform
IN: Insights

Please add a Jira issue key to your PR title.

services/apps/snowflake_connectors/src/integrations/committees/committees/transformer.ts

Copilot

Pull request overview

Adds “committees” as a first-class platform across the pipeline (types/config, Snowflake connector extraction+transform, and DB activity type registration) so committee membership events can be exported and represented as activities.

Changes:

Added committees to platform/org source enums and organization attribute source priority.
Implemented Snowflake connector source query + transformer for committee membership events.
Added DB migration to register new committee membership activity types.

Reviewed changes

Copilot reviewed 10 out of 11 changed files in this pull request and generated 2 comments.

Show a summary per file

File	Description
services/libs/types/src/enums/platforms.ts	Adds `PlatformType.COMMITTEES`.
services/libs/types/src/enums/organizations.ts	Adds `COMMITTEES` to org source enums.
services/libs/integrations/src/integrations/index.ts	Re-exports committees integration types.
services/libs/integrations/src/integrations/committees/types.ts	Defines committee activity types + scoring grid.
services/libs/data-access-layer/src/organizations/attributesConfig.ts	Adds committees to org attribute source priority list.
services/apps/snowflake_connectors/src/integrations/types.ts	Adds committees datasource name.
services/apps/snowflake_connectors/src/integrations/index.ts	Registers committees platform and datasource.
services/apps/snowflake_connectors/src/integrations/committees/committees/buildSourceQuery.ts	Adds Snowflake query for committee membership export.
services/apps/snowflake_connectors/src/integrations/committees/committees/transformer.ts	Transforms exported rows into committee membership activities.
backend/src/database/migrations/V1775064222__addCommitteesActivityTypes.sql	Inserts committees activity types into `activityTypes`.
backend/src/database/migrations/U1775064222__addCommitteesActivityTypes.sql	Undo migration placeholder (empty).

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

services/apps/snowflake_connectors/src/integrations/committees/committees/transformer.ts

Copilot · 2026-04-02T14:04:56Z

services/apps/snowflake_connectors/src/integrations/committees/committees/transformer.ts

+      return null
+    }
+
+    const committeeId = (row.COMMITTEE_ID as string).trim()


committeeId is computed via (row.COMMITTEE_ID as string).trim() without null/undefined protection. If COMMITTEE_ID is missing or not a string for any row, this will throw and safeTransformRow will skip the row. Consider using optional chaining + explicit skip/log when COMMITTEE_ID is not present.

Suggested change

const committeeId = (row.COMMITTEE_ID as string).trim()

const rawCommitteeId = row.COMMITTEE_ID

const committeeId =

typeof rawCommitteeId === 'string' && rawCommitteeId.trim().length > 0

? rawCommitteeId.trim()

: null

if (!committeeId) {

log.warn(

{ sfid: row.SFID, rawCommitteeId: row.COMMITTEE_ID, email },

'Skipping row: missing or invalid committeeId',

)

return null

}

Signed-off-by: Mouad BANI <mouad-mb@outlook.com>

github-actions · 2026-04-02T14:47:29Z

⚠️ Jira Issue Key Missing

Your PR title doesn't contain a Jira issue key. Consider adding it for better traceability.

Example:

feat: add user authentication (CM-123)
feat: add user authentication (IN-123)

Projects:

CM: Community Data Platform
IN: Insights

Please add a Jira issue key to your PR title.

CLAassistant · 2026-04-02T14:51:09Z

Thank you for your submission! We really appreciate it. Like many open source projects, we ask that you sign our Contributor License Agreement before we can accept your contribution.
_{You have signed the CLA already but the status is still pending? Let us recheck it.}

services/apps/snowflake_connectors/src/integrations/committees/committees/transformer.ts

Signed-off-by: Mouad BANI <mouad-mb@outlook.com>

services/apps/snowflake_connectors/src/integrations/committees/committees/transformer.ts

Signed-off-by: Mouad BANI <mouad-mb@outlook.com>

cursor

Cursor Bugbot has reviewed your changes and found 3 potential issues.

^{Bugbot Autofix is OFF. To automatically fix reported issues with cloud agents, have a team admin enable autofix in the Cursor dashboard.}

services/apps/snowflake_connectors/src/integrations/committees/committees/transformer.ts

cursor · 2026-04-03T12:15:36Z

services/libs/integrations/src/integrations/committees/types.ts

+
+export const COMMITTEES_GRID: Record<CommitteesActivityType, IActivityScoringGrid> = {
+  [CommitteesActivityType.ADDED_TO_COMMITTEE]: { score: 1 },
+  [CommitteesActivityType.REMOVED_FROM_COMMITTEE]: { score: 1 },


Removal activity score inconsistent with codebase pattern

Medium Severity

REMOVED_FROM_COMMITTEE has score: 1, which is inconsistent with the established codebase scoring convention. Analogous removal activities use negative scores: UNSTAR is -2 (reversal of STAR at 2), and MEMBER_LEAVE is -2 (reversal of MEMBER_JOIN at 2). A committee removal with a positive score would incorrectly boost member engagement metrics.

services/apps/snowflake_connectors/src/integrations/committees/committees/buildSourceQuery.ts

joanagmaia · 2026-04-06T10:19:35Z

services/apps/snowflake_connectors/src/integrations/committees/committees/transformer.ts

+    const activityTimestamp =
+      type === CommitteesActivityType.ADDED_TO_COMMITTEE
+        ? (row.CREATEDDATE as string | null) || null
+        : (row.FIVETRAN_SYNCED as string | null) || null


Should it be fivetran_deleted instead?

Not really, fivetran_deleted is a bool. While FIVETRAN_SYNCED is a timestamp, which reflects the deletion timestamp, I believe

services/apps/snowflake_connectors/src/integrations/committees/committees/transformer.ts

Signed-off-by: Mouad BANI <mouad-mb@outlook.com>

feat: committees implementation (WIP)

5f5e903

Signed-off-by: Mouad BANI <mouad-mb@outlook.com>

mbani01 self-assigned this Apr 2, 2026

Copilot AI review requested due to automatic review settings April 2, 2026 13:58

Copilot started reviewing on behalf of mbani01 April 2, 2026 13:58 View session

cursor bot reviewed Apr 2, 2026

View reviewed changes

services/apps/snowflake_connectors/src/integrations/committees/committees/transformer.ts Outdated Show resolved Hide resolved

Copilot AI reviewed Apr 2, 2026

View reviewed changes

fix: activity timestamp and use cncf for staging tests

bdc9da9

Signed-off-by: Mouad BANI <mouad-mb@outlook.com>

mbani01 changed the title ~~feat: committees implementation~~ feat: committees implementation [CM-1066] Apr 2, 2026

mbani01 requested a review from joanagmaia April 2, 2026 14:48

joanagmaia reviewed Apr 2, 2026

View reviewed changes

fix: add channel & use unique sourceId

2f5b708

Signed-off-by: Mouad BANI <mouad-mb@outlook.com>

mbani01 force-pushed the feat/snowflake_committees_implementation branch from cb61fdd to 2f5b708 Compare April 3, 2026 11:50

Merge branch 'main' into feat/snowflake_committees_implementation

5f1703e

cursor bot reviewed Apr 3, 2026

View reviewed changes

services/apps/snowflake_connectors/src/integrations/committees/committees/transformer.ts Outdated Show resolved Hide resolved

mbani01 added 3 commits April 3, 2026 13:05

fix: member identity

6debc97

Signed-off-by: Mouad BANI <mouad-mb@outlook.com>

chore: pass explicity null platform username

510704b

Signed-off-by: Mouad BANI <mouad-mb@outlook.com>

fix: lint

b3da22c

Signed-off-by: Mouad BANI <mouad-mb@outlook.com>

cursor bot reviewed Apr 3, 2026

View reviewed changes

joanagmaia reviewed Apr 6, 2026

View reviewed changes

fix: include soft-delete records in incremental export

f110c43

Signed-off-by: Mouad BANI <mouad-mb@outlook.com>

-    const committeeId = (row.COMMITTEE_ID as string).trim()
+    const rawCommitteeId = row.COMMITTEE_ID
+    const committeeId =
+      typeof rawCommitteeId === 'string' && rawCommitteeId.trim().length > 0
+        ? rawCommitteeId.trim()
+        : null
+    if (!committeeId) {
+      log.warn(
+        { sfid: row.SFID, rawCommitteeId: row.COMMITTEE_ID, email },
+        'Skipping row: missing or invalid committeeId',
+      )
+      return null
+    }

Conversation

mbani01 commented Apr 2, 2026 • edited by cursor bot Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

github-actions bot commented Apr 2, 2026

Uh oh!

github-actions bot commented Apr 2, 2026

Uh oh!

github-actions bot commented Apr 2, 2026

Uh oh!

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Reviewed changes

Uh oh!

Uh oh!

Copilot AI Apr 2, 2026

Choose a reason for hiding this comment

Uh oh!

github-actions bot commented Apr 2, 2026

Uh oh!

CLAassistant commented Apr 2, 2026

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

cursor bot left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

cursor bot Apr 3, 2026

Choose a reason for hiding this comment

Removal activity score inconsistent with codebase pattern

Uh oh!

Uh oh!

Uh oh!

joanagmaia Apr 6, 2026

Choose a reason for hiding this comment

Uh oh!

mbani01 Apr 6, 2026

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

mbani01 commented Apr 2, 2026 •

edited by cursor bot

Loading