Add OCSF pipelines by orenma-dd · Pull Request #22267 · DataDog/integrations-core

orenma-dd · 2026-01-06T13:41:20Z

Create OCSF mappings for Cisco Umbrella pipeline

chatgpt-codex-connector

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: da32c6c597

ℹ️ About Codex in GitHub

Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you

Open a pull request for review
Mark a draft as ready
Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".

…f-mapping

github-actions · 2026-01-08T11:41:58Z

⚠️ Recommendation: Add qa/skip-qa label

This PR does not modify any files shipped with the agent.

To help streamline the release process, please consider adding the qa/skip-qa label if these changes do not require QA testing.

orenma-dd · 2026-01-08T12:49:34Z

orenma-dd · 2026-01-08T13:28:29Z

jbfeldman-dd · 2026-01-08T14:12:58Z

+            className: HTTP Activity
+            classUid: 4002
+            extensions: []
+            profiles: []


Should probably have the network proxy and security control profiles. If you had the security_control profile, you could map info on malware, DLS prevention, etc.

Good point will add it

jbfeldman-dd · 2026-01-08T14:17:24Z

+              targetFormat: string
+              preserveSource: true
+              overrideOnConflict: true
+              type: schema-remapper


If you're using the Network Proxy Profile, you could map externalip to proxy_endpoint.ip

jbfeldman-dd · 2026-01-08T14:25:45Z

+              preserveSource: true
+              overrideOnConflict: true
+              type: schema-remapper
+            - name: Map `domain` to `ocsf.query.hostname`


You could map info in the identities array to additional fields in ocsf.src_endpoint (like hw_info). Not required though, so up to you and not blocking

I am not sure how consistent is hw_info

jbfeldman-dd · 2026-01-08T14:34:51Z

+              preserveSource: true
+              overrideOnConflict: true
+              type: schema-remapper
+            - name: Map `internalip` to `ocsf.src_endpoint.ip`


You should map destination IP as well as source IP

jbfeldman-dd · 2026-01-08T14:36:09Z

+              targetFormat: string
+              preserveSource: true
+              overrideOnConflict: true
+              type: schema-remapper


You could map various file attributes to ocsf.file as well, it would extract indicators that could be useful for threat intel matching

This logs for proxy is kinda weird since they have field blockedfilename\responsefilename which I am not sure is what the ocsf.file is looking for
The File object represents the metadata associated with a file stored in a computer system
https://schema.ocsf.io/1.3.0/objects/file

…f-mapping

jbfeldman-dd

Non-blocking comments. LGTM

jbfeldman-dd · 2026-01-14T14:33:54Z

+          targetType: attribute
+          targetFormat: integer
+          preserveSource: false
+          overrideOnConflict: false


The preferred design here I think is to have OCSF sub pipeline for HTTP Activity [4002] malware transformations come before the other 4002 sub pipeline, and then map ocsf.disposition_id to itself in the schema-processor. This will ensure it's always mapped to the correct type based on the OCSF schema, and you don't have to map it yourself. But ultimately a stylistic choice so up to you

Is the query that specifically has service:proxy not enough to ensure that?

jbfeldman-dd · 2026-01-14T14:35:10Z

+            source: ocsf.malware.classification_id
+            target: ocsf.malware.classification_ids
+            preserveSource: false
+            type: append


similar to above comment - mapping ocsf.malware.classification_ids to itself in the schema-processor is a slightly more elegant way of doing this

Issue is if there is not malware its an issue since I cant remove a field/

Review from jbfeldman-dd is dismissed. Related teams and files:

logs-integrations-reviewers
- cisco_umbrella_dns/assets/logs/cisco-umbrella-dns.yaml

…f-mapping

* Add OCSF pipelines * update test file * update processor name * remove metadata placeholder and cahnge test file * chnage schema processor name * add facets * remove conflicting facets * update activity id mapping * update activity id mapping * preserve source * update test file * add is_ocsf flag * address feedback * fix test file * add malware sub pipeline * fix test file * add endpoint proxy to dns * Bring back ocsf.metadata field * add ocsf.metadata Signed-off-by: lukepatrick <lukephilips@gmail.com>

Add OCSF pipelines

da32c6c

orenma-dd requested a review from a team as a code owner January 6, 2026 13:41

temporal-github-worker-1 Bot added ecosystems/review-requested product/review-requested labels Jan 6, 2026

datadog-agent-integrations-bot Bot added integration/cisco_umbrella_dns team/logs-integrations-reviewers labels Jan 6, 2026

chatgpt-codex-connector Bot reviewed Jan 6, 2026

View reviewed changes

Comment thread cisco_umbrella_dns/assets/logs/cisco-umbrella-dns.yaml Outdated

Comment thread cisco_umbrella_dns/assets/logs/cisco-umbrella-dns.yaml Outdated

orenma-dd added 3 commits January 6, 2026 16:00

update test file

8e8cd79

update processor name

3b58ce4

remove metadata placeholder and cahnge test file

7329388

orenma-dd added assets/no-dry-run Run asset publishing github checks in staging and removed assets/no-dry-run Run asset publishing github checks in staging labels Jan 7, 2026

Merge branch 'master' into oren.margalit/SCI2-5670-cisco-umbrella-ocs…

6220c79

…f-mapping

orenma-dd added 3 commits January 8, 2026 13:57

chnage schema processor name

d4d0228

add facets

e5c23c6

remove conflicting facets

74c07ea

update activity id mapping

c98bc0a

orenma-dd removed the assets/no-dry-run Run asset publishing github checks in staging label Jan 8, 2026

orenma-dd added 3 commits January 8, 2026 15:12

update activity id mapping

ce9985e

preserve source

7c427bc

update test file

5cd7927

jbfeldman-dd reviewed Jan 8, 2026

View reviewed changes

orenma-dd added the assets/no-dry-run Run asset publishing github checks in staging label Jan 8, 2026

add is_ocsf flag

8278662

orenma-dd removed the assets/no-dry-run Run asset publishing github checks in staging label Jan 11, 2026

orenma-dd added 2 commits January 11, 2026 17:35

address feedback

bbe44fa

fix test file

6d39b59

orenma-dd added the assets/no-dry-run Run asset publishing github checks in staging label Jan 14, 2026

orenma-dd and others added 3 commits January 14, 2026 14:47

add malware sub pipeline

a4ca2f5

fix test file

f06a05c

Merge branch 'master' into oren.margalit/SCI2-5670-cisco-umbrella-ocs…

58e63cf

…f-mapping

orenma-dd removed the assets/no-dry-run Run asset publishing github checks in staging label Jan 14, 2026

add endpoint proxy to dns

40d13c6

jbfeldman-dd previously approved these changes Jan 14, 2026

View reviewed changes

Bring back ocsf.metadata field

1d7c0ce

orenma-dd and others added 2 commits January 15, 2026 12:30

add ocsf.metadata

cfe3ce8

Merge branch 'master' into oren.margalit/SCI2-5670-cisco-umbrella-ocs…

0d2e662

…f-mapping

jbfeldman-dd approved these changes Jan 21, 2026

View reviewed changes

Merge branch 'master' into oren.margalit/SCI2-5670-cisco-umbrella-ocs…

fa5b31f

…f-mapping

orenma-dd added this pull request to the merge queue Jan 26, 2026

Merged via the queue into master with commit 9a3216f Jan 26, 2026
46 checks passed

orenma-dd deleted the oren.margalit/SCI2-5670-cisco-umbrella-ocsf-mapping branch January 26, 2026 07:43

Conversation

orenma-dd commented Jan 6, 2026

Uh oh!

chatgpt-codex-connector Bot left a comment

Choose a reason for hiding this comment

💡 Codex Review

Uh oh!

Uh oh!

Uh oh!

github-actions Bot commented Jan 8, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

orenma-dd commented Jan 8, 2026

Uh oh!

orenma-dd commented Jan 8, 2026

Uh oh!

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

jbfeldman-dd left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

github-actions Bot commented Jan 8, 2026 •

edited

Loading