diff --git a/config/_default/menus/main.en.yaml b/config/_default/menus/main.en.yaml index a835a18c00d..7ccaff42184 100644 --- a/config/_default/menus/main.en.yaml +++ b/config/_default/menus/main.en.yaml @@ -6844,7 +6844,7 @@ menu: identifier: observability_pipelines_socket weight: 421 - name: Splunk HEC - url: observability_pipelines/destinations/splunk_hec + url: observability_pipelines/destinations/splunk_hec/logs/ parent: observability_pipelines_destinations identifier: observability_pipelines_splunk_hec weight: 422 diff --git a/content/en/observability_pipelines/destinations/_index.md b/content/en/observability_pipelines/destinations/_index.md index c06f7f6f54f..b6fbb03d08f 100644 --- a/content/en/observability_pipelines/destinations/_index.md +++ b/content/en/observability_pipelines/destinations/_index.md @@ -62,7 +62,7 @@ These are the available destinations: [16]: /observability_pipelines/destinations/opensearch/ [17]: /observability_pipelines/destinations/sentinelone/ [18]: /observability_pipelines/destinations/socket/ -[19]: /observability_pipelines/destinations/splunk_hec/ +[19]: /observability_pipelines/destinations/splunk_hec/logs/ [20]: /observability_pipelines/destinations/sumo_logic_hosted_collector/ [21]: /observability_pipelines/destinations/syslog/ [22]: /observability_pipelines/destinations/amazon_s3/ diff --git a/content/en/observability_pipelines/destinations/splunk_hec/_index.md b/content/en/observability_pipelines/destinations/splunk_hec/_index.md new file mode 100644 index 00000000000..d59a419fa88 --- /dev/null +++ b/content/en/observability_pipelines/destinations/splunk_hec/_index.md @@ -0,0 +1,5 @@ +--- +title: Splunk HEC +type: multi-code-lang +external_redirect: /observability_pipelines/destinations/splunk_hec/logs/ +--- \ No newline at end of file diff --git a/content/en/observability_pipelines/destinations/splunk_hec.md b/content/en/observability_pipelines/destinations/splunk_hec/logs.md similarity index 85% rename from content/en/observability_pipelines/destinations/splunk_hec.md rename to content/en/observability_pipelines/destinations/splunk_hec/logs.md index ecc720bdac0..7ebaa6fd0b9 100644 --- a/content/en/observability_pipelines/destinations/splunk_hec.md +++ b/content/en/observability_pipelines/destinations/splunk_hec/logs.md @@ -1,14 +1,14 @@ --- title: Splunk HTTP Event Collector (HEC) Destination +description: Learn how to set up the Splunk HEC destination for logs in Observability Pipelines. disable_toc: false -products: -- name: Logs - icon: logs - url: /observability_pipelines/configuration/?tab=logs#pipeline-types +aliases: +- /observability_pipelines/destinations/splunk_hec/ +code_lang: logs +type: multi-code-lang +weight: 1 --- -{{< product-availability >}} - ## Overview Use Observability Pipelines' Splunk HTTP Event Collector (HEC) destination to send logs to Splunk HEC. @@ -63,13 +63,7 @@ Select the **Encoding** in the dropdown menu (**JSON** or **Raw**). {{< tabs >}} {{% tab "Secrets Management" %}} -- Splunk HEC token identifier: - - References the Splunk HEC token for the Splunk indexer. - - The default identifier is `DESTINATION_SPLUNK_HEC_TOKEN`. -- Splunk HEC endpoint URL identifier: - - References the Splunk HTTP Event Collector endpoint your Observability Pipelines Worker sends processed logs to. For example, `https://hec.splunkcloud.com:8088`. - - **Note**: `/services/collector/event` path is automatically appended to the endpoint. - - The default identifier is `DESTINATION_SPLUNK_HEC_ENDPOINT_URL`. +{{% observability_pipelines/splunk_hec_secrets %}} {{% /tab %}} diff --git a/content/en/observability_pipelines/destinations/splunk_hec/metrics.md b/content/en/observability_pipelines/destinations/splunk_hec/metrics.md new file mode 100644 index 00000000000..744a700cd50 --- /dev/null +++ b/content/en/observability_pipelines/destinations/splunk_hec/metrics.md @@ -0,0 +1,96 @@ +--- +title: Splunk HTTP Event Collector (HEC) Destination +description: Learn how to set up the Splunk HEC destination for metrics in Observability Pipelines. +disable_toc: false +code_lang: metrics +type: multi-code-lang +weight: 2 +--- + +## Overview + +Use Observability Pipelines' Splunk HTTP Event Collector (HEC) destination to send metrics to Splunk HEC. + +## Setup + +Configure the Splunk HEC destination when you [set up a pipeline][1]. You can set up a pipeline in the [UI][3], using the [API][4], or with [Terraform][5]. The steps in this section are configured in the UI. + +**Notes**: +- The Splunk index you send your metrics to must be a metrics index. If you send them to an events index, you can't view the metrics in Splunk using any metrics type queries, such as `mcatalog` and `mstats`. +- If you don't attach your index to the Splunk authentication token you are using for Observability Pipelines, you must enter the name of the [index](#splunk-index) when you set up the destination. + +
For Secrets Management: Only enter the identifiers for the Splunk HEC token, endpoint, and if applicable, the TLS key pass. Do not enter the actual values.
+ +{{% observability_pipelines/secrets_env_var_note %}} + +After you select the Splunk HEC destination in the pipeline UI: + +1. Enter the identifier for your token. If you leave it blank, the [default](#secret-defaults) is used. +1. Enter the identifier for your endpoint URL. If you leave it blank, the [default](#secret-defaults) is used. + +### Optional settings + +#### Default namespace + +Enter a default namespace. + +#### Compression + +If you want to compress your metrics with gzip, select **gzip** in the dropdown menu. The default compression is **None**. + +#### Splunk index + +Enter the name of the Splunk metrics index to which you are sending metrics. See [template syntax][6] if you want to route metrics to different indexes based on specific fields in your metrics. + +**Note**: This **Index** field is only optional if you have an index attached to the Splunk authentication token you are using for Observability Pipelines. Otherwise, you must enter the name of the Splunk metrics index. + +#### Source + +Enter a source value if you want the source field added to your metrics. + +#### Source type override + +Set the `sourcetype` to override Splunk's default value, which is `httpevent` for HEC data. See [template syntax][6] if you want to route metrics to different source types based on specific fields in your metrics. + +#### Buffering + +{{% observability_pipelines/destination_buffer %}} + +#### Enable TLS + +{{% observability_pipelines/tls_settings %}} + +## Secret defaults + +{{% observability_pipelines/set_secrets_intro %}} + +{{< tabs >}} +{{% tab "Secrets Management" %}} + +{{% observability_pipelines/splunk_hec_secrets %}} + +{{% /tab %}} + +{{% tab "Environment Variables" %}} + +{{% observability_pipelines/configure_existing_pipelines/destination_env_vars/splunk_hec %}} + +{{% /tab %}} +{{< /tabs >}} + +### How the destination works + +#### Event batching + +A batch of events is flushed when one of these parameters is met. See [event batching][2] for more information. + +| Maximum Events | Maximum Size (MB) | Timeout (seconds) | +|----------------|-------------------|---------------------| +| None | 1 | 1 | + +[1]: /observability_pipelines/configuration/set_up_pipelines/ +[2]: /observability_pipelines/destinations/#event-batching +[3]: https://app.datadoghq.com/observability-pipelines +[4]: /api/latest/observability-pipelines/ +[5]: https://registry.terraform.io/providers/datadog/datadog/latest/docs/resources/observability_pipeline +[6]: /observability_pipelines/destinations/#template-syntax \ No newline at end of file diff --git a/layouts/shortcodes/observability_pipelines/configure_existing_pipelines/destination_env_vars/splunk_hec.en.md b/layouts/shortcodes/observability_pipelines/configure_existing_pipelines/destination_env_vars/splunk_hec.en.md index f775154f6fa..82e4f75b57e 100644 --- a/layouts/shortcodes/observability_pipelines/configure_existing_pipelines/destination_env_vars/splunk_hec.en.md +++ b/layouts/shortcodes/observability_pipelines/configure_existing_pipelines/destination_env_vars/splunk_hec.en.md @@ -2,6 +2,6 @@ - The Splunk HEC token for the Splunk indexer. **Note**: Depending on your shell and environment, you may not want to wrap your environment variable in quotes. - The default environment variable is `DD_OP_DESTINATION_SPLUNK_HEC_TOKEN`. - Base URL of the Splunk instance: - - The Splunk HTTP Event Collector endpoint your Observability Pipelines Worker sends processed logs to. For example, `https://hec.splunkcloud.com:8088`. + - The Splunk HTTP Event Collector endpoint your Observability Pipelines Worker sends processed data to. For example, `https://hec.splunkcloud.com:8088`. **Note**: `/services/collector/event` path is automatically appended to the endpoint. - The default environment variable is `DD_OP_DESTINATION_SPLUNK_HEC_ENDPOINT_URL`. \ No newline at end of file diff --git a/layouts/shortcodes/observability_pipelines/splunk_hec_secrets.en.md b/layouts/shortcodes/observability_pipelines/splunk_hec_secrets.en.md new file mode 100644 index 00000000000..7bb52750be9 --- /dev/null +++ b/layouts/shortcodes/observability_pipelines/splunk_hec_secrets.en.md @@ -0,0 +1,7 @@ +- Splunk HEC token identifier: + - References the Splunk HEC token for the Splunk indexer. + - The default identifier is `DESTINATION_SPLUNK_HEC_TOKEN`. +- Splunk HEC endpoint URL identifier: + - References the Splunk HTTP Event Collector endpoint your Observability Pipelines Worker sends processed data to. For example, `https://hec.splunkcloud.com:8088`. + - **Note**: `/services/collector/event` path is automatically appended to the endpoint. + - The default identifier is `DESTINATION_SPLUNK_HEC_ENDPOINT_URL`. \ No newline at end of file