Skip to content

Add documentation for Elastic Agent as an OTel Collector#5545

Open
vishaangelova wants to merge 2 commits intomainfrom
1882-document-hybrid-agent
Open

Add documentation for Elastic Agent as an OTel Collector#5545
vishaangelova wants to merge 2 commits intomainfrom
1882-document-hybrid-agent

Conversation

@vishaangelova
Copy link
Contributor

Summary

This PR adds a new “Elastic Agent as an OTel Collector” document with details about:

  • Elastic Agent using OTel runtime for agent self-monitoring from v9.3
  • Architectural changes and the Beats receivers as OTel receivers
  • Elastic Agent as a hybrid agent and OTel integrations
  • A comparison between the collector options available for ingesting OTel data
  • FAQ

Resolves elastic/ingest-docs#1882

Generative AI disclosure

  1. Did you use a generative AI (GenAI) tool to assist in creating this contribution?
  • Yes
  • No
  1. If you answered "Yes" to the previous question, please specify the tool(s) and model(s) used (e.g., Google Gemini, OpenAI ChatGPT-4, etc.).

Tool(s) and model(s) used: Cursor with claude-4.5-sonnet and Auto models

@github-actions
Copy link
Contributor

github-actions bot commented Mar 18, 2026

✅ Vale Linting Results

No issues found on modified lines!


The Vale linter checks documentation changes against the Elastic Docs style guide.

To use Vale locally or report issues, refer to Elastic style guide for Vale.

@github-actions
Copy link
Contributor

github-actions bot commented Mar 18, 2026

🔍 Preview links for changed docs

endpoints:
- https://example.com

exporters:
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should the exporter section include an elasticsearch exporter if the output section already includes the ES output? For this configuration example I used the snippet from https://github.com/elastic/elastic-agent/blob/main/docs/hybrid-agent-beats-receivers.md#hybrid-agent.


Data collected by a Beat receiver is written by the {{es}} exporter. As with traditional {{agent}} and {{beats}}, this data is processed by ingest pipelines, schematized, and stored in the appropriate data stream.

Data collected by an OTel-native receiver follows OpenTelemetry semantic conventions. When this data arrives at the Elastic cluster, it bypasses ingest pipelines and is stored directly in an OTel-specific data stream.
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@nimarezainia is this statement correct? You had a comment in your draft google doc which said this might need clarification.

A standalone hybrid {{agent}} can enroll into {{fleet}} in the field if an in-place upgrade to {{fleet}}-managed is required. The standalone EDOT Collector does not support enrollment into {{fleet}}.
:::

## Frequently asked questions [faq]
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should we rather move this section to a separate doc positioned under this one? I think keeping the FAQ here makes the Q&A pairs more visible.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'd rather not have an FAQ as a pattern unless we deem it strictly necessary.

@vishaangelova vishaangelova marked this pull request as ready for review March 18, 2026 15:58
@vishaangelova vishaangelova requested a review from a team as a code owner March 18, 2026 15:58
@karenzone karenzone self-requested a review March 19, 2026 12:57
Copy link
Contributor

@theletterf theletterf left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Great work! First review pass.

---

# {{agent}} as an OpenTelemetry Collector [elastic-agent-otel-collector]

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Guess you thought about this, but... Section-level applies to?


# {{agent}} as an OpenTelemetry Collector [elastic-agent-otel-collector]

Starting with version 9.3, {{agent}} runs on the [Elastic Distribution of OpenTelemetry (EDOT) Collector](elastic-agent://reference/edot-collector/index.md) internally. Rather than managing separate {{beats}} sub-processes, the agent now runs data collection inside an embedded OpenTelemetry (OTel) Collector process, leveraging the extensibility and interoperability of the OTel ecosystem. This new architecture brings OTel capabilities while maintaining compatibility with existing {{beats}}-based workflows and integrations.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
Starting with version 9.3, {{agent}} runs on the [Elastic Distribution of OpenTelemetry (EDOT) Collector](elastic-agent://reference/edot-collector/index.md) internally. Rather than managing separate {{beats}} sub-processes, the agent now runs data collection inside an embedded OpenTelemetry (OTel) Collector process, leveraging the extensibility and interoperability of the OTel ecosystem. This new architecture brings OTel capabilities while maintaining compatibility with existing {{beats}}-based workflows and integrations.
Starting with version 9.3, {{agent}} runs the [Elastic Distribution of OpenTelemetry (EDOT) Collector](elastic-agent://reference/edot-collector/index.md). Rather than managing separate {{beats}} sub-processes, the agent collects data through an embedded OpenTelemetry (OTel) Collector process, leveraging the extensibility and interoperability of the OTel ecosystem. This architecture brings OTel capabilities while maintaining compatibility with existing {{beats}}-based workflows and integrations.

Comment on lines +19 to +21
Previously {{agent}} acted as a supervisor that launched and managed individual {{beats}} processes to collect telemetry ({{filebeat}}, {{metricbeat}}, and others), each running as a separate sub-process.

With the new {{agent}} OTel Collector architecture:
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I would focus just on the present. I'm not sure if the past architecture is relevant.

With the new {{agent}} OTel Collector architecture:

* {{agent}} embeds the EDOT Collector as its runtime, eliminating the overhead associated with managing separate sub-processes.
* Instead of running as individual {{beats}} processes, Beat inputs can run inside the EDOT Collector as [Beat receivers](#beat-receivers).
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
* Instead of running as individual {{beats}} processes, Beat inputs can run inside the EDOT Collector as [Beat receivers](#beat-receivers).
* Instead of running as individual {{beats}} processes, Beats inputs can run inside the EDOT Collector as [Beat receivers](#beat-receivers).

Beat can be used without 's'?

:alt: Diagram showing data ingestion with Beat receivers and OTel receivers as part of an OTel Collector
:::

:::{note}
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Applies_to for admonitions?


## Beat receivers [beat-receivers]

A _Beat receiver_ is a Beat input and its associated processors, wrapped to run as an OTel receiver inside the EDOT Collector. Beat receivers produce the exact same data, formatted according to the [Elastic Common Schema](ecs://reference/index.md) (ECS), as current Beat inputs — they do not output data in the OTLP schema.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
A _Beat receiver_ is a Beat input and its associated processors, wrapped to run as an OTel receiver inside the EDOT Collector. Beat receivers produce the exact same data, formatted according to the [Elastic Common Schema](ecs://reference/index.md) (ECS), as current Beat inputs — they do not output data in the OTLP schema.
A _Beat receiver_ is a Beat input and its associated processors, wrapped to run as an OTel receiver inside the EDOT Collector. Beat receivers produce the exact same data, formatted according to the [Elastic Common Schema](ecs://reference/index.md) (ECS), as current Beat inputs. Beat receivers don't output data in the OTLP schema.

:alt: Beat input configuration translated into an OTel configuration
:::

## {{agent}} as a hybrid agent [hybrid-elastic-agent]
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This in my reads different than "EA as an OTel Collector". Could we make it more compatible, like "Hybrid data collection"?

A standalone hybrid {{agent}} can enroll into {{fleet}} in the field if an in-place upgrade to {{fleet}}-managed is required. The standalone EDOT Collector does not support enrollment into {{fleet}}.
:::

## Frequently asked questions [faq]
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'd rather not have an FAQ as a pattern unless we deem it strictly necessary.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Document details of Hybrid Agent

2 participants