[FEATURE] Propagate A2A message metadata as OTEL span attribute

[karma-king-350]

📋 Prerequisites

• I have searched the existing issues to avoid creating a duplicate
• By submitting this issue, you agree to follow our Code of Conduct

📝 Feature Summary

Add support for propagating custom metadata from A2A 'message' payloads as attributes on OTEL spans, enabling external callers to attach contextual data (e.g. approver identity) to traces.

❓ Problem Statement / Motivation

• Current limitation: kagent's OTEL instrumentation only emits spans for its own internal operations (invoke_agent, execute_tool,
  call_llm.. etc). Any metadata included in the A2A message/send payload (at params.message.metadata or within
  params.message.parts[].data) is silently dropped and never surfaces in any OTEL span.
• Who is affected: Anyone building human-in-the-loop or external-trigger workflows on top of kagent where the caller needs to attach
  contextual attributes to the trace (e.g. who approved an action, which system triggered the request, audit metadata).
• Why it's needed: In approval workflows (e.g. a Slack bot that gates agent actions), the approver's identity is captured outside
  kagent. There is currently no way to record that identity in OTEL traces, making audit and observability impossible without running
  a completely separate OTEL pipeline from the external service.

💡 Proposed Solution

When kagent processes an incoming message/send request, any key-value pairs found in params.message.metadata should be set as
attributes on the active OTEL span (e.g. under a a2a.message.metadata.* prefix).

Example: if the caller sends metadata: { "approver_email": "[user@example.com](mailto:user@example.com)" }, the span should include the attribute
a2a.message.metadata.approver_email = [user@example.com](mail to:user@example.com).

Requirements:

• Only scalar values (string, number, bool) should be set as span attributes
• The prefix a2a.message.metadata. should be used to avoid collisions with kagent's own span attributes
• Nested objects in metadata can be skipped or flattened (up to implementation discretion)

🔄 Alternatives Considered

• External OTEL instrumentation: The calling service (e.g. Slack handler) emits its own spans. This works but requires trace context
  (traceparent) to be propagated through the A2A payload to stitch spans into the same trace
  trace context injection/extraction on A2A messages.
• Storing metadata in PostgreSQL: The event table stores A2A message payloads, so approver data can be queried post-hoc. This is a workaround but not a real-time observability solution.

🎯 Affected Service(s)

None

📚 Additional Context

Use case: Human-in-the-loop Slack approval flow where a Slack bot sends the approver's email back to kagent via message/send. The
approver identity needs to appear in OTEL traces for audit and compliance purposes.

🙋 Are you willing to contribute?

• I am willing to submit a PR for this feature