Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
9 changes: 9 additions & 0 deletions sdk/voicelive/azure-ai-voicelive/CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -4,6 +4,15 @@

### Features Added

- Added built-in OpenTelemetry tracing support for voice sessions following GenAI Semantic Conventions:
- `VoiceLiveClientBuilder.openTelemetry(OpenTelemetry)` method for providing a custom OpenTelemetry instance
- Defaults to `GlobalOpenTelemetry.getOrNoop()` for automatic Java agent detection with zero-cost no-op fallback
- Emits spans for `connect`, `send`, `recv`, and `close` operations with voice-specific attributes
- Session-level counters: turn count, interruption count, audio bytes sent/received, first token latency
- Per-message attributes: token usage, event types, error details
- Content recording controlled via `enableContentRecording(boolean)` or `AZURE_TRACING_GEN_AI_CONTENT_RECORDING_ENABLED` environment variable
- Added `TelemetrySample.java` demonstrating OpenTelemetry integration patterns

### Breaking Changes

### Bugs Fixed
Expand Down
75 changes: 75 additions & 0 deletions sdk/voicelive/azure-ai-voicelive/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -126,6 +126,7 @@ The following sections provide code snippets for common scenarios:
* [Handle event types](#handle-event-types)
* [Voice configuration](#voice-configuration)
* [Function calling](#function-calling)
* [Telemetry and tracing](#telemetry-and-tracing)
* [Complete voice assistant with microphone](#complete-voice-assistant-with-microphone)

### Focused Sample Files
Expand Down Expand Up @@ -166,6 +167,12 @@ For easier learning, explore these focused samples in order:
- Execute functions locally and return results
- Continue conversation with function results

7. **TelemetrySample.java** - OpenTelemetry tracing integration
- Automatic tracing via GlobalOpenTelemetry (zero-config)
- Explicit OpenTelemetry instance via builder
- Span structure and session-level attributes
- Azure Monitor integration example

> **Note:** To run audio samples (AudioPlaybackSample, MicrophoneInputSample, VoiceAssistantSample, FunctionCallingSample):
> ```bash
> mvn exec:java -Dexec.mainClass=com.azure.ai.voicelive.FunctionCallingSample -Dexec.classpathScope=test
Expand Down Expand Up @@ -397,6 +404,74 @@ client.startSession("gpt-4o-realtime-preview")
* Results are sent back to continue the conversation
* See `FunctionCallingSample.java` for a complete working example

### Telemetry and tracing

The SDK has built-in [OpenTelemetry](https://opentelemetry.io/) tracing that emits spans for every WebSocket operation. When no OpenTelemetry SDK is present, all tracing calls are automatically no-op with zero performance impact.

#### Automatic tracing (recommended)

If the [OpenTelemetry Java agent](https://opentelemetry.io/docs/languages/java/automatic/) is attached, or `GlobalOpenTelemetry` is configured, tracing works automatically with no code changes:

```java com.azure.ai.voicelive.tracing.automatic
// No special configuration needed — tracing is picked up from GlobalOpenTelemetry
VoiceLiveAsyncClient client = new VoiceLiveClientBuilder()
.endpoint(endpoint)
.credential(new AzureKeyCredential(apiKey))
.buildAsyncClient();
```

#### Explicit OpenTelemetry instance

Provide your own `OpenTelemetry` instance to control trace export:

```java com.azure.ai.voicelive.tracing.explicit
VoiceLiveAsyncClient client = new VoiceLiveClientBuilder()
.endpoint(endpoint)
.credential(new AzureKeyCredential(apiKey))
.openTelemetry(otel)
.buildAsyncClient();
```

#### Span structure

When tracing is active, the following span hierarchy is emitted for each voice session:

```
connect gpt-4o-realtime-preview ← session lifetime span
├── send session.update ← one span per sent event
├── send response.create
├── recv session.created ← one span per received event
├── recv response.audio.delta
├── recv response.done ← includes token usage attributes
└── close
```

**Session-level attributes** (on the connect span):
- `gen_ai.system` — `az.ai.voicelive`
- `gen_ai.request.model` — Model name (e.g., `gpt-4o-realtime-preview`)
- `server.address` — Service endpoint
- `gen_ai.voice.session_id` — Voice session ID
- `gen_ai.voice.turn_count` — Completed response turns
- `gen_ai.voice.interruption_count` — User interruptions
- `gen_ai.voice.audio_bytes_sent` / `gen_ai.voice.audio_bytes_received` — Audio payload bytes
- `gen_ai.voice.first_token_latency_ms` — Time to first audio response

#### Content recording

By default, message payloads are not recorded in spans for privacy. Enable content recording via the builder or environment variable:

```java
// Via builder
new VoiceLiveClientBuilder()
.enableContentRecording(true)
// ...

// Or via environment variable
// AZURE_TRACING_GEN_AI_CONTENT_RECORDING_ENABLED=true
```

> See `TelemetrySample.java` for complete tracing examples including Azure Monitor integration.

### Complete voice assistant with microphone

A full example demonstrating real-time microphone input and audio playback:
Expand Down
12 changes: 12 additions & 0 deletions sdk/voicelive/azure-ai-voicelive/checkstyle-suppressions.xml
Original file line number Diff line number Diff line change
Expand Up @@ -3,6 +3,18 @@
<!-- This file is generated by the /eng/scripts/linting_suppression_generator.py script. -->

<suppressions>
<!-- OpenTelemetry tracing imports -->
<suppress files="com.azure.ai.voicelive.VoiceLiveTracer.java" checks="IllegalImportCheck" />
<suppress files="com.azure.ai.voicelive.VoiceLiveAsyncClient.java" checks="IllegalImportCheck" />
<suppress files="com.azure.ai.voicelive.VoiceLiveClientBuilder.java" checks="IllegalImportCheck" />
<suppress files="com.azure.ai.voicelive.VoiceLiveClientBuilder.java" checks="io.clientcore.linting.extensions.checkstyle.checks.ExternalDependencyExposedCheck" />
<suppress files="com.azure.ai.voicelive.VoiceLiveSessionAsyncClient.java" checks="IllegalImportCheck" />
<suppress files="com.azure.ai.voicelive.VoiceLiveTracerTest.java" checks="IllegalImportCheck" />
<suppress files="com.azure.ai.voicelive.VoiceLiveClientBuilderTest.java" checks="IllegalImportCheck" />
<suppress files="com.azure.ai.voicelive.ReadmeSamples.java" checks="IllegalImportCheck" />
<suppress files="com.azure.ai.voicelive.TelemetrySample.java" checks="IllegalImportCheck" />
<suppress files="com.azure.ai.voicelive.VoiceAssistantSample.java" checks="IllegalImportCheck" />

<suppress files="com.azure.ai.voicelive.models.AssistantMessageItem.java" checks="io.clientcore.linting.extensions.checkstyle.checks.EnforceFinalFieldsCheck" />
<suppress files="com.azure.ai.voicelive.models.MessageItem.java" checks="io.clientcore.linting.extensions.checkstyle.checks.EnforceFinalFieldsCheck" />
<suppress files="com.azure.ai.voicelive.models.SystemMessageItem.java" checks="io.clientcore.linting.extensions.checkstyle.checks.EnforceFinalFieldsCheck" />
Expand Down
42 changes: 42 additions & 0 deletions sdk/voicelive/azure-ai-voicelive/pom.xml
Original file line number Diff line number Diff line change
Expand Up @@ -56,6 +56,11 @@ Code generated by Microsoft (R) TypeSpec Code Generator.
<artifactId>azure-core-http-netty</artifactId>
<version>1.16.3</version> <!-- {x-version-update;com.azure:azure-core-http-netty;dependency} -->
</dependency>
<dependency>
<groupId>io.opentelemetry</groupId>
<artifactId>opentelemetry-api</artifactId>
<version>1.58.0</version> <!-- {x-version-update;io.opentelemetry:opentelemetry-api;external_dependency} -->
</dependency>
<dependency>
<groupId>com.azure</groupId>
<artifactId>azure-core-test</artifactId>
Expand All @@ -82,5 +87,42 @@ Code generated by Microsoft (R) TypeSpec Code Generator.
<version>3.7.11</version> <!-- {x-version-update;io.projectreactor:reactor-test;external_dependency} -->
<scope>test</scope>
</dependency>
<dependency>
<groupId>io.opentelemetry</groupId>
<artifactId>opentelemetry-sdk</artifactId>
<version>1.58.0</version> <!-- {x-version-update;io.opentelemetry:opentelemetry-sdk;external_dependency} -->
<scope>test</scope>
</dependency>
<dependency>
<groupId>io.opentelemetry</groupId>
<artifactId>opentelemetry-sdk-testing</artifactId>
<version>1.58.0</version> <!-- {x-version-update;io.opentelemetry:opentelemetry-sdk-testing;external_dependency} -->
<scope>test</scope>
</dependency>
<dependency>
<groupId>io.opentelemetry</groupId>
<artifactId>opentelemetry-exporter-logging</artifactId>
<version>1.58.0</version> <!-- {x-version-update;io.opentelemetry:opentelemetry-exporter-logging;external_dependency} -->
<scope>test</scope>
</dependency>
</dependencies>

<build>
<plugins>
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-enforcer-plugin</artifactId>
<version>3.6.1</version> <!-- {x-version-update;org.apache.maven.plugins:maven-enforcer-plugin;external_dependency} -->
<configuration>
<rules>
<bannedDependencies>
<includes>
<include>io.opentelemetry:opentelemetry-api:[1.58.0]</include> <!-- {x-include-update;io.opentelemetry:opentelemetry-api;external_dependency} -->
</includes>
</bannedDependencies>
</rules>
</configuration>
</plugin>
</plugins>
</build>
</project>
Original file line number Diff line number Diff line change
Expand Up @@ -18,6 +18,7 @@
import com.azure.core.http.HttpHeaderName;
import com.azure.core.http.HttpHeaders;
import com.azure.core.util.logging.ClientLogger;
import io.opentelemetry.api.trace.Tracer;

import reactor.core.publisher.Mono;

Expand All @@ -34,6 +35,8 @@ public final class VoiceLiveAsyncClient {
private final TokenCredential tokenCredential;
private final String apiVersion;
private final HttpHeaders additionalHeaders;
private final Tracer tracer;
private final Boolean enableContentRecording;

/**
* Creates a VoiceLiveAsyncClient with API key authentication.
Expand All @@ -44,11 +47,28 @@ public final class VoiceLiveAsyncClient {
* @param additionalHeaders Additional headers to include in requests.
*/
VoiceLiveAsyncClient(URI endpoint, KeyCredential keyCredential, String apiVersion, HttpHeaders additionalHeaders) {
this(endpoint, keyCredential, apiVersion, additionalHeaders, null, null);
}

/**
* Creates a VoiceLiveAsyncClient with API key authentication and tracing.
*
* @param endpoint The service endpoint.
* @param keyCredential The API key credential.
* @param apiVersion The API version.
* @param additionalHeaders Additional headers to include in requests.
* @param tracer The OpenTelemetry Tracer for instrumentation (may be a no-op tracer).
* @param enableContentRecording Override for content recording, or null to use env var.
*/
VoiceLiveAsyncClient(URI endpoint, KeyCredential keyCredential, String apiVersion, HttpHeaders additionalHeaders,
Tracer tracer, Boolean enableContentRecording) {
this.endpoint = Objects.requireNonNull(endpoint, "'endpoint' cannot be null");
this.keyCredential = Objects.requireNonNull(keyCredential, "'keyCredential' cannot be null");
this.tokenCredential = null;
this.apiVersion = Objects.requireNonNull(apiVersion, "'apiVersion' cannot be null");
this.additionalHeaders = additionalHeaders != null ? additionalHeaders : new HttpHeaders();
this.tracer = tracer;
this.enableContentRecording = enableContentRecording;
}

/**
Expand All @@ -61,11 +81,28 @@ public final class VoiceLiveAsyncClient {
*/
VoiceLiveAsyncClient(URI endpoint, TokenCredential tokenCredential, String apiVersion,
HttpHeaders additionalHeaders) {
this(endpoint, tokenCredential, apiVersion, additionalHeaders, null, null);
}

/**
* Creates a VoiceLiveAsyncClient with token authentication and tracing.
*
* @param endpoint The service endpoint.
* @param tokenCredential The token credential.
* @param apiVersion The API version.
* @param additionalHeaders Additional headers to include in requests.
* @param tracer The OpenTelemetry Tracer for instrumentation (may be a no-op tracer).
* @param enableContentRecording Override for content recording, or null to use env var.
*/
VoiceLiveAsyncClient(URI endpoint, TokenCredential tokenCredential, String apiVersion,
HttpHeaders additionalHeaders, Tracer tracer, Boolean enableContentRecording) {
this.endpoint = Objects.requireNonNull(endpoint, "'endpoint' cannot be null");
this.keyCredential = null;
this.tokenCredential = Objects.requireNonNull(tokenCredential, "'tokenCredential' cannot be null");
this.apiVersion = Objects.requireNonNull(apiVersion, "'apiVersion' cannot be null");
this.additionalHeaders = additionalHeaders != null ? additionalHeaders : new HttpHeaders();
this.tracer = tracer;
this.enableContentRecording = enableContentRecording;
}

/**
Expand All @@ -79,12 +116,7 @@ public Mono<VoiceLiveSessionAsyncClient> startSession(String model) {
Objects.requireNonNull(model, "'model' cannot be null");

return Mono.fromCallable(() -> convertToWebSocketEndpoint(endpoint, model)).flatMap(wsEndpoint -> {
VoiceLiveSessionAsyncClient session;
if (keyCredential != null) {
session = new VoiceLiveSessionAsyncClient(wsEndpoint, keyCredential);
} else {
session = new VoiceLiveSessionAsyncClient(wsEndpoint, tokenCredential);
}
VoiceLiveSessionAsyncClient session = createSessionClient(wsEndpoint, model);
return session.connect(additionalHeaders).thenReturn(session);
});
}
Expand All @@ -97,12 +129,7 @@ public Mono<VoiceLiveSessionAsyncClient> startSession(String model) {
*/
public Mono<VoiceLiveSessionAsyncClient> startSession() {
return Mono.fromCallable(() -> convertToWebSocketEndpoint(endpoint, null)).flatMap(wsEndpoint -> {
VoiceLiveSessionAsyncClient session;
if (keyCredential != null) {
session = new VoiceLiveSessionAsyncClient(wsEndpoint, keyCredential);
} else {
session = new VoiceLiveSessionAsyncClient(wsEndpoint, tokenCredential);
}
VoiceLiveSessionAsyncClient session = createSessionClient(wsEndpoint, null);
return session.connect(additionalHeaders).thenReturn(session);
});
}
Expand All @@ -122,12 +149,7 @@ public Mono<VoiceLiveSessionAsyncClient> startSession(String model, VoiceLiveReq
return Mono
.fromCallable(() -> convertToWebSocketEndpoint(endpoint, model, requestOptions.getCustomQueryParameters()))
.flatMap(wsEndpoint -> {
VoiceLiveSessionAsyncClient session;
if (keyCredential != null) {
session = new VoiceLiveSessionAsyncClient(wsEndpoint, keyCredential);
} else {
session = new VoiceLiveSessionAsyncClient(wsEndpoint, tokenCredential);
}
VoiceLiveSessionAsyncClient session = createSessionClient(wsEndpoint, model);
// Merge additional headers with custom headers from requestOptions
HttpHeaders mergedHeaders = mergeHeaders(additionalHeaders, requestOptions.getCustomHeaders());
return session.connect(mergedHeaders).thenReturn(session);
Expand All @@ -148,12 +170,7 @@ public Mono<VoiceLiveSessionAsyncClient> startSession(VoiceLiveRequestOptions re
return Mono
.fromCallable(() -> convertToWebSocketEndpoint(endpoint, null, requestOptions.getCustomQueryParameters()))
.flatMap(wsEndpoint -> {
VoiceLiveSessionAsyncClient session;
if (keyCredential != null) {
session = new VoiceLiveSessionAsyncClient(wsEndpoint, keyCredential);
} else {
session = new VoiceLiveSessionAsyncClient(wsEndpoint, tokenCredential);
}
VoiceLiveSessionAsyncClient session = createSessionClient(wsEndpoint, null);
// Merge additional headers with custom headers from requestOptions
HttpHeaders mergedHeaders = mergeHeaders(additionalHeaders, requestOptions.getCustomHeaders());
return session.connect(mergedHeaders).thenReturn(session);
Expand All @@ -176,12 +193,7 @@ public Mono<VoiceLiveSessionAsyncClient> startSession(AgentSessionConfig agentCo

return Mono.fromCallable(() -> convertToWebSocketEndpoint(endpoint, null, agentConfig.toQueryParameters()))
.flatMap(wsEndpoint -> {
VoiceLiveSessionAsyncClient session;
if (keyCredential != null) {
session = new VoiceLiveSessionAsyncClient(wsEndpoint, keyCredential);
} else {
session = new VoiceLiveSessionAsyncClient(wsEndpoint, tokenCredential);
}
VoiceLiveSessionAsyncClient session = createSessionClient(wsEndpoint, null);
return session.connect(additionalHeaders).thenReturn(session);
});
}
Expand Down Expand Up @@ -211,18 +223,28 @@ public Mono<VoiceLiveSessionAsyncClient> startSession(AgentSessionConfig agentCo
}

return Mono.fromCallable(() -> convertToWebSocketEndpoint(endpoint, null, mergedParams)).flatMap(wsEndpoint -> {
VoiceLiveSessionAsyncClient session;
if (keyCredential != null) {
session = new VoiceLiveSessionAsyncClient(wsEndpoint, keyCredential);
} else {
session = new VoiceLiveSessionAsyncClient(wsEndpoint, tokenCredential);
}
VoiceLiveSessionAsyncClient session = createSessionClient(wsEndpoint, null);
// Merge additional headers with custom headers from requestOptions
HttpHeaders mergedHeaders = mergeHeaders(additionalHeaders, requestOptions.getCustomHeaders());
return session.connect(mergedHeaders).thenReturn(session);
});
}

/**
* Creates a VoiceLiveSessionAsyncClient with the appropriate credentials and optional tracing.
*
* @param wsEndpoint The WebSocket endpoint URI.
* @param model The model name, used for tracing span names.
* @return A new VoiceLiveSessionAsyncClient instance.
*/
private VoiceLiveSessionAsyncClient createSessionClient(URI wsEndpoint, String model) {
if (keyCredential != null) {
return new VoiceLiveSessionAsyncClient(wsEndpoint, keyCredential, tracer, model, enableContentRecording);
} else {
return new VoiceLiveSessionAsyncClient(wsEndpoint, tokenCredential, tracer, model, enableContentRecording);
}
}

/**
* Merges two HttpHeaders objects, with custom headers taking precedence.
*
Expand Down
Loading