Skip to content
Merged
Show file tree
Hide file tree
Changes from 1 commit
Commits
Show all changes
18 commits
Select commit Hold shift + click to select a range
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
9 changes: 9 additions & 0 deletions sdk/voicelive/azure-ai-voicelive/CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -4,6 +4,15 @@

### Features Added

- Added built-in OpenTelemetry tracing support for voice sessions following GenAI Semantic Conventions:
- `VoiceLiveClientBuilder.openTelemetry(OpenTelemetry)` method for providing a custom OpenTelemetry instance
- Defaults to `GlobalOpenTelemetry.getOrNoop()` for automatic Java agent detection with zero-cost no-op fallback
- Emits spans for `connect`, `send`, `recv`, and `close` operations with voice-specific attributes
- Session-level counters: turn count, interruption count, audio bytes sent/received, first token latency
- Per-message attributes: token usage, event types, error details
- Content recording controlled via `enableContentRecording(boolean)` or `AZURE_TRACING_GEN_AI_CONTENT_RECORDING_ENABLED` environment variable
- Added `TelemetrySample.java` demonstrating OpenTelemetry integration patterns

### Breaking Changes

### Bugs Fixed
Expand Down
75 changes: 75 additions & 0 deletions sdk/voicelive/azure-ai-voicelive/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -126,6 +126,7 @@ The following sections provide code snippets for common scenarios:
* [Handle event types](#handle-event-types)
* [Voice configuration](#voice-configuration)
* [Function calling](#function-calling)
* [Telemetry and tracing](#telemetry-and-tracing)
* [Complete voice assistant with microphone](#complete-voice-assistant-with-microphone)

### Focused Sample Files
Expand Down Expand Up @@ -166,6 +167,12 @@ For easier learning, explore these focused samples in order:
- Execute functions locally and return results
- Continue conversation with function results

7. **TelemetrySample.java** - OpenTelemetry tracing integration
- Automatic tracing via GlobalOpenTelemetry (zero-config)
- Explicit OpenTelemetry instance via builder
- Span structure and session-level attributes
- Azure Monitor integration example

> **Note:** To run audio samples (AudioPlaybackSample, MicrophoneInputSample, VoiceAssistantSample, FunctionCallingSample):
> ```bash
> mvn exec:java -Dexec.mainClass=com.azure.ai.voicelive.FunctionCallingSample -Dexec.classpathScope=test
Expand Down Expand Up @@ -397,6 +404,74 @@ client.startSession("gpt-4o-realtime-preview")
* Results are sent back to continue the conversation
* See `FunctionCallingSample.java` for a complete working example

### Telemetry and tracing

The SDK has built-in [OpenTelemetry](https://opentelemetry.io/) tracing that emits spans for every WebSocket operation. When no OpenTelemetry SDK is present, all tracing calls are automatically no-op with zero performance impact.

#### Automatic tracing (recommended)

If the [OpenTelemetry Java agent](https://opentelemetry.io/docs/languages/java/automatic/) is attached, or `GlobalOpenTelemetry` is configured, tracing works automatically with no code changes:

```java com.azure.ai.voicelive.tracing.automatic
// No special configuration needed — tracing is picked up from GlobalOpenTelemetry
VoiceLiveAsyncClient client = new VoiceLiveClientBuilder()
.endpoint(endpoint)
.credential(new AzureKeyCredential(apiKey))
.buildAsyncClient();
```

#### Explicit OpenTelemetry instance

Provide your own `OpenTelemetry` instance to control trace export:

```java com.azure.ai.voicelive.tracing.explicit
VoiceLiveAsyncClient client = new VoiceLiveClientBuilder()
.endpoint(endpoint)
.credential(new AzureKeyCredential(apiKey))
.openTelemetry(otel)
.buildAsyncClient();
```

#### Span structure

When tracing is active, the following span hierarchy is emitted for each voice session:

```
connect gpt-4o-realtime-preview ← session lifetime span
├── send session.update ← one span per sent event
├── send response.create
├── recv session.created ← one span per received event
├── recv response.audio.delta
├── recv response.done ← includes token usage attributes
└── close
```

**Session-level attributes** (on the connect span):
- `gen_ai.system` — `openai`
- `gen_ai.request.model` — Model name (e.g., `gpt-4o-realtime-preview`)
- `server.address` — Service endpoint
- `gen_ai.voice.session_id` — Voice session ID
- `gen_ai.voice.turn_count` — Completed response turns
- `gen_ai.voice.interruption_count` — User interruptions
- `gen_ai.voice.audio_bytes_sent` / `audio_bytes_received` — Audio payload bytes
Comment thread
xitzhang marked this conversation as resolved.
Outdated
- `gen_ai.voice.first_token_latency_ms` — Time to first audio response

#### Content recording

By default, message payloads are not recorded in spans for privacy. Enable content recording via the builder or environment variable:

```java
// Via builder
new VoiceLiveClientBuilder()
.enableContentRecording(true)
// ...

// Or via environment variable
// AZURE_TRACING_GEN_AI_CONTENT_RECORDING_ENABLED=true
```

> See `TelemetrySample.java` for complete tracing examples including Azure Monitor integration.

### Complete voice assistant with microphone

A full example demonstrating real-time microphone input and audio playback:
Expand Down
12 changes: 12 additions & 0 deletions sdk/voicelive/azure-ai-voicelive/checkstyle-suppressions.xml
Original file line number Diff line number Diff line change
Expand Up @@ -3,6 +3,18 @@
<!-- This file is generated by the /eng/scripts/linting_suppression_generator.py script. -->

<suppressions>
<!-- OpenTelemetry tracing imports -->
<suppress files="com.azure.ai.voicelive.VoiceLiveTracer.java" checks="IllegalImportCheck" />
<suppress files="com.azure.ai.voicelive.VoiceLiveAsyncClient.java" checks="IllegalImportCheck" />
<suppress files="com.azure.ai.voicelive.VoiceLiveClientBuilder.java" checks="IllegalImportCheck" />
<suppress files="com.azure.ai.voicelive.VoiceLiveClientBuilder.java" checks="io.clientcore.linting.extensions.checkstyle.checks.ExternalDependencyExposedCheck" />
Comment thread
xitzhang marked this conversation as resolved.
Outdated
Comment thread
xitzhang marked this conversation as resolved.
Outdated
<suppress files="com.azure.ai.voicelive.VoiceLiveSessionAsyncClient.java" checks="IllegalImportCheck" />
<suppress files="com.azure.ai.voicelive.VoiceLiveTracerTest.java" checks="IllegalImportCheck" />
<suppress files="com.azure.ai.voicelive.VoiceLiveClientBuilderTest.java" checks="IllegalImportCheck" />
<suppress files="com.azure.ai.voicelive.ReadmeSamples.java" checks="IllegalImportCheck" />
<suppress files="com.azure.ai.voicelive.TelemetrySample.java" checks="IllegalImportCheck" />
<suppress files="com.azure.ai.voicelive.VoiceAssistantSample.java" checks="IllegalImportCheck" />

<suppress files="com.azure.ai.voicelive.models.AssistantMessageItem.java" checks="io.clientcore.linting.extensions.checkstyle.checks.EnforceFinalFieldsCheck" />
<suppress files="com.azure.ai.voicelive.models.MessageItem.java" checks="io.clientcore.linting.extensions.checkstyle.checks.EnforceFinalFieldsCheck" />
<suppress files="com.azure.ai.voicelive.models.SystemMessageItem.java" checks="io.clientcore.linting.extensions.checkstyle.checks.EnforceFinalFieldsCheck" />
Expand Down
42 changes: 42 additions & 0 deletions sdk/voicelive/azure-ai-voicelive/pom.xml
Original file line number Diff line number Diff line change
Expand Up @@ -56,6 +56,11 @@ Code generated by Microsoft (R) TypeSpec Code Generator.
<artifactId>azure-core-http-netty</artifactId>
<version>1.16.3</version> <!-- {x-version-update;com.azure:azure-core-http-netty;dependency} -->
</dependency>
<dependency>
<groupId>io.opentelemetry</groupId>
<artifactId>opentelemetry-api</artifactId>
<version>1.58.0</version> <!-- {x-version-update;io.opentelemetry:opentelemetry-api;external_dependency} -->
</dependency>
<dependency>
<groupId>com.azure</groupId>
<artifactId>azure-core-test</artifactId>
Expand All @@ -82,5 +87,42 @@ Code generated by Microsoft (R) TypeSpec Code Generator.
<version>3.7.11</version> <!-- {x-version-update;io.projectreactor:reactor-test;external_dependency} -->
<scope>test</scope>
</dependency>
<dependency>
<groupId>io.opentelemetry</groupId>
<artifactId>opentelemetry-sdk</artifactId>
<version>1.58.0</version> <!-- {x-version-update;io.opentelemetry:opentelemetry-sdk;external_dependency} -->
<scope>test</scope>
</dependency>
<dependency>
<groupId>io.opentelemetry</groupId>
<artifactId>opentelemetry-sdk-testing</artifactId>
<version>1.58.0</version> <!-- {x-version-update;io.opentelemetry:opentelemetry-sdk-testing;external_dependency} -->
<scope>test</scope>
</dependency>
<dependency>
<groupId>io.opentelemetry</groupId>
<artifactId>opentelemetry-exporter-logging</artifactId>
<version>1.58.0</version> <!-- {x-version-update;io.opentelemetry:opentelemetry-exporter-logging;external_dependency} -->
<scope>test</scope>
</dependency>
</dependencies>

<build>
<plugins>
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-enforcer-plugin</artifactId>
<version>3.6.1</version> <!-- {x-version-update;org.apache.maven.plugins:maven-enforcer-plugin;external_dependency} -->
<configuration>
<rules>
<bannedDependencies>
<includes>
<include>io.opentelemetry:opentelemetry-api:[1.58.0]</include> <!-- {x-include-update;io.opentelemetry:opentelemetry-api;external_dependency} -->
Comment thread
xitzhang marked this conversation as resolved.
</includes>
</bannedDependencies>
</rules>
</configuration>
</plugin>
</plugins>
</build>
</project>
Original file line number Diff line number Diff line change
Expand Up @@ -18,6 +18,7 @@
import com.azure.core.http.HttpHeaderName;
import com.azure.core.http.HttpHeaders;
import com.azure.core.util.logging.ClientLogger;
import io.opentelemetry.api.trace.Tracer;

import reactor.core.publisher.Mono;

Expand All @@ -34,6 +35,8 @@ public final class VoiceLiveAsyncClient {
private final TokenCredential tokenCredential;
private final String apiVersion;
private final HttpHeaders additionalHeaders;
private final Tracer tracer;
private final Boolean enableContentRecording;

/**
* Creates a VoiceLiveAsyncClient with API key authentication.
Expand All @@ -44,11 +47,28 @@ public final class VoiceLiveAsyncClient {
* @param additionalHeaders Additional headers to include in requests.
*/
VoiceLiveAsyncClient(URI endpoint, KeyCredential keyCredential, String apiVersion, HttpHeaders additionalHeaders) {
this(endpoint, keyCredential, apiVersion, additionalHeaders, null, null);
}

/**
* Creates a VoiceLiveAsyncClient with API key authentication and tracing.
*
* @param endpoint The service endpoint.
* @param keyCredential The API key credential.
* @param apiVersion The API version.
* @param additionalHeaders Additional headers to include in requests.
* @param tracer The OpenTelemetry Tracer for instrumentation (may be a no-op tracer).
* @param enableContentRecording Override for content recording, or null to use env var.
*/
VoiceLiveAsyncClient(URI endpoint, KeyCredential keyCredential, String apiVersion, HttpHeaders additionalHeaders,
Tracer tracer, Boolean enableContentRecording) {
this.endpoint = Objects.requireNonNull(endpoint, "'endpoint' cannot be null");
this.keyCredential = Objects.requireNonNull(keyCredential, "'keyCredential' cannot be null");
this.tokenCredential = null;
this.apiVersion = Objects.requireNonNull(apiVersion, "'apiVersion' cannot be null");
this.additionalHeaders = additionalHeaders != null ? additionalHeaders : new HttpHeaders();
this.tracer = tracer;
this.enableContentRecording = enableContentRecording;
}

/**
Expand All @@ -61,11 +81,28 @@ public final class VoiceLiveAsyncClient {
*/
VoiceLiveAsyncClient(URI endpoint, TokenCredential tokenCredential, String apiVersion,
HttpHeaders additionalHeaders) {
this(endpoint, tokenCredential, apiVersion, additionalHeaders, null, null);
}

/**
* Creates a VoiceLiveAsyncClient with token authentication and tracing.
*
* @param endpoint The service endpoint.
* @param tokenCredential The token credential.
* @param apiVersion The API version.
* @param additionalHeaders Additional headers to include in requests.
* @param tracer The OpenTelemetry Tracer for instrumentation (may be a no-op tracer).
* @param enableContentRecording Override for content recording, or null to use env var.
*/
VoiceLiveAsyncClient(URI endpoint, TokenCredential tokenCredential, String apiVersion,
HttpHeaders additionalHeaders, Tracer tracer, Boolean enableContentRecording) {
this.endpoint = Objects.requireNonNull(endpoint, "'endpoint' cannot be null");
this.keyCredential = null;
this.tokenCredential = Objects.requireNonNull(tokenCredential, "'tokenCredential' cannot be null");
this.apiVersion = Objects.requireNonNull(apiVersion, "'apiVersion' cannot be null");
this.additionalHeaders = additionalHeaders != null ? additionalHeaders : new HttpHeaders();
this.tracer = tracer;
this.enableContentRecording = enableContentRecording;
}

/**
Expand All @@ -79,12 +116,7 @@ public Mono<VoiceLiveSessionAsyncClient> startSession(String model) {
Objects.requireNonNull(model, "'model' cannot be null");

return Mono.fromCallable(() -> convertToWebSocketEndpoint(endpoint, model)).flatMap(wsEndpoint -> {
VoiceLiveSessionAsyncClient session;
if (keyCredential != null) {
session = new VoiceLiveSessionAsyncClient(wsEndpoint, keyCredential);
} else {
session = new VoiceLiveSessionAsyncClient(wsEndpoint, tokenCredential);
}
VoiceLiveSessionAsyncClient session = createSessionClient(wsEndpoint, model);
return session.connect(additionalHeaders).thenReturn(session);
});
}
Expand All @@ -97,12 +129,7 @@ public Mono<VoiceLiveSessionAsyncClient> startSession(String model) {
*/
public Mono<VoiceLiveSessionAsyncClient> startSession() {
return Mono.fromCallable(() -> convertToWebSocketEndpoint(endpoint, null)).flatMap(wsEndpoint -> {
VoiceLiveSessionAsyncClient session;
if (keyCredential != null) {
session = new VoiceLiveSessionAsyncClient(wsEndpoint, keyCredential);
} else {
session = new VoiceLiveSessionAsyncClient(wsEndpoint, tokenCredential);
}
VoiceLiveSessionAsyncClient session = createSessionClient(wsEndpoint, null);
return session.connect(additionalHeaders).thenReturn(session);
});
}
Expand All @@ -122,12 +149,7 @@ public Mono<VoiceLiveSessionAsyncClient> startSession(String model, VoiceLiveReq
return Mono
.fromCallable(() -> convertToWebSocketEndpoint(endpoint, model, requestOptions.getCustomQueryParameters()))
.flatMap(wsEndpoint -> {
VoiceLiveSessionAsyncClient session;
if (keyCredential != null) {
session = new VoiceLiveSessionAsyncClient(wsEndpoint, keyCredential);
} else {
session = new VoiceLiveSessionAsyncClient(wsEndpoint, tokenCredential);
}
VoiceLiveSessionAsyncClient session = createSessionClient(wsEndpoint, model);
// Merge additional headers with custom headers from requestOptions
HttpHeaders mergedHeaders = mergeHeaders(additionalHeaders, requestOptions.getCustomHeaders());
return session.connect(mergedHeaders).thenReturn(session);
Expand All @@ -148,12 +170,7 @@ public Mono<VoiceLiveSessionAsyncClient> startSession(VoiceLiveRequestOptions re
return Mono
.fromCallable(() -> convertToWebSocketEndpoint(endpoint, null, requestOptions.getCustomQueryParameters()))
.flatMap(wsEndpoint -> {
VoiceLiveSessionAsyncClient session;
if (keyCredential != null) {
session = new VoiceLiveSessionAsyncClient(wsEndpoint, keyCredential);
} else {
session = new VoiceLiveSessionAsyncClient(wsEndpoint, tokenCredential);
}
VoiceLiveSessionAsyncClient session = createSessionClient(wsEndpoint, null);
// Merge additional headers with custom headers from requestOptions
HttpHeaders mergedHeaders = mergeHeaders(additionalHeaders, requestOptions.getCustomHeaders());
return session.connect(mergedHeaders).thenReturn(session);
Expand All @@ -176,12 +193,7 @@ public Mono<VoiceLiveSessionAsyncClient> startSession(AgentSessionConfig agentCo

return Mono.fromCallable(() -> convertToWebSocketEndpoint(endpoint, null, agentConfig.toQueryParameters()))
.flatMap(wsEndpoint -> {
VoiceLiveSessionAsyncClient session;
if (keyCredential != null) {
session = new VoiceLiveSessionAsyncClient(wsEndpoint, keyCredential);
} else {
session = new VoiceLiveSessionAsyncClient(wsEndpoint, tokenCredential);
}
VoiceLiveSessionAsyncClient session = createSessionClient(wsEndpoint, null);
return session.connect(additionalHeaders).thenReturn(session);
});
}
Expand Down Expand Up @@ -211,18 +223,28 @@ public Mono<VoiceLiveSessionAsyncClient> startSession(AgentSessionConfig agentCo
}

return Mono.fromCallable(() -> convertToWebSocketEndpoint(endpoint, null, mergedParams)).flatMap(wsEndpoint -> {
VoiceLiveSessionAsyncClient session;
if (keyCredential != null) {
session = new VoiceLiveSessionAsyncClient(wsEndpoint, keyCredential);
} else {
session = new VoiceLiveSessionAsyncClient(wsEndpoint, tokenCredential);
}
VoiceLiveSessionAsyncClient session = createSessionClient(wsEndpoint, null);
// Merge additional headers with custom headers from requestOptions
HttpHeaders mergedHeaders = mergeHeaders(additionalHeaders, requestOptions.getCustomHeaders());
return session.connect(mergedHeaders).thenReturn(session);
});
}

/**
* Creates a VoiceLiveSessionAsyncClient with the appropriate credentials and optional tracing.
*
* @param wsEndpoint The WebSocket endpoint URI.
* @param model The model name, used for tracing span names.
* @return A new VoiceLiveSessionAsyncClient instance.
*/
private VoiceLiveSessionAsyncClient createSessionClient(URI wsEndpoint, String model) {
if (keyCredential != null) {
return new VoiceLiveSessionAsyncClient(wsEndpoint, keyCredential, tracer, model, enableContentRecording);
} else {
return new VoiceLiveSessionAsyncClient(wsEndpoint, tokenCredential, tracer, model, enableContentRecording);
}
}

/**
* Merges two HttpHeaders objects, with custom headers taking precedence.
*
Expand Down
Loading
Loading