From 983238388e701fa36655f355c82df3780991e9b2 Mon Sep 17 00:00:00 2001
From: Prathik Rao
Date: Wed, 25 Mar 2026 11:19:54 -0700
Subject: [PATCH 01/21] secure supply chain analysis fixes (#549)
Fixes the following errors I encountered when migrating our
packaging/publishing pipelines to onnxruntime-release-pipelines
```
Starting: Secure Supply Chain Analysis (auto-injected by policy)
==============================================================================
Task : Secure Supply Chain Analysis
Description : A task to scan for vulnerabilities in your software supply chain. Formerly "NuGet Security Analysis".
Version : 0.2.216
Author : Microsoft Corporation
Help : See https://aka.ms/sscatask for more information.
==============================================================================
Telemetry ID: 29518951-f4fb-4d5c-a56e-110cbb97c51b
For more information please visit: https://aka.ms/sscatask
Scanning repository contents at source path: E:\_work\1\s
> Starting Multifeed Nuget Security Analysis:
##[warning]samples/cs/GettingStarted/nuget.config - Multiple feeds declared. (https://aka.ms/cfs/nuget)
##[warning]sdk/cs/NuGet.config - Multiple feeds declared. (https://aka.ms/cfs/nuget)
> Starting Multifeed Corext Analysis:
> Starting Multifeed Python Security Analysis:
> Starting CFS NuGet Analysis:
##[warning]samples/cs/GettingStarted/nuget.config - CFS0013: Package source has value that is not an Azure Artifacts feed. (https://aka.ms/cfs/nuget)
##[warning]sdk/cs/NuGet.config - CFS0013: Package source has value that is not an Azure Artifacts feed. (https://aka.ms/cfs/nuget)
##[warning]sdk_legacy/cs/samples/TestApp/TestApp.csproj - CFS0011: Missing in scope NuGet.config file(s). (https://aka.ms/cfs/nuget)
##[warning]sdk_legacy/cs/src/Microsoft.AI.Foundry.Local.csproj - CFS0011: Missing in scope NuGet.config file(s). (https://aka.ms/cfs/nuget)
##[warning]sdk_legacy/cs/test/FoundryLocal.Tests/FoundryLocal.Tests.csproj - CFS0011: Missing in scope NuGet.config file(s). (https://aka.ms/cfs/nuget)
> Starting CFS NPM Analysis:
##[warning]www/.npmrc - CFS0002: Missing default registry. (https://aka.ms/cfs/npm)
##[warning]samples/js/chat-and-audio-foundry-local/package.json - CFS0001: Missing sibling .npmrc file. (https://aka.ms/cfs/npm)
##[warning]samples/js/copilot-sdk-foundry-local/package.json - CFS0001: Missing sibling .npmrc file. (https://aka.ms/cfs/npm)
##[warning]samples/js/electron-chat-application/package.json - CFS0001: Missing sibling .npmrc file. (https://aka.ms/cfs/npm)
##[warning]samples/js/tool-calling-foundry-local/package.json - CFS0001: Missing sibling .npmrc file. (https://aka.ms/cfs/npm)
##[warning]sdk/js/package.json - CFS0001: Missing sibling .npmrc file. (https://aka.ms/cfs/npm)
##[warning]sdk_legacy/js/package.json - CFS0001: Missing sibling .npmrc file. (https://aka.ms/cfs/npm)
> Starting CFS Maven Analysis:
> Starting CFS Cargo Analysis:
##[warning]samples/rust/Cargo.toml - CFS0041: Missing associated .cargo/config.toml file. (https://aka.ms/cfs/cargo)
##[warning]samples/rust/audio-transcription-example/Cargo.toml - CFS0041: Missing associated .cargo/config.toml file. (https://aka.ms/cfs/cargo)
##[warning]samples/rust/foundry-local-webserver/Cargo.toml - CFS0041: Missing associated .cargo/config.toml file. (https://aka.ms/cfs/cargo)
##[warning]samples/rust/native-chat-completions/Cargo.toml - CFS0041: Missing associated .cargo/config.toml file. (https://aka.ms/cfs/cargo)
##[warning]samples/rust/tool-calling-foundry-local/Cargo.toml - CFS0041: Missing associated .cargo/config.toml file. (https://aka.ms/cfs/cargo)
##[warning]sdk/rust/Cargo.toml - CFS0041: Missing associated .cargo/config.toml file. (https://aka.ms/cfs/cargo)
##[warning]sdk_legacy/rust/Cargo.toml - CFS0041: Missing associated .cargo/config.toml file. (https://aka.ms/cfs/cargo)
> Starting CFS CoreXT Analysis:
> Starting CFS CDPx Analysis:
> Starting DockerFile Analysis:
> Starting Kubernetes Deployment File Analysis:
> Starting Helm Charts Analysis:
> Starting Pipeline Configuration Security Analysis:
Azure Artifacts Configuration Analysis found 19 package configuration files in the repository which do not comply with Microsoft package feed security policies. The specific problems and links to their mitigations are listed above. If you need further assistance, please visit https://aka.ms/cfs/detectors .
##[error]NuGet Security Analysis found 2 NuGet package configuration files in the repository which do not comply with Microsoft package feed security policies. The specific problems are listed above. Please visit https://aka.ms/cfs/nuget for more details.
```
---------
Co-authored-by: Prathik Rao
---
.github/workflows/build-cs-steps.yml | 3 +++
.github/workflows/build-js-steps.yml | 12 +++++++-----
.github/workflows/build-rust-steps.yml | 12 ++++++++++++
samples/cs/GettingStarted/nuget.config | 11 +----------
samples/js/chat-and-audio-foundry-local/.npmrc | 2 ++
samples/js/copilot-sdk-foundry-local/.npmrc | 2 ++
samples/js/electron-chat-application/.npmrc | 2 ++
samples/js/tool-calling-foundry-local/.npmrc | 2 ++
samples/rust/.cargo/config.toml | 7 +++++++
sdk/cs/NuGet.config | 1 -
.../Microsoft.AI.Foundry.Local.Tests.csproj | 4 ++--
sdk/js/.npmrc | 2 ++
sdk/js/package.json | 4 ++--
sdk/js/script/install.cjs | 6 +++++-
sdk/rust/.cargo/config.toml | 7 +++++++
sdk_legacy/cs/NuGet.config | 7 +++++++
sdk_legacy/js/.npmrc | 2 ++
sdk_legacy/rust/.cargo/config.toml | 7 +++++++
www/.npmrc | 2 ++
19 files changed, 74 insertions(+), 21 deletions(-)
create mode 100644 samples/js/chat-and-audio-foundry-local/.npmrc
create mode 100644 samples/js/copilot-sdk-foundry-local/.npmrc
create mode 100644 samples/js/electron-chat-application/.npmrc
create mode 100644 samples/js/tool-calling-foundry-local/.npmrc
create mode 100644 samples/rust/.cargo/config.toml
create mode 100644 sdk/js/.npmrc
create mode 100644 sdk/rust/.cargo/config.toml
create mode 100644 sdk_legacy/cs/NuGet.config
create mode 100644 sdk_legacy/js/.npmrc
create mode 100644 sdk_legacy/rust/.cargo/config.toml
diff --git a/.github/workflows/build-cs-steps.yml b/.github/workflows/build-cs-steps.yml
index 9b089bc6..dcfed979 100644
--- a/.github/workflows/build-cs-steps.yml
+++ b/.github/workflows/build-cs-steps.yml
@@ -43,6 +43,9 @@ jobs:
# TODO: once the nightly packaging is fixed, add back the commented out lines with /p:FoundryLocalCoreVersion="*-*"
# /p:FoundryLocalCoreVersion="*-*" to always use nightly version of Foundry Local Core
+ - name: Authenticate to Azure Artifacts NuGet feed
+ run: dotnet nuget update source ORT-Nightly --username az --password ${{ secrets.AZURE_DEVOPS_PAT }} --store-password-in-clear-text --configfile sdk/cs/NuGet.config
+
- name: Restore dependencies
run: |
# dotnet restore sdk/cs/src/Microsoft.AI.Foundry.Local.csproj /p:UseWinML=${{ inputs.useWinML }} /p:FoundryLocalCoreVersion="*-*" --configfile sdk/cs/NuGet.config
diff --git a/.github/workflows/build-js-steps.yml b/.github/workflows/build-js-steps.yml
index a806933c..d7a568a3 100644
--- a/.github/workflows/build-js-steps.yml
+++ b/.github/workflows/build-js-steps.yml
@@ -84,6 +84,13 @@ jobs:
Write-Host "`nDirectory contents:"
Get-ChildItem -Recurse -Depth 2 | ForEach-Object { Write-Host " $($_.FullName)" }
+ # The .npmrc points to an Azure Artifacts feed for CFS compliance.
+ # Remove it in CI so npm uses the public registry directly.
+ - name: Remove .npmrc (use public registry)
+ shell: pwsh
+ working-directory: sdk/js
+ run: |
+ if (Test-Path .npmrc) { Remove-Item .npmrc -Force; Write-Host "Removed .npmrc" }
- name: npm install (WinML)
if: ${{ inputs.useWinML == true }}
@@ -95,11 +102,6 @@ jobs:
working-directory: sdk/js
run: npm install
- # Verify that installing new packages doesn't strip custom native binary folders
- - name: npm install openai (verify persistence)
- working-directory: sdk/js
- run: npm install openai
-
- name: Set package version
working-directory: sdk/js
run: npm version ${{ env.ProjectVersion }} --no-git-tag-version --allow-same-version
diff --git a/.github/workflows/build-rust-steps.yml b/.github/workflows/build-rust-steps.yml
index 7649acaa..27c22da8 100644
--- a/.github/workflows/build-rust-steps.yml
+++ b/.github/workflows/build-rust-steps.yml
@@ -46,6 +46,18 @@ jobs:
with:
workspaces: sdk/rust -> target
+ # The .cargo/config.toml redirects crates-io to an Azure Artifacts feed
+ # for CFS compliance. Remove the redirect in CI so cargo can fetch from
+ # crates.io directly without Azure DevOps auth.
+ - name: Use crates.io directly
+ shell: pwsh
+ working-directory: sdk/rust
+ run: |
+ if (Test-Path .cargo/config.toml) {
+ Remove-Item .cargo/config.toml
+ Write-Host "Removed .cargo/config.toml crates-io redirect"
+ }
+
- name: Checkout test-data-shared from Azure DevOps
if: ${{ inputs.run-integration-tests }}
shell: pwsh
diff --git a/samples/cs/GettingStarted/nuget.config b/samples/cs/GettingStarted/nuget.config
index 5cf1e78e..b5c4e511 100644
--- a/samples/cs/GettingStarted/nuget.config
+++ b/samples/cs/GettingStarted/nuget.config
@@ -2,15 +2,6 @@
-
-
+
-
-
-
-
-
-
-
-
\ No newline at end of file
diff --git a/samples/js/chat-and-audio-foundry-local/.npmrc b/samples/js/chat-and-audio-foundry-local/.npmrc
new file mode 100644
index 00000000..114ea2a4
--- /dev/null
+++ b/samples/js/chat-and-audio-foundry-local/.npmrc
@@ -0,0 +1,2 @@
+registry=https://pkgs.dev.azure.com/aiinfra/PublicPackages/_packaging/ORT-Nightly/npm/registry/
+always-auth=true
diff --git a/samples/js/copilot-sdk-foundry-local/.npmrc b/samples/js/copilot-sdk-foundry-local/.npmrc
new file mode 100644
index 00000000..114ea2a4
--- /dev/null
+++ b/samples/js/copilot-sdk-foundry-local/.npmrc
@@ -0,0 +1,2 @@
+registry=https://pkgs.dev.azure.com/aiinfra/PublicPackages/_packaging/ORT-Nightly/npm/registry/
+always-auth=true
diff --git a/samples/js/electron-chat-application/.npmrc b/samples/js/electron-chat-application/.npmrc
new file mode 100644
index 00000000..114ea2a4
--- /dev/null
+++ b/samples/js/electron-chat-application/.npmrc
@@ -0,0 +1,2 @@
+registry=https://pkgs.dev.azure.com/aiinfra/PublicPackages/_packaging/ORT-Nightly/npm/registry/
+always-auth=true
diff --git a/samples/js/tool-calling-foundry-local/.npmrc b/samples/js/tool-calling-foundry-local/.npmrc
new file mode 100644
index 00000000..114ea2a4
--- /dev/null
+++ b/samples/js/tool-calling-foundry-local/.npmrc
@@ -0,0 +1,2 @@
+registry=https://pkgs.dev.azure.com/aiinfra/PublicPackages/_packaging/ORT-Nightly/npm/registry/
+always-auth=true
diff --git a/samples/rust/.cargo/config.toml b/samples/rust/.cargo/config.toml
new file mode 100644
index 00000000..84c57445
--- /dev/null
+++ b/samples/rust/.cargo/config.toml
@@ -0,0 +1,7 @@
+[registries]
+
+[source.crates-io]
+replace-with = "ORT-Nightly"
+
+[source.ORT-Nightly]
+registry = "sparse+https://pkgs.dev.azure.com/aiinfra/PublicPackages/_packaging/ORT-Nightly/Cargo/index/"
diff --git a/sdk/cs/NuGet.config b/sdk/cs/NuGet.config
index 294478a7..420497e9 100644
--- a/sdk/cs/NuGet.config
+++ b/sdk/cs/NuGet.config
@@ -2,7 +2,6 @@
-
diff --git a/sdk/cs/test/FoundryLocal.Tests/Microsoft.AI.Foundry.Local.Tests.csproj b/sdk/cs/test/FoundryLocal.Tests/Microsoft.AI.Foundry.Local.Tests.csproj
index b0bd3cd0..5f0c7cf2 100644
--- a/sdk/cs/test/FoundryLocal.Tests/Microsoft.AI.Foundry.Local.Tests.csproj
+++ b/sdk/cs/test/FoundryLocal.Tests/Microsoft.AI.Foundry.Local.Tests.csproj
@@ -1,7 +1,7 @@
- net9.0
+ net10.0
enable
enable
false
@@ -19,7 +19,7 @@
- net9.0-windows10.0.26100.0
+ net10.0-windows10.0.26100.0
10.0.17763.0
None
true
diff --git a/sdk/js/.npmrc b/sdk/js/.npmrc
new file mode 100644
index 00000000..114ea2a4
--- /dev/null
+++ b/sdk/js/.npmrc
@@ -0,0 +1,2 @@
+registry=https://pkgs.dev.azure.com/aiinfra/PublicPackages/_packaging/ORT-Nightly/npm/registry/
+always-auth=true
diff --git a/sdk/js/package.json b/sdk/js/package.json
index bdfadf5e..46ae6ce5 100644
--- a/sdk/js/package.json
+++ b/sdk/js/package.json
@@ -1,6 +1,6 @@
{
- "name": "@prathikrao/foundry-local-sdk",
- "version": "0.0.3",
+ "name": "foundry-local-sdk",
+ "version": "0.9.0",
"description": "Foundry Local JavaScript SDK",
"main": "dist/index.js",
"types": "dist/index.d.ts",
diff --git a/sdk/js/script/install.cjs b/sdk/js/script/install.cjs
index 3db771b8..cdf5531d 100644
--- a/sdk/js/script/install.cjs
+++ b/sdk/js/script/install.cjs
@@ -40,6 +40,7 @@ const REQUIRED_FILES = [
// Instead, it sets an environment variable named npm_config_winml to 'true'.
const useWinML = process.env.npm_config_winml === 'true';
const useNightly = process.env.npm_config_nightly === 'true';
+const noDeps = process.env.npm_config_nodeps === 'true';
console.log(`[foundry-local] WinML enabled: ${useWinML}`);
console.log(`[foundry-local] Nightly enabled: ${useNightly}`);
@@ -120,7 +121,10 @@ const LINUX_ARTIFACTS = [
];
let ARTIFACTS = [];
-if (useWinML) {
+if (noDeps) {
+ console.log(`[foundry-local] Skipping dependencies install...`);
+ ARTIFACTS = [];
+} else if (useWinML) {
console.log(`[foundry-local] Using WinML artifacts...`);
ARTIFACTS = WINML_ARTIFACTS;
} else if (os.platform() === 'linux') {
diff --git a/sdk/rust/.cargo/config.toml b/sdk/rust/.cargo/config.toml
new file mode 100644
index 00000000..84c57445
--- /dev/null
+++ b/sdk/rust/.cargo/config.toml
@@ -0,0 +1,7 @@
+[registries]
+
+[source.crates-io]
+replace-with = "ORT-Nightly"
+
+[source.ORT-Nightly]
+registry = "sparse+https://pkgs.dev.azure.com/aiinfra/PublicPackages/_packaging/ORT-Nightly/Cargo/index/"
diff --git a/sdk_legacy/cs/NuGet.config b/sdk_legacy/cs/NuGet.config
new file mode 100644
index 00000000..420497e9
--- /dev/null
+++ b/sdk_legacy/cs/NuGet.config
@@ -0,0 +1,7 @@
+
+
+
+
+
+
+
diff --git a/sdk_legacy/js/.npmrc b/sdk_legacy/js/.npmrc
new file mode 100644
index 00000000..114ea2a4
--- /dev/null
+++ b/sdk_legacy/js/.npmrc
@@ -0,0 +1,2 @@
+registry=https://pkgs.dev.azure.com/aiinfra/PublicPackages/_packaging/ORT-Nightly/npm/registry/
+always-auth=true
diff --git a/sdk_legacy/rust/.cargo/config.toml b/sdk_legacy/rust/.cargo/config.toml
new file mode 100644
index 00000000..84c57445
--- /dev/null
+++ b/sdk_legacy/rust/.cargo/config.toml
@@ -0,0 +1,7 @@
+[registries]
+
+[source.crates-io]
+replace-with = "ORT-Nightly"
+
+[source.ORT-Nightly]
+registry = "sparse+https://pkgs.dev.azure.com/aiinfra/PublicPackages/_packaging/ORT-Nightly/Cargo/index/"
diff --git a/www/.npmrc b/www/.npmrc
index b6f27f13..06fe7275 100644
--- a/www/.npmrc
+++ b/www/.npmrc
@@ -1 +1,3 @@
+registry=https://pkgs.dev.azure.com/aiinfra/PublicPackages/_packaging/ORT-Nightly/npm/registry/
+always-auth=true
engine-strict=true
From 021b632931b7bf2bb6ae26200dc6de8905aeb529 Mon Sep 17 00:00:00 2001
From: Prathik Rao
Date: Wed, 25 Mar 2026 14:20:08 -0700
Subject: [PATCH 02/21] init dummy ADO packaging pipeline for FLC & SDK (#553)
Co-authored-by: Prathik Rao
---
.pipelines/foundry-local-packaging.yml | 9 +++++++++
1 file changed, 9 insertions(+)
create mode 100644 .pipelines/foundry-local-packaging.yml
diff --git a/.pipelines/foundry-local-packaging.yml b/.pipelines/foundry-local-packaging.yml
new file mode 100644
index 00000000..b87eb70e
--- /dev/null
+++ b/.pipelines/foundry-local-packaging.yml
@@ -0,0 +1,9 @@
+# Foundry Local SDK Packaging Pipeline (placeholder)
+trigger: none
+
+pool:
+ vmImage: 'windows-latest'
+
+steps:
+- script: echo "Foundry Local packaging pipeline - placeholder"
+ displayName: 'Placeholder'
\ No newline at end of file
From 31d31d3f5217b2ba5abe138db95650bf8556b454 Mon Sep 17 00:00:00 2001
From: Copilot <198982749+Copilot@users.noreply.github.com>
Date: Thu, 26 Mar 2026 09:40:54 -0700
Subject: [PATCH 03/21] Convert JS SDK streaming APIs from callbacks to async
iterables (#545)
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit
- [x] Convert JS SDK streaming APIs from callbacks to async iterables
- [x] Add `return()` hook to async iterators to prevent unbounded
buffering on early break
- [x] Add guards in streaming callbacks to skip work after error or
cancellation
- [x] Fix test assertions to assert synchronous throws directly
- [x] Replace O(n) `chunks.shift()` with O(1) head-index dequeue with
compaction
- [x] Guard against concurrent `next()` calls with `nextInFlight` flag
- [x] Add comment explaining native stream cancellation limitation in
`return()`
- [x] Fix docs example for `completeStreamingChat(messages, tools)`
overload to pass `tools`
- [x] Regenerate TypeDoc API docs
- [x] Type-check, code review, and security scan
- [x] Add comments explaining why local variable captures are needed
(closures lose `this`)
- [x] Add comments clarifying promise-resolve wake-up pattern in
`.then()` handler
- [x] Add structural comments explaining the AsyncIterable/AsyncIterator
factory pattern
- [x] Apply same readability improvements to chatClient.ts
---
⚡ Quickly spin up Copilot coding agent tasks from anywhere on your macOS
or Windows machine with [Raycast](https://gh.io/cca-raycast-docs).
---------
Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com>
Co-authored-by: baijumeswani <12852605+baijumeswani@users.noreply.github.com>
---
README.md | 4 +-
samples/js/audio-transcription-example/app.js | 6 +-
.../chat-and-audio-foundry-local/src/app.js | 27 ++-
samples/js/native-chat-completions/app.js | 15 +-
sdk/js/README.md | 34 ++-
sdk/js/docs/README.md | 178 ++++++++--------
sdk/js/docs/classes/AudioClient.md | 19 +-
sdk/js/docs/classes/AudioClientSettings.md | 4 +-
sdk/js/docs/classes/ChatClient.md | 59 ++++--
sdk/js/docs/classes/ChatClientSettings.md | 20 +-
sdk/js/docs/classes/Model.md | 2 +-
.../docs/classes/ResponsesClientSettings.md | 28 +--
sdk/js/examples/audio-transcription.ts | 4 +-
sdk/js/examples/chat-completion.ts | 15 +-
sdk/js/examples/tool-calling.ts | 40 ++--
sdk/js/src/openai/audioClient.ts | 179 +++++++++++-----
sdk/js/src/openai/chatClient.ts | 194 +++++++++++++-----
sdk/js/test/openai/audioClient.test.ts | 27 +--
sdk/js/test/openai/chatClient.test.ts | 36 +---
19 files changed, 528 insertions(+), 363 deletions(-)
diff --git a/README.md b/README.md
index 14c53229..07bc9b4d 100644
--- a/README.md
+++ b/README.md
@@ -232,9 +232,9 @@ const result = await audioClient.transcribe('recording.wav');
console.log('Transcription:', result.text);
// Or stream in real-time
-await audioClient.transcribeStreaming('recording.wav', (chunk) => {
+for await (const chunk of audioClient.transcribeStreaming('recording.wav')) {
process.stdout.write(chunk.text);
-});
+}
await whisperModel.unload();
```
diff --git a/samples/js/audio-transcription-example/app.js b/samples/js/audio-transcription-example/app.js
index fe441d1b..78efc8af 100644
--- a/samples/js/audio-transcription-example/app.js
+++ b/samples/js/audio-transcription-example/app.js
@@ -39,12 +39,12 @@ console.log('\nAudio transcription result:');
console.log(transcription.text);
console.log('✓ Audio transcription completed');
-// Same example but with streaming transcription using callback
+// Same example but with streaming transcription using async iteration
console.log('\nTesting streaming audio transcription...');
-await audioClient.transcribeStreaming('./Recording.mp3', (result) => {
+for await (const result of audioClient.transcribeStreaming('./Recording.mp3')) {
// Output the intermediate transcription results as they are received without line ending
process.stdout.write(result.text);
-});
+}
console.log('\n✓ Streaming transcription completed');
// Unload the model
diff --git a/samples/js/chat-and-audio-foundry-local/src/app.js b/samples/js/chat-and-audio-foundry-local/src/app.js
index b3084816..49ce199c 100644
--- a/samples/js/chat-and-audio-foundry-local/src/app.js
+++ b/samples/js/chat-and-audio-foundry-local/src/app.js
@@ -76,22 +76,19 @@ async function main() {
// Summarize the transcription
console.log("Generating summary...\n");
- await chatClient.completeStreamingChat(
- [
- {
- role: "system",
- content:
- "You are a helpful assistant. Summarize the following transcribed audio and extract key themes and action items.",
- },
- { role: "user", content: transcription.text },
- ],
- (chunk) => {
- const content = chunk.choices?.[0]?.message?.content;
- if (content) {
- process.stdout.write(content);
- }
+ for await (const chunk of chatClient.completeStreamingChat([
+ {
+ role: "system",
+ content:
+ "You are a helpful assistant. Summarize the following transcribed audio and extract key themes and action items.",
+ },
+ { role: "user", content: transcription.text },
+ ])) {
+ const content = chunk.choices?.[0]?.message?.content;
+ if (content) {
+ process.stdout.write(content);
}
- );
+ }
console.log("\n");
// --- Clean up ---
diff --git a/samples/js/native-chat-completions/app.js b/samples/js/native-chat-completions/app.js
index af566ef7..67348e8c 100644
--- a/samples/js/native-chat-completions/app.js
+++ b/samples/js/native-chat-completions/app.js
@@ -41,15 +41,14 @@ console.log(completion.choices[0]?.message?.content);
// Example streaming completion
console.log('\nTesting streaming completion...');
-await chatClient.completeStreamingChat(
- [{ role: 'user', content: 'Write a short poem about programming.' }],
- (chunk) => {
- const content = chunk.choices?.[0]?.message?.content;
- if (content) {
- process.stdout.write(content);
- }
+for await (const chunk of chatClient.completeStreamingChat(
+ [{ role: 'user', content: 'Write a short poem about programming.' }]
+)) {
+ const content = chunk.choices?.[0]?.message?.content;
+ if (content) {
+ process.stdout.write(content);
}
-);
+}
console.log('\n');
// Unload the model
diff --git a/sdk/js/README.md b/sdk/js/README.md
index 3308c9d8..9b08f9ac 100644
--- a/sdk/js/README.md
+++ b/sdk/js/README.md
@@ -69,15 +69,14 @@ console.log(completion.choices[0]?.message?.content);
// Example streaming completion
console.log('\nTesting streaming completion...');
-await chatClient.completeStreamingChat(
- [{ role: 'user', content: 'Write a short poem about programming.' }],
- (chunk) => {
- const content = chunk.choices?.[0]?.message?.content;
- if (content) {
- process.stdout.write(content);
- }
+for await (const chunk of chatClient.completeStreamingChat(
+ [{ role: 'user', content: 'Write a short poem about programming.' }]
+)) {
+ const content = chunk.choices?.[0]?.message?.content;
+ if (content) {
+ process.stdout.write(content);
}
-);
+}
console.log('\n');
// Unload the model
@@ -157,15 +156,14 @@ console.log(response.choices[0].message.content);
For real-time output, use streaming:
```typescript
-await chatClient.completeStreamingChat(
- [{ role: 'user', content: 'Write a short poem about programming.' }],
- (chunk) => {
- const content = chunk.choices?.[0]?.message?.content;
- if (content) {
- process.stdout.write(content);
- }
+for await (const chunk of chatClient.completeStreamingChat(
+ [{ role: 'user', content: 'Write a short poem about programming.' }]
+)) {
+ const content = chunk.choices?.[0]?.message?.content;
+ if (content) {
+ process.stdout.write(content);
}
-);
+}
```
### Audio Transcription
@@ -180,9 +178,9 @@ audioClient.settings.language = 'en';
const result = await audioClient.transcribe('/path/to/audio.wav');
// Streaming transcription
-await audioClient.transcribeStreaming('/path/to/audio.wav', (chunk) => {
+for await (const chunk of audioClient.transcribeStreaming('/path/to/audio.wav')) {
console.log(chunk);
-});
+}
```
### Embedded Web Service
diff --git a/sdk/js/docs/README.md b/sdk/js/docs/README.md
index e79be84d..58218628 100644
--- a/sdk/js/docs/README.md
+++ b/sdk/js/docs/README.md
@@ -163,7 +163,7 @@ Use a plain object with these properties to configure the SDK.
##### additionalSettings?
```ts
-optional additionalSettings: {
+optional additionalSettings?: {
[key: string]: string;
};
```
@@ -180,7 +180,7 @@ Optional. Internal use only.
##### appDataDir?
```ts
-optional appDataDir: string;
+optional appDataDir?: string;
```
The directory where application data should be stored.
@@ -198,7 +198,7 @@ Used for identifying the application in logs and telemetry.
##### libraryPath?
```ts
-optional libraryPath: string;
+optional libraryPath?: string;
```
The path to the directory containing the native Foundry Local Core libraries.
@@ -208,7 +208,7 @@ If not provided, the SDK attempts to discover them in standard locations.
##### logLevel?
```ts
-optional logLevel: "trace" | "debug" | "info" | "warn" | "error" | "fatal";
+optional logLevel?: "trace" | "debug" | "info" | "warn" | "error" | "fatal";
```
The logging level for the SDK.
@@ -218,7 +218,7 @@ Defaults to 'warn'.
##### logsDir?
```ts
-optional logsDir: string;
+optional logsDir?: string;
```
The directory where log files are written.
@@ -227,7 +227,7 @@ Optional. Defaults to `{appDataDir}/logs`.
##### modelCacheDir?
```ts
-optional modelCacheDir: string;
+optional modelCacheDir?: string;
```
The directory where models are downloaded and cached.
@@ -236,7 +236,7 @@ Optional. Defaults to `{appDataDir}/cache/models`.
##### serviceEndpoint?
```ts
-optional serviceEndpoint: string;
+optional serviceEndpoint?: string;
```
The external URL if the web service is running in a separate process.
@@ -245,7 +245,7 @@ Optional. This is used to connect to an existing service instance.
##### webServiceUrls?
```ts
-optional webServiceUrls: string;
+optional webServiceUrls?: string;
```
The URL(s) for the local web service to bind to.
@@ -351,7 +351,7 @@ call_id: string;
##### id?
```ts
-optional id: string;
+optional id?: string;
```
##### name
@@ -363,7 +363,7 @@ name: string;
##### status?
```ts
-optional status: ResponseItemStatus;
+optional status?: ResponseItemStatus;
```
##### type
@@ -387,7 +387,7 @@ call_id: string;
##### id?
```ts
-optional id: string;
+optional id?: string;
```
##### output
@@ -399,7 +399,7 @@ output: string | ContentPart[];
##### status?
```ts
-optional status: ResponseItemStatus;
+optional status?: ResponseItemStatus;
```
##### type
@@ -417,7 +417,7 @@ type: "function_call_output";
##### description?
```ts
-optional description: string;
+optional description?: string;
```
##### name
@@ -429,13 +429,13 @@ name: string;
##### parameters?
```ts
-optional parameters: Record;
+optional parameters?: Record;
```
##### strict?
```ts
-optional strict: boolean;
+optional strict?: boolean;
```
##### type
@@ -671,7 +671,7 @@ type: "item_reference";
##### bytes?
```ts
-optional bytes: number[];
+optional bytes?: number[];
```
##### logprob
@@ -701,7 +701,7 @@ content: string | ContentPart[];
##### id?
```ts
-optional id: string;
+optional id?: string;
```
##### role
@@ -713,7 +713,7 @@ role: MessageRole;
##### status?
```ts
-optional status: ResponseItemStatus;
+optional status?: ResponseItemStatus;
```
##### type
@@ -749,13 +749,13 @@ createdAtUnix: number;
##### displayName?
```ts
-optional displayName: string | null;
+optional displayName?: string | null;
```
##### fileSizeMb?
```ts
-optional fileSizeMb: number | null;
+optional fileSizeMb?: number | null;
```
##### id
@@ -767,31 +767,31 @@ id: string;
##### license?
```ts
-optional license: string | null;
+optional license?: string | null;
```
##### licenseDescription?
```ts
-optional licenseDescription: string | null;
+optional licenseDescription?: string | null;
```
##### maxOutputTokens?
```ts
-optional maxOutputTokens: number | null;
+optional maxOutputTokens?: number | null;
```
##### minFLVersion?
```ts
-optional minFLVersion: string | null;
+optional minFLVersion?: string | null;
```
##### modelSettings?
```ts
-optional modelSettings: ModelSettings | null;
+optional modelSettings?: ModelSettings | null;
```
##### modelType
@@ -809,7 +809,7 @@ name: string;
##### promptTemplate?
```ts
-optional promptTemplate: PromptTemplate | null;
+optional promptTemplate?: PromptTemplate | null;
```
##### providerType
@@ -821,25 +821,25 @@ providerType: string;
##### publisher?
```ts
-optional publisher: string | null;
+optional publisher?: string | null;
```
##### runtime?
```ts
-optional runtime: Runtime | null;
+optional runtime?: Runtime | null;
```
##### supportsToolCalling?
```ts
-optional supportsToolCalling: boolean | null;
+optional supportsToolCalling?: boolean | null;
```
##### task?
```ts
-optional task: string | null;
+optional task?: string | null;
```
##### uri
@@ -863,7 +863,7 @@ version: number;
##### parameters?
```ts
-optional parameters: Parameter[] | null;
+optional parameters?: Parameter[] | null;
```
***
@@ -947,13 +947,13 @@ type: "response.output_item.done";
##### annotations?
```ts
-optional annotations: Annotation[];
+optional annotations?: Annotation[];
```
##### logprobs?
```ts
-optional logprobs: LogProb[];
+optional logprobs?: LogProb[];
```
##### text
@@ -1067,7 +1067,7 @@ name: string;
##### value?
```ts
-optional value: string | null;
+optional value?: string | null;
```
***
@@ -1091,13 +1091,13 @@ prompt: string;
##### system?
```ts
-optional system: string | null;
+optional system?: string | null;
```
##### user?
```ts
-optional user: string | null;
+optional user?: string | null;
```
***
@@ -1109,13 +1109,13 @@ optional user: string | null;
##### effort?
```ts
-optional effort: string;
+optional effort?: string;
```
##### summary?
```ts
-optional summary: string;
+optional summary?: string;
```
***
@@ -1127,31 +1127,31 @@ optional summary: string;
##### content?
```ts
-optional content: ContentPart[];
+optional content?: ContentPart[];
```
##### encrypted\_content?
```ts
-optional encrypted_content: string;
+optional encrypted_content?: string;
```
##### id?
```ts
-optional id: string;
+optional id?: string;
```
##### status?
```ts
-optional status: ResponseItemStatus;
+optional status?: ResponseItemStatus;
```
##### summary?
```ts
-optional summary: string;
+optional summary?: string;
```
##### type
@@ -1259,121 +1259,121 @@ type: "response.refusal.done";
##### frequency\_penalty?
```ts
-optional frequency_penalty: number;
+optional frequency_penalty?: number;
```
##### input?
```ts
-optional input: string | ResponseInputItem[];
+optional input?: string | ResponseInputItem[];
```
##### instructions?
```ts
-optional instructions: string;
+optional instructions?: string;
```
##### max\_output\_tokens?
```ts
-optional max_output_tokens: number;
+optional max_output_tokens?: number;
```
##### metadata?
```ts
-optional metadata: Record;
+optional metadata?: Record;
```
##### model?
```ts
-optional model: string;
+optional model?: string;
```
##### parallel\_tool\_calls?
```ts
-optional parallel_tool_calls: boolean;
+optional parallel_tool_calls?: boolean;
```
##### presence\_penalty?
```ts
-optional presence_penalty: number;
+optional presence_penalty?: number;
```
##### previous\_response\_id?
```ts
-optional previous_response_id: string;
+optional previous_response_id?: string;
```
##### reasoning?
```ts
-optional reasoning: ReasoningConfig;
+optional reasoning?: ReasoningConfig;
```
##### seed?
```ts
-optional seed: number;
+optional seed?: number;
```
##### store?
```ts
-optional store: boolean;
+optional store?: boolean;
```
##### stream?
```ts
-optional stream: boolean;
+optional stream?: boolean;
```
##### temperature?
```ts
-optional temperature: number;
+optional temperature?: number;
```
##### text?
```ts
-optional text: TextConfig;
+optional text?: TextConfig;
```
##### tool\_choice?
```ts
-optional tool_choice: ResponseToolChoice;
+optional tool_choice?: ResponseToolChoice;
```
##### tools?
```ts
-optional tools: FunctionToolDefinition[];
+optional tools?: FunctionToolDefinition[];
```
##### top\_p?
```ts
-optional top_p: number;
+optional top_p?: number;
```
##### truncation?
```ts
-optional truncation: TruncationStrategy;
+optional truncation?: TruncationStrategy;
```
##### user?
```ts
-optional user: string;
+optional user?: string;
```
***
@@ -1403,13 +1403,13 @@ message: string;
##### jsonSchema?
```ts
-optional jsonSchema: string;
+optional jsonSchema?: string;
```
##### larkGrammar?
```ts
-optional larkGrammar: string;
+optional larkGrammar?: string;
```
##### type
@@ -1457,13 +1457,13 @@ type:
##### cancelled\_at?
```ts
-optional cancelled_at: number | null;
+optional cancelled_at?: number | null;
```
##### completed\_at?
```ts
-optional completed_at: number | null;
+optional completed_at?: number | null;
```
##### created\_at
@@ -1475,13 +1475,13 @@ created_at: number;
##### error?
```ts
-optional error: ResponseError | null;
+optional error?: ResponseError | null;
```
##### failed\_at?
```ts
-optional failed_at: number | null;
+optional failed_at?: number | null;
```
##### frequency\_penalty
@@ -1499,25 +1499,25 @@ id: string;
##### incomplete\_details?
```ts
-optional incomplete_details: IncompleteDetails | null;
+optional incomplete_details?: IncompleteDetails | null;
```
##### instructions?
```ts
-optional instructions: string | null;
+optional instructions?: string | null;
```
##### max\_output\_tokens?
```ts
-optional max_output_tokens: number | null;
+optional max_output_tokens?: number | null;
```
##### metadata?
```ts
-optional metadata: Record | null;
+optional metadata?: Record | null;
```
##### model
@@ -1553,13 +1553,13 @@ presence_penalty: number;
##### previous\_response\_id?
```ts
-optional previous_response_id: string | null;
+optional previous_response_id?: string | null;
```
##### reasoning?
```ts
-optional reasoning: ReasoningConfig | null;
+optional reasoning?: ReasoningConfig | null;
```
##### status
@@ -1613,13 +1613,13 @@ truncation: TruncationStrategy;
##### usage?
```ts
-optional usage: ResponseUsage | null;
+optional usage?: ResponseUsage | null;
```
##### user?
```ts
-optional user: string | null;
+optional user?: string | null;
```
***
@@ -1655,7 +1655,7 @@ input_tokens: number;
##### input\_tokens\_details?
```ts
-optional input_tokens_details: {
+optional input_tokens_details?: {
cached_tokens: number;
};
```
@@ -1675,7 +1675,7 @@ output_tokens: number;
##### output\_tokens\_details?
```ts
-optional output_tokens_details: {
+optional output_tokens_details?: {
reasoning_tokens: number;
};
```
@@ -1719,19 +1719,19 @@ executionProvider: string;
##### code?
```ts
-optional code: string;
+optional code?: string;
```
##### message?
```ts
-optional message: string;
+optional message?: string;
```
##### param?
```ts
-optional param: string;
+optional param?: string;
```
##### sequence\_number
@@ -1755,13 +1755,13 @@ type: "error";
##### format?
```ts
-optional format: TextFormat;
+optional format?: TextFormat;
```
##### verbosity?
```ts
-optional verbosity: string;
+optional verbosity?: string;
```
***
@@ -1773,25 +1773,25 @@ optional verbosity: string;
##### description?
```ts
-optional description: string;
+optional description?: string;
```
##### name?
```ts
-optional name: string;
+optional name?: string;
```
##### schema?
```ts
-optional schema: unknown;
+optional schema?: unknown;
```
##### strict?
```ts
-optional strict: boolean;
+optional strict?: boolean;
```
##### type
@@ -1809,7 +1809,7 @@ type: string;
##### name?
```ts
-optional name: string;
+optional name?: string;
```
##### type
diff --git a/sdk/js/docs/classes/AudioClient.md b/sdk/js/docs/classes/AudioClient.md
index 7fd13bd8..12e79de5 100644
--- a/sdk/js/docs/classes/AudioClient.md
+++ b/sdk/js/docs/classes/AudioClient.md
@@ -46,24 +46,31 @@ Error - If audioFilePath is invalid or transcription fails.
### transcribeStreaming()
```ts
-transcribeStreaming(audioFilePath, callback): Promise;
+transcribeStreaming(audioFilePath): AsyncIterable;
```
-Transcribes audio into the input language using streaming.
+Transcribes audio into the input language using streaming, returning an async iterable of chunks.
#### Parameters
| Parameter | Type | Description |
| ------ | ------ | ------ |
| `audioFilePath` | `string` | Path to the audio file to transcribe. |
-| `callback` | (`chunk`) => `void` | A callback function that receives each chunk of the streaming response. |
#### Returns
-`Promise`\<`void`\>
+`AsyncIterable`\<`any`\>
-A promise that resolves when the stream is complete.
+An async iterable that yields parsed streaming transcription chunks.
#### Throws
-Error - If audioFilePath or callback are invalid, or streaming fails.
+Error - If audioFilePath is invalid, or streaming fails.
+
+#### Example
+
+```typescript
+for await (const chunk of audioClient.transcribeStreaming('recording.wav')) {
+ process.stdout.write(chunk.text);
+}
+```
diff --git a/sdk/js/docs/classes/AudioClientSettings.md b/sdk/js/docs/classes/AudioClientSettings.md
index 619c526b..dae7cbbe 100644
--- a/sdk/js/docs/classes/AudioClientSettings.md
+++ b/sdk/js/docs/classes/AudioClientSettings.md
@@ -19,7 +19,7 @@ new AudioClientSettings(): AudioClientSettings;
### language?
```ts
-optional language: string;
+optional language?: string;
```
***
@@ -27,5 +27,5 @@ optional language: string;
### temperature?
```ts
-optional temperature: number;
+optional temperature?: number;
```
diff --git a/sdk/js/docs/classes/ChatClient.md b/sdk/js/docs/classes/ChatClient.md
index 91e877aa..c3120f0b 100644
--- a/sdk/js/docs/classes/ChatClient.md
+++ b/sdk/js/docs/classes/ChatClient.md
@@ -75,53 +75,80 @@ Error - If messages or tools are invalid or completion fails.
#### Call Signature
```ts
-completeStreamingChat(messages, callback): Promise;
+completeStreamingChat(messages): AsyncIterable;
```
-Performs a streaming chat completion.
+Performs a streaming chat completion, returning an async iterable of chunks.
##### Parameters
| Parameter | Type | Description |
| ------ | ------ | ------ |
| `messages` | `any`[] | An array of message objects. |
-| `callback` | (`chunk`) => `void` | A callback function that receives each chunk of the streaming response. |
##### Returns
-`Promise`\<`void`\>
+`AsyncIterable`\<`any`\>
-A promise that resolves when the stream is complete.
+An async iterable that yields parsed streaming response chunks.
##### Throws
-Error - If messages, tools, or callback are invalid, or streaming fails.
+Error - If messages or tools are invalid, or streaming fails.
+
+##### Example
+
+```typescript
+// Without tools:
+for await (const chunk of chatClient.completeStreamingChat(messages)) {
+ const content = chunk.choices?.[0]?.delta?.content;
+ if (content) process.stdout.write(content);
+}
+
+// With tools:
+for await (const chunk of chatClient.completeStreamingChat(messages, tools)) {
+ const content = chunk.choices?.[0]?.delta?.content;
+ if (content) process.stdout.write(content);
+}
+```
#### Call Signature
```ts
-completeStreamingChat(
- messages,
- tools,
-callback): Promise;
+completeStreamingChat(messages, tools): AsyncIterable;
```
-Performs a streaming chat completion.
+Performs a streaming chat completion, returning an async iterable of chunks.
##### Parameters
| Parameter | Type | Description |
| ------ | ------ | ------ |
| `messages` | `any`[] | An array of message objects. |
-| `tools` | `any`[] | An array of tool objects. |
-| `callback` | (`chunk`) => `void` | A callback function that receives each chunk of the streaming response. |
+| `tools` | `any`[] | An optional array of tool objects. |
##### Returns
-`Promise`\<`void`\>
+`AsyncIterable`\<`any`\>
-A promise that resolves when the stream is complete.
+An async iterable that yields parsed streaming response chunks.
##### Throws
-Error - If messages, tools, or callback are invalid, or streaming fails.
+Error - If messages or tools are invalid, or streaming fails.
+
+##### Example
+
+```typescript
+// Without tools:
+for await (const chunk of chatClient.completeStreamingChat(messages)) {
+ const content = chunk.choices?.[0]?.delta?.content;
+ if (content) process.stdout.write(content);
+}
+
+// With tools:
+for await (const chunk of chatClient.completeStreamingChat(messages, tools)) {
+ const content = chunk.choices?.[0]?.delta?.content;
+ if (content) process.stdout.write(content);
+}
+```
diff --git a/sdk/js/docs/classes/ChatClientSettings.md b/sdk/js/docs/classes/ChatClientSettings.md
index 7fed8a46..7d48bcca 100644
--- a/sdk/js/docs/classes/ChatClientSettings.md
+++ b/sdk/js/docs/classes/ChatClientSettings.md
@@ -19,7 +19,7 @@ new ChatClientSettings(): ChatClientSettings;
### frequencyPenalty?
```ts
-optional frequencyPenalty: number;
+optional frequencyPenalty?: number;
```
***
@@ -27,7 +27,7 @@ optional frequencyPenalty: number;
### maxTokens?
```ts
-optional maxTokens: number;
+optional maxTokens?: number;
```
***
@@ -35,7 +35,7 @@ optional maxTokens: number;
### n?
```ts
-optional n: number;
+optional n?: number;
```
***
@@ -43,7 +43,7 @@ optional n: number;
### presencePenalty?
```ts
-optional presencePenalty: number;
+optional presencePenalty?: number;
```
***
@@ -51,7 +51,7 @@ optional presencePenalty: number;
### randomSeed?
```ts
-optional randomSeed: number;
+optional randomSeed?: number;
```
***
@@ -59,7 +59,7 @@ optional randomSeed: number;
### responseFormat?
```ts
-optional responseFormat: ResponseFormat;
+optional responseFormat?: ResponseFormat;
```
***
@@ -67,7 +67,7 @@ optional responseFormat: ResponseFormat;
### temperature?
```ts
-optional temperature: number;
+optional temperature?: number;
```
***
@@ -75,7 +75,7 @@ optional temperature: number;
### toolChoice?
```ts
-optional toolChoice: ToolChoice;
+optional toolChoice?: ToolChoice;
```
***
@@ -83,7 +83,7 @@ optional toolChoice: ToolChoice;
### topK?
```ts
-optional topK: number;
+optional topK?: number;
```
***
@@ -91,5 +91,5 @@ optional topK: number;
### topP?
```ts
-optional topP: number;
+optional topP?: number;
```
diff --git a/sdk/js/docs/classes/Model.md b/sdk/js/docs/classes/Model.md
index 48340dae..424d673b 100644
--- a/sdk/js/docs/classes/Model.md
+++ b/sdk/js/docs/classes/Model.md
@@ -156,7 +156,7 @@ Automatically selects the new variant if it is cached and the current one is not
#### Throws
-Error - If the variant's alias does not match the model's alias.
+Error - If the argument is not a ModelVariant object, or if the variant's alias does not match the model's alias.
***
diff --git a/sdk/js/docs/classes/ResponsesClientSettings.md b/sdk/js/docs/classes/ResponsesClientSettings.md
index 08b9ea94..8401faf1 100644
--- a/sdk/js/docs/classes/ResponsesClientSettings.md
+++ b/sdk/js/docs/classes/ResponsesClientSettings.md
@@ -22,7 +22,7 @@ new ResponsesClientSettings(): ResponsesClientSettings;
### frequencyPenalty?
```ts
-optional frequencyPenalty: number;
+optional frequencyPenalty?: number;
```
***
@@ -30,7 +30,7 @@ optional frequencyPenalty: number;
### instructions?
```ts
-optional instructions: string;
+optional instructions?: string;
```
System-level instructions to guide the model.
@@ -40,7 +40,7 @@ System-level instructions to guide the model.
### maxOutputTokens?
```ts
-optional maxOutputTokens: number;
+optional maxOutputTokens?: number;
```
***
@@ -48,7 +48,7 @@ optional maxOutputTokens: number;
### metadata?
```ts
-optional metadata: Record;
+optional metadata?: Record;
```
***
@@ -56,7 +56,7 @@ optional metadata: Record;
### parallelToolCalls?
```ts
-optional parallelToolCalls: boolean;
+optional parallelToolCalls?: boolean;
```
***
@@ -64,7 +64,7 @@ optional parallelToolCalls: boolean;
### presencePenalty?
```ts
-optional presencePenalty: number;
+optional presencePenalty?: number;
```
***
@@ -72,7 +72,7 @@ optional presencePenalty: number;
### reasoning?
```ts
-optional reasoning: ReasoningConfig;
+optional reasoning?: ReasoningConfig;
```
***
@@ -80,7 +80,7 @@ optional reasoning: ReasoningConfig;
### seed?
```ts
-optional seed: number;
+optional seed?: number;
```
***
@@ -88,7 +88,7 @@ optional seed: number;
### store?
```ts
-optional store: boolean;
+optional store?: boolean;
```
***
@@ -96,7 +96,7 @@ optional store: boolean;
### temperature?
```ts
-optional temperature: number;
+optional temperature?: number;
```
***
@@ -104,7 +104,7 @@ optional temperature: number;
### text?
```ts
-optional text: TextConfig;
+optional text?: TextConfig;
```
***
@@ -112,7 +112,7 @@ optional text: TextConfig;
### toolChoice?
```ts
-optional toolChoice: ResponseToolChoice;
+optional toolChoice?: ResponseToolChoice;
```
***
@@ -120,7 +120,7 @@ optional toolChoice: ResponseToolChoice;
### topP?
```ts
-optional topP: number;
+optional topP?: number;
```
***
@@ -128,5 +128,5 @@ optional topP: number;
### truncation?
```ts
-optional truncation: TruncationStrategy;
+optional truncation?: TruncationStrategy;
```
diff --git a/sdk/js/examples/audio-transcription.ts b/sdk/js/examples/audio-transcription.ts
index 7fddf2d8..4e4fc2d4 100644
--- a/sdk/js/examples/audio-transcription.ts
+++ b/sdk/js/examples/audio-transcription.ts
@@ -72,9 +72,9 @@ async function main() {
// Example: Streaming transcription
console.log('\nTesting streaming transcription...');
- await audioClient.transcribeStreaming(audioFilePath, (chunk: any) => {
+ for await (const chunk of audioClient.transcribeStreaming(audioFilePath)) {
process.stdout.write(chunk.text);
- });
+ }
console.log('\n');
// Unload the model
diff --git a/sdk/js/examples/chat-completion.ts b/sdk/js/examples/chat-completion.ts
index 2c283e23..a9e2d59a 100644
--- a/sdk/js/examples/chat-completion.ts
+++ b/sdk/js/examples/chat-completion.ts
@@ -70,15 +70,14 @@ async function main() {
// Example streaming completion
console.log('\nTesting streaming completion...');
- await chatClient.completeStreamingChat(
- [{ role: 'user', content: 'Write a short poem about programming.' }],
- (chunk) => {
- const content = chunk.choices?.[0]?.message?.content;
- if (content) {
- process.stdout.write(content);
- }
+ for await (const chunk of chatClient.completeStreamingChat(
+ [{ role: 'user', content: 'Write a short poem about programming.' }]
+ )) {
+ const content = chunk.choices?.[0]?.message?.content;
+ if (content) {
+ process.stdout.write(content);
}
- );
+ }
console.log('\n');
// Model management example
diff --git a/sdk/js/examples/tool-calling.ts b/sdk/js/examples/tool-calling.ts
index bb4ed541..c3640a8f 100644
--- a/sdk/js/examples/tool-calling.ts
+++ b/sdk/js/examples/tool-calling.ts
@@ -109,22 +109,18 @@ async function main() {
let toolCallData: any = null;
console.log('Chat completion response:');
- await chatClient.completeStreamingChat(
- messages,
- tools,
- (chunk: any) => {
- const content = chunk.choices?.[0]?.message?.content;
- if (content) {
- process.stdout.write(content);
- }
-
- // Capture tool call data
- const toolCalls = chunk.choices?.[0]?.message?.tool_calls;
- if (toolCalls && toolCalls.length > 0) {
- toolCallData = toolCalls[0];
- }
+ for await (const chunk of chatClient.completeStreamingChat(messages, tools)) {
+ const content = chunk.choices?.[0]?.message?.content;
+ if (content) {
+ process.stdout.write(content);
+ }
+
+ // Capture tool call data
+ const toolCalls = chunk.choices?.[0]?.message?.tool_calls;
+ if (toolCalls && toolCalls.length > 0) {
+ toolCallData = toolCalls[0];
}
- );
+ }
console.log('\n');
// Handle tool invocation
@@ -159,16 +155,12 @@ async function main() {
};
console.log('Chat completion response:');
- await chatClient.completeStreamingChat(
- messages,
- tools,
- (chunk: any) => {
- const content = chunk.choices?.[0]?.message?.content;
- if (content) {
- process.stdout.write(content);
- }
+ for await (const chunk of chatClient.completeStreamingChat(messages, tools)) {
+ const content = chunk.choices?.[0]?.message?.content;
+ if (content) {
+ process.stdout.write(content);
}
- );
+ }
console.log('\n');
console.log('\n✓ Example completed successfully');
diff --git a/sdk/js/src/openai/audioClient.ts b/sdk/js/src/openai/audioClient.ts
index 59267015..7b174924 100644
--- a/sdk/js/src/openai/audioClient.ts
+++ b/sdk/js/src/openai/audioClient.ts
@@ -89,66 +89,153 @@ export class AudioClient {
}
/**
- * Transcribes audio into the input language using streaming.
+ * Transcribes audio into the input language using streaming, returning an async iterable of chunks.
* @param audioFilePath - Path to the audio file to transcribe.
- * @param callback - A callback function that receives each chunk of the streaming response.
- * @returns A promise that resolves when the stream is complete.
- * @throws Error - If audioFilePath or callback are invalid, or streaming fails.
+ * @returns An async iterable that yields parsed streaming transcription chunks.
+ * @throws Error - If audioFilePath is invalid, or streaming fails.
+ *
+ * @example
+ * ```typescript
+ * for await (const chunk of audioClient.transcribeStreaming('recording.wav')) {
+ * process.stdout.write(chunk.text);
+ * }
+ * ```
*/
- public async transcribeStreaming(audioFilePath: string, callback: (chunk: any) => void): Promise {
+ public transcribeStreaming(audioFilePath: string): AsyncIterable {
this.validateAudioFilePath(audioFilePath);
- if (!callback || typeof callback !== 'function') {
- throw new Error('Callback must be a valid function.');
- }
+
const request = {
Model: this.modelId,
FileName: audioFilePath,
...this.settings._serialize()
};
-
- let error: Error | null = null;
- try {
- await this.coreInterop.executeCommandStreaming(
- "audio_transcribe",
- { Params: { OpenAICreateRequest: JSON.stringify(request) } },
- (chunkStr: string) => {
- // Skip processing if we already encountered an error
- if (error) {
- return;
- }
-
- if (chunkStr) {
- let chunk: any;
- try {
- chunk = JSON.parse(chunkStr);
- } catch (e) {
- // Don't throw from callback - store first error and stop processing
- error = new Error(`Failed to parse streaming chunk: ${e instanceof Error ? e.message : String(e)}`, { cause: e });
- return;
+ // Capture instance properties to local variables because `this` is not
+ // accessible inside the [Symbol.asyncIterator]() method below — it's a
+ // regular method on the returned object literal, not on the AudioClient.
+ const coreInterop = this.coreInterop;
+ const modelId = this.modelId;
+
+ // Return an AsyncIterable object. The [Symbol.asyncIterator]() factory
+ // is called once when the consumer starts a `for await` loop, and it
+ // returns the AsyncIterator (with next() / return() methods).
+ return {
+ [Symbol.asyncIterator](): AsyncIterator {
+ // Buffer for chunks received from the native callback.
+ // Uses a head index for O(1) dequeue instead of Array.shift() which is O(n).
+ // JavaScript's single-threaded event loop ensures no race conditions
+ // between the callback pushing chunks and next() consuming them.
+ const chunks: any[] = [];
+ let head = 0;
+ let done = false;
+ let cancelled = false;
+ let error: Error | null = null;
+ let resolve: (() => void) | null = null;
+ let nextInFlight = false;
+
+ const streamingPromise = coreInterop.executeCommandStreaming(
+ "audio_transcribe",
+ { Params: { OpenAICreateRequest: JSON.stringify(request) } },
+ (chunkStr: string) => {
+ if (cancelled || error) return;
+ if (chunkStr) {
+ try {
+ const chunk = JSON.parse(chunkStr);
+ chunks.push(chunk);
+ } catch (e) {
+ if (!error) {
+ error = new Error(
+ `Failed to parse streaming chunk: ${e instanceof Error ? e.message : String(e)}`,
+ { cause: e }
+ );
+ }
+ }
+ }
+ // Wake up any waiting next() call
+ if (resolve) {
+ const r = resolve;
+ resolve = null;
+ r();
}
+ }
+ // When the native stream completes, mark done and wake up any
+ // pending next() call so it can see that iteration has ended.
+ ).then(() => {
+ done = true;
+ if (resolve) {
+ const r = resolve;
+ resolve = null;
+ r(); // resolve the pending next() promise
+ }
+ }).catch((err) => {
+ if (!error) {
+ const underlyingError = err instanceof Error ? err : new Error(String(err));
+ error = new Error(
+ `Streaming audio transcription failed for model '${modelId}': ${underlyingError.message}`,
+ { cause: underlyingError }
+ );
+ }
+ done = true;
+ if (resolve) {
+ const r = resolve;
+ resolve = null;
+ r();
+ }
+ });
+ // Return the AsyncIterator object consumed by `for await`.
+ // next() yields buffered chunks one at a time; return() is
+ // called automatically when the consumer breaks out early.
+ return {
+ async next(): Promise> {
+ if (nextInFlight) {
+ throw new Error('next() called concurrently on streaming iterator; await each call before invoking next().');
+ }
+ nextInFlight = true;
try {
- callback(chunk);
- } catch (e) {
- // Don't throw from callback - store first error and stop processing
- error = new Error(`User callback threw an error: ${e instanceof Error ? e.message : String(e)}`, { cause: e });
- return;
+ while (true) {
+ if (head < chunks.length) {
+ const value = chunks[head];
+ chunks[head] = undefined; // allow GC
+ head++;
+ // Compact the array when all buffered chunks have been consumed
+ if (head === chunks.length) {
+ chunks.length = 0;
+ head = 0;
+ }
+ return { value, done: false };
+ }
+ if (error) {
+ throw error;
+ }
+ if (done || cancelled) {
+ return { value: undefined, done: true };
+ }
+ // Wait for the next chunk or completion
+ await new Promise((r) => { resolve = r; });
+ }
+ } finally {
+ nextInFlight = false;
}
+ },
+ async return(): Promise> {
+ // Mark cancelled so the callback stops buffering.
+ // Note: the underlying native stream cannot be cancelled
+ // (CoreInterop.executeCommandStreaming has no abort support),
+ // so the koffi callback may still fire but will no-op due
+ // to the cancelled guard above.
+ cancelled = true;
+ chunks.length = 0;
+ head = 0;
+ if (resolve) {
+ const r = resolve;
+ resolve = null;
+ r();
+ }
+ return { value: undefined, done: true };
}
- }
- );
-
- // If we encountered an error during streaming, reject now
- if (error) {
- throw error;
+ };
}
- } catch (err) {
- const underlyingError = err instanceof Error ? err : new Error(String(err));
- throw new Error(
- `Streaming audio transcription failed for model '${this.modelId}': ${underlyingError.message}`,
- { cause: underlyingError }
- );
- }
+ };
}
}
diff --git a/sdk/js/src/openai/chatClient.ts b/sdk/js/src/openai/chatClient.ts
index 7aa77170..f844da41 100644
--- a/sdk/js/src/openai/chatClient.ts
+++ b/sdk/js/src/openai/chatClient.ts
@@ -211,26 +211,33 @@ export class ChatClient {
}
/**
- * Performs a streaming chat completion.
+ * Performs a streaming chat completion, returning an async iterable of chunks.
* @param messages - An array of message objects.
- * @param tools - An array of tool objects.
- * @param callback - A callback function that receives each chunk of the streaming response.
- * @returns A promise that resolves when the stream is complete.
- * @throws Error - If messages, tools, or callback are invalid, or streaming fails.
+ * @param tools - An optional array of tool objects.
+ * @returns An async iterable that yields parsed streaming response chunks.
+ * @throws Error - If messages or tools are invalid, or streaming fails.
+ *
+ * @example
+ * ```typescript
+ * // Without tools:
+ * for await (const chunk of chatClient.completeStreamingChat(messages)) {
+ * const content = chunk.choices?.[0]?.delta?.content;
+ * if (content) process.stdout.write(content);
+ * }
+ *
+ * // With tools:
+ * for await (const chunk of chatClient.completeStreamingChat(messages, tools)) {
+ * const content = chunk.choices?.[0]?.delta?.content;
+ * if (content) process.stdout.write(content);
+ * }
+ * ```
*/
- public async completeStreamingChat(messages: any[], callback: (chunk: any) => void): Promise;
- public async completeStreamingChat(messages: any[], tools: any[], callback: (chunk: any) => void): Promise;
- public async completeStreamingChat(messages: any[], toolsOrCallback: any[] | ((chunk: any) => void), maybeCallback?: (chunk: any) => void): Promise {
- const tools = Array.isArray(toolsOrCallback) ? toolsOrCallback : undefined;
- const callback = (Array.isArray(toolsOrCallback) ? maybeCallback : toolsOrCallback) as ((chunk: any) => void) | undefined;
-
+ public completeStreamingChat(messages: any[]): AsyncIterable;
+ public completeStreamingChat(messages: any[], tools: any[]): AsyncIterable;
+ public completeStreamingChat(messages: any[], tools?: any[]): AsyncIterable {
this.validateMessages(messages);
this.validateTools(tools);
- if (!callback || typeof callback !== 'function') {
- throw new Error('Callback must be a valid function.');
- }
-
const request = {
model: this.modelId,
messages,
@@ -239,49 +246,132 @@ export class ChatClient {
...this.settings._serialize()
};
- let error: Error | null = null;
+ // Capture instance properties to local variables because `this` is not
+ // accessible inside the [Symbol.asyncIterator]() method below — it's a
+ // regular method on the returned object literal, not on the ChatClient.
+ const coreInterop = this.coreInterop;
+ const modelId = this.modelId;
- try {
- await this.coreInterop.executeCommandStreaming(
- 'chat_completions',
- { Params: { OpenAICreateRequest: JSON.stringify(request) } },
- (chunkStr: string) => {
- // Skip processing if we already encountered an error
- if (error) return;
+ // Return an AsyncIterable object. The [Symbol.asyncIterator]() factory
+ // is called once when the consumer starts a `for await` loop, and it
+ // returns the AsyncIterator (with next() / return() methods).
+ return {
+ [Symbol.asyncIterator](): AsyncIterator {
+ // Buffer for chunks received from the native callback.
+ // Uses a head index for O(1) dequeue instead of Array.shift() which is O(n).
+ // JavaScript's single-threaded event loop ensures no race conditions
+ // between the callback pushing chunks and next() consuming them.
+ const chunks: any[] = [];
+ let head = 0;
+ let done = false;
+ let cancelled = false;
+ let error: Error | null = null;
+ let resolve: (() => void) | null = null;
+ let nextInFlight = false;
- if (chunkStr) {
- let chunk: any;
- try {
- chunk = JSON.parse(chunkStr);
- } catch (e) {
- // Don't throw from callback - store first error and stop processing
- error = new Error(
- `Failed to parse streaming chunk: ${e instanceof Error ? e.message : String(e)}`,
- { cause: e }
- );
- return;
+ const streamingPromise = coreInterop.executeCommandStreaming(
+ 'chat_completions',
+ { Params: { OpenAICreateRequest: JSON.stringify(request) } },
+ (chunkStr: string) => {
+ if (cancelled || error) return;
+ if (chunkStr) {
+ try {
+ const chunk = JSON.parse(chunkStr);
+ chunks.push(chunk);
+ } catch (e) {
+ if (!error) {
+ error = new Error(
+ `Failed to parse streaming chunk: ${e instanceof Error ? e.message : String(e)}`,
+ { cause: e }
+ );
+ }
+ }
}
+ // Wake up any waiting next() call
+ if (resolve) {
+ const r = resolve;
+ resolve = null;
+ r();
+ }
+ }
+ // When the native stream completes, mark done and wake up any
+ // pending next() call so it can see that iteration has ended.
+ ).then(() => {
+ done = true;
+ if (resolve) {
+ const r = resolve;
+ resolve = null;
+ r(); // resolve the pending next() promise
+ }
+ }).catch((err) => {
+ if (!error) {
+ const underlyingError = err instanceof Error ? err : new Error(String(err));
+ error = new Error(
+ `Streaming chat completion failed for model '${modelId}': ${underlyingError.message}`,
+ { cause: underlyingError }
+ );
+ }
+ done = true;
+ if (resolve) {
+ const r = resolve;
+ resolve = null;
+ r();
+ }
+ });
+ // Return the AsyncIterator object consumed by `for await`.
+ // next() yields buffered chunks one at a time; return() is
+ // called automatically when the consumer breaks out early.
+ return {
+ async next(): Promise> {
+ if (nextInFlight) {
+ throw new Error('next() called concurrently on streaming iterator; await each call before invoking next().');
+ }
+ nextInFlight = true;
try {
- callback(chunk);
- } catch (e) {
- // Don't throw from callback - store first error and stop processing
- error = new Error(
- `User callback threw an error: ${e instanceof Error ? e.message : String(e)}`,
- { cause: e }
- );
+ while (true) {
+ if (head < chunks.length) {
+ const value = chunks[head];
+ chunks[head] = undefined; // allow GC
+ head++;
+ // Compact the array when all buffered chunks have been consumed
+ if (head === chunks.length) {
+ chunks.length = 0;
+ head = 0;
+ }
+ return { value, done: false };
+ }
+ if (error) {
+ throw error;
+ }
+ if (done || cancelled) {
+ return { value: undefined, done: true };
+ }
+ // Wait for the next chunk or completion
+ await new Promise((r) => { resolve = r; });
+ }
+ } finally {
+ nextInFlight = false;
+ }
+ },
+ async return(): Promise> {
+ // Mark cancelled so the callback stops buffering.
+ // Note: the underlying native stream cannot be cancelled
+ // (CoreInterop.executeCommandStreaming has no abort support),
+ // so the koffi callback may still fire but will no-op due
+ // to the cancelled guard above.
+ cancelled = true;
+ chunks.length = 0;
+ head = 0;
+ if (resolve) {
+ const r = resolve;
+ resolve = null;
+ r();
}
+ return { value: undefined, done: true };
}
- }
- );
-
- // If we encountered an error during streaming, reject now
- if (error) throw error;
- } catch (err) {
- const underlyingError = err instanceof Error ? err : new Error(String(err));
- throw new Error(`Streaming chat completion failed for model '${this.modelId}': ${underlyingError.message}`, {
- cause: underlyingError
- });
- }
+ };
+ }
+ };
}
}
diff --git a/sdk/js/test/openai/audioClient.test.ts b/sdk/js/test/openai/audioClient.test.ts
index a57c02e5..10da05be 100644
--- a/sdk/js/test/openai/audioClient.test.ts
+++ b/sdk/js/test/openai/audioClient.test.ts
@@ -110,13 +110,13 @@ describe('Audio Client Tests', () => {
audioClient.settings.temperature = 0.0; // for deterministic results
let fullResponse = '';
- await audioClient.transcribeStreaming(AUDIO_FILE_PATH, (chunk) => {
+ for await (const chunk of audioClient.transcribeStreaming(AUDIO_FILE_PATH)) {
expect(chunk).to.not.be.undefined;
expect(chunk.text).to.not.be.undefined;
expect(chunk.text).to.be.a('string');
expect(chunk.text.length).to.be.greaterThan(0);
fullResponse += chunk.text;
- });
+ }
console.log(`Full response: ${fullResponse}`);
expect(fullResponse).to.equal(EXPECTED_TEXT);
@@ -151,13 +151,13 @@ describe('Audio Client Tests', () => {
audioClient.settings.temperature = 0.0; // for deterministic results
let fullResponse = '';
- await audioClient.transcribeStreaming(AUDIO_FILE_PATH, (chunk) => {
+ for await (const chunk of audioClient.transcribeStreaming(AUDIO_FILE_PATH)) {
expect(chunk).to.not.be.undefined;
expect(chunk.text).to.not.be.undefined;
expect(chunk.text).to.be.a('string');
expect(chunk.text.length).to.be.greaterThan(0);
fullResponse += chunk.text;
- });
+ }
console.log(`Full response: ${fullResponse}`);
expect(fullResponse).to.equal(EXPECTED_TEXT);
@@ -190,27 +190,12 @@ describe('Audio Client Tests', () => {
const audioClient = model.createAudioClient();
try {
- await audioClient.transcribeStreaming('', () => {});
+ // transcribeStreaming validates synchronously before returning the AsyncIterable
+ audioClient.transcribeStreaming('');
expect.fail('Should have thrown an error for empty audio file path');
} catch (error) {
expect(error).to.be.instanceOf(Error);
expect((error as Error).message).to.include('Audio file path must be a non-empty string');
}
});
-
- it('should throw when transcribing streaming with invalid callback', async function() {
- const manager = getTestManager();
- const catalog = manager.catalog;
- const model = await catalog.getModel(WHISPER_MODEL_ALIAS);
- const audioClient = model.createAudioClient();
- const invalidCallbacks: any[] = [null, undefined, 42, {}, 'not-a-function'];
- for (const invalidCallback of invalidCallbacks) {
- try {
- await audioClient.transcribeStreaming(AUDIO_FILE_PATH, invalidCallback as any);
- expect.fail('Should have thrown an error for invalid callback');
- } catch (error) {
- expect(error).to.be.instanceOf(Error);
- }
- }
- });
});
\ No newline at end of file
diff --git a/sdk/js/test/openai/chatClient.test.ts b/sdk/js/test/openai/chatClient.test.ts
index 5f612845..7be190ce 100644
--- a/sdk/js/test/openai/chatClient.test.ts
+++ b/sdk/js/test/openai/chatClient.test.ts
@@ -81,13 +81,13 @@ describe('Chat Client Tests', () => {
let fullContent = '';
let chunkCount = 0;
- await client.completeStreamingChat(messages, (chunk: any) => {
+ for await (const chunk of client.completeStreamingChat(messages)) {
chunkCount++;
const content = chunk.choices?.[0]?.delta?.content;
if (content) {
fullContent += content;
}
- });
+ }
expect(chunkCount).to.be.greaterThan(0);
expect(fullContent).to.be.a('string');
@@ -102,13 +102,13 @@ describe('Chat Client Tests', () => {
fullContent = '';
chunkCount = 0;
- await client.completeStreamingChat(messages, (chunk: any) => {
+ for await (const chunk of client.completeStreamingChat(messages)) {
chunkCount++;
const content = chunk.choices?.[0]?.delta?.content;
if (content) {
fullContent += content;
}
- });
+ }
expect(chunkCount).to.be.greaterThan(0);
expect(fullContent).to.be.a('string');
@@ -172,7 +172,8 @@ describe('Chat Client Tests', () => {
const invalidMessages: any[] = [[], null, undefined];
for (const invalidMessage of invalidMessages) {
try {
- await client.completeStreamingChat(invalidMessage, () => {});
+ // completeStreamingChat validates synchronously before returning the AsyncIterable
+ client.completeStreamingChat(invalidMessage);
expect.fail(`Should have thrown an error for ${Array.isArray(invalidMessage) ? 'empty' : invalidMessage} messages`);
} catch (error) {
expect(error).to.be.instanceOf(Error);
@@ -181,23 +182,6 @@ describe('Chat Client Tests', () => {
}
});
- it('should throw when completing streaming chat with invalid callback', async function() {
- const manager = getTestManager();
- const catalog = manager.catalog;
- const model = await catalog.getModel(TEST_MODEL_ALIAS);
- const client = model.createChatClient();
- const messages = [{ role: 'user', content: 'Hello' }];
- const invalidCallbacks: any[] = [null, undefined, {} as any, 'not a function' as any];
- for (const invalidCallback of invalidCallbacks) {
- try {
- await client.completeStreamingChat(messages as any, invalidCallback as any);
- expect.fail('Should have thrown an error for invalid callback');
- } catch (error) {
- expect(error).to.be.instanceOf(Error);
- }
- }
- });
-
it('should perform tool calling chat completion (non-streaming)', async function() {
this.timeout(20000);
const manager = getTestManager();
@@ -305,7 +289,7 @@ describe('Chat Client Tests', () => {
let lastToolCallChunk: any = null;
// Check that each response chunk contains the expected information
- await client.completeStreamingChat(messages, tools, (chunk: any) => {
+ for await (const chunk of client.completeStreamingChat(messages, tools)) {
const content = chunk.choices?.[0]?.message?.content ?? chunk.choices?.[0]?.delta?.content;
if (content) {
fullResponse += content;
@@ -314,7 +298,7 @@ describe('Chat Client Tests', () => {
if (toolCalls && toolCalls.length > 0) {
lastToolCallChunk = chunk;
}
- });
+ }
expect(fullResponse).to.be.a('string').and.not.equal('');
expect(lastToolCallChunk).to.not.be.null;
@@ -341,12 +325,12 @@ describe('Chat Client Tests', () => {
// Run the next turn of the conversation
fullResponse = '';
- await client.completeStreamingChat(messages, tools, (chunk: any) => {
+ for await (const chunk of client.completeStreamingChat(messages, tools)) {
const content = chunk.choices?.[0]?.message?.content ?? chunk.choices?.[0]?.delta?.content;
if (content) {
fullResponse += content;
}
- });
+ }
// Check that the conversation continued
expect(fullResponse).to.be.a('string').and.not.equal('');
From e570724a3300f6663d8f3f39e234231f5bf13ae1 Mon Sep 17 00:00:00 2001
From: Prathik Rao
Date: Fri, 27 Mar 2026 01:24:44 -0400
Subject: [PATCH 04/21] separates js sdk into foundry-local-sdk and
foundry-local-sdk-winml packages (#555)
no longer need `npm install --winml` as `npm install` with the separate
packages will fetch the appropriate binaries
---------
Co-authored-by: Prathik Rao
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
---
.github/workflows/build-js-steps.yml | 26 +-
sdk/js/docs/README.md | 2 +-
sdk/js/package.json | 12 +-
sdk/js/script/install-standard.cjs | 26 ++
sdk/js/script/install-utils.cjs | 193 +++++++++++++++
sdk/js/script/install-winml.cjs | 25 ++
sdk/js/script/install.cjs | 357 ---------------------------
sdk/js/script/pack.cjs | 32 +++
8 files changed, 293 insertions(+), 380 deletions(-)
create mode 100644 sdk/js/script/install-standard.cjs
create mode 100644 sdk/js/script/install-utils.cjs
create mode 100644 sdk/js/script/install-winml.cjs
delete mode 100644 sdk/js/script/install.cjs
create mode 100644 sdk/js/script/pack.cjs
diff --git a/.github/workflows/build-js-steps.yml b/.github/workflows/build-js-steps.yml
index d7a568a3..55f3ebf8 100644
--- a/.github/workflows/build-js-steps.yml
+++ b/.github/workflows/build-js-steps.yml
@@ -92,13 +92,7 @@ jobs:
run: |
if (Test-Path .npmrc) { Remove-Item .npmrc -Force; Write-Host "Removed .npmrc" }
- - name: npm install (WinML)
- if: ${{ inputs.useWinML == true }}
- working-directory: sdk/js
- run: npm install --winml
-
- - name: npm install (Standard)
- if: ${{ inputs.useWinML == false }}
+ - name: npm install
working-directory: sdk/js
run: npm install
@@ -114,21 +108,15 @@ jobs:
working-directory: sdk/js
run: npm run build
- - name: Pack npm package
+ - name: Pack npm package (WinML)
+ if: ${{ inputs.useWinML == true }}
working-directory: sdk/js
- run: npm pack
+ run: npm run pack:winml
- - name: Rename WinML artifact
- if: ${{ inputs.useWinML == true }}
- shell: pwsh
+ - name: Pack npm package (Standard)
+ if: ${{ inputs.useWinML == false }}
working-directory: sdk/js
- run: |
- $tgz = Get-ChildItem *.tgz | Select-Object -First 1
- if ($tgz) {
- $newName = $tgz.Name -replace '^foundry-local-sdk-', 'foundry-local-sdk-winml-'
- Rename-Item -Path $tgz.FullName -NewName $newName
- Write-Host "Renamed $($tgz.Name) to $newName"
- }
+ run: npm run pack
- name: Upload npm packages
uses: actions/upload-artifact@v4
diff --git a/sdk/js/docs/README.md b/sdk/js/docs/README.md
index 58218628..dd483aa4 100644
--- a/sdk/js/docs/README.md
+++ b/sdk/js/docs/README.md
@@ -1,4 +1,4 @@
-# @prathikrao/foundry-local-sdk
+# foundry-local-sdk
## Enumerations
diff --git a/sdk/js/package.json b/sdk/js/package.json
index 46ae6ce5..5830e3fe 100644
--- a/sdk/js/package.json
+++ b/sdk/js/package.json
@@ -7,13 +7,19 @@
"type": "module",
"files": [
"dist",
- "script"
+ "script/install-standard.cjs",
+ "script/install-winml.cjs",
+ "script/install-utils.cjs",
+ "script/pack.cjs",
+ "script/preinstall.cjs"
],
"scripts": {
"build": "tsc -p tsconfig.build.json",
"docs": "typedoc",
"example": "tsx examples/chat-completion.ts",
- "install": "node script/install.cjs",
+ "install": "node script/install-standard.cjs",
+ "pack": "node script/pack.cjs",
+ "pack:winml": "node script/pack.cjs winml",
"preinstall": "node script/preinstall.cjs",
"test": "mocha --import=tsx test/**/*.test.ts"
},
@@ -45,4 +51,4 @@
},
"author": "",
"license": "ISC"
-}
+}
\ No newline at end of file
diff --git a/sdk/js/script/install-standard.cjs b/sdk/js/script/install-standard.cjs
new file mode 100644
index 00000000..319a33d1
--- /dev/null
+++ b/sdk/js/script/install-standard.cjs
@@ -0,0 +1,26 @@
+// Copyright (c) Microsoft Corporation. All rights reserved.
+// Licensed under the MIT License.
+
+// Install script for foundry-local-sdk (standard variant).
+
+'use strict';
+
+const os = require('os');
+const { NUGET_FEED, ORT_NIGHTLY_FEED, runInstall } = require('./install-utils.cjs');
+
+const useNightly = process.env.npm_config_nightly === 'true';
+
+const ARTIFACTS = [
+ { name: 'Microsoft.AI.Foundry.Local.Core', version: '0.9.0.8-rc3', feed: ORT_NIGHTLY_FEED, nightly: useNightly },
+ { name: os.platform() === 'linux' ? 'Microsoft.ML.OnnxRuntime.Gpu.Linux' : 'Microsoft.ML.OnnxRuntime.Foundry', version: '1.24.3', feed: NUGET_FEED, nightly: false },
+ { name: 'Microsoft.ML.OnnxRuntimeGenAI.Foundry', version: '0.12.2', feed: NUGET_FEED, nightly: false },
+];
+
+(async () => {
+ try {
+ await runInstall(ARTIFACTS);
+ } catch (err) {
+ console.error('[foundry-local] Installation failed:', err instanceof Error ? err.message : err);
+ process.exit(1);
+ }
+})();
diff --git a/sdk/js/script/install-utils.cjs b/sdk/js/script/install-utils.cjs
new file mode 100644
index 00000000..f9a5186c
--- /dev/null
+++ b/sdk/js/script/install-utils.cjs
@@ -0,0 +1,193 @@
+// Copyright (c) Microsoft Corporation. All rights reserved.
+// Licensed under the MIT License.
+
+// Shared NuGet download and extraction utilities for install scripts.
+
+'use strict';
+
+const fs = require('fs');
+const path = require('path');
+const os = require('os');
+const https = require('https');
+const AdmZip = require('adm-zip');
+
+const PLATFORM_MAP = {
+ 'win32-x64': 'win-x64',
+ 'win32-arm64': 'win-arm64',
+ 'linux-x64': 'linux-x64',
+ 'darwin-arm64': 'osx-arm64',
+};
+const platformKey = `${os.platform()}-${os.arch()}`;
+const RID = PLATFORM_MAP[platformKey];
+const BIN_DIR = path.join(__dirname, '..', 'packages', '@foundry-local-core', platformKey);
+const EXT = os.platform() === 'win32' ? '.dll' : os.platform() === 'darwin' ? '.dylib' : '.so';
+
+const REQUIRED_FILES = [
+ `Microsoft.AI.Foundry.Local.Core${EXT}`,
+ `${os.platform() === 'win32' ? '' : 'lib'}onnxruntime${EXT}`,
+ `${os.platform() === 'win32' ? '' : 'lib'}onnxruntime-genai${EXT}`,
+];
+
+const NUGET_FEED = 'https://api.nuget.org/v3/index.json';
+const ORT_NIGHTLY_FEED = 'https://pkgs.dev.azure.com/aiinfra/PublicPackages/_packaging/ORT-Nightly/nuget/v3/index.json';
+
+// --- Download helpers ---
+
+async function downloadWithRetryAndRedirects(url, destStream = null) {
+ const maxRedirects = 5;
+ let currentUrl = url;
+ let redirects = 0;
+
+ while (redirects < maxRedirects) {
+ const response = await new Promise((resolve, reject) => {
+ https.get(currentUrl, (res) => resolve(res))
+ .on('error', reject);
+ });
+
+ if (response.statusCode >= 300 && response.statusCode < 400 && response.headers.location) {
+ currentUrl = response.headers.location;
+ response.resume();
+ redirects++;
+ console.log(` Following redirect to ${new URL(currentUrl).host}...`);
+ continue;
+ }
+
+ if (response.statusCode !== 200) {
+ throw new Error(`Download failed with status ${response.statusCode}: ${currentUrl}`);
+ }
+
+ if (destStream) {
+ response.pipe(destStream);
+ return new Promise((resolve, reject) => {
+ destStream.on('finish', resolve);
+ destStream.on('error', reject);
+ response.on('error', reject);
+ });
+ } else {
+ let data = '';
+ response.on('data', chunk => data += chunk);
+ return new Promise((resolve, reject) => {
+ response.on('end', () => resolve(data));
+ response.on('error', reject);
+ });
+ }
+ }
+ throw new Error('Too many redirects');
+}
+
+async function downloadJson(url) {
+ return JSON.parse(await downloadWithRetryAndRedirects(url));
+}
+
+async function downloadFile(url, dest) {
+ const file = fs.createWriteStream(dest);
+ try {
+ await downloadWithRetryAndRedirects(url, file);
+ file.close();
+ } catch (e) {
+ file.close();
+ if (fs.existsSync(dest)) fs.unlinkSync(dest);
+ throw e;
+ }
+}
+
+const serviceIndexCache = new Map();
+
+async function getBaseAddress(feedUrl) {
+ if (!serviceIndexCache.has(feedUrl)) {
+ serviceIndexCache.set(feedUrl, await downloadJson(feedUrl));
+ }
+ const resources = serviceIndexCache.get(feedUrl).resources || [];
+ const res = resources.find(r => r['@type'] && r['@type'].startsWith('PackageBaseAddress/3.0.0'));
+ if (!res) throw new Error('Could not find PackageBaseAddress/3.0.0 in NuGet feed.');
+ const baseAddress = res['@id'];
+ return baseAddress.endsWith('/') ? baseAddress : baseAddress + '/';
+}
+
+async function resolveLatestVersion(feedUrl, packageName) {
+ const baseAddress = await getBaseAddress(feedUrl);
+ const versionsUrl = `${baseAddress}${packageName.toLowerCase()}/index.json`;
+ const versionData = await downloadJson(versionsUrl);
+ const versions = versionData.versions || [];
+ if (versions.length === 0) throw new Error(`No versions found for ${packageName}`);
+ versions.sort((a, b) => b.localeCompare(a));
+ console.log(`[foundry-local] Latest version of ${packageName}: ${versions[0]}`);
+ return versions[0];
+}
+
+async function installPackage(artifact, tempDir) {
+ const pkgName = artifact.name;
+ let pkgVer = artifact.version;
+ if (artifact.nightly) {
+ console.log(` Resolving latest version for ${pkgName}...`);
+ pkgVer = await resolveLatestVersion(artifact.feed, pkgName);
+ }
+
+ const baseAddress = await getBaseAddress(artifact.feed);
+ const nameLower = pkgName.toLowerCase();
+ const verLower = pkgVer.toLowerCase();
+ const downloadUrl = `${baseAddress}${nameLower}/${verLower}/${nameLower}.${verLower}.nupkg`;
+
+ const nupkgPath = path.join(tempDir, `${pkgName}.${pkgVer}.nupkg`);
+ console.log(` Downloading ${pkgName} ${pkgVer}...`);
+ await downloadFile(downloadUrl, nupkgPath);
+
+ console.log(` Extracting...`);
+ const zip = new AdmZip(nupkgPath);
+ const targetPathPrefix = `runtimes/${RID}/native/`.toLowerCase();
+ const entries = zip.getEntries().filter(e => {
+ const p = e.entryName.toLowerCase();
+ return p.includes(targetPathPrefix) && p.endsWith(EXT);
+ });
+
+ if (entries.length > 0) {
+ entries.forEach(entry => {
+ zip.extractEntryTo(entry, BIN_DIR, false, true);
+ console.log(` Extracted ${entry.name}`);
+ });
+ } else {
+ console.warn(` No files found for RID ${RID} in ${pkgName}.`);
+ }
+
+ // Update platform package.json version for Core packages
+ if (pkgName.startsWith('Microsoft.AI.Foundry.Local.Core')) {
+ const pkgJsonPath = path.join(BIN_DIR, 'package.json');
+ if (fs.existsSync(pkgJsonPath)) {
+ const pkgJson = JSON.parse(fs.readFileSync(pkgJsonPath, 'utf8'));
+ pkgJson.version = pkgVer;
+ fs.writeFileSync(pkgJsonPath, JSON.stringify(pkgJson, null, 2));
+ }
+ }
+}
+
+async function runInstall(artifacts) {
+ if (!RID) {
+ console.warn(`[foundry-local] Unsupported platform: ${platformKey}. Skipping.`);
+ return;
+ }
+
+ if (fs.existsSync(BIN_DIR) && REQUIRED_FILES.every(f => fs.existsSync(path.join(BIN_DIR, f)))) {
+ if (process.env.npm_config_nightly === 'true') {
+ console.log(`[foundry-local] Nightly requested. Forcing reinstall...`);
+ fs.rmSync(BIN_DIR, { recursive: true, force: true });
+ } else {
+ console.log(`[foundry-local] Native libraries already installed.`);
+ return;
+ }
+ }
+
+ console.log(`[foundry-local] Installing native libraries for ${RID}...`);
+ fs.mkdirSync(BIN_DIR, { recursive: true });
+
+ const tempDir = fs.mkdtempSync(path.join(os.tmpdir(), 'foundry-install-'));
+ try {
+ for (const artifact of artifacts) {
+ await installPackage(artifact, tempDir);
+ }
+ console.log('[foundry-local] Installation complete.');
+ } finally {
+ try { fs.rmSync(tempDir, { recursive: true, force: true }); } catch {}
+ }
+}
+
+module.exports = { NUGET_FEED, ORT_NIGHTLY_FEED, runInstall };
diff --git a/sdk/js/script/install-winml.cjs b/sdk/js/script/install-winml.cjs
new file mode 100644
index 00000000..b46770ca
--- /dev/null
+++ b/sdk/js/script/install-winml.cjs
@@ -0,0 +1,25 @@
+// Copyright (c) Microsoft Corporation. All rights reserved.
+// Licensed under the MIT License.
+
+// Install script for foundry-local-sdk-winml variant.
+
+'use strict';
+
+const { NUGET_FEED, ORT_NIGHTLY_FEED, runInstall } = require('./install-utils.cjs');
+
+const useNightly = process.env.npm_config_nightly === 'true';
+
+const ARTIFACTS = [
+ { name: 'Microsoft.AI.Foundry.Local.Core.WinML', version: '0.9.0.8-rc3', feed: ORT_NIGHTLY_FEED, nightly: useNightly },
+ { name: 'Microsoft.ML.OnnxRuntime.Foundry', version: '1.23.2.3', feed: NUGET_FEED, nightly: false },
+ { name: 'Microsoft.ML.OnnxRuntimeGenAI.WinML', version: '0.12.2', feed: NUGET_FEED, nightly: false },
+];
+
+(async () => {
+ try {
+ await runInstall(ARTIFACTS);
+ } catch (err) {
+ console.error('Failed to install WinML artifacts:', err);
+ process.exit(1);
+ }
+})();
diff --git a/sdk/js/script/install.cjs b/sdk/js/script/install.cjs
deleted file mode 100644
index cdf5531d..00000000
--- a/sdk/js/script/install.cjs
+++ /dev/null
@@ -1,357 +0,0 @@
-// Copyright (c) Microsoft Corporation. All rights reserved.
-// Licensed under the MIT License.
-
-// Adapted from onnxruntime\js\node\script\install-utils.js
-// The file in packages/ are the original source of truth that we are downloading and "installing" into our project's source tree.
-// The file in node_modules/... is a symlink created by NPM to mark them as dependencies of the overall package.
-
-'use strict';
-
-const fs = require('fs');
-const path = require('path');
-const os = require('os');
-const https = require('https');
-const AdmZip = require('adm-zip');
-
-// Determine platform
-const PLATFORM_MAP = {
- 'win32-x64': 'win-x64',
- 'win32-arm64': 'win-arm64',
- 'linux-x64': 'linux-x64',
- 'darwin-arm64': 'osx-arm64',
-};
-const platformKey = `${os.platform()}-${os.arch()}`;
-const RID = PLATFORM_MAP[platformKey];
-
-if (!RID) {
- console.warn(`[foundry-local] Unsupported platform: ${platformKey}. Skipping native library installation.`);
- process.exit(0);
-}
-
-// Write to the source 'packages' directory so binaries persist and link correctly via package.json
-const BIN_DIR = path.join(__dirname, '..', 'packages', '@foundry-local-core', platformKey);
-const REQUIRED_FILES = [
- 'Microsoft.AI.Foundry.Local.Core.dll',
- 'onnxruntime.dll',
- 'onnxruntime-genai.dll',
-].map(f => f.replace('.dll', os.platform() === 'win32' ? '.dll' : os.platform() === 'darwin' ? '.dylib' : '.so'));
-
-// When you run npm install --winml, npm does not pass --winml as a command-line argument to your script.
-// Instead, it sets an environment variable named npm_config_winml to 'true'.
-const useWinML = process.env.npm_config_winml === 'true';
-const useNightly = process.env.npm_config_nightly === 'true';
-const noDeps = process.env.npm_config_nodeps === 'true';
-
-console.log(`[foundry-local] WinML enabled: ${useWinML}`);
-console.log(`[foundry-local] Nightly enabled: ${useNightly}`);
-
-const NUGET_FEED = 'https://api.nuget.org/v3/index.json';
-const ORT_FEED = 'https://pkgs.dev.azure.com/aiinfra/PublicPackages/_packaging/ORT/nuget/v3/index.json';
-const ORT_NIGHTLY_FEED = 'https://pkgs.dev.azure.com/aiinfra/PublicPackages/_packaging/ORT-Nightly/nuget/v3/index.json';
-
-// If nightly is requested, pull Core/GenAI from the ORT-Nightly feed where nightly builds are published.
-// Otherwise use the standard NuGet.org feed.
-const CORE_FEED = useNightly ? ORT_NIGHTLY_FEED : NUGET_FEED;
-
-const FOUNDRY_LOCAL_CORE_ARTIFACT = {
- name: 'Microsoft.AI.Foundry.Local.Core',
- version: '0.9.0.8-rc3',
- feed: ORT_NIGHTLY_FEED,
- nightly: useNightly
-}
-
-const FOUNDRY_LOCAL_CORE_WINML_ARTIFACT = {
- name: 'Microsoft.AI.Foundry.Local.Core.WinML',
- version: '0.9.0.8-rc3',
- feed: ORT_NIGHTLY_FEED,
- nightly: useNightly
-}
-
-const ONNX_RUNTIME_FOUNDRY_ARTIFACT = {
- name: 'Microsoft.ML.OnnxRuntime.Foundry',
- version: '1.24.3',
- feed: NUGET_FEED,
- nightly: false
-}
-
-const ONNX_RUNTIME_WINML_ARTIFACT = {
- name: 'Microsoft.ML.OnnxRuntime.Foundry',
- version: '1.23.2.3',
- feed: NUGET_FEED,
- nightly: false
-}
-
-const ONNX_RUNTIME_LINUX_ARTIFACT = {
- name: 'Microsoft.ML.OnnxRuntime.Gpu.Linux',
- version: '1.24.3',
- feed: NUGET_FEED,
- nightly: false
-}
-
-const ONNX_RUNTIME_GENAI_FOUNDRY_ARTIFACT = {
- name: 'Microsoft.ML.OnnxRuntimeGenAI.Foundry',
- version: '0.12.2',
- feed: NUGET_FEED,
- nightly: false
-}
-
-const ONNX_RUNTIME_GENAI_WINML_ARTIFACT = {
- name: 'Microsoft.ML.OnnxRuntimeGenAI.WinML',
- version: '0.12.2',
- feed: NUGET_FEED,
- nightly: false
-}
-
-const WINML_ARTIFACTS = [
- FOUNDRY_LOCAL_CORE_WINML_ARTIFACT,
- ONNX_RUNTIME_WINML_ARTIFACT,
- ONNX_RUNTIME_GENAI_WINML_ARTIFACT
-];
-
-const NON_WINML_ARTIFACTS = [
- FOUNDRY_LOCAL_CORE_ARTIFACT,
- ONNX_RUNTIME_FOUNDRY_ARTIFACT,
- ONNX_RUNTIME_GENAI_FOUNDRY_ARTIFACT
-];
-
-const LINUX_ARTIFACTS = [
- FOUNDRY_LOCAL_CORE_ARTIFACT,
- ONNX_RUNTIME_LINUX_ARTIFACT,
- ONNX_RUNTIME_GENAI_FOUNDRY_ARTIFACT
-];
-
-let ARTIFACTS = [];
-if (noDeps) {
- console.log(`[foundry-local] Skipping dependencies install...`);
- ARTIFACTS = [];
-} else if (useWinML) {
- console.log(`[foundry-local] Using WinML artifacts...`);
- ARTIFACTS = WINML_ARTIFACTS;
-} else if (os.platform() === 'linux') {
- console.log(`[foundry-local] Using Linux GPU artifacts...`);
- ARTIFACTS = LINUX_ARTIFACTS;
-} else {
- console.log(`[foundry-local] Using standard artifacts...`);
- ARTIFACTS = NON_WINML_ARTIFACTS;
-}
-
-// Check if already installed
-if (fs.existsSync(BIN_DIR) && REQUIRED_FILES.every(f => fs.existsSync(path.join(BIN_DIR, f)))) {
- if (useNightly) {
- console.log(`[foundry-local] Nightly requested. Forcing reinstall...`);
- fs.rmSync(BIN_DIR, { recursive: true, force: true });
- } else {
- console.log(`[foundry-local] Native libraries already installed.`);
- process.exit(0);
- }
-}
-
-console.log(`[foundry-local] Installing native libraries for ${RID}...`);
-fs.mkdirSync(BIN_DIR, { recursive: true });
-
-async function downloadWithRetryAndRedirects(url, destStream = null) {
- const maxRedirects = 5;
- let currentUrl = url;
- let redirects = 0;
-
- while (redirects < maxRedirects) {
- const response = await new Promise((resolve, reject) => {
- https.get(currentUrl, (res) => resolve(res))
- .on('error', reject);
- });
-
- // When you request a file from api.nuget.org, it rarely serves the file directly.
- // Instead, it usually responds with a 302 Found or 307 Temporary Redirect pointing to a Content Delivery Network (CDN)
- // or a specific Storage Account where the actual file lives. Node.js treats a redirect as a completed request so we
- // need to explicitly handle it here.
- if (response.statusCode >= 300 && response.statusCode < 400 && response.headers.location) {
- currentUrl = response.headers.location;
- response.resume(); // Consume/discard response data to free up socket
- redirects++;
- console.log(` Following redirect to ${new URL(currentUrl).host}...`);
- continue;
- }
-
- if (response.statusCode !== 200) {
- throw new Error(`Download failed with status ${response.statusCode}: ${currentUrl}`);
- }
-
- // destStream is null when the function is used to download JSON data (like NuGet feed index or package metadata) rather than a file
- if (destStream) {
- response.pipe(destStream);
- return new Promise((resolve, reject) => {
- destStream.on('finish', resolve);
- destStream.on('error', reject);
- response.on('error', reject);
- });
- } else {
- let data = '';
- response.on('data', chunk => data += chunk);
- return new Promise((resolve, reject) => {
- response.on('end', () => resolve(data));
- response.on('error', reject);
- });
- }
- }
- throw new Error('Too many redirects');
-}
-
-async function downloadJson(url) {
- const data = await downloadWithRetryAndRedirects(url);
- return JSON.parse(data);
-}
-
-async function downloadFile(url, dest) {
- const file = fs.createWriteStream(dest);
- try {
- await downloadWithRetryAndRedirects(url, file);
- file.close();
- } catch (e) {
- file.close();
- if (fs.existsSync(dest)) fs.unlinkSync(dest);
- throw e;
- }
-}
-
-
-// Map to cache service index resources
-const serviceIndexCache = new Map();
-
-async function getBaseAddress(feedUrl) {
- // 1. Get Service Index
- if (!serviceIndexCache.has(feedUrl)) {
- const index = await downloadJson(feedUrl);
- serviceIndexCache.set(feedUrl, index);
- }
-
- const serviceIndex = serviceIndexCache.get(feedUrl);
-
- // 2. Find PackageBaseAddress/3.0.0
- const resources = serviceIndex.resources || [];
- const baseAddressRes = resources.find(r => r['@type'] && r['@type'].startsWith('PackageBaseAddress/3.0.0'));
-
- if (!baseAddressRes) {
- throw new Error('Could not find PackageBaseAddress/3.0.0 in NuGet feed.');
- }
-
- const baseAddress = baseAddressRes['@id'];
- // Ensure trailing slash
- return baseAddress.endsWith('/') ? baseAddress : baseAddress + '/';
-}
-
-async function resolveLatestVersion(feedUrl, packageName) {
- const baseAddress = await getBaseAddress(feedUrl);
- const nameLower = packageName.toLowerCase();
-
- // Fetch version list: {baseAddress}/{lower_id}/index.json
- const versionsUrl = `${baseAddress}${nameLower}/index.json`;
- try {
- const versionData = await downloadJson(versionsUrl);
- const versions = versionData.versions || [];
-
- if (versions.length === 0) {
- throw new Error('No versions found');
- }
-
- // Sort descending to prioritize latest date-based versions (e.g. 0.9.0-dev.YYYYMMDD...)
- versions.sort((a, b) => b.localeCompare(a));
-
- const latestVersion = versions[0];
- console.log(`[foundry-local] Installing latest version of Foundry Local Core: ${latestVersion}`);
- return latestVersion;
- } catch (e) {
- throw new Error(`Failed to fetch versions for ${packageName} from ${versionsUrl}: ${e.message}`);
- }
-}
-
-async function resolvePackageRawUrl(feedUrl, packageName, version) {
- const properBase = await getBaseAddress(feedUrl);
-
- // 3. Construct .nupkg URL (lowercase is standard for V3)
- const nameLower = packageName.toLowerCase();
- const verLower = version.toLowerCase();
-
- return `${properBase}${nameLower}/${verLower}/${nameLower}.${verLower}.nupkg`;
-}
-
-async function installPackage(artifact, tempDir) {
- const pkgName = artifact.name;
- const feedUrl = artifact.feed;
-
- // Resolve version if not specified
- let pkgVer = artifact.version;
- let isNightly = artifact.nightly;
- if (isNightly) {
- console.log(` Resolving latest version for ${pkgName}...`);
- pkgVer = await resolveLatestVersion(feedUrl, pkgName);
- }
-
- console.log(` Resolving ${pkgName} ${pkgVer}...`);
- const downloadUrl = await resolvePackageRawUrl(feedUrl, pkgName, pkgVer);
-
- const nupkgPath = path.join(tempDir, `${pkgName}.${pkgVer}.nupkg`);
-
- console.log(` Downloading ${downloadUrl}...`);
- await downloadFile(downloadUrl, nupkgPath);
-
- console.log(` Extracting...`);
- const zip = new AdmZip(nupkgPath);
- const zipEntries = zip.getEntries();
-
- // Pattern: runtimes/{RID}/native/{file}.{ext}
- const ext = os.platform() === 'win32' ? '.dll' : os.platform() === 'darwin' ? '.dylib' : '.so';
- const targetPathPrefix = `runtimes/${RID}/native/`.toLowerCase();
-
- let found = false;
-
- console.log(` Scanning for all ${ext} files in ${targetPathPrefix}...`);
- const entries = zipEntries.filter(e => {
- const entryPathLower = e.entryName.toLowerCase();
- return entryPathLower.includes(targetPathPrefix) && entryPathLower.endsWith(ext);
- });
-
- if (entries.length > 0) {
- entries.forEach(entry => {
- console.log(` Found ${entry.entryName}`);
- zip.extractEntryTo(entry, BIN_DIR, false, true);
- console.log(` Extracted ${entry.name}`);
- });
- found = true;
- } else {
- console.warn(` ⚠ No files found for RID ${RID} in package.`);
- }
-
- // After extracting, update the packages/@foundry-local-core/RID/package.json version to match the downloaded artifact
- if (found && pkgName.startsWith('Microsoft.AI.Foundry.Local.Core')) {
- const pkgJsonPath = path.join(BIN_DIR, 'package.json');
- try {
- if (fs.existsSync(pkgJsonPath)) {
- const pkgJson = JSON.parse(fs.readFileSync(pkgJsonPath, 'utf8'));
- pkgJson.version = pkgVer;
- fs.writeFileSync(pkgJsonPath, JSON.stringify(pkgJson, null, 2));
- console.log(` Updated package.json version to ${pkgVer}`);
- }
- } catch (e) {
- console.warn(` Failed to update package.json version: ${e.message}`);
- }
- }
-}
-
-async function main() {
- const tempDir = fs.mkdtempSync(path.join(os.tmpdir(), 'foundry-install-'));
- try {
- for (const artifact of ARTIFACTS) {
- await installPackage(artifact, tempDir);
- }
- console.log('[foundry-local] ✓ Installation complete.');
- } catch (e) {
- console.error(`[foundry-local] Installation failed: ${e.message}`);
- process.exit(1);
- } finally {
- try {
- fs.rmSync(tempDir, { recursive: true, force: true });
- } catch {}
- }
-}
-
-main();
diff --git a/sdk/js/script/pack.cjs b/sdk/js/script/pack.cjs
new file mode 100644
index 00000000..32057c7e
--- /dev/null
+++ b/sdk/js/script/pack.cjs
@@ -0,0 +1,32 @@
+// Copyright (c) Microsoft Corporation. All rights reserved.
+// Licensed under the MIT License.
+
+// Usage:
+// node script/pack.cjs -> foundry-local-sdk-.tgz
+// node script/pack.cjs winml -> foundry-local-sdk-winml-.tgz
+
+'use strict';
+
+const fs = require('fs');
+const path = require('path');
+const { execSync } = require('child_process');
+
+const pkgPath = path.join(__dirname, '..', 'package.json');
+const original = fs.readFileSync(pkgPath, 'utf8');
+const isWinML = process.argv[2] === 'winml';
+
+try {
+ const pkg = JSON.parse(original);
+ if (isWinML) {
+ pkg.name = 'foundry-local-sdk-winml';
+ pkg.scripts.install = 'node script/install-winml.cjs';
+ pkg.files = ['dist', 'script/install-winml.cjs', 'script/install-utils.cjs', 'script/preinstall.cjs'];
+ } else {
+ pkg.files = ['dist', 'script/install-standard.cjs', 'script/install-utils.cjs', 'script/preinstall.cjs'];
+ }
+ fs.writeFileSync(pkgPath, JSON.stringify(pkg, null, 2));
+ execSync('npm pack', { cwd: path.join(__dirname, '..'), stdio: 'inherit' });
+} finally {
+ // Always restore original package.json
+ fs.writeFileSync(pkgPath, original);
+}
From 62dc8a7b7fcd1002068af1d856d0f3e9ceb74e19 Mon Sep 17 00:00:00 2001
From: Prathik Rao
Date: Fri, 27 Mar 2026 11:47:54 -0400
Subject: [PATCH 05/21] implements python sdk (#533)
mvp
---------
Co-authored-by: Prathik Rao
Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>
---
.github/workflows/build-python-steps.yml | 110 +++++++
.github/workflows/foundry-local-sdk-build.yml | 19 ++
sdk/python/.gitignore | 20 ++
sdk/python/LICENSE.txt | 21 ++
sdk/python/README.md | 243 ++++++++++++++
sdk/python/build_backend.py | 157 +++++++++
sdk/python/examples/chat_completion.py | 83 +++++
sdk/python/pyproject.toml | 55 ++++
sdk/python/requirements-dev.txt | 5 +
sdk/python/requirements-winml.txt | 7 +
sdk/python/requirements.txt | 7 +
sdk/python/src/__init__.py | 23 ++
sdk/python/src/catalog.py | 144 +++++++++
sdk/python/src/configuration.py | 163 ++++++++++
sdk/python/src/detail/__init__.py | 25 ++
sdk/python/src/detail/core_interop.py | 306 ++++++++++++++++++
sdk/python/src/detail/model_data_types.py | 76 +++++
sdk/python/src/detail/model_load_manager.py | 166 ++++++++++
sdk/python/src/detail/utils.py | 294 +++++++++++++++++
sdk/python/src/exception.py | 7 +
sdk/python/src/foundry_local_manager.py | 118 +++++++
sdk/python/src/imodel.py | 91 ++++++
sdk/python/src/logging_helper.py | 30 ++
sdk/python/src/model.py | 133 ++++++++
sdk/python/src/model_variant.py | 130 ++++++++
sdk/python/src/openai/__init__.py | 10 +
sdk/python/src/openai/audio_client.py | 153 +++++++++
sdk/python/src/openai/chat_client.py | 290 +++++++++++++++++
sdk/python/src/version.py | 6 +
sdk/python/test/README.md | 79 +++++
sdk/python/test/__init__.py | 0
sdk/python/test/conftest.py | 145 +++++++++
sdk/python/test/detail/__init__.py | 0
.../test/detail/test_model_load_manager.py | 144 +++++++++
sdk/python/test/openai/__init__.py | 0
sdk/python/test/openai/test_audio_client.py | 156 +++++++++
sdk/python/test/openai/test_chat_client.py | 243 ++++++++++++++
sdk/python/test/test_catalog.py | 74 +++++
sdk/python/test/test_foundry_local_manager.py | 22 ++
sdk/python/test/test_model.py | 58 ++++
40 files changed, 3813 insertions(+)
create mode 100644 .github/workflows/build-python-steps.yml
create mode 100644 sdk/python/.gitignore
create mode 100644 sdk/python/LICENSE.txt
create mode 100644 sdk/python/README.md
create mode 100644 sdk/python/build_backend.py
create mode 100644 sdk/python/examples/chat_completion.py
create mode 100644 sdk/python/pyproject.toml
create mode 100644 sdk/python/requirements-dev.txt
create mode 100644 sdk/python/requirements-winml.txt
create mode 100644 sdk/python/requirements.txt
create mode 100644 sdk/python/src/__init__.py
create mode 100644 sdk/python/src/catalog.py
create mode 100644 sdk/python/src/configuration.py
create mode 100644 sdk/python/src/detail/__init__.py
create mode 100644 sdk/python/src/detail/core_interop.py
create mode 100644 sdk/python/src/detail/model_data_types.py
create mode 100644 sdk/python/src/detail/model_load_manager.py
create mode 100644 sdk/python/src/detail/utils.py
create mode 100644 sdk/python/src/exception.py
create mode 100644 sdk/python/src/foundry_local_manager.py
create mode 100644 sdk/python/src/imodel.py
create mode 100644 sdk/python/src/logging_helper.py
create mode 100644 sdk/python/src/model.py
create mode 100644 sdk/python/src/model_variant.py
create mode 100644 sdk/python/src/openai/__init__.py
create mode 100644 sdk/python/src/openai/audio_client.py
create mode 100644 sdk/python/src/openai/chat_client.py
create mode 100644 sdk/python/src/version.py
create mode 100644 sdk/python/test/README.md
create mode 100644 sdk/python/test/__init__.py
create mode 100644 sdk/python/test/conftest.py
create mode 100644 sdk/python/test/detail/__init__.py
create mode 100644 sdk/python/test/detail/test_model_load_manager.py
create mode 100644 sdk/python/test/openai/__init__.py
create mode 100644 sdk/python/test/openai/test_audio_client.py
create mode 100644 sdk/python/test/openai/test_chat_client.py
create mode 100644 sdk/python/test/test_catalog.py
create mode 100644 sdk/python/test/test_foundry_local_manager.py
create mode 100644 sdk/python/test/test_model.py
diff --git a/.github/workflows/build-python-steps.yml b/.github/workflows/build-python-steps.yml
new file mode 100644
index 00000000..dc180bb4
--- /dev/null
+++ b/.github/workflows/build-python-steps.yml
@@ -0,0 +1,110 @@
+name: Build Python SDK
+
+on:
+ workflow_call:
+ inputs:
+ version:
+ required: true
+ type: string
+ useWinML:
+ required: false
+ type: boolean
+ default: false
+ platform:
+ required: false
+ type: string
+ default: 'windows'
+
+permissions:
+ contents: read
+
+jobs:
+ build:
+ runs-on: ${{ inputs.platform }}-latest
+
+ steps:
+ - name: Checkout repository
+ uses: actions/checkout@v4
+ with:
+ clean: true
+
+ - name: Setup Python
+ uses: actions/setup-python@v5
+ with:
+ python-version: '3.12'
+
+ # Clone test-data-shared from Azure DevOps (models for integration tests)
+ - name: Checkout test-data-shared from Azure DevOps
+ shell: pwsh
+ working-directory: ${{ github.workspace }}/..
+ run: |
+ $pat = "${{ secrets.AZURE_DEVOPS_PAT }}"
+ $encodedPat = [Convert]::ToBase64String([Text.Encoding]::ASCII.GetBytes(":$pat"))
+
+ git config --global http.https://dev.azure.com.extraheader "AUTHORIZATION: Basic $encodedPat"
+
+ git lfs install
+ git clone --depth 1 https://dev.azure.com/microsoft/windows.ai.toolkit/_git/test-data-shared test-data-shared
+
+ Write-Host "Clone completed successfully to ${{ github.workspace }}/../test-data-shared"
+
+ - name: Checkout specific commit in test-data-shared
+ shell: pwsh
+ working-directory: ${{ github.workspace }}/../test-data-shared
+ run: |
+ git checkout 231f820fe285145b7ea4a449b112c1228ce66a41
+ if ($LASTEXITCODE -ne 0) {
+ Write-Error "Git checkout failed."
+ exit 1
+ }
+
+ - name: Install build tool
+ run: |
+ python -m pip install build
+
+ - name: Configure pip for Azure Artifacts
+ run: |
+ pip config set global.index-url https://pkgs.dev.azure.com/aiinfra/PublicPackages/_packaging/ORT-Nightly/pypi/simple/
+ pip config set global.extra-index-url https://pypi.org/simple/
+ pip config set global.pre true
+
+ - name: Set package version
+ working-directory: sdk/python
+ run: echo '__version__ = "${{ inputs.version }}"' > src/version.py
+
+ - name: Build wheel (Cross-Platform)
+ if: ${{ inputs.useWinML == false }}
+ working-directory: sdk/python
+ run: python -m build --wheel --outdir dist/
+
+ - name: Build wheel (WinML)
+ if: ${{ inputs.useWinML == true }}
+ working-directory: sdk/python
+ run: python -m build --wheel -C winml=true --outdir dist/
+
+ - name: Install built wheel
+ working-directory: sdk/python
+ shell: pwsh
+ run: |
+ $wheel = (Get-ChildItem dist/*.whl | Select-Object -First 1).FullName
+ pip install $wheel
+
+ - name: Install test dependencies
+ run: pip install coverage pytest>=7.0.0 pytest-timeout>=2.1.0
+
+ - name: Run tests
+ working-directory: sdk/python
+ run: python -m pytest test/ -v
+
+ - name: Upload Python packages
+ uses: actions/upload-artifact@v4
+ with:
+ name: python-sdk-${{ inputs.platform }}${{ inputs.useWinML == true && '-winml' || '' }}
+ path: sdk/python/dist/*
+
+ - name: Upload flcore logs
+ uses: actions/upload-artifact@v4
+ if: always()
+ with:
+ name: python-sdk-${{ inputs.platform }}${{ inputs.useWinML == true && '-winml' || '' }}-logs
+ path: sdk/python/logs/**
diff --git a/.github/workflows/foundry-local-sdk-build.yml b/.github/workflows/foundry-local-sdk-build.yml
index 9ac5fe04..13eddf6d 100644
--- a/.github/workflows/foundry-local-sdk-build.yml
+++ b/.github/workflows/foundry-local-sdk-build.yml
@@ -29,6 +29,12 @@ jobs:
version: '0.9.0.${{ github.run_number }}'
platform: 'windows'
secrets: inherit
+ build-python-windows:
+ uses: ./.github/workflows/build-python-steps.yml
+ with:
+ version: '0.9.0.${{ github.run_number }}'
+ platform: 'windows'
+ secrets: inherit
build-rust-windows:
uses: ./.github/workflows/build-rust-steps.yml
with:
@@ -50,6 +56,13 @@ jobs:
platform: 'windows'
useWinML: true
secrets: inherit
+ build-python-windows-WinML:
+ uses: ./.github/workflows/build-python-steps.yml
+ with:
+ version: '0.9.0.${{ github.run_number }}'
+ platform: 'windows'
+ useWinML: true
+ secrets: inherit
build-rust-windows-WinML:
uses: ./.github/workflows/build-rust-steps.yml
with:
@@ -70,6 +83,12 @@ jobs:
version: '0.9.0.${{ github.run_number }}'
platform: 'macos'
secrets: inherit
+ build-python-macos:
+ uses: ./.github/workflows/build-python-steps.yml
+ with:
+ version: '0.9.0.${{ github.run_number }}'
+ platform: 'macos'
+ secrets: inherit
build-rust-macos:
uses: ./.github/workflows/build-rust-steps.yml
with:
diff --git a/sdk/python/.gitignore b/sdk/python/.gitignore
new file mode 100644
index 00000000..543c109e
--- /dev/null
+++ b/sdk/python/.gitignore
@@ -0,0 +1,20 @@
+# Native binaries downloaded from NuGet (per-platform)
+packages/
+
+# Build / egg info
+*.egg-info/
+dist/
+build/
+*.whl
+*.tar.gz
+__pycache__/
+
+# Logs
+logs/
+
+# IDE
+.vscode/
+.idea/
+
+# pytest
+.pytest_cache/
diff --git a/sdk/python/LICENSE.txt b/sdk/python/LICENSE.txt
new file mode 100644
index 00000000..48bc6bb4
--- /dev/null
+++ b/sdk/python/LICENSE.txt
@@ -0,0 +1,21 @@
+MIT License
+
+Copyright (c) Microsoft Corporation
+
+Permission is hereby granted, free of charge, to any person obtaining a copy
+of this software and associated documentation files (the "Software"), to deal
+in the Software without restriction, including without limitation the rights
+to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+copies of the Software, and to permit persons to whom the Software is
+furnished to do so, subject to the following conditions:
+
+The above copyright notice and this permission notice shall be included in all
+copies or substantial portions of the Software.
+
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+SOFTWARE.
diff --git a/sdk/python/README.md b/sdk/python/README.md
new file mode 100644
index 00000000..7cc8b44c
--- /dev/null
+++ b/sdk/python/README.md
@@ -0,0 +1,243 @@
+# Foundry Local Python SDK
+
+The Foundry Local Python SDK provides a Python interface for interacting with local AI models via the Foundry Local Core native library. It allows you to discover, download, load, and run inference on models directly on your local machine — no cloud required.
+
+## Features
+
+- **Model Discovery** – browse and search the model catalog
+- **Model Management** – download, cache, load, and unload models
+- **Chat Completions** – OpenAI-compatible chat API (non-streaming and streaming)
+- **Tool Calling** – function-calling support with chat completions
+- **Audio Transcription** – Whisper-based speech-to-text (non-streaming and streaming)
+- **Built-in Web Service** – optional HTTP endpoint for multi-process scenarios
+- **Native Performance** – ctypes FFI to AOT-compiled Foundry Local Core
+
+## Installation
+
+Two package variants are published — choose the one that matches your target hardware:
+
+| Variant | Package | Native backends |
+|---|---|---|
+| Standard (cross-platform) | `foundry-local-sdk` | CPU / DirectML / CUDA |
+| WinML (Windows only) | `foundry-local-sdk-winml` | Windows ML + all standard backends |
+
+```bash
+# Standard (cross-platform — Linux, macOS, Windows)
+pip install foundry-local-sdk
+
+# WinML (Windows only)
+pip install foundry-local-sdk-winml
+```
+
+Each package installs the correct native binaries (`foundry-local-core`, `onnxruntime-core`, `onnxruntime-genai-core`) as wheel dependencies. They are mutually exclusive — install only one per environment. WinML is auto-detected at runtime: if the WinML package is installed, the SDK automatically enables the Windows App Runtime Bootstrap.
+
+### Building from source
+
+```bash
+cd sdk/python
+
+# Standard wheel
+python -m build --wheel
+
+# WinML wheel (uses the build_backend.py shim)
+python -m build --wheel -C winml=true
+```
+
+For editable installs during development (native packages installed separately via `foundry-local-install`):
+
+```bash
+pip install -e .
+```
+
+### Installing native binaries for development / CI
+
+When working from source the native packages are not pulled in automatically. Use the `foundry-local-install` CLI to install them:
+
+```bash
+# Standard
+foundry-local-install
+
+# WinML (Windows only)
+foundry-local-install --winml
+```
+
+Add `--verbose` to print the resolved binary paths after installation:
+
+```bash
+foundry-local-install --verbose
+foundry-local-install --winml --verbose
+```
+
+> **Note:** The standard and WinML native packages use different PyPI package names (`foundry-local-core` vs `foundry-local-core-winml`) so they can coexist in the same pip index, but they should not be installed in the same Python environment simultaneously.
+
+## Quick Start
+
+```python
+from foundry_local_sdk import Configuration, FoundryLocalManager
+
+# 1. Initialize
+config = Configuration(app_name="MyApp")
+FoundryLocalManager.initialize(config)
+manager = FoundryLocalManager.instance
+
+# 2. Discover models
+catalog = manager.catalog
+models = catalog.list_models()
+for m in models:
+ print(f" {m.alias}")
+
+# 3. Load a model
+model = catalog.get_model("phi-3.5-mini")
+model.load()
+
+# 4. Chat
+client = model.get_chat_client()
+response = client.complete_chat([
+ {"role": "user", "content": "Why is the sky blue?"}
+])
+print(response.choices[0].message.content)
+
+# 5. Cleanup
+model.unload()
+```
+
+## Usage
+
+### Initialization
+
+Create a `Configuration` and initialize the singleton `FoundryLocalManager`.
+
+```python
+from foundry_local_sdk import Configuration, FoundryLocalManager
+from foundry_local_sdk.configuration import LogLevel
+
+config = Configuration(
+ app_name="MyApp",
+ model_cache_dir="/path/to/cache", # optional
+ log_level=LogLevel.INFORMATION, # optional (default: Warning)
+ additional_settings={"Bootstrap": "false"}, # optional
+)
+FoundryLocalManager.initialize(config)
+manager = FoundryLocalManager.instance
+```
+
+### Discovering Models
+
+```python
+catalog = manager.catalog
+
+# List all models in the catalog
+models = catalog.list_models()
+
+# Get a specific model by alias
+model = catalog.get_model("qwen2.5-0.5b")
+
+# Get a specific variant by ID
+variant = catalog.get_model_variant("qwen2.5-0.5b-instruct-generic-cpu:4")
+
+# List locally cached models
+cached = catalog.get_cached_models()
+
+# List currently loaded models
+loaded = catalog.get_loaded_models()
+```
+
+### Loading and Running a Model
+
+```python
+model = catalog.get_model("qwen2.5-0.5b")
+
+# Select a specific variant (optional – defaults to highest-priority cached variant)
+cached = catalog.get_cached_models()
+variant = next(v for v in cached if v.alias == "qwen2.5-0.5b")
+model.select_variant(variant)
+
+# Load into memory
+model.load()
+
+# Non-streaming chat
+client = model.get_chat_client()
+client.settings.temperature = 0.0
+client.settings.max_tokens = 500
+
+result = client.complete_chat([
+ {"role": "user", "content": "What is 7 multiplied by 6?"}
+])
+print(result.choices[0].message.content) # "42"
+
+# Streaming chat
+messages = [{"role": "user", "content": "Tell me a joke"}]
+
+def on_chunk(chunk):
+ delta = chunk.choices[0].delta
+ if delta and delta.content:
+ print(delta.content, end="", flush=True)
+
+client.complete_streaming_chat(messages, on_chunk)
+
+# Unload when done
+model.unload()
+```
+
+### Web Service (Optional)
+
+Start a built-in HTTP server for multi-process access.
+
+```python
+manager.start_web_service()
+print(f"Listening on: {manager.urls}")
+
+# ... use the service ...
+
+manager.stop_web_service()
+```
+
+## API Reference
+
+### Core Classes
+
+| Class | Description |
+|---|---|
+| `Configuration` | SDK configuration (app name, cache dir, log level, web service settings) |
+| `FoundryLocalManager` | Singleton entry point – initialization, catalog access, web service |
+| `Catalog` | Model discovery – listing, lookup by alias/ID, cached/loaded queries |
+| `Model` | Groups variants under one alias – select, load, unload, create clients |
+| `ModelVariant` | Specific model variant – download, cache, load/unload, create clients |
+
+### OpenAI Clients
+
+| Class | Description |
+|---|---|
+| `ChatClient` | Chat completions (non-streaming and streaming) with tool calling |
+| `AudioClient` | Audio transcription (non-streaming and streaming) |
+
+### Internal / Detail
+
+| Class | Description |
+|---|---|
+| `CoreInterop` | ctypes FFI layer to the native Foundry Local Core library |
+| `ModelLoadManager` | Load/unload via core interop or external web service |
+| `ModelInfo` | Pydantic model for catalog entries |
+
+### CLI entry point
+
+| Function | CLI name | Description |
+|---|---|---|
+| `foundry_local_sdk.detail.utils.foundry_local_install` | `foundry-local-install` | Install and verify native binaries (`--winml` for WinML variant) |
+
+> **Migration note:** The function was previously named `verify_native_install`. The public CLI name (`foundry-local-install`) and its behaviour are unchanged; only the Python function name in `foundry_local_sdk.detail.utils` was updated to `foundry_local_install` for consistency.
+
+## Running Tests
+
+```bash
+pip install -r requirements-dev.txt
+python -m pytest test/ -v
+```
+
+See [test/README.md](test/README.md) for detailed test setup and structure.
+
+## Running Examples
+
+```bash
+python examples/chat_completion.py
+```
\ No newline at end of file
diff --git a/sdk/python/build_backend.py b/sdk/python/build_backend.py
new file mode 100644
index 00000000..b4b91a1b
--- /dev/null
+++ b/sdk/python/build_backend.py
@@ -0,0 +1,157 @@
+# -------------------------------------------------------------------------
+# Copyright (c) Microsoft Corporation. All rights reserved.
+# Licensed under the MIT License.
+# --------------------------------------------------------------------------
+"""PEP 517 build backend shim for foundry-local-sdk.
+
+Delegates all hooks to ``setuptools.build_meta`` after optionally
+patching ``pyproject.toml`` and ``requirements.txt`` in-place for the
+WinML variant build.
+
+Usage
+-----
+Standard (default)::
+
+ python -m build --wheel
+
+WinML variant::
+
+ python -m build --wheel -C winml=true
+
+Environment variable fallback (useful in CI pipelines)::
+
+ FOUNDRY_VARIANT=winml python -m build --wheel
+"""
+
+from __future__ import annotations
+
+import contextlib
+import os
+import shutil
+from collections.abc import Generator
+from pathlib import Path
+
+import setuptools.build_meta as _sb
+
+# ---------------------------------------------------------------------------
+# Paths
+# ---------------------------------------------------------------------------
+
+_PROJECT_ROOT = Path(__file__).parent
+_PYPROJECT = _PROJECT_ROOT / "pyproject.toml"
+_REQUIREMENTS = _PROJECT_ROOT / "requirements.txt"
+_REQUIREMENTS_WINML = _PROJECT_ROOT / "requirements-winml.txt"
+
+# The exact string in pyproject.toml to patch for the WinML variant.
+_STANDARD_NAME = 'name = "foundry-local-sdk"'
+_WINML_NAME = 'name = "foundry-local-sdk-winml"'
+
+
+# ---------------------------------------------------------------------------
+# Variant detection
+# ---------------------------------------------------------------------------
+
+
+def _is_winml(config_settings: dict | None) -> bool:
+ """Return True when the WinML variant should be built.
+
+ Checks ``config_settings["winml"]`` first (set via ``-C winml=true``),
+ then falls back to the ``FOUNDRY_VARIANT`` environment variable.
+ """
+ if config_settings and str(config_settings.get("winml", "")).lower() == "true":
+ return True
+ return os.environ.get("FOUNDRY_VARIANT", "").lower() == "winml"
+
+
+# ---------------------------------------------------------------------------
+# In-place patching context manager
+# ---------------------------------------------------------------------------
+
+
+@contextlib.contextmanager
+def _patch_for_winml() -> Generator[None, None, None]:
+ """Temporarily patch ``pyproject.toml`` and ``requirements.txt`` for WinML.
+
+ Both files are restored to their original content in the ``finally``
+ block, even if the build raises an exception.
+ """
+ pyproject_original = _PYPROJECT.read_text(encoding="utf-8")
+ requirements_original = _REQUIREMENTS.read_text(encoding="utf-8")
+ try:
+ # Patch package name (simple string replacement — no TOML writer needed)
+ patched_pyproject = pyproject_original.replace(_STANDARD_NAME, _WINML_NAME, 1)
+ if patched_pyproject == pyproject_original:
+ raise RuntimeError(
+ f"Could not find {_STANDARD_NAME!r} in pyproject.toml — "
+ "WinML name patch failed."
+ )
+ _PYPROJECT.write_text(patched_pyproject, encoding="utf-8")
+
+ # Swap requirements.txt with the WinML variant
+ shutil.copy2(_REQUIREMENTS_WINML, _REQUIREMENTS)
+
+ yield
+ finally:
+ _PYPROJECT.write_text(pyproject_original, encoding="utf-8")
+ _REQUIREMENTS.write_text(requirements_original, encoding="utf-8")
+
+
+# ---------------------------------------------------------------------------
+# PEP 517 hook delegation
+# ---------------------------------------------------------------------------
+
+
+def get_requires_for_build_wheel(config_settings=None):
+ if _is_winml(config_settings):
+ with _patch_for_winml():
+ return _sb.get_requires_for_build_wheel(config_settings)
+ return _sb.get_requires_for_build_wheel(config_settings)
+
+
+def prepare_metadata_for_build_wheel(metadata_directory, config_settings=None):
+ if _is_winml(config_settings):
+ with _patch_for_winml():
+ return _sb.prepare_metadata_for_build_wheel(metadata_directory, config_settings)
+ return _sb.prepare_metadata_for_build_wheel(metadata_directory, config_settings)
+
+
+def build_wheel(wheel_directory, config_settings=None, metadata_directory=None):
+ if _is_winml(config_settings):
+ with _patch_for_winml():
+ return _sb.build_wheel(wheel_directory, config_settings, metadata_directory)
+ return _sb.build_wheel(wheel_directory, config_settings, metadata_directory)
+
+
+def get_requires_for_build_editable(config_settings=None):
+ if _is_winml(config_settings):
+ with _patch_for_winml():
+ return _sb.get_requires_for_build_editable(config_settings)
+ return _sb.get_requires_for_build_editable(config_settings)
+
+
+def prepare_metadata_for_build_editable(metadata_directory, config_settings=None):
+ if _is_winml(config_settings):
+ with _patch_for_winml():
+ return _sb.prepare_metadata_for_build_editable(metadata_directory, config_settings)
+ return _sb.prepare_metadata_for_build_editable(metadata_directory, config_settings)
+
+
+def build_editable(wheel_directory, config_settings=None, metadata_directory=None):
+ if _is_winml(config_settings):
+ with _patch_for_winml():
+ return _sb.build_editable(wheel_directory, config_settings, metadata_directory)
+ return _sb.build_editable(wheel_directory, config_settings, metadata_directory)
+
+
+def get_requires_for_build_sdist(config_settings=None):
+ if _is_winml(config_settings):
+ with _patch_for_winml():
+ return _sb.get_requires_for_build_sdist(config_settings)
+ return _sb.get_requires_for_build_sdist(config_settings)
+
+
+def build_sdist(sdist_directory, config_settings=None):
+ if _is_winml(config_settings):
+ with _patch_for_winml():
+ return _sb.build_sdist(sdist_directory, config_settings)
+ return _sb.build_sdist(sdist_directory, config_settings)
diff --git a/sdk/python/examples/chat_completion.py b/sdk/python/examples/chat_completion.py
new file mode 100644
index 00000000..60eefd5e
--- /dev/null
+++ b/sdk/python/examples/chat_completion.py
@@ -0,0 +1,83 @@
+#!/usr/bin/env python3
+# -------------------------------------------------------------------------
+# Copyright (c) Microsoft Corporation. All rights reserved.
+# Licensed under the MIT License.
+# --------------------------------------------------------------------------
+
+"""Example: Chat completion using Foundry Local Python SDK.
+
+Demonstrates basic chat completion with the Foundry Local runtime,
+including model discovery, loading, and inference.
+"""
+
+from foundry_local_sdk import Configuration, FoundryLocalManager
+
+def main():
+ # 1. Initialize the SDK
+ config = Configuration(app_name="ChatCompletionExample")
+ print("Initializing Foundry Local Manager")
+ FoundryLocalManager.initialize(config)
+ manager = FoundryLocalManager.instance
+
+ # 2. Print available models in the catalog and cache
+ models = manager.catalog.list_models()
+ print("Available models in catalog:")
+ for m in models:
+ print(f" - {m.alias} ({m.id})")
+
+ cached_models = manager.catalog.get_cached_models()
+ print("\nCached models:")
+ for m in cached_models:
+ print(f" - {m.alias} ({m.id})")
+
+ CACHED_MODEL_ALIAS = "qwen2.5-0.5b"
+
+ # 3. Find a model from the cache (+ download if not cached)
+ model = manager.catalog.get_model(CACHED_MODEL_ALIAS)
+ if model is None:
+ print(f"Model '{CACHED_MODEL_ALIAS}' not found in catalog.")
+ print("Available models:")
+ for m in manager.catalog.list_models():
+ print(f" - {m.alias} ({m.id})")
+ return
+
+ if not model.is_cached:
+ print(f"Downloading {model.alias}...")
+ model.download(progress_callback=lambda pct: print(f" {pct:.1f}%", end="\r"))
+ print()
+
+ # 4. Load the model
+ print(f"Loading {model.alias}...", end="")
+ model.load()
+ print("loaded!")
+
+ try:
+ # 5. Create a chat client and send a message
+ client = model.get_chat_client()
+
+ print("\n--- Non-streaming ---")
+ response = client.complete_chat(
+ messages=[{"role": "user", "content": "What is the capital of France? Reply briefly."}]
+ )
+ print(f"Response: {response.choices[0].message.content}")
+
+ # 6. Streaming
+ print("\n--- Streaming ---")
+ for chunk in client.complete_streaming_chat(
+ [{"role": "user", "content": "Tell me a short joke."}]
+ ):
+ if chunk.choices and chunk.choices[0].delta and chunk.choices[0].delta.content:
+ print(chunk.choices[0].delta.content, end="", flush=True)
+ print() # newline after streaming
+
+ except Exception as e:
+ print(f"Error during inference: {e}")
+
+ finally:
+ # 7. Cleanup
+ model.unload()
+ print("\nModel unloaded.")
+
+
+if __name__ == "__main__":
+ main()
diff --git a/sdk/python/pyproject.toml b/sdk/python/pyproject.toml
new file mode 100644
index 00000000..ef93b6f7
--- /dev/null
+++ b/sdk/python/pyproject.toml
@@ -0,0 +1,55 @@
+[build-system]
+requires = ["setuptools>=61.0", "wheel"]
+build-backend = "build_backend"
+backend-path = ["."]
+
+[project]
+name = "foundry-local-sdk"
+dynamic = ["version", "dependencies"]
+description = "Foundry Local Manager Python SDK: Control-plane SDK for Foundry Local."
+readme = "README.md"
+requires-python = ">=3.11"
+license = "MIT"
+license-files = ["LICENSE.txt"]
+authors = [
+ {name = "Microsoft Corporation", email = "foundrylocaldevs@microsoft.com"},
+]
+classifiers = [
+ "Development Status :: 3 - Alpha",
+ "Intended Audience :: Developers",
+ "Topic :: Scientific/Engineering",
+ "Topic :: Scientific/Engineering :: Artificial Intelligence",
+ "Topic :: Software Development",
+ "Topic :: Software Development :: Libraries",
+ "Topic :: Software Development :: Libraries :: Python Modules",
+ "Programming Language :: Python",
+ "Programming Language :: Python :: 3 :: Only",
+ "Programming Language :: Python :: 3.11",
+ "Programming Language :: Python :: 3.12",
+ "Programming Language :: Python :: 3.13",
+]
+
+[project.urls]
+Homepage = "https://github.com/microsoft/Foundry-Local"
+
+[project.scripts]
+foundry-local-install = "foundry_local_sdk.detail.utils:foundry_local_install"
+
+[tool.setuptools.package-dir]
+foundry_local_sdk = "src"
+"foundry_local_sdk.detail" = "src/detail"
+"foundry_local_sdk.openai" = "src/openai"
+
+[tool.setuptools]
+packages = ["foundry_local_sdk", "foundry_local_sdk.detail", "foundry_local_sdk.openai"]
+
+[tool.setuptools.dynamic]
+version = {attr = "foundry_local_sdk.version.__version__"}
+dependencies = {file = ["requirements.txt"]}
+
+[tool.pytest.ini_options]
+testpaths = ["test"]
+python_files = ["test_*.py"]
+python_classes = ["Test*"]
+python_functions = ["test_*"]
+timeout = 60
diff --git a/sdk/python/requirements-dev.txt b/sdk/python/requirements-dev.txt
new file mode 100644
index 00000000..aea40875
--- /dev/null
+++ b/sdk/python/requirements-dev.txt
@@ -0,0 +1,5 @@
+-r requirements.txt
+build
+coverage
+pytest
+pytest-timeout
diff --git a/sdk/python/requirements-winml.txt b/sdk/python/requirements-winml.txt
new file mode 100644
index 00000000..0fb9f9c2
--- /dev/null
+++ b/sdk/python/requirements-winml.txt
@@ -0,0 +1,7 @@
+pydantic>=2.0.0
+requests>=2.32.4
+openai>=2.24.0
+# WinML native binary packages from the ORT-Nightly PyPI feed.
+foundry-local-core-winml
+onnxruntime-core==1.24.3
+onnxruntime-genai-core==0.12.1
\ No newline at end of file
diff --git a/sdk/python/requirements.txt b/sdk/python/requirements.txt
new file mode 100644
index 00000000..801f577d
--- /dev/null
+++ b/sdk/python/requirements.txt
@@ -0,0 +1,7 @@
+pydantic>=2.0.0
+requests>=2.32.4
+openai>=2.24.0
+# Standard native binary packages from the ORT-Nightly PyPI feed.
+foundry-local-core==0.9.0.dev20260327060216
+onnxruntime-core==1.24.3
+onnxruntime-genai-core==0.12.1
\ No newline at end of file
diff --git a/sdk/python/src/__init__.py b/sdk/python/src/__init__.py
new file mode 100644
index 00000000..14534d19
--- /dev/null
+++ b/sdk/python/src/__init__.py
@@ -0,0 +1,23 @@
+# -------------------------------------------------------------------------
+# Copyright (c) Microsoft Corporation. All rights reserved.
+# Licensed under the MIT License.
+# --------------------------------------------------------------------------
+import logging
+import sys
+
+from .configuration import Configuration
+from .foundry_local_manager import FoundryLocalManager
+from .version import __version__
+
+_logger = logging.getLogger(__name__)
+_logger.setLevel(logging.WARNING)
+
+_sc = logging.StreamHandler(stream=sys.stdout)
+_formatter = logging.Formatter(
+ "[foundry-local] | %(asctime)s | %(levelname)-8s | %(message)s", datefmt="%Y-%m-%d %H:%M:%S"
+)
+_sc.setFormatter(_formatter)
+_logger.addHandler(_sc)
+_logger.propagate = False
+
+__all__ = ["Configuration", "FoundryLocalManager", "__version__"]
diff --git a/sdk/python/src/catalog.py b/sdk/python/src/catalog.py
new file mode 100644
index 00000000..767a9f08
--- /dev/null
+++ b/sdk/python/src/catalog.py
@@ -0,0 +1,144 @@
+# -------------------------------------------------------------------------
+# Copyright (c) Microsoft Corporation. All rights reserved.
+# Licensed under the MIT License.
+# --------------------------------------------------------------------------
+
+from __future__ import annotations
+
+import datetime
+import logging
+import threading
+from typing import List, Optional
+from pydantic import TypeAdapter
+
+from .model import Model
+from .model_variant import ModelVariant
+
+from .detail.core_interop import CoreInterop, get_cached_model_ids
+from .detail.model_data_types import ModelInfo
+from .detail.model_load_manager import ModelLoadManager
+from .exception import FoundryLocalException
+
+logger = logging.getLogger(__name__)
+
+class Catalog():
+ """Model catalog for discovering and querying available models.
+
+ Provides methods to list models, look up by alias or ID, and query
+ cached or loaded models. The model list is refreshed every 6 hours.
+ """
+
+ def __init__(self, model_load_manager: ModelLoadManager, core_interop: CoreInterop):
+ """Initialize the Catalog.
+
+ Args:
+ model_load_manager: Manager for loading/unloading models.
+ core_interop: Native interop layer for Foundry Local Core.
+ """
+ self._core_interop = core_interop
+ self._model_load_manager = model_load_manager
+ self._lock = threading.Lock()
+
+ self._models: List[ModelInfo] = []
+ self._model_alias_to_model = {}
+ self._model_id_to_model_variant = {}
+ self._last_fetch = datetime.datetime.min
+
+ response = core_interop.execute_command("get_catalog_name")
+ if response.error is not None:
+ raise FoundryLocalException(f"Failed to get catalog name: {response.error}")
+
+ self.name = response.data
+
+ def _update_models(self):
+ with self._lock:
+ # refresh every 6 hours
+ if (datetime.datetime.now() - self._last_fetch) < datetime.timedelta(hours=6):
+ return
+
+ response = self._core_interop.execute_command("get_model_list")
+ if response.error is not None:
+ raise FoundryLocalException(f"Failed to get model list: {response.error}")
+
+ model_list_json = response.data
+
+ adapter = TypeAdapter(list[ModelInfo])
+ models: List[ModelInfo] = adapter.validate_json(model_list_json)
+
+ self._model_alias_to_model.clear()
+ self._model_id_to_model_variant.clear()
+
+ for model_info in models:
+ variant = ModelVariant(model_info, self._model_load_manager, self._core_interop)
+
+ value = self._model_alias_to_model.get(model_info.alias)
+ if value is None:
+ value = Model(variant, self._core_interop)
+ self._model_alias_to_model[model_info.alias] = value
+ else:
+ value._add_variant(variant)
+
+ self._model_id_to_model_variant[variant.id] = variant
+
+ self._last_fetch = datetime.datetime.now()
+ self._models = models
+
+ def list_models(self) -> List[Model]:
+ """
+ List the available models in the catalog.
+ :return: List of Model instances.
+ """
+ self._update_models()
+ return list(self._model_alias_to_model.values())
+
+ def get_model(self, model_alias: str) -> Optional[Model]:
+ """
+ Lookup a model by its alias.
+ :param model_alias: Model alias.
+ :return: Model if found.
+ """
+ self._update_models()
+ return self._model_alias_to_model.get(model_alias)
+
+ def get_model_variant(self, model_id: str) -> Optional[ModelVariant]:
+ """
+ Lookup a model variant by its unique model id.
+ :param model_id: Model id.
+ :return: Model variant if found.
+ """
+ self._update_models()
+ return self._model_id_to_model_variant.get(model_id)
+
+ def get_cached_models(self) -> List[ModelVariant]:
+ """
+ Get a list of currently downloaded models from the model cache.
+ :return: List of ModelVariant instances.
+ """
+ self._update_models()
+
+ cached_model_ids = get_cached_model_ids(self._core_interop)
+
+ cached_models = []
+ for model_id in cached_model_ids:
+ model_variant = self._model_id_to_model_variant.get(model_id)
+ if model_variant is not None:
+ cached_models.append(model_variant)
+
+ return cached_models
+
+ def get_loaded_models(self) -> List[ModelVariant]:
+ """
+ Get a list of the currently loaded models.
+ :return: List of ModelVariant instances.
+ """
+ self._update_models()
+
+ loaded_model_ids = self._model_load_manager.list_loaded()
+ loaded_models = []
+
+ for model_id in loaded_model_ids:
+ model_variant = self._model_id_to_model_variant.get(model_id)
+ if model_variant is not None:
+ loaded_models.append(model_variant)
+
+ return loaded_models
\ No newline at end of file
diff --git a/sdk/python/src/configuration.py b/sdk/python/src/configuration.py
new file mode 100644
index 00000000..23967efb
--- /dev/null
+++ b/sdk/python/src/configuration.py
@@ -0,0 +1,163 @@
+# -------------------------------------------------------------------------
+# Copyright (c) Microsoft Corporation. All rights reserved.
+# Licensed under the MIT License.
+# --------------------------------------------------------------------------
+
+import logging
+import re
+
+from typing import Optional, Dict
+from urllib.parse import urlparse
+
+from .exception import FoundryLocalException
+
+from .logging_helper import LogLevel
+
+logger = logging.getLogger(__name__)
+
+
+class Configuration:
+ """Configuration for Foundry Local SDK.
+
+ Configuration values:
+ app_name: Your application name. MUST be set to a valid name.
+ foundry_local_core_path: Path to the Foundry Local Core native library.
+ app_data_dir: Application data directory.
+ Default: {home}/.{appname}, where {home} is the user's home directory
+ and {appname} is the app_name value.
+ model_cache_dir: Model cache directory.
+ Default: {appdata}/cache/models, where {appdata} is the app_data_dir value.
+ logs_dir: Log directory.
+ Default: {appdata}/logs
+ log_level: Logging level.
+ Valid values are: Verbose, Debug, Information, Warning, Error, Fatal.
+ Default: LogLevel.WARNING
+ web: Optional configuration for the built-in web service.
+ NOTE: This is not included in all builds.
+ additional_settings: Additional settings that Foundry Local Core can consume.
+ Keys and values are strings.
+ """
+
+ class WebService:
+ """Configuration settings if the optional web service is used."""
+
+ def __init__(
+ self,
+ urls: Optional[str] = None,
+ external_url: Optional[str] = None
+ ):
+ """Initialize WebService configuration.
+
+ Args:
+ urls: Url/s to bind to the web service when
+ FoundryLocalManager.start_web_service() is called.
+ After startup, FoundryLocalManager.urls will contain the actual URL/s
+ the service is listening on.
+ Default: 127.0.0.1:0, which binds to a random ephemeral port.
+ Multiple URLs can be specified as a semi-colon separated list.
+ external_url: If the web service is running in a separate process,
+ it will be accessed using this URI.
+ Both processes should be using the same version of the SDK.
+ If a random port is assigned when creating the web service in the
+ external process the actual port must be provided here.
+ """
+ self.urls = urls
+ self.external_url = external_url
+
+ def __init__(
+ self,
+ app_name: str,
+ foundry_local_core_path: Optional[str] = None,
+ app_data_dir: Optional[str] = None,
+ model_cache_dir: Optional[str] = None,
+ logs_dir: Optional[str] = None,
+ log_level: Optional[LogLevel] = LogLevel.WARNING,
+ web: Optional['Configuration.WebService'] = None,
+ additional_settings: Optional[Dict[str, str]] = None
+ ):
+ """Initialize Configuration.
+
+ Args:
+ app_name: Your application name. MUST be set to a valid name.
+ app_data_dir: Application data directory. Optional.
+ model_cache_dir: Model cache directory. Optional.
+ logs_dir: Log directory. Optional.
+ log_level: Logging level. Default: LogLevel.WARNING
+ web: Optional configuration for the built-in web service.
+ additional_settings: Additional settings dictionary. Optional.
+ """
+ self.app_name = app_name
+ self.foundry_local_core_path = foundry_local_core_path
+ self.app_data_dir = app_data_dir
+ self.model_cache_dir = model_cache_dir
+ self.logs_dir = logs_dir
+ self.log_level = log_level
+ self.web = web
+ self.additional_settings = additional_settings
+
+ # make sure app name only has safe characters as it's used as a directory name
+ self._safe_app_name_chars = re.compile(r'^[A-Za-z0-9._-]+$')
+
+ def validate(self) -> None:
+ """Validate the configuration.
+
+ Raises:
+ FoundryLocalException: If configuration is invalid.
+ """
+ if not self.app_name:
+ raise FoundryLocalException(
+ "Configuration AppName must be set to a valid application name."
+ )
+
+ # Check for invalid filename characters
+ if not bool(self._safe_app_name_chars.match(self.app_name)):
+ raise FoundryLocalException("Configuration AppName value contains invalid characters.")
+
+ if self.web is not None and self.web.external_url is not None:
+ parsed = urlparse(self.web.external_url)
+ if not parsed.port or parsed.port == 0:
+ raise FoundryLocalException("Configuration Web.ExternalUrl has invalid port.")
+
+ def as_dictionary(self) -> Dict[str, str]:
+ """Convert configuration to a dictionary of string key-value pairs.
+
+ Returns:
+ Dictionary containing configuration values as strings.
+
+ Raises:
+ FoundryLocalException: If AppName is not set to a valid value.
+ """
+ if not self.app_name:
+ raise FoundryLocalException(
+ "Configuration AppName must be set to a valid application name."
+ )
+
+ config_values = {
+ "AppName": self.app_name,
+ "LogLevel": str(self.log_level)
+ }
+
+ if self.app_data_dir:
+ config_values["AppDataDir"] = self.app_data_dir
+
+ if self.model_cache_dir:
+ config_values["ModelCacheDir"] = self.model_cache_dir
+
+ if self.logs_dir:
+ config_values["LogsDir"] = self.logs_dir
+
+ if self.foundry_local_core_path:
+ config_values["FoundryLocalCorePath"] = self.foundry_local_core_path
+
+ if self.web is not None:
+ if self.web.urls is not None:
+ config_values["WebServiceUrls"] = self.web.urls
+
+ # Emit any additional settings.
+ if self.additional_settings is not None:
+ for key, value in self.additional_settings.items():
+ if not key:
+ continue # skip empty keys
+ config_values[key] = value if value is not None else ""
+
+ return config_values
diff --git a/sdk/python/src/detail/__init__.py b/sdk/python/src/detail/__init__.py
new file mode 100644
index 00000000..d9a7cbc0
--- /dev/null
+++ b/sdk/python/src/detail/__init__.py
@@ -0,0 +1,25 @@
+# -------------------------------------------------------------------------
+# Copyright (c) Microsoft Corporation. All rights reserved.
+# Licensed under the MIT License.
+# --------------------------------------------------------------------------
+"""This file is required for Python to treat this directory as a package,
+enabling dotted imports such as ``foundry_local_sdk.detail.core_interop``.
+
+The re-exports below are optional convenience aliases so callers can write
+``from foundry_local_sdk.detail import CoreInterop`` instead of importing
+from the individual submodule directly.
+"""
+
+from .core_interop import CoreInterop, InteropRequest, Response
+from .model_data_types import ModelInfo, DeviceType, Runtime
+from .model_load_manager import ModelLoadManager
+
+__all__ = [
+ "CoreInterop",
+ "DeviceType",
+ "InteropRequest",
+ "ModelInfo",
+ "ModelLoadManager",
+ "Response",
+ "Runtime",
+]
diff --git a/sdk/python/src/detail/core_interop.py b/sdk/python/src/detail/core_interop.py
new file mode 100644
index 00000000..7a6bb08c
--- /dev/null
+++ b/sdk/python/src/detail/core_interop.py
@@ -0,0 +1,306 @@
+# -------------------------------------------------------------------------
+# Copyright (c) Microsoft Corporation. All rights reserved.
+# Licensed under the MIT License.
+# --------------------------------------------------------------------------
+
+from __future__ import annotations
+
+import ctypes
+import json
+import logging
+import os
+import sys
+
+from dataclasses import dataclass
+from pathlib import Path
+from typing import Callable, Dict, Optional
+from ..configuration import Configuration
+from ..exception import FoundryLocalException
+from .utils import get_native_binary_paths, NativeBinaryPaths, create_ort_symlinks, _get_ext
+
+logger = logging.getLogger(__name__)
+
+class InteropRequest:
+ """Request payload for a Foundry Local Core command.
+
+ Args:
+ params: Dictionary of key-value string parameters.
+ """
+
+ def __init__(self, params: Dict[str, str] = None):
+ self.params = params or {}
+
+ def to_json(self) -> str:
+ """Serialize the request to a JSON string."""
+ return json.dumps({"Params": self.params}, ensure_ascii=False) # FLC expects UTF-8 encoded JSON (not ascii)
+
+
+class RequestBuffer(ctypes.Structure):
+ """ctypes Structure matching the native ``RequestBuffer`` C struct."""
+
+ _fields_ = [
+ ("Command", ctypes.c_void_p),
+ ("CommandLength", ctypes.c_int),
+ ("Data", ctypes.c_void_p),
+ ("DataLength", ctypes.c_int),
+ ]
+
+
+class ResponseBuffer(ctypes.Structure):
+ """ctypes Structure matching the native ``ResponseBuffer`` C struct."""
+
+ _fields_ = [
+ ("Data", ctypes.c_void_p),
+ ("DataLength", ctypes.c_int),
+ ("Error", ctypes.c_void_p),
+ ("ErrorLength", ctypes.c_int),
+ ]
+
+
+@dataclass
+class Response:
+ """Result from a Foundry Local Core command.
+ Either ``data`` or ``error`` will be set, never both.
+ """
+
+ data: Optional[str] = None
+ error: Optional[str] = None
+
+
+class CallbackHelper:
+ """Internal helper class to convert the callback from ctypes to a str and call the python callback."""
+ @staticmethod
+ def callback(data_ptr, length, self_ptr):
+ self = None
+ try:
+ self = ctypes.cast(self_ptr, ctypes.POINTER(ctypes.py_object)).contents.value
+
+ # convert to a string and pass to the python callback
+ data_bytes = ctypes.string_at(data_ptr, length)
+ data_str = data_bytes.decode('utf-8')
+ self._py_callback(data_str)
+ except Exception as e:
+ if self is not None and self.exception is None:
+ self.exception = e # keep the first only as they are likely all the same
+
+ def __init__(self, py_callback: Callable[[str], None]):
+ self._py_callback = py_callback
+ self.exception = None
+
+
+class CoreInterop:
+ """ctypes FFI layer for the Foundry Local Core native library.
+
+ Provides ``execute_command`` and ``execute_command_with_callback`` to
+ invoke native commands exposed by ``Microsoft.AI.Foundry.Local.Core``.
+ """
+
+ _initialized = False
+ _flcore_library = None
+ _genai_library = None
+ _ort_library = None
+
+ instance = None
+
+ # Callback function for native interop.
+ # This returns a string and its length, and an optional user provided object.
+ CALLBACK_TYPE = ctypes.CFUNCTYPE(None, ctypes.c_void_p, ctypes.c_int, ctypes.c_void_p)
+
+ @staticmethod
+ def _initialize_native_libraries() -> 'NativeBinaryPaths':
+ """Load the native Foundry Local Core library and its dependencies.
+
+ Locates the binaries from the installed Python packages
+ ``foundry-local-core``, ``onnxruntime-core``, and
+ ``onnxruntime-genai-core`` using :func:`get_native_binary_paths`.
+
+ Returns:
+ NativeBinaryPaths with resolved paths to all native binaries.
+ """
+ paths = get_native_binary_paths()
+ if paths is None:
+ raise RuntimeError(
+ "Could not locate native libraries.\n"
+ " Standard variant : pip install foundry-local-sdk\n"
+ " WinML variant : pip install foundry-local-sdk-winml\n"
+ " Dev/CI install : foundry-local-install (or --winml)"
+ )
+
+ logger.info("Native libraries found — Core: %s ORT: %s GenAI: %s",
+ paths.core, paths.ort, paths.genai)
+
+ # Create the onnxruntime.dll symlink on Linux/macOS if needed.
+ # create_ort_symlinks(paths)
+ os.environ["ORT_LIB_PATH"] = str(paths.ort) # For ORT-GENAI to find ORT dependency
+
+ if sys.platform.startswith("win"):
+ # Register every binary directory so the .NET AOT Core library
+ # can resolve sibling DLLs via P/Invoke.
+ for native_dir in paths.all_dirs():
+ os.add_dll_directory(str(native_dir))
+
+ # Explicitly pre-load ORT and GenAI so their symbols are globally
+ # available when Core does P/Invoke lookups at runtime.
+ # On Windows the PATH manipulation above is sufficient; on
+ # Linux/macOS we need RTLD_GLOBAL so that dlopen() within the
+ # Core native code can resolve ORT/GenAI symbols.
+ # ORT must be loaded before GenAI (GenAI depends on ORT).
+ if sys.platform.startswith("win"):
+ CoreInterop._ort_library = ctypes.CDLL(str(paths.ort))
+ CoreInterop._genai_library = ctypes.CDLL(str(paths.genai))
+ else:
+ CoreInterop._ort_library = ctypes.CDLL(str(paths.ort), mode=os.RTLD_GLOBAL)
+ CoreInterop._genai_library = ctypes.CDLL(str(paths.genai), mode=os.RTLD_GLOBAL)
+
+ CoreInterop._flcore_library = ctypes.CDLL(str(paths.core))
+
+ # Set the function signatures
+ lib = CoreInterop._flcore_library
+ lib.execute_command.argtypes = [ctypes.POINTER(RequestBuffer),
+ ctypes.POINTER(ResponseBuffer)]
+ lib.execute_command.restype = None
+
+ lib.free_response.argtypes = [ctypes.POINTER(ResponseBuffer)]
+ lib.free_response.restype = None
+
+ # Set the callback function signature and delegate info
+ lib.execute_command_with_callback.argtypes = [ctypes.POINTER(RequestBuffer),
+ ctypes.POINTER(ResponseBuffer),
+ ctypes.c_void_p, # callback_fn
+ ctypes.c_void_p] # user_data
+ lib.execute_command_with_callback.restype = None
+
+ return paths
+
+ @staticmethod
+ def _to_c_buffer(s: str):
+ # Helper: encodes strings into unmanaged memory
+ if s is None:
+ return ctypes.c_void_p(0), 0, None
+
+ buf = s.encode("utf-8")
+ ptr = ctypes.create_string_buffer(buf) # keeps memory alive in Python
+ return ctypes.cast(ptr, ctypes.c_void_p), len(buf), ptr
+
+ def __init__(self, config: Configuration):
+ if not CoreInterop._initialized:
+ paths = CoreInterop._initialize_native_libraries()
+ CoreInterop._initialized = True
+
+ # Pass the full path to the Core DLL so the native layer can
+ # discover sibling DLLs via Path.GetDirectoryName(FoundryLocalCorePath).
+ flcore_lib_name = f"Microsoft.AI.Foundry.Local.Core{_get_ext()}"
+ config.foundry_local_core_path = str(paths.core_dir / flcore_lib_name)
+
+ # Pass ORT and GenAI library paths so the C# native library resolver
+ # can search their directories (they may be in separate pip packages).
+ if config.additional_settings is None:
+ config.additional_settings = {}
+ config.additional_settings["OrtLibraryPath"] = str(paths.ort)
+ config.additional_settings["OrtGenAILibraryPath"] = str(paths.genai)
+
+ # Auto-detect WinML Bootstrap: if the Bootstrap DLL is present
+ # in the native binaries directory and the user hasn't explicitly
+ # set the Bootstrap config, enable it automatically.
+ if sys.platform.startswith("win"):
+ bootstrap_dll = paths.core_dir / "Microsoft.WindowsAppRuntime.Bootstrap.dll"
+ if bootstrap_dll.exists():
+ if config.additional_settings is None:
+ config.additional_settings = {}
+ if "Bootstrap" not in config.additional_settings:
+ logger.info("WinML Bootstrap DLL detected — enabling Bootstrap")
+ config.additional_settings["Bootstrap"] = "true"
+
+ request = InteropRequest(params=config.as_dictionary())
+ response = self.execute_command("initialize", request)
+ if response.error is not None:
+ raise FoundryLocalException(f"Failed to initialize Foundry.Local.Core: {response.error}")
+
+ logger.info("Foundry.Local.Core initialized successfully: %s", response.data)
+
+ def _execute_command(self, command: str, interop_request: InteropRequest = None,
+ callback: CoreInterop.CALLBACK_TYPE = None):
+ cmd_ptr, cmd_len, cmd_buf = CoreInterop._to_c_buffer(command)
+ data_ptr, data_len, data_buf = CoreInterop._to_c_buffer(interop_request.to_json() if interop_request else None)
+
+ req = RequestBuffer(Command=cmd_ptr, CommandLength=cmd_len, Data=data_ptr, DataLength=data_len)
+ resp = ResponseBuffer()
+ lib = CoreInterop._flcore_library
+
+ if (callback is not None):
+ # If a callback is provided, use the execute_command_with_callback method
+ # We need a helper to do the initial conversion from ctypes to Python and pass it through to the
+ # provided callback function
+ callback_helper = CallbackHelper(callback)
+ callback_py_obj = ctypes.py_object(callback_helper)
+ callback_helper_ptr = ctypes.cast(ctypes.pointer(callback_py_obj), ctypes.c_void_p)
+ callback_fn = CoreInterop.CALLBACK_TYPE(CallbackHelper.callback)
+
+ lib.execute_command_with_callback(ctypes.byref(req), ctypes.byref(resp), callback_fn, callback_helper_ptr)
+
+ if callback_helper.exception is not None:
+ raise callback_helper.exception
+ else:
+ lib.execute_command(ctypes.byref(req), ctypes.byref(resp))
+
+ req = None # Free Python reference to request
+
+ response_str = ctypes.string_at(resp.Data, resp.DataLength).decode("utf-8") if resp.Data else None
+ error_str = ctypes.string_at(resp.Error, resp.ErrorLength).decode("utf-8") if resp.Error else None
+
+ # C# owns the memory in the response so we need to free it explicitly
+ lib.free_response(resp)
+
+ return Response(data=response_str, error=error_str)
+
+ def execute_command(self, command_name: str, command_input: Optional[InteropRequest] = None) -> Response:
+ """Execute a command synchronously.
+
+ Args:
+ command_name: The native command name (e.g. ``"get_model_list"``).
+ command_input: Optional request parameters.
+
+ Returns:
+ A ``Response`` with ``data`` on success or ``error`` on failure.
+ """
+ logger.debug("Executing command: %s Input: %s", command_name,
+ command_input.params if command_input else None)
+
+ response = self._execute_command(command_name, command_input)
+ return response
+
+ def execute_command_with_callback(self, command_name: str, command_input: Optional[InteropRequest],
+ callback: Callable[[str], None]) -> Response:
+ """Execute a command with a streaming callback.
+
+ The ``callback`` receives incremental string data from the native layer
+ (e.g. streaming chat tokens or download progress).
+
+ Args:
+ command_name: The native command name.
+ command_input: Optional request parameters.
+ callback: Called with each incremental string response.
+
+ Returns:
+ A ``Response`` with ``data`` on success or ``error`` on failure.
+ """
+ logger.debug("Executing command with callback: %s Input: %s", command_name,
+ command_input.params if command_input else None)
+ response = self._execute_command(command_name, command_input, callback)
+ return response
+
+
+def get_cached_model_ids(core_interop: CoreInterop) -> list[str]:
+ """Get the list of models that have been downloaded and are cached."""
+
+ response = core_interop.execute_command("get_cached_models")
+ if response.error is not None:
+ raise FoundryLocalException(f"Failed to get cached models: {response.error}")
+
+ try:
+ model_ids = json.loads(response.data)
+ except json.JSONDecodeError as e:
+ raise FoundryLocalException(f"Failed to decode JSON response: Response was: {response.data}") from e
+
+ return model_ids
+
diff --git a/sdk/python/src/detail/model_data_types.py b/sdk/python/src/detail/model_data_types.py
new file mode 100644
index 00000000..b8b9e8d6
--- /dev/null
+++ b/sdk/python/src/detail/model_data_types.py
@@ -0,0 +1,76 @@
+# -------------------------------------------------------------------------
+# Copyright (c) Microsoft Corporation. All rights reserved.
+# Licensed under the MIT License.
+# --------------------------------------------------------------------------
+
+from typing import Optional, List
+from pydantic import BaseModel, Field
+
+from enum import StrEnum
+
+# ---------- ENUMS ----------
+class DeviceType(StrEnum):
+ """Device types supported by model variants."""
+
+ CPU = "CPU"
+ GPU = "GPU"
+ NPU = "NPU"
+
+# ---------- DATA MODELS ----------
+
+class PromptTemplate(BaseModel):
+ """Prompt template strings for system, user, assistant, and raw prompt roles."""
+
+ system: Optional[str] = Field(default=None, alias="system")
+ user: Optional[str] = Field(default=None, alias="user")
+ assistant: Optional[str] = Field(default=None, alias="assistant")
+ prompt: Optional[str] = Field(default=None, alias="prompt")
+
+
+class Runtime(BaseModel):
+ """Runtime configuration specifying the device type and execution provider."""
+
+ device_type: DeviceType = Field(alias="deviceType")
+ execution_provider: str = Field(alias="executionProvider")
+
+
+class Parameter(BaseModel):
+ """A named parameter with an optional string value."""
+
+ name: str
+ value: Optional[str] = None
+
+
+class ModelSettings(BaseModel):
+ """Model-specific settings containing a list of parameters."""
+
+ parameters: Optional[List[Parameter]] = Field(default=None, alias="parameters")
+
+
+class ModelInfo(BaseModel):
+ """Catalog metadata for a single model variant.
+
+ Fields are populated from the JSON response of the ``get_model_list`` command.
+ """
+
+ id: str = Field(alias="id", description="Unique identifier of the model. Generally :")
+ name: str = Field(alias="name", description="Model variant name")
+ version: int = Field(alias="version")
+ alias: str = Field(..., description="Alias of the model")
+ display_name: Optional[str] = Field(alias="displayName")
+ provider_type: str = Field(alias="providerType")
+ uri: str = Field(alias="uri")
+ model_type: str = Field(alias="modelType")
+ prompt_template: Optional[PromptTemplate] = Field(default=None, alias="promptTemplate")
+ publisher: Optional[str] = Field(alias="publisher")
+ model_settings: Optional[ModelSettings] = Field(default=None, alias="modelSettings")
+ license: Optional[str] = Field(alias="license")
+ license_description: Optional[str] = Field(alias="licenseDescription")
+ cached: bool = Field(alias="cached")
+ task: Optional[str] = Field(alias="task")
+ runtime: Optional[Runtime] = Field(alias="runtime")
+ file_size_mb: Optional[int] = Field(alias="fileSizeMb")
+ supports_tool_calling: Optional[bool] = Field(alias="supportsToolCalling")
+ max_output_tokens: Optional[int] = Field(alias="maxOutputTokens")
+ min_fl_version: Optional[str] = Field(alias="minFLVersion")
+ created_at_unix: int = Field(alias="createdAt")
diff --git a/sdk/python/src/detail/model_load_manager.py b/sdk/python/src/detail/model_load_manager.py
new file mode 100644
index 00000000..8ffd087a
--- /dev/null
+++ b/sdk/python/src/detail/model_load_manager.py
@@ -0,0 +1,166 @@
+# -------------------------------------------------------------------------
+# Copyright (c) Microsoft Corporation. All rights reserved.
+# Licensed under the MIT License.
+# --------------------------------------------------------------------------
+from __future__ import annotations
+
+import json
+import logging
+import requests
+
+from typing import List
+from urllib.parse import quote
+
+from ..exception import FoundryLocalException
+from ..version import __version__ as sdk_version
+from .core_interop import CoreInterop, InteropRequest
+
+logger = logging.getLogger(__name__)
+
+
+class ModelLoadManager:
+ """Manages loading and unloading of models in Foundry Local.
+
+ Can operate in two modes: direct interop with Foundry Local Core, or via
+ an external web service if the configuration provides a
+ ``WebServiceExternalUrl`` value.
+ """
+
+ _headers = {"user-agent": f"foundry-local-python-sdk/{sdk_version}"}
+
+ def __init__(self, core_interop: CoreInterop, external_service_url: str = None):
+ self._core_interop = core_interop
+ self._external_service_url = external_service_url
+
+ def load(self, model_id: str) -> None:
+ """
+ Load a model by its ID.
+
+ This method loads a model either via direct interop with Foundry Local Core
+ or, if an external service URL is configured, by calling the external web
+ service.
+
+ :param model_id: The ID of the model to load.
+ :raises FoundryLocalException: If the model cannot be loaded successfully,
+ for example due to an error returned from Foundry Local Core or from
+ the external service, including underlying HTTP or network errors when
+ communicating with the external service.
+ """
+ if self._external_service_url:
+ self._web_load_model(model_id)
+ return
+
+ request = InteropRequest({"Model": model_id})
+ response = self._core_interop.execute_command("load_model", request)
+ if response.error is not None:
+ raise FoundryLocalException(f"Failed to load model {model_id}: {response.error}")
+
+ def unload(self, model_id: str) -> None:
+ """
+ Unload a model by its ID.
+ :param model_id: The ID of the model to unload.
+ """
+ if self._external_service_url:
+ self._web_unload_model(model_id)
+ return
+
+ request = InteropRequest({"Model": model_id})
+ response = self._core_interop.execute_command("unload_model", request)
+ if response.error is not None:
+ raise FoundryLocalException(f"Failed to unload model {model_id}: {response.error}")
+
+ def list_loaded(self) -> list[str]:
+ """
+ List loaded models.
+ :return: List of loaded model IDs
+ """
+ if self._external_service_url:
+ return self._web_list_loaded_models()
+
+ response = self._core_interop.execute_command("list_loaded_models")
+ if response.error is not None:
+ raise FoundryLocalException(f"Failed to list loaded models: {response.error}")
+
+ try:
+ model_ids = json.loads(response.data)
+ except json.JSONDecodeError as e:
+ raise FoundryLocalException(f"Failed to decode JSON response: Response was: {response.data}") from e
+
+ return model_ids
+
+ def _web_list_loaded_models(self) -> List[str]:
+ try:
+ response = requests.get(f"{self._external_service_url}/models/loaded", headers=self._headers, timeout=10)
+
+ if not response.ok:
+ raise FoundryLocalException(
+ f"Error listing loaded models from {self._external_service_url}: {response.reason}"
+ )
+
+ content = response.text
+ logger.debug("Loaded models json from %s: %s", self._external_service_url, content)
+
+ model_list = json.loads(content)
+ return model_list if model_list is not None else []
+ except requests.RequestException as e:
+ raise FoundryLocalException(
+ f"HTTP request failed when listing loaded models from {self._external_service_url}"
+ ) from e
+ except json.JSONDecodeError as e:
+ raise FoundryLocalException(f"Failed to decode JSON response: Response was: {content}") from e
+
+ def _web_load_model(self, model_id: str) -> None:
+ """
+ Load a model via the external web service.
+
+ :param model_id: The ID of the model to load
+ :raises FoundryLocalException: If the HTTP request fails or response is invalid
+ """
+ try:
+ encoded_model_id = quote(model_id)
+ url = f"{self._external_service_url}/models/load/{encoded_model_id}"
+
+ # Future: add query params like load timeout
+ # query_params = {
+ # # "timeout": "30"
+ # }
+ # response = requests.get(url, params=query_params)
+
+ response = requests.get(url, headers=self._headers, timeout=10)
+
+ if not response.ok:
+ raise FoundryLocalException(
+ f"Error loading model {model_id} from {self._external_service_url}: "
+ f"{response.reason}"
+ )
+
+ content = response.text
+ logger.info("Model %s loaded successfully from %s: %s",
+ model_id, self._external_service_url, content)
+
+ except requests.RequestException as e:
+ raise FoundryLocalException(
+ f"HTTP request failed when loading model {model_id} from {self._external_service_url}: {e}"
+ ) from e
+
+ def _web_unload_model(self, model_id: str) -> None:
+ try:
+ encoded_model_id = quote(model_id)
+ url = f"{self._external_service_url}/models/unload/{encoded_model_id}"
+
+ response = requests.get(url, headers=self._headers, timeout=10)
+
+ if not response.ok:
+ raise FoundryLocalException(
+ f"Error unloading model {model_id} from {self._external_service_url}: "
+ f"{response.reason}"
+ )
+
+ content = response.text
+ logger.info("Model %s unloaded successfully from %s: %s",
+ model_id, self._external_service_url, content)
+
+ except requests.RequestException as e:
+ raise FoundryLocalException(
+ f"HTTP request failed when unloading model {model_id} from {self._external_service_url}: {e}"
+ ) from e
diff --git a/sdk/python/src/detail/utils.py b/sdk/python/src/detail/utils.py
new file mode 100644
index 00000000..5a054610
--- /dev/null
+++ b/sdk/python/src/detail/utils.py
@@ -0,0 +1,294 @@
+# -------------------------------------------------------------------------
+# Copyright (c) Microsoft Corporation. All rights reserved.
+# Licensed under the MIT License.
+# --------------------------------------------------------------------------
+"""Utility functions for the Foundry Local SDK.
+
+Includes native library locator logic and helper functions used by
+other SDK modules.
+"""
+
+from __future__ import annotations
+
+import argparse
+import importlib.util
+import json
+import logging
+import os
+import sys
+
+from dataclasses import dataclass
+from pathlib import Path
+
+from enum import StrEnum
+from ..exception import FoundryLocalException
+
+logger = logging.getLogger(__name__)
+
+
+# ---------------------------------------------------------------------------
+# Platform helpers
+# ---------------------------------------------------------------------------
+
+# Maps Python sys.platform to native shared library extension
+EXT_MAP: dict[str, str] = {
+ "win32": ".dll",
+ "linux": ".so",
+ "darwin": ".dylib",
+}
+
+
+def _get_ext() -> str:
+ """Get the native library file extension for the current platform."""
+ for plat_prefix, ext in EXT_MAP.items():
+ if sys.platform.startswith(plat_prefix):
+ return ext
+ raise RuntimeError(f"Unsupported platform: {sys.platform}")
+
+
+# ---------------------------------------------------------------------------
+# Package-based binary discovery
+# ---------------------------------------------------------------------------
+
+# On Linux/macOS the ORT shared libraries carry the "lib" prefix while the
+# Core library refers to them without it — a symlink "onnxruntime.dll" →
+# "libonnxruntime.so/.dylib" is created to bridge the gap (see below).
+_ORT_PREFIX = "" if sys.platform == "win32" else "lib"
+
+
+def _native_binary_names() -> tuple[str, str, str]:
+ """Return the expected native binary filenames for the current platform."""
+ ext = _get_ext()
+ return (
+ f"Microsoft.AI.Foundry.Local.Core{ext}",
+ f"{_ORT_PREFIX}onnxruntime{ext}",
+ f"{_ORT_PREFIX}onnxruntime-genai{ext}",
+ )
+
+
+def _find_file_in_package(package_name: str, filename: str) -> Path | None:
+ """Locate a native binary *filename* inside an installed Python package.
+
+ Searches the package root and common sub-directories (``capi/``,
+ ``native/``, ``lib/``). Falls back to a recursive ``rglob`` scan of
+ the entire package tree when none of the quick paths match.
+
+ Args:
+ package_name: The PyPI package name (hyphens or underscores accepted;
+ e.g. ``"onnxruntime-genai-core"`` or ``"onnxruntime_genai_core"``).
+ filename: The filename to look for (e.g. ``"onnxruntime-genai.dll"``).
+
+ Returns:
+ Absolute ``Path`` to the file, or ``None`` if not found.
+ """
+ import_name = package_name.replace("-", "_")
+ spec = importlib.util.find_spec(import_name)
+ if spec is None or spec.origin is None:
+ return None
+
+ pkg_root = Path(spec.origin).parent
+
+ # Quick checks for well-known sub-directories first
+ for candidate_dir in (pkg_root, pkg_root / "capi", pkg_root / "native", pkg_root / "lib", pkg_root / "bin"):
+ candidate = candidate_dir / filename
+ if candidate.exists():
+ return candidate
+
+ # Recursive fallback
+ for match in pkg_root.rglob(filename):
+ return match
+
+ return None
+
+
+@dataclass
+class NativeBinaryPaths:
+ """Resolved paths to the three native binaries required by the SDK."""
+
+ core: Path
+ ort: Path
+ genai: Path
+
+ @property
+ def core_dir(self) -> Path:
+ """Directory that contains the Core binary."""
+ return self.core.parent
+
+ @property
+ def ort_dir(self) -> Path:
+ """Directory that contains the OnnxRuntime binary."""
+ return self.ort.parent
+
+ @property
+ def genai_dir(self) -> Path:
+ """Directory that contains the OnnxRuntimeGenAI binary."""
+ return self.genai.parent
+
+ def all_dirs(self) -> list[Path]:
+ """Return a deduplicated list of directories that contain the binaries."""
+ seen: list[Path] = []
+ for d in (self.core_dir, self.ort_dir, self.genai_dir):
+ if d not in seen:
+ seen.append(d)
+ return seen
+
+
+def get_native_binary_paths() -> NativeBinaryPaths | None:
+ """Locate native binaries from installed Python packages.
+
+ Returns:
+ A :class:`NativeBinaryPaths` instance if all three binaries were
+ found, or ``None`` if any is missing.
+ """
+ core_name, ort_name, genai_name = _native_binary_names()
+
+ # Probe WinML packages first; fall back to standard if not installed.
+ core_path = _find_file_in_package("foundry-local-core-winml", core_name) or _find_file_in_package("foundry-local-core", core_name)
+ ort_path = _find_file_in_package("onnxruntime-core", ort_name)
+ genai_path = _find_file_in_package("onnxruntime-genai-core", genai_name)
+
+ if core_path and ort_path and genai_path:
+ return NativeBinaryPaths(core=core_path, ort=ort_path, genai=genai_path)
+
+ return None
+
+def create_ort_symlinks(paths: NativeBinaryPaths) -> None:
+ """Create compatibility symlinks for ORT in the Core library directory on Linux/macOS.
+
+ Workaround for ORT issue https://github.com/microsoft/onnxruntime/issues/27263.
+
+ On Linux/macOS the native packages ship ORT binaries with a ``lib`` prefix
+ (e.g. ``libonnxruntime.dylib``) in their own package directories, while the
+ .NET AOT Core library P/Invokes ``onnxruntime.dylib`` / ``onnxruntime-genai.dylib``
+ and searches its *own* directory first (matching the JS SDK behaviour where all
+ binaries live in a single ``coreDir``).
+
+ This function creates ``onnxruntime{ext}`` and ``onnxruntime-genai{ext}`` symlinks
+ in ``paths.core_dir`` pointing at the absolute paths of the respective binaries so
+ the Core DLL can resolve them via ``dlopen`` without needing ``DYLD_LIBRARY_PATH``.
+ """
+ if sys.platform == "win32":
+ return
+
+ ext = ".dylib" if sys.platform == "darwin" else ".so"
+
+ # Pairs of (actual binary path, link stem to create in core_dir)
+ links: list[tuple[Path, str]] = [
+ (paths.ort, "onnxruntime"),
+ (paths.genai, "onnxruntime-genai"),
+ ]
+
+ for src_path, link_stem in links:
+ link_path = paths.core_dir / f"{link_stem}{ext}"
+ if not link_path.exists():
+ if src_path.exists():
+ os.symlink(str(src_path), link_path)
+ logger.info("Created symlink: %s -> %s", link_path, src_path)
+ else:
+ logger.warning("Cannot create symlink %s: source %s not found", link_path, src_path)
+
+ # Create a libonnxruntime symlink in genai_dir pointing to the real ORT
+ # binary so the dynamic linker can resolve GenAI's dependency.
+ if paths.genai_dir != paths.ort_dir:
+ ort_link_in_genai = paths.genai_dir / paths.ort.name
+ if not ort_link_in_genai.exists():
+ if paths.ort.exists():
+ os.symlink(str(paths.ort), ort_link_in_genai)
+ logger.info("Created symlink: %s -> %s", ort_link_in_genai, paths.ort)
+ else:
+ logger.warning("Cannot create symlink %s: source %s not found",
+ ort_link_in_genai, paths.ort)
+
+
+# ---------------------------------------------------------------------------
+# CLI entry point for verifying native binary installation
+# ---------------------------------------------------------------------------
+
+
+def foundry_local_install(args: list[str] | None = None) -> None:
+ """CLI entry point for installing and verifying native binaries.
+
+ Usage::
+
+ foundry-local-install [--winml] [--verbose]
+
+ Installs the platform-specific native libraries required by the SDK via
+ pip, then verifies they can be located. Use ``--winml`` to install the
+ WinML variants of the native packages (Windows only).
+
+ Standard variant (default)::
+
+ foundry-local-install
+ # installs: foundry-local-core, onnxruntime-core, onnxruntime-genai-core
+
+ WinML variant::
+
+ foundry-local-install --winml
+ # installs: foundry-local-core-winml, onnxruntime-core, onnxruntime-genai-core
+ """
+ import subprocess
+
+ parser = argparse.ArgumentParser(
+ description=(
+ "Install and verify the platform-specific native libraries required by "
+ "the Foundry Local SDK via pip. Use --winml to install the WinML variants "
+ "(Windows only). Without --winml the standard cross-platform packages are installed."
+ ),
+ prog="foundry-local-install",
+ )
+ parser.add_argument(
+ "--winml",
+ action="store_true",
+ help=(
+ "Install WinML native package (foundry-local-core-winml) "
+ "instead of the standard cross-platform package."
+ ),
+ )
+ parser.add_argument(
+ "--verbose",
+ action="store_true",
+ help="Print the resolved path for each binary after installation.",
+ )
+ parsed = parser.parse_args(args)
+
+ if parsed.winml:
+ variant = "WinML"
+ packages = ["foundry-local-core-winml", "onnxruntime-core", "onnxruntime-genai-core"]
+ else:
+ variant = "standard"
+ packages = ["foundry-local-core", "onnxruntime-core", "onnxruntime-genai-core"]
+
+ print(f"[foundry-local] Installing {variant} native packages: {', '.join(packages)}")
+ subprocess.check_call([sys.executable, "-m", "pip", "install", *packages])
+
+ paths = get_native_binary_paths()
+ if paths is None:
+ core_name, ort_name, genai_name = _native_binary_names()
+ missing: list[str] = []
+ if parsed.winml:
+ if _find_file_in_package("foundry-local-core-winml", core_name) is None:
+ missing.append("foundry-local-core-winml")
+ else:
+ if _find_file_in_package("foundry-local-core", core_name) is None:
+ missing.append("foundry-local-core")
+ if _find_file_in_package("onnxruntime-core", ort_name) is None:
+ missing.append("onnxruntime-core")
+ if _find_file_in_package("onnxruntime-genai-core", genai_name) is None:
+ missing.append("onnxruntime-genai-core")
+ print(
+ "[foundry-local] ERROR: Could not locate native binaries after installation. "
+ f"Missing: {', '.join(missing)}",
+ file=sys.stderr,
+ )
+ hint = "pip install foundry-local-sdk-winml" if parsed.winml else "pip install foundry-local-sdk"
+ print(f" Try: {hint}", file=sys.stderr)
+ sys.exit(1)
+
+ print(f"[foundry-local] {variant.capitalize()} native libraries installed and verified.")
+ if parsed.verbose:
+ print(f" Core : {paths.core}")
+ print(f" ORT : {paths.ort}")
+ print(f" GenAI : {paths.genai}")
+
+
+
diff --git a/sdk/python/src/exception.py b/sdk/python/src/exception.py
new file mode 100644
index 00000000..0cff6a90
--- /dev/null
+++ b/sdk/python/src/exception.py
@@ -0,0 +1,7 @@
+# -------------------------------------------------------------------------
+# Copyright (c) Microsoft Corporation. All rights reserved.
+# Licensed under the MIT License.
+# --------------------------------------------------------------------------
+
+class FoundryLocalException(Exception):
+ """Base exception for Foundry Local SDK errors."""
diff --git a/sdk/python/src/foundry_local_manager.py b/sdk/python/src/foundry_local_manager.py
new file mode 100644
index 00000000..4486eaf1
--- /dev/null
+++ b/sdk/python/src/foundry_local_manager.py
@@ -0,0 +1,118 @@
+# -------------------------------------------------------------------------
+# Copyright (c) Microsoft Corporation. All rights reserved.
+# Licensed under the MIT License.
+# --------------------------------------------------------------------------
+
+from __future__ import annotations
+
+import json
+import logging
+import threading
+
+from .catalog import Catalog
+from .configuration import Configuration
+from .logging_helper import set_default_logger_severity
+from .detail.core_interop import CoreInterop
+from .detail.model_load_manager import ModelLoadManager
+from .exception import FoundryLocalException
+
+logger = logging.getLogger(__name__)
+
+
+class FoundryLocalManager:
+ """Singleton manager for Foundry Local SDK operations.
+
+ Call ``FoundryLocalManager.initialize(config)`` once at startup, then access
+ the singleton via ``FoundryLocalManager.instance``.
+
+ Attributes:
+ instance: The singleton ``FoundryLocalManager`` instance (set after ``initialize``).
+ catalog: The model ``Catalog`` for discovering and managing models.
+ urls: Bound URL(s) after ``start_web_service()`` is called, or ``None``.
+ """
+
+ _lock = threading.Lock()
+ instance: FoundryLocalManager = None
+
+ @staticmethod
+ def initialize(config: Configuration):
+ """Initialize the Foundry Local SDK with the given configuration.
+
+ This method must be called before using any other part of the SDK.
+
+ Args:
+ config: Configuration object for the SDK.
+ """
+ # Delegate singleton creation to the constructor, which enforces
+ # the singleton invariant under a lock and sets `instance`.
+ FoundryLocalManager(config)
+
+ def __init__(self, config: Configuration):
+ # Enforce singleton creation under a class-level lock and ensure
+ # that `FoundryLocalManager.instance` is set exactly once.
+ with FoundryLocalManager._lock:
+ if FoundryLocalManager.instance is not None:
+ raise FoundryLocalException(
+ "FoundryLocalManager is a singleton and has already been initialized."
+ )
+ config.validate()
+ self.config = config
+ self._initialize()
+ FoundryLocalManager.instance = self
+
+ self.urls = None
+
+ def _initialize(self):
+ set_default_logger_severity(self.config.log_level)
+
+ external_service_url = self.config.web.external_url if self.config.web else None
+
+ self._core_interop = CoreInterop(self.config)
+ self._model_load_manager = ModelLoadManager(self._core_interop, external_service_url)
+ self.catalog = Catalog(self._model_load_manager, self._core_interop)
+
+ def ensure_eps_downloaded(self) -> None:
+ """Ensure execution providers are downloaded and registered (synchronous).
+ Only relevant when using WinML.
+
+ Raises:
+ FoundryLocalException: If execution provider download fails.
+ """
+ result = self._core_interop.execute_command("ensure_eps_downloaded")
+
+ if result.error is not None:
+ raise FoundryLocalException(f"Error ensuring execution providers downloaded: {result.error}")
+
+ def start_web_service(self):
+ """Start the optional web service.
+
+ If provided, the service will be bound to the value of Configuration.web.urls.
+ The default of http://127.0.0.1:0 will be used otherwise, which binds to a random ephemeral port.
+
+ FoundryLocalManager.urls will be updated with the actual URL/s the service is listening on.
+ """
+ with FoundryLocalManager._lock:
+ response = self._core_interop.execute_command("start_service")
+
+ if response.error is not None:
+ raise FoundryLocalException(f"Error starting web service: {response.error}")
+
+ bound_urls = json.loads(response.data)
+ if bound_urls is None or len(bound_urls) == 0:
+ raise FoundryLocalException("Failed to get bound URLs from web service start response.")
+
+ self.urls = bound_urls
+
+ def stop_web_service(self):
+ """Stop the optional web service."""
+
+ with FoundryLocalManager._lock:
+ if self.urls is None:
+ raise FoundryLocalException("Web service is not running.")
+
+ response = self._core_interop.execute_command("stop_service")
+
+ if response.error is not None:
+ raise FoundryLocalException(f"Error stopping web service: {response.error}")
+
+ self.urls = None
diff --git a/sdk/python/src/imodel.py b/sdk/python/src/imodel.py
new file mode 100644
index 00000000..a092b98e
--- /dev/null
+++ b/sdk/python/src/imodel.py
@@ -0,0 +1,91 @@
+# -------------------------------------------------------------------------
+# Copyright (c) Microsoft Corporation. All rights reserved.
+# Licensed under the MIT License.
+# --------------------------------------------------------------------------
+from __future__ import annotations
+
+from abc import ABC, abstractmethod
+from typing import Callable, Optional
+
+from .openai.chat_client import ChatClient
+from .openai.audio_client import AudioClient
+
+class IModel(ABC):
+ """Abstract interface for a model that can be downloaded, loaded, and used for inference."""
+
+ @property
+ @abstractmethod
+ def id(self) -> str:
+ """Unique model id."""
+ pass
+
+ @property
+ @abstractmethod
+ def alias(self) -> str:
+ """Model alias."""
+ pass
+
+ @property
+ @abstractmethod
+ def is_cached(self) -> bool:
+ """True if the model is present in the local cache."""
+ pass
+
+ @property
+ @abstractmethod
+ def is_loaded(self) -> bool:
+ """True if the model is loaded into memory."""
+ pass
+
+ @abstractmethod
+ def download(self, progress_callback: Callable[[float], None] = None) -> None:
+ """
+ Download the model to local cache if not already present.
+ :param progress_callback: Optional callback function for download progress as a percentage (0.0 to 100.0).
+ """
+ pass
+
+ @abstractmethod
+ def get_path(self) -> str:
+ """
+ Gets the model path if cached.
+ :return: Path of model directory.
+ """
+ pass
+
+ @abstractmethod
+ def load(self) -> None:
+ """
+ Load the model into memory if not already loaded.
+ """
+ pass
+
+ @abstractmethod
+ def remove_from_cache(self) -> None:
+ """
+ Remove the model from the local cache.
+ """
+ pass
+
+ @abstractmethod
+ def unload(self) -> None:
+ """
+ Unload the model if loaded.
+ """
+ pass
+
+ @abstractmethod
+ def get_chat_client(self) -> ChatClient:
+ """
+ Get an OpenAI API based ChatClient.
+ :return: ChatClient instance.
+ """
+ pass
+
+ @abstractmethod
+ def get_audio_client(self) -> AudioClient:
+ """
+ Get an OpenAI API based AudioClient.
+ :return: AudioClient instance.
+ """
+ pass
diff --git a/sdk/python/src/logging_helper.py b/sdk/python/src/logging_helper.py
new file mode 100644
index 00000000..e476f62b
--- /dev/null
+++ b/sdk/python/src/logging_helper.py
@@ -0,0 +1,30 @@
+# -------------------------------------------------------------------------
+# Copyright (c) Microsoft Corporation. All rights reserved.
+# Licensed under the MIT License.
+# --------------------------------------------------------------------------
+import logging
+
+from enum import StrEnum
+
+# Map the python logging levels to the Foundry Local Core names
+class LogLevel(StrEnum):
+ VERBOSE = "Verbose"
+ DEBUG = "Debug"
+ INFORMATION = "Information"
+ WARNING = "Warning"
+ ERROR = "Error"
+ FATAL = "Fatal"
+
+LOG_LEVEL_MAP = {
+ LogLevel.VERBOSE: logging.DEBUG, # No direct equivalent for Trace in Python logging
+ LogLevel.DEBUG: logging.DEBUG,
+ LogLevel.INFORMATION: logging.INFO,
+ LogLevel.WARNING: logging.WARNING,
+ LogLevel.ERROR: logging.ERROR,
+ LogLevel.FATAL: logging.CRITICAL,
+}
+
+def set_default_logger_severity(config_level: LogLevel):
+ py_level = LOG_LEVEL_MAP.get(config_level, logging.INFO)
+ logger = logging.getLogger(__name__.split(".", maxsplit=1)[0])
+ logger.setLevel(py_level)
diff --git a/sdk/python/src/model.py b/sdk/python/src/model.py
new file mode 100644
index 00000000..4c8750ca
--- /dev/null
+++ b/sdk/python/src/model.py
@@ -0,0 +1,133 @@
+# -------------------------------------------------------------------------
+# Copyright (c) Microsoft Corporation. All rights reserved.
+# Licensed under the MIT License.
+# --------------------------------------------------------------------------
+from __future__ import annotations
+
+import logging
+from typing import Callable, List, Optional
+
+from .imodel import IModel
+from .openai.chat_client import ChatClient
+from .openai.audio_client import AudioClient
+from .model_variant import ModelVariant
+from .exception import FoundryLocalException
+from .detail.core_interop import CoreInterop
+
+logger = logging.getLogger(__name__)
+
+
+class Model(IModel):
+ """A model identified by an alias that groups one or more ``ModelVariant`` instances.
+
+ Operations are delegated to the currently selected variant.
+ """
+
+ def __init__(self, model_variant: ModelVariant, core_interop: CoreInterop):
+ self._alias = model_variant.alias
+ self._variants: List[ModelVariant] = [model_variant]
+ # Variants are sorted by Core, so the first one added is the default
+ self._selected_variant = model_variant
+ self._core_interop = core_interop
+
+ def _add_variant(self, variant: ModelVariant) -> None:
+ if variant.alias != self._alias:
+ raise FoundryLocalException(
+ f"Variant alias {variant.alias} does not match model alias {self._alias}"
+ )
+
+ self._variants.append(variant)
+
+ # Prefer the highest priority locally cached variant
+ if variant.info.cached and not self._selected_variant.info.cached:
+ self._selected_variant = variant
+
+ def select_variant(self, variant: ModelVariant) -> None:
+ """
+ Select a specific model variant by its ModelVariant object.
+ The selected variant will be used for IModel operations.
+
+ :param variant: ModelVariant to select
+ :raises FoundryLocalException: If variant is not valid for this model
+ """
+ if variant not in self._variants:
+ raise FoundryLocalException(
+ f"Model {self._alias} does not have a {variant.id} variant."
+ )
+
+ self._selected_variant = variant
+
+ def get_latest_version(self, variant: ModelVariant) -> ModelVariant:
+ """
+ Get the latest version of the specified model variant.
+
+ :param variant: Model variant
+ :return: ModelVariant for latest version. Same as variant if that is the latest version
+ :raises FoundryLocalException: If variant is not valid for this model
+ """
+ # Variants are sorted by version, so the first one matching the name is the latest version
+ for v in self._variants:
+ if v.info.name == variant.info.name:
+ return v
+
+ raise FoundryLocalException(
+ f"Model {self._alias} does not have a {variant.id} variant."
+ )
+
+ @property
+ def variants(self) -> List[ModelVariant]:
+ """List of all variants for this model."""
+ return self._variants.copy() # Return a copy to prevent external modification
+
+ @property
+ def selected_variant(self) -> ModelVariant:
+ """Currently selected variant."""
+ return self._selected_variant
+
+ @property
+ def id(self) -> str:
+ """Model Id of the currently selected variant."""
+ return self._selected_variant.id
+
+ @property
+ def alias(self) -> str:
+ """Alias of this model."""
+ return self._alias
+
+ @property
+ def is_cached(self) -> bool:
+ """Is the currently selected variant cached locally?"""
+ return self._selected_variant.is_cached
+
+ @property
+ def is_loaded(self) -> bool:
+ """Is the currently selected variant loaded in memory?"""
+ return self._selected_variant.is_loaded
+
+ def download(self, progress_callback: Optional[Callable[[float], None]] = None) -> None:
+ """Download the currently selected variant."""
+ self._selected_variant.download(progress_callback)
+
+ def get_path(self) -> str:
+ """Get the path to the currently selected variant."""
+ return self._selected_variant.get_path()
+
+ def load(self) -> None:
+ """Load the currently selected variant into memory."""
+ self._selected_variant.load()
+
+ def unload(self) -> None:
+ """Unload the currently selected variant from memory."""
+ self._selected_variant.unload()
+
+ def remove_from_cache(self) -> None:
+ """Remove the currently selected variant from the local cache."""
+ self._selected_variant.remove_from_cache()
+
+ def get_chat_client(self) -> ChatClient:
+ """Get a chat client for the currently selected variant."""
+ return self._selected_variant.get_chat_client()
+
+ def get_audio_client(self) -> AudioClient:
+ """Get an audio client for the currently selected variant."""
+ return self._selected_variant.get_audio_client()
diff --git a/sdk/python/src/model_variant.py b/sdk/python/src/model_variant.py
new file mode 100644
index 00000000..f0d40109
--- /dev/null
+++ b/sdk/python/src/model_variant.py
@@ -0,0 +1,130 @@
+# -------------------------------------------------------------------------
+# Copyright (c) Microsoft Corporation. All rights reserved.
+# Licensed under the MIT License.
+# --------------------------------------------------------------------------
+from __future__ import annotations
+
+import logging
+from typing import Callable, Optional
+
+from .imodel import IModel
+from .exception import FoundryLocalException
+
+from .detail.core_interop import CoreInterop, InteropRequest
+from .detail.model_data_types import ModelInfo
+from .detail.core_interop import get_cached_model_ids
+from .detail.model_load_manager import ModelLoadManager
+from .openai.audio_client import AudioClient
+from .openai.chat_client import ChatClient
+
+logger = logging.getLogger(__name__)
+
+
+class ModelVariant(IModel):
+ """A specific variant of a model (e.g. a particular device type, version, or quantization).
+
+ Implements ``IModel`` and provides download, cache, load/unload, and
+ client-creation operations for a single model variant.
+ """
+
+ def __init__(self, model_info: ModelInfo, model_load_manager: ModelLoadManager, core_interop: CoreInterop):
+ """Initialize a ModelVariant.
+
+ Args:
+ model_info: Catalog metadata for this variant.
+ model_load_manager: Manager for loading/unloading models.
+ core_interop: Native interop layer for Foundry Local Core.
+ """
+ self._model_info = model_info
+ self._model_load_manager = model_load_manager
+ self._core_interop = core_interop
+
+ self._id = model_info.id
+ self._alias = model_info.alias
+
+ @property
+ def id(self) -> str:
+ """Unique model variant ID (e.g. ``name:version``)."""
+ return self._id
+
+ @property
+ def alias(self) -> str:
+ """Model alias shared across variants."""
+ return self._alias
+
+ @property
+ def info(self) -> ModelInfo:
+ """Full catalog metadata for this variant."""
+ return self._model_info
+
+ @property
+ def is_cached(self) -> bool:
+ """``True`` if this variant is present in the local model cache."""
+ cached_model_ids = get_cached_model_ids(self._core_interop)
+ return self.id in cached_model_ids
+
+ @property
+ def is_loaded(self) -> bool:
+ """``True`` if this variant is currently loaded into memory."""
+ loaded_model_ids = self._model_load_manager.list_loaded()
+ return self.id in loaded_model_ids
+
+ def download(self, progress_callback: Callable[[float], None] = None):
+ """Download this variant to the local cache.
+
+ Args:
+ progress_callback: Optional callback receiving download progress as a
+ percentage (0.0 to 100.0).
+ """
+ request = InteropRequest(params={"Model": self.id})
+ if progress_callback is None:
+ response = self._core_interop.execute_command("download_model", request)
+ else:
+ response = self._core_interop.execute_command_with_callback(
+ "download_model", request,
+ lambda pct_str: progress_callback(float(pct_str))
+ )
+
+ logger.info("Download response: %s", response)
+ if response.error is not None:
+ raise FoundryLocalException(f"Failed to download model: {response.error}")
+
+ def get_path(self) -> str:
+ """Get the local file-system path to this variant if cached.
+
+ Returns:
+ Path to the model directory.
+
+ Raises:
+ FoundryLocalException: If the model path cannot be retrieved.
+ """
+ request = InteropRequest(params={"Model": self.id})
+ response = self._core_interop.execute_command("get_model_path", request)
+ if response.error is not None:
+ raise FoundryLocalException(f"Failed to get model path: {response.error}")
+
+ return response.data
+
+ def load(self) -> None:
+ """Load this variant into memory for inference."""
+ self._model_load_manager.load(self.id)
+
+ def remove_from_cache(self) -> None:
+ """Remove this variant from the local model cache."""
+ request = InteropRequest(params={"Model": self.id})
+ response = self._core_interop.execute_command("remove_cached_model", request)
+ if response.error is not None:
+ raise FoundryLocalException(f"Failed to remove model from cache: {response.error}")
+
+
+ def unload(self) -> None:
+ """Unload this variant from memory."""
+ self._model_load_manager.unload(self.id)
+
+ def get_chat_client(self) -> ChatClient:
+ """Create an OpenAI-compatible ``ChatClient`` for this variant."""
+ return ChatClient(self.id, self._core_interop)
+
+ def get_audio_client(self) -> AudioClient:
+ """Create an OpenAI-compatible ``AudioClient`` for this variant."""
+ return AudioClient(self.id, self._core_interop)
\ No newline at end of file
diff --git a/sdk/python/src/openai/__init__.py b/sdk/python/src/openai/__init__.py
new file mode 100644
index 00000000..e445ba1d
--- /dev/null
+++ b/sdk/python/src/openai/__init__.py
@@ -0,0 +1,10 @@
+# -------------------------------------------------------------------------
+# Copyright (c) Microsoft Corporation. All rights reserved.
+# Licensed under the MIT License.
+# --------------------------------------------------------------------------
+"""OpenAI-compatible clients for chat completions and audio transcription."""
+
+from .chat_client import ChatClient, ChatClientSettings
+from .audio_client import AudioClient
+
+__all__ = ["AudioClient", "ChatClient", "ChatClientSettings"]
diff --git a/sdk/python/src/openai/audio_client.py b/sdk/python/src/openai/audio_client.py
new file mode 100644
index 00000000..8d3ffa29
--- /dev/null
+++ b/sdk/python/src/openai/audio_client.py
@@ -0,0 +1,153 @@
+# -------------------------------------------------------------------------
+# Copyright (c) Microsoft Corporation. All rights reserved.
+# Licensed under the MIT License.
+# --------------------------------------------------------------------------
+
+from __future__ import annotations
+
+import json
+import logging
+from dataclasses import dataclass
+from typing import Callable, Optional
+
+from ..detail.core_interop import CoreInterop, InteropRequest
+from ..exception import FoundryLocalException
+
+logger = logging.getLogger(__name__)
+
+
+class AudioSettings:
+ """Settings supported by Foundry Local for audio transcription.
+
+ Attributes:
+ language: Language of the audio (e.g. ``"en"``).
+ temperature: Sampling temperature (0.0 for deterministic results).
+ """
+
+ def __init__(
+ self,
+ language: Optional[str] = None,
+ temperature: Optional[float] = None,
+ ):
+ self.language = language
+ self.temperature = temperature
+
+
+@dataclass
+class AudioTranscriptionResponse:
+ """Response from an audio transcription request.
+
+ Attributes:
+ text: The transcribed text.
+ """
+
+ text: str
+
+
+class AudioClient:
+ """OpenAI-compatible audio transcription client backed by Foundry Local Core.
+
+ Supports non-streaming and streaming transcription of audio files.
+
+ Attributes:
+ model_id: The ID of the loaded Whisper model variant.
+ settings: Tunable ``AudioSettings`` (language, temperature).
+ """
+
+ def __init__(self, model_id: str, core_interop: CoreInterop):
+ self.model_id = model_id
+ self.settings = AudioSettings()
+ self._core_interop = core_interop
+
+ @staticmethod
+ def _validate_audio_file_path(audio_file_path: str) -> None:
+ """Validate that the audio file path is a non-empty string."""
+ if not isinstance(audio_file_path, str) or audio_file_path.strip() == "":
+ raise ValueError("Audio file path must be a non-empty string.")
+
+ def _create_request_json(self, audio_file_path: str) -> str:
+ """Build the JSON payload for the ``audio_transcribe`` native command."""
+ request: dict = {
+ "Model": self.model_id,
+ "FileName": audio_file_path,
+ }
+
+ metadata: dict[str, str] = {}
+
+ if self.settings.language is not None:
+ request["Language"] = self.settings.language
+ metadata["language"] = self.settings.language
+
+ if self.settings.temperature is not None:
+ request["Temperature"] = self.settings.temperature
+ metadata["temperature"] = str(self.settings.temperature)
+
+ if metadata:
+ request["metadata"] = metadata
+
+ return json.dumps(request)
+
+ def transcribe(self, audio_file_path: str) -> AudioTranscriptionResponse:
+ """Transcribe an audio file (non-streaming).
+
+ Args:
+ audio_file_path: Path to the audio file to transcribe.
+
+ Returns:
+ An ``AudioTranscriptionResponse`` containing the transcribed text.
+
+ Raises:
+ ValueError: If *audio_file_path* is not a non-empty string.
+ FoundryLocalException: If the underlying native transcription command fails.
+ """
+ self._validate_audio_file_path(audio_file_path)
+
+ request_json = self._create_request_json(audio_file_path)
+ request = InteropRequest(params={"OpenAICreateRequest": request_json})
+
+ response = self._core_interop.execute_command("audio_transcribe", request)
+ if response.error is not None:
+ raise FoundryLocalException(
+ f"Audio transcription failed for model '{self.model_id}': {response.error}"
+ )
+
+ data = json.loads(response.data)
+ return AudioTranscriptionResponse(text=data.get("text", ""))
+
+ def transcribe_streaming(
+ self,
+ audio_file_path: str,
+ callback: Callable[[AudioTranscriptionResponse], None],
+ ) -> None:
+ """Transcribe an audio file with streaming chunks.
+
+ Each chunk is passed to *callback* as an ``AudioTranscriptionResponse``.
+
+ Args:
+ audio_file_path: Path to the audio file to transcribe.
+ callback: Called with each incremental transcription chunk.
+
+ Raises:
+ ValueError: If *audio_file_path* is not a non-empty string.
+ FoundryLocalException: If the underlying native transcription command fails.
+ """
+ self._validate_audio_file_path(audio_file_path)
+
+ if not callable(callback):
+ raise TypeError("Callback must be a valid function.")
+
+ request_json = self._create_request_json(audio_file_path)
+ request = InteropRequest(params={"OpenAICreateRequest": request_json})
+
+ def callback_handler(chunk_str: str):
+ chunk_data = json.loads(chunk_str)
+ chunk = AudioTranscriptionResponse(text=chunk_data.get("text", ""))
+ callback(chunk)
+
+ response = self._core_interop.execute_command_with_callback(
+ "audio_transcribe", request, callback_handler
+ )
+ if response.error is not None:
+ raise FoundryLocalException(
+ f"Streaming audio transcription failed for model '{self.model_id}': {response.error}"
+ )
\ No newline at end of file
diff --git a/sdk/python/src/openai/chat_client.py b/sdk/python/src/openai/chat_client.py
new file mode 100644
index 00000000..0b0d58bc
--- /dev/null
+++ b/sdk/python/src/openai/chat_client.py
@@ -0,0 +1,290 @@
+# -------------------------------------------------------------------------
+# Copyright (c) Microsoft Corporation. All rights reserved.
+# Licensed under the MIT License.
+# --------------------------------------------------------------------------
+
+from __future__ import annotations
+
+import logging
+import json
+import queue
+import threading
+
+from ..detail.core_interop import CoreInterop, InteropRequest
+from ..exception import FoundryLocalException
+from openai.types.chat.chat_completion_message_param import ChatCompletionMessageParam
+from openai.types.chat.completion_create_params import CompletionCreateParamsBase, \
+ CompletionCreateParamsNonStreaming, \
+ CompletionCreateParamsStreaming
+from openai.types.chat import ChatCompletion
+from openai.types.chat.chat_completion_chunk import ChatCompletionChunk
+from typing import Any, Dict, Generator, List, Optional
+
+logger = logging.getLogger(__name__)
+
+
+class ChatClientSettings:
+ """Settings for chat completion requests.
+
+ Attributes match the OpenAI chat completion API parameters.
+ Foundry-specific settings (``top_k``, ``random_seed``) are sent via metadata.
+ """
+
+ def __init__(
+ self,
+ frequency_penalty: Optional[float] = None,
+ max_tokens: Optional[int] = None,
+ n: Optional[int] = None,
+ temperature: Optional[float] = None,
+ presence_penalty: Optional[float] = None,
+ random_seed: Optional[int] = None,
+ top_k: Optional[int] = None,
+ top_p: Optional[float] = None,
+ response_format: Optional[Dict[str, Any]] = None,
+ tool_choice: Optional[Dict[str, Any]] = None,
+ ):
+ self.frequency_penalty = frequency_penalty
+ self.max_tokens = max_tokens
+ self.n = n
+ self.temperature = temperature
+ self.presence_penalty = presence_penalty
+ self.random_seed = random_seed
+ self.top_k = top_k
+ self.top_p = top_p
+ self.response_format = response_format
+ self.tool_choice = tool_choice
+
+ def _serialize(self) -> Dict[str, Any]:
+ """Serialize settings into an OpenAI-compatible request dict."""
+ self._validate_response_format(self.response_format)
+ self._validate_tool_choice(self.tool_choice)
+
+ result: Dict[str, Any] = {
+ k: v for k, v in {
+ "frequency_penalty": self.frequency_penalty,
+ "max_tokens": self.max_tokens,
+ "n": self.n,
+ "presence_penalty": self.presence_penalty,
+ "temperature": self.temperature,
+ "top_p": self.top_p,
+ "response_format": self.response_format,
+ "tool_choice": self.tool_choice,
+ }.items() if v is not None
+ }
+
+ metadata: Dict[str, str] = {}
+ if self.top_k is not None:
+ metadata["top_k"] = str(self.top_k)
+ if self.random_seed is not None:
+ metadata["random_seed"] = str(self.random_seed)
+
+ if metadata:
+ result["metadata"] = metadata
+
+ return result
+
+ def _validate_response_format(self, response_format: Optional[Dict[str, Any]]) -> None:
+ if response_format is None:
+ return
+ valid_types = ["text", "json_object", "json_schema", "lark_grammar"]
+ fmt_type = response_format.get("type")
+ if fmt_type not in valid_types:
+ raise ValueError(f"ResponseFormat type must be one of: {', '.join(valid_types)}")
+ grammar_types = ["json_schema", "lark_grammar"]
+ if fmt_type in grammar_types:
+ if fmt_type == "json_schema" and (
+ not isinstance(response_format.get("json_schema"), str)
+ or not response_format["json_schema"].strip()
+ ):
+ raise ValueError('ResponseFormat with type "json_schema" must have a valid json_schema string.')
+ if fmt_type == "lark_grammar" and (
+ not isinstance(response_format.get("lark_grammar"), str)
+ or not response_format["lark_grammar"].strip()
+ ):
+ raise ValueError('ResponseFormat with type "lark_grammar" must have a valid lark_grammar string.')
+ elif response_format.get("json_schema") or response_format.get("lark_grammar"):
+ raise ValueError(
+ f'ResponseFormat with type "{fmt_type}" should not have json_schema or lark_grammar properties.'
+ )
+
+ def _validate_tool_choice(self, tool_choice: Optional[Dict[str, Any]]) -> None:
+ if tool_choice is None:
+ return
+ valid_types = ["none", "auto", "required", "function"]
+ choice_type = tool_choice.get("type")
+ if choice_type not in valid_types:
+ raise ValueError(f"ToolChoice type must be one of: {', '.join(valid_types)}")
+ if choice_type == "function" and (
+ not isinstance(tool_choice.get("name"), str) or not tool_choice.get("name", "").strip()
+ ):
+ raise ValueError('ToolChoice with type "function" must have a valid name string.')
+ elif choice_type != "function" and tool_choice.get("name"):
+ raise ValueError(f'ToolChoice with type "{choice_type}" should not have a name property.')
+
+class ChatClient:
+ """OpenAI-compatible chat completions client backed by Foundry Local Core.
+
+ Supports non-streaming and streaming completions with optional tool calling.
+
+ Attributes:
+ model_id: The ID of the loaded model variant.
+ settings: Tunable ``ChatClientSettings`` (temperature, max tokens, etc.).
+ """
+
+ def __init__(self, model_id: str, core_interop: CoreInterop):
+ self.model_id = model_id
+ self.settings = ChatClientSettings()
+ self._core_interop = core_interop
+
+ def _validate_messages(self, messages: List[ChatCompletionMessageParam]) -> None:
+ """Validate the messages list before sending to the native layer."""
+ if not messages:
+ raise ValueError("messages must be a non-empty list.")
+ for i, msg in enumerate(messages):
+ if not isinstance(msg, dict):
+ raise ValueError(f"messages[{i}] must be a dict, got {type(msg).__name__}.")
+ if "role" not in msg:
+ raise ValueError(f"messages[{i}] is missing required key 'role'.")
+ if "content" not in msg:
+ raise ValueError(f"messages[{i}] is missing required key 'content'.")
+
+ def _validate_tools(self, tools: Optional[List[Dict[str, Any]]]) -> None:
+ """Validate the tools list before sending to the native layer."""
+ if not tools:
+ return
+ if not isinstance(tools, list):
+ raise ValueError("tools must be a list if provided.")
+ for i, tool in enumerate(tools):
+ if not isinstance(tool, dict) or not tool:
+ raise ValueError(
+ f"tools[{i}] must be a non-null object with a valid 'type' and 'function' definition."
+ )
+ if not isinstance(tool.get("type"), str) or not tool["type"].strip():
+ raise ValueError(f"tools[{i}] must have a 'type' property that is a non-empty string.")
+ fn = tool.get("function")
+ if not isinstance(fn, dict):
+ raise ValueError(f"tools[{i}] must have a 'function' property that is a non-empty object.")
+ if not isinstance(fn.get("name"), str) or not fn["name"].strip():
+ raise ValueError(
+ f"tools[{i}]'s function must have a 'name' property that is a non-empty string."
+ )
+
+ def _create_request(
+ self,
+ messages: List[ChatCompletionMessageParam],
+ streaming: bool,
+ tools: Optional[List[Dict[str, Any]]] = None,
+ ) -> str:
+ request: Dict[str, Any] = {
+ "model": self.model_id,
+ "messages": messages,
+ **({
+ "tools": tools} if tools else {}),
+ **({
+ "stream": True} if streaming else {}),
+ **self.settings._serialize(),
+ }
+
+ if streaming:
+ chat_request = CompletionCreateParamsStreaming(request)
+ else:
+ chat_request = CompletionCreateParamsNonStreaming(request)
+
+ return json.dumps(chat_request)
+
+ def complete_chat(self, messages: List[ChatCompletionMessageParam], tools: Optional[List[Dict[str, Any]]] = None):
+ """Perform a non-streaming chat completion.
+
+ Args:
+ messages: Conversation history as a list of OpenAI message dicts.
+ tools: Optional list of tool definitions for function calling.
+
+ Returns:
+ A ``ChatCompletion`` response.
+
+ Raises:
+ ValueError: If messages is None, empty, or contains malformed entries.
+ FoundryLocalException: If the native command returns an error.
+ """
+ self._validate_messages(messages)
+ self._validate_tools(tools)
+ chat_request_json = self._create_request(messages, streaming=False, tools=tools)
+
+ # Send the request to the chat API
+ request = InteropRequest(params={"OpenAICreateRequest": chat_request_json})
+ response = self._core_interop.execute_command("chat_completions", request)
+ if response.error is not None:
+ raise FoundryLocalException(f"Error during chat completion: {response.error}")
+
+ completion = ChatCompletion.model_validate_json(response.data)
+
+ return completion
+
+ def _stream_chunks(self, chat_request_json: str) -> Generator[ChatCompletionChunk, None, None]:
+ """Background-thread generator that yields parsed chunks from the native streaming call."""
+ _SENTINEL = object()
+ chunk_queue: queue.Queue = queue.Queue()
+ errors: List[Exception] = []
+
+ def _on_chunk(response_str: str) -> None:
+ raw = json.loads(response_str)
+ # Foundry Local returns tool call chunks with "message.tool_calls" instead
+ # of the standard streaming "delta.tool_calls". Normalize to delta format
+ # so ChatCompletionChunk parses correctly.
+ for choice in raw.get("choices", []):
+ if "message" in choice and "delta" not in choice:
+ msg = choice.pop("message")
+ # ChoiceDeltaToolCall requires "index"; add if missing
+ for i, tc in enumerate(msg.get("tool_calls", [])):
+ tc.setdefault("index", i)
+ choice["delta"] = msg
+ chunk_queue.put(ChatCompletionChunk.model_validate(raw))
+
+ def _run() -> None:
+ try:
+ resp = self._core_interop.execute_command_with_callback(
+ "chat_completions",
+ InteropRequest(params={"OpenAICreateRequest": chat_request_json}),
+ _on_chunk,
+ )
+ if resp.error is not None:
+ errors.append(FoundryLocalException(f"Error during streaming chat completion: {resp.error}"))
+ except Exception as exc:
+ errors.append(exc)
+ finally:
+ chunk_queue.put(_SENTINEL)
+
+ threading.Thread(target=_run, daemon=True).start()
+ while (item := chunk_queue.get()) is not _SENTINEL:
+ yield item
+ if errors:
+ raise errors[0]
+
+ def complete_streaming_chat(
+ self,
+ messages: List[ChatCompletionMessageParam],
+ tools: Optional[List[Dict[str, Any]]] = None,
+ ) -> Generator[ChatCompletionChunk, None, None]:
+ """Perform a streaming chat completion, yielding chunks as they arrive.
+
+ Consume with a standard ``for`` loop::
+
+ for chunk in client.complete_streaming_chat(messages):
+ if chunk.choices[0].delta.content:
+ print(chunk.choices[0].delta.content, end="", flush=True)
+
+ Args:
+ messages: Conversation history as a list of OpenAI message dicts.
+ tools: Optional list of tool definitions for function calling.
+
+ Returns:
+ A generator of ``ChatCompletionChunk`` objects.
+
+ Raises:
+ ValueError: If messages or tools are malformed.
+ FoundryLocalException: If the native layer returns an error.
+ """
+ self._validate_messages(messages)
+ self._validate_tools(tools)
+ chat_request_json = self._create_request(messages, streaming=True, tools=tools)
+ return self._stream_chunks(chat_request_json)
diff --git a/sdk/python/src/version.py b/sdk/python/src/version.py
new file mode 100644
index 00000000..f198d448
--- /dev/null
+++ b/sdk/python/src/version.py
@@ -0,0 +1,6 @@
+# -------------------------------------------------------------------------
+# Copyright (c) Microsoft Corporation. All rights reserved.
+# Licensed under the MIT License.
+# --------------------------------------------------------------------------
+
+__version__ = "0.9.0.dev0"
diff --git a/sdk/python/test/README.md b/sdk/python/test/README.md
new file mode 100644
index 00000000..92f389a8
--- /dev/null
+++ b/sdk/python/test/README.md
@@ -0,0 +1,79 @@
+# Foundry Local Python SDK – Test Suite
+
+This test suite mirrors the structure of the JS (`sdk_v2/js/test/`) and C# (`sdk_v2/cs/test/`) SDK test suites.
+
+## Prerequisites
+
+1. **Python 3.10+** (tested with 3.12/3.13)
+2. **SDK installed in editable mode** from the `sdk/python` directory:
+ ```bash
+ pip install -e .
+ ```
+3. **Test dependencies**:
+ ```bash
+ pip install -r requirements-dev.txt
+ ```
+4. **Test model data** – the `test-data-shared` folder must exist as a sibling of the git repo root
+ (e.g. `../test-data-shared` relative to the repo). It should contain cached models for
+ `qwen2.5-0.5b` and `whisper-tiny`.
+
+## Running the tests
+
+From the `sdk/python` directory:
+
+```bash
+# Run all tests
+python -m pytest test/
+
+# Run with verbose output
+python -m pytest test/ -v
+
+# Run a specific test file
+python -m pytest test/test_catalog.py
+
+# Run a specific test class or function
+python -m pytest test/test_catalog.py::TestCatalog::test_should_list_models
+
+# List all collected tests without running them
+python -m pytest test/ --collect-only
+```
+
+## Test structure
+
+```
+test/
+├── conftest.py # Shared fixtures & config (equivalent to testUtils.ts)
+├── test_foundry_local_manager.py # FoundryLocalManager initialization (2 tests)
+├── test_catalog.py # Catalog listing, lookup, error cases (9 tests)
+├── test_model.py # Model caching & load/unload lifecycle (2 tests)
+├── detail/
+│ └── test_model_load_manager.py # ModelLoadManager core interop & web service (5 tests)
+└── openai/
+ ├── test_chat_client.py # Chat completions, streaming, error validation (7 tests)
+ └── test_audio_client.py # Audio transcription (7 tests)
+```
+
+**Total: 32 tests**
+
+## Key conventions
+
+| Concept | Python (pytest) | JS (Mocha) | C# (TUnit) |
+|---|---|---|---|
+| Shared setup | `conftest.py` (auto-discovered) | `testUtils.ts` (explicit import) | `Utils.cs` (`[Before(Assembly)]`) |
+| Session fixture | `@pytest.fixture(scope="session")` | manual singleton | `[Before(Assembly)]` static |
+| Teardown | `yield` + cleanup in fixture | `after()` hook | `[After(Assembly)]` |
+| Skip in CI | `@skip_in_ci` marker | `IS_RUNNING_IN_CI` + `this.skip()` | `[SkipInCI]` attribute |
+| Expected failure | `@pytest.mark.xfail` | N/A | N/A |
+| Timeout | `@pytest.mark.timeout(30)` | `this.timeout(30000)` | `[Timeout(30000)]` |
+
+## CI environment detection
+
+Tests that require the web service are skipped when either `TF_BUILD=true` (Azure DevOps) or
+`GITHUB_ACTIONS=true` is set.
+
+## Test models
+
+| Alias | Use | Variant |
+|---|---|---|
+| `qwen2.5-0.5b` | Chat completions | `qwen2.5-0.5b-instruct-generic-cpu:4` |
+| `whisper-tiny` | Audio transcription | `openai-whisper-tiny-generic-cpu:2` |
diff --git a/sdk/python/test/__init__.py b/sdk/python/test/__init__.py
new file mode 100644
index 00000000..e69de29b
diff --git a/sdk/python/test/conftest.py b/sdk/python/test/conftest.py
new file mode 100644
index 00000000..b7e22c97
--- /dev/null
+++ b/sdk/python/test/conftest.py
@@ -0,0 +1,145 @@
+# -------------------------------------------------------------------------
+# Copyright (c) Microsoft Corporation. All rights reserved.
+# Licensed under the MIT License.
+# --------------------------------------------------------------------------
+"""Shared test configuration and fixtures for Foundry Local Python SDK tests.
+
+NOTE: "conftest.py" is a special filename that pytest uses to auto-discover
+fixtures and shared utilities. All fixtures defined here are automatically
+available to every test file without needing an explicit import.
+This serves the same role as testUtils.ts in the JS SDK.
+"""
+
+from __future__ import annotations
+
+import os
+import logging
+
+import pytest
+
+from pathlib import Path
+
+from foundry_local_sdk.configuration import Configuration, LogLevel
+from foundry_local_sdk.foundry_local_manager import FoundryLocalManager
+
+logger = logging.getLogger(__name__)
+
+TEST_MODEL_ALIAS = "qwen2.5-0.5b"
+AUDIO_MODEL_ALIAS = "whisper-tiny"
+
+def get_git_repo_root() -> Path:
+ """Walk upward from __file__ until we find a .git directory."""
+ current = Path(__file__).resolve().parent
+ while True:
+ if (current / ".git").exists():
+ return current
+ parent = current.parent
+ if parent == current:
+ raise RuntimeError("Could not find git repo root")
+ current = parent
+
+
+def get_test_data_shared_path() -> str:
+ """Return absolute path to the test-data-shared folder (sibling of the repo root)."""
+ repo_root = get_git_repo_root()
+ return str(repo_root.parent / "test-data-shared")
+
+
+def is_running_in_ci() -> bool:
+ """Check TF_BUILD (Azure DevOps) and GITHUB_ACTIONS env vars."""
+ azure_devops = os.environ.get("TF_BUILD", "false").lower() == "true"
+ github_actions = os.environ.get("GITHUB_ACTIONS", "false").lower() == "true"
+ return azure_devops or github_actions
+
+
+IS_RUNNING_IN_CI = is_running_in_ci()
+
+skip_in_ci = pytest.mark.skipif(IS_RUNNING_IN_CI, reason="Skipped in CI environments")
+
+
+def get_test_config() -> Configuration:
+ """Build a Configuration suitable for integration tests."""
+ repo_root = get_git_repo_root()
+ return Configuration(
+ app_name="FoundryLocalTest",
+ model_cache_dir=get_test_data_shared_path(),
+ log_level=LogLevel.WARNING,
+ logs_dir=str(repo_root / "sdk" / "python" / "logs"),
+ additional_settings={"Bootstrap": "false"},
+ )
+
+
+def get_multiply_tool():
+ """Tool definition for the multiply_numbers function-calling test."""
+ return {
+ "type": "function",
+ "function": {
+ "name": "multiply_numbers",
+ "description": "A tool for multiplying two numbers.",
+ "parameters": {
+ "type": "object",
+ "properties": {
+ "first": {
+ "type": "integer",
+ "description": "The first number in the operation",
+ },
+ "second": {
+ "type": "integer",
+ "description": "The second number in the operation",
+ },
+ },
+ "required": ["first", "second"],
+ },
+ },
+ }
+
+
+# ---------------------------------------------------------------------------
+# Session-scoped fixtures
+# ---------------------------------------------------------------------------
+
+@pytest.fixture(scope="session")
+def manager():
+ """Initialize FoundryLocalManager once for the entire test session."""
+ # Reset singleton in case a previous run left state
+ FoundryLocalManager.instance = None
+
+ config = get_test_config()
+ FoundryLocalManager.initialize(config)
+ mgr = FoundryLocalManager.instance
+ assert mgr is not None, "FoundryLocalManager.initialize did not set instance"
+
+ yield mgr
+
+ # Teardown: unload all loaded models
+ try:
+ catalog = mgr.catalog
+ loaded = catalog.get_loaded_models()
+ for model_variant in loaded:
+ try:
+ model_variant.unload()
+ except Exception as e:
+ logger.warning("Failed to unload model %s during teardown: %s", model_variant.id, e)
+ except Exception as e:
+ logger.warning("Failed to get loaded models during teardown: %s", e)
+
+ # Reset the singleton so that other test sessions start clean
+ FoundryLocalManager.instance = None
+
+
+@pytest.fixture(scope="session")
+def catalog(manager):
+ """Return the Catalog from the session-scoped manager."""
+ return manager.catalog
+
+
+@pytest.fixture(scope="session")
+def core_interop(manager):
+ """Return the CoreInterop from the session-scoped manager (internal, for component tests)."""
+ return manager._core_interop
+
+
+@pytest.fixture(scope="session")
+def model_load_manager(manager):
+ """Return the ModelLoadManager from the session-scoped manager (internal, for component tests)."""
+ return manager._model_load_manager
diff --git a/sdk/python/test/detail/__init__.py b/sdk/python/test/detail/__init__.py
new file mode 100644
index 00000000..e69de29b
diff --git a/sdk/python/test/detail/test_model_load_manager.py b/sdk/python/test/detail/test_model_load_manager.py
new file mode 100644
index 00000000..a5a231e3
--- /dev/null
+++ b/sdk/python/test/detail/test_model_load_manager.py
@@ -0,0 +1,144 @@
+# -------------------------------------------------------------------------
+# Copyright (c) Microsoft Corporation. All rights reserved.
+# Licensed under the MIT License.
+# --------------------------------------------------------------------------
+"""Tests for ModelLoadManager – mirrors modelLoadManager.test.ts."""
+
+from __future__ import annotations
+
+import pytest
+
+from foundry_local_sdk.detail.model_load_manager import ModelLoadManager
+from ..conftest import TEST_MODEL_ALIAS, IS_RUNNING_IN_CI, skip_in_ci
+
+
+class TestModelLoadManagerCoreInterop:
+ """ModelLoadManager tests using Core Interop (no external URL)."""
+
+ def _get_model_id(self, catalog) -> str:
+ """Resolve the variant ID for the test model alias."""
+ cached = catalog.get_cached_models()
+ variant = next((m for m in cached if m.alias == TEST_MODEL_ALIAS), None)
+ assert variant is not None, f"{TEST_MODEL_ALIAS} should be cached"
+ return variant.id
+
+ def test_should_load_model(self, catalog, core_interop):
+ """Load model via core interop and verify it appears in loaded list."""
+ model_id = self._get_model_id(catalog)
+ mlm = ModelLoadManager(core_interop)
+
+ mlm.load(model_id)
+ loaded = mlm.list_loaded()
+ assert model_id in loaded
+
+ # Cleanup
+ mlm.unload(model_id)
+
+ def test_should_unload_model(self, catalog, core_interop):
+ """Load then unload model via core interop."""
+ model_id = self._get_model_id(catalog)
+ mlm = ModelLoadManager(core_interop)
+
+ mlm.load(model_id)
+ loaded = mlm.list_loaded()
+ assert model_id in loaded
+
+ mlm.unload(model_id)
+ loaded = mlm.list_loaded()
+ assert model_id not in loaded
+
+ def test_should_list_loaded_models(self, catalog, core_interop):
+ """list_loaded() should return an array containing the loaded model."""
+ model_id = self._get_model_id(catalog)
+ mlm = ModelLoadManager(core_interop)
+
+ mlm.load(model_id)
+ loaded = mlm.list_loaded()
+
+ assert isinstance(loaded, list)
+ assert model_id in loaded
+
+ # Cleanup
+ mlm.unload(model_id)
+
+
+class TestModelLoadManagerExternalService:
+ """ModelLoadManager tests using external web service URL (skipped in CI)."""
+
+ @skip_in_ci
+ def test_should_load_and_unload_via_external_service(self, manager, catalog, core_interop):
+ """Load/unload model through the web service endpoint."""
+ cached = catalog.get_cached_models()
+ variant = next((m for m in cached if m.alias == TEST_MODEL_ALIAS), None)
+ assert variant is not None
+ model_id = variant.id
+
+ # Start web service
+ try:
+ manager.start_web_service()
+ except Exception as e:
+ pytest.skip(f"Failed to start web service: {e}")
+
+ urls = manager.urls
+ if not urls or len(urls) == 0:
+ pytest.skip("Web service started but no URLs returned")
+
+ service_url = urls[0]
+
+ try:
+ # Setup: load via core interop
+ setup_mlm = ModelLoadManager(core_interop)
+ setup_mlm.load(model_id)
+ loaded = setup_mlm.list_loaded()
+ assert model_id in loaded
+
+ # Unload via external service
+ ext_mlm = ModelLoadManager(core_interop, service_url)
+ ext_mlm.unload(model_id)
+
+ # Verify via core interop
+ loaded = setup_mlm.list_loaded()
+ assert model_id not in loaded
+ finally:
+ try:
+ manager.stop_web_service()
+ except Exception:
+ pass
+
+ @skip_in_ci
+ def test_should_list_loaded_via_external_service(self, manager, catalog, core_interop):
+ """list_loaded() through the web service endpoint should match core interop."""
+ cached = catalog.get_cached_models()
+ variant = next((m for m in cached if m.alias == TEST_MODEL_ALIAS), None)
+ assert variant is not None
+ model_id = variant.id
+
+ try:
+ manager.start_web_service()
+ except Exception as e:
+ pytest.skip(f"Failed to start web service: {e}")
+
+ urls = manager.urls
+ if not urls or len(urls) == 0:
+ pytest.skip("Web service started but no URLs returned")
+
+ service_url = urls[0]
+
+ try:
+ # Setup: load via core
+ setup_mlm = ModelLoadManager(core_interop)
+ setup_mlm.load(model_id)
+
+ # Verify via external service
+ ext_mlm = ModelLoadManager(core_interop, service_url)
+ loaded = ext_mlm.list_loaded()
+ assert isinstance(loaded, list)
+ assert model_id in loaded
+
+ # Cleanup
+ setup_mlm.unload(model_id)
+ finally:
+ try:
+ manager.stop_web_service()
+ except Exception:
+ pass
diff --git a/sdk/python/test/openai/__init__.py b/sdk/python/test/openai/__init__.py
new file mode 100644
index 00000000..e69de29b
diff --git a/sdk/python/test/openai/test_audio_client.py b/sdk/python/test/openai/test_audio_client.py
new file mode 100644
index 00000000..f430d8d5
--- /dev/null
+++ b/sdk/python/test/openai/test_audio_client.py
@@ -0,0 +1,156 @@
+# -------------------------------------------------------------------------
+# Copyright (c) Microsoft Corporation. All rights reserved.
+# Licensed under the MIT License.
+# --------------------------------------------------------------------------
+"""Tests for AudioClient – mirrors audioClient.test.ts."""
+
+from __future__ import annotations
+
+import pytest
+
+from ..conftest import AUDIO_MODEL_ALIAS, get_git_repo_root
+
+# Recording.mp3 lives at sdk/testdata/Recording.mp3 relative to the repo root
+AUDIO_FILE_PATH = str(get_git_repo_root() / "sdk" / "testdata" / "Recording.mp3")
+EXPECTED_TEXT = (
+ " And lots of times you need to give people more than one link at a time."
+ " You a band could give their fans a couple new videos from the live concert"
+ " behind the scenes photo gallery and album to purchase like these next few links."
+)
+
+
+def _get_loaded_audio_model(catalog):
+ """Helper: ensure the whisper model is selected, loaded, and return Model."""
+ cached = catalog.get_cached_models()
+ assert len(cached) > 0
+
+ cached_variant = next((m for m in cached if m.alias == AUDIO_MODEL_ALIAS), None)
+ assert cached_variant is not None, f"{AUDIO_MODEL_ALIAS} should be cached"
+
+ model = catalog.get_model(AUDIO_MODEL_ALIAS)
+ assert model is not None
+
+ model.select_variant(cached_variant)
+ model.load()
+ return model
+
+
+class TestAudioClient:
+ """Audio Client Tests."""
+
+ def test_should_transcribe_audio(self, catalog):
+ """Non-streaming transcription of Recording.mp3."""
+ model = _get_loaded_audio_model(catalog)
+ try:
+ audio_client = model.get_audio_client()
+ assert audio_client is not None
+
+ audio_client.settings.language = "en"
+ audio_client.settings.temperature = 0.0
+
+ response = audio_client.transcribe(AUDIO_FILE_PATH)
+
+ assert response is not None
+ assert hasattr(response, "text")
+ assert isinstance(response.text, str)
+ assert len(response.text) > 0
+ assert response.text == EXPECTED_TEXT
+ finally:
+ model.unload()
+
+ def test_should_transcribe_audio_with_temperature(self, catalog):
+ """Non-streaming transcription with explicit temperature."""
+ model = _get_loaded_audio_model(catalog)
+ try:
+ audio_client = model.get_audio_client()
+ assert audio_client is not None
+
+ audio_client.settings.language = "en"
+ audio_client.settings.temperature = 0.0
+
+ response = audio_client.transcribe(AUDIO_FILE_PATH)
+
+ assert response is not None
+ assert isinstance(response.text, str)
+ assert len(response.text) > 0
+ assert response.text == EXPECTED_TEXT
+ finally:
+ model.unload()
+
+ def test_should_transcribe_audio_streaming(self, catalog):
+ """Streaming transcription of Recording.mp3."""
+ model = _get_loaded_audio_model(catalog)
+ try:
+ audio_client = model.get_audio_client()
+ assert audio_client is not None
+
+ audio_client.settings.language = "en"
+ audio_client.settings.temperature = 0.0
+
+ chunks = []
+
+ def on_chunk(chunk):
+ assert chunk is not None
+ assert hasattr(chunk, "text")
+ assert isinstance(chunk.text, str)
+ assert len(chunk.text) > 0
+ chunks.append(chunk.text)
+
+ audio_client.transcribe_streaming(AUDIO_FILE_PATH, on_chunk)
+
+ full_text = "".join(chunks)
+ assert full_text == EXPECTED_TEXT
+ finally:
+ model.unload()
+
+ def test_should_transcribe_audio_streaming_with_temperature(self, catalog):
+ """Streaming transcription with explicit temperature."""
+ model = _get_loaded_audio_model(catalog)
+ try:
+ audio_client = model.get_audio_client()
+ assert audio_client is not None
+
+ audio_client.settings.language = "en"
+ audio_client.settings.temperature = 0.0
+
+ chunks = []
+
+ def on_chunk(chunk):
+ assert chunk is not None
+ assert isinstance(chunk.text, str)
+ chunks.append(chunk.text)
+
+ audio_client.transcribe_streaming(AUDIO_FILE_PATH, on_chunk)
+
+ full_text = "".join(chunks)
+ assert full_text == EXPECTED_TEXT
+ finally:
+ model.unload()
+
+ def test_should_raise_for_empty_audio_file_path(self, catalog):
+ """transcribe('') should raise."""
+ model = catalog.get_model(AUDIO_MODEL_ALIAS)
+ assert model is not None
+ audio_client = model.get_audio_client()
+
+ with pytest.raises(ValueError, match="Audio file path must be a non-empty string"):
+ audio_client.transcribe("")
+
+ def test_should_raise_for_streaming_empty_audio_file_path(self, catalog):
+ """transcribe_streaming('') should raise."""
+ model = catalog.get_model(AUDIO_MODEL_ALIAS)
+ assert model is not None
+ audio_client = model.get_audio_client()
+
+ with pytest.raises(ValueError, match="Audio file path must be a non-empty string"):
+ audio_client.transcribe_streaming("", lambda chunk: None)
+
+ def test_should_raise_for_streaming_invalid_callback(self, catalog):
+ """transcribe_streaming with invalid callback should raise."""
+ model = catalog.get_model(AUDIO_MODEL_ALIAS)
+ assert model is not None
+ audio_client = model.get_audio_client()
+
+ for invalid_callback in [None, 42, {}, "not a function"]:
+ with pytest.raises(TypeError, match="Callback must be a valid function"):
+ audio_client.transcribe_streaming(AUDIO_FILE_PATH, invalid_callback)
diff --git a/sdk/python/test/openai/test_chat_client.py b/sdk/python/test/openai/test_chat_client.py
new file mode 100644
index 00000000..d96891b9
--- /dev/null
+++ b/sdk/python/test/openai/test_chat_client.py
@@ -0,0 +1,243 @@
+# -------------------------------------------------------------------------
+# Copyright (c) Microsoft Corporation. All rights reserved.
+# Licensed under the MIT License.
+# --------------------------------------------------------------------------
+"""Tests for ChatClient – mirrors chatClient.test.ts."""
+
+from __future__ import annotations
+
+import json
+
+import pytest
+
+from ..conftest import TEST_MODEL_ALIAS, get_multiply_tool
+
+
+def _get_loaded_chat_model(catalog):
+ """Helper: ensure the test model is selected, loaded, and return Model + ChatClient."""
+ cached = catalog.get_cached_models()
+ assert len(cached) > 0
+
+ cached_variant = next((m for m in cached if m.alias == TEST_MODEL_ALIAS), None)
+ assert cached_variant is not None, f"{TEST_MODEL_ALIAS} should be cached"
+
+ model = catalog.get_model(TEST_MODEL_ALIAS)
+ assert model is not None
+
+ model.select_variant(cached_variant)
+ model.load()
+ return model
+
+
+class TestChatClient:
+ """Chat Client Tests."""
+
+ def test_should_perform_chat_completion(self, catalog):
+ """Non-streaming chat: 7 * 6 should include '42' in the response."""
+ model = _get_loaded_chat_model(catalog)
+ try:
+ client = model.get_chat_client()
+ client.settings.max_tokens = 500
+ client.settings.temperature = 0.0 # deterministic
+
+ result = client.complete_chat([
+ {"role": "user",
+ "content": "You are a calculator. Be precise. What is the answer to 7 multiplied by 6?"}
+ ])
+
+ assert result is not None
+ assert result.choices is not None
+ assert len(result.choices) > 0
+ assert result.choices[0].message is not None
+ content = result.choices[0].message.content
+ assert isinstance(content, str)
+ assert "42" in content
+ finally:
+ model.unload()
+
+ def test_should_perform_streaming_chat_completion(self, catalog):
+ """Streaming chat: 7 * 6 = 42, then follow-up +25 = 67."""
+ model = _get_loaded_chat_model(catalog)
+ try:
+ client = model.get_chat_client()
+ client.settings.max_tokens = 500
+ client.settings.temperature = 0.0
+
+ messages = [
+ {"role": "user",
+ "content": "You are a calculator. Be precise. What is the answer to 7 multiplied by 6?"}
+ ]
+
+ # ---- First question ----
+ chunks = list(client.complete_streaming_chat(messages))
+ assert len(chunks) > 0
+ first_response = "".join(
+ c.choices[0].delta.content
+ for c in chunks
+ if c.choices and c.choices[0].delta and c.choices[0].delta.content
+ )
+ assert "42" in first_response
+
+ # ---- Follow-up question ----
+ messages.append({"role": "assistant", "content": first_response})
+ messages.append({"role": "user", "content": "Add 25 to the previous answer. Think hard to be sure of the answer."})
+
+ chunks = list(client.complete_streaming_chat(messages))
+ assert len(chunks) > 0
+ second_response = "".join(
+ c.choices[0].delta.content
+ for c in chunks
+ if c.choices and c.choices[0].delta and c.choices[0].delta.content
+ )
+ assert "67" in second_response
+ finally:
+ model.unload()
+
+ def test_should_raise_for_empty_messages(self, catalog):
+ """complete_chat with empty list should raise."""
+ model = catalog.get_model(TEST_MODEL_ALIAS)
+ assert model is not None
+ client = model.get_chat_client()
+
+ with pytest.raises(ValueError):
+ client.complete_chat([])
+
+ def test_should_raise_for_none_messages(self, catalog):
+ """complete_chat with None should raise."""
+ model = catalog.get_model(TEST_MODEL_ALIAS)
+ assert model is not None
+ client = model.get_chat_client()
+
+ with pytest.raises(ValueError):
+ client.complete_chat(None)
+
+ def test_should_raise_for_streaming_empty_messages(self, catalog):
+ """complete_streaming_chat with empty list should raise."""
+ model = catalog.get_model(TEST_MODEL_ALIAS)
+ assert model is not None
+ client = model.get_chat_client()
+
+ with pytest.raises(ValueError):
+ client.complete_streaming_chat([])
+
+ def test_should_raise_for_streaming_none_messages(self, catalog):
+ """complete_streaming_chat with None should raise."""
+ model = catalog.get_model(TEST_MODEL_ALIAS)
+ assert model is not None
+ client = model.get_chat_client()
+
+ with pytest.raises(ValueError):
+ client.complete_streaming_chat(None)
+
+ def test_should_perform_tool_calling_chat_completion(self, catalog):
+ """Tool calling (non-streaming): model uses multiply_numbers tool to answer 7 * 6."""
+ model = _get_loaded_chat_model(catalog)
+ try:
+ client = model.get_chat_client()
+ client.settings.max_tokens = 500
+ client.settings.temperature = 0.0
+ client.settings.tool_choice = {"type": "required"}
+
+ messages = [
+ {"role": "system", "content": "You are a helpful AI assistant. If necessary, you can use any provided tools to answer the question."},
+ {"role": "user", "content": "What is the answer to 7 multiplied by 6?"},
+ ]
+ tools = [get_multiply_tool()]
+
+ # First turn: model should respond with a tool call
+ response = client.complete_chat(messages, tools)
+
+ assert response is not None
+ assert response.choices is not None
+ assert len(response.choices) > 0
+ assert response.choices[0].finish_reason == "tool_calls"
+ assert response.choices[0].message is not None
+ assert response.choices[0].message.tool_calls is not None
+ assert len(response.choices[0].message.tool_calls) > 0
+
+ tool_call = response.choices[0].message.tool_calls[0]
+ assert tool_call.type == "function"
+ assert tool_call.function.name == "multiply_numbers"
+
+ args = json.loads(tool_call.function.arguments)
+ assert args["first"] == 7
+ assert args["second"] == 6
+
+ # Second turn: provide tool result and ask model to continue
+ messages.append({"role": "tool", "content": "7 x 6 = 42."})
+ messages.append({"role": "system", "content": "Respond only with the answer generated by the tool."})
+
+ client.settings.tool_choice = {"type": "auto"}
+ response = client.complete_chat(messages, tools)
+
+ assert response.choices[0].message.content is not None
+ assert "42" in response.choices[0].message.content
+ finally:
+ model.unload()
+
+ def test_should_perform_tool_calling_streaming_chat_completion(self, catalog):
+ """Tool calling (streaming): model uses multiply_numbers tool, then continue conversation."""
+ model = _get_loaded_chat_model(catalog)
+ try:
+ client = model.get_chat_client()
+ client.settings.max_tokens = 500
+ client.settings.temperature = 0.0
+ client.settings.tool_choice = {"type": "required"}
+
+ messages = [
+ {"role": "system", "content": "You are a helpful AI assistant. If necessary, you can use any provided tools to answer the question."},
+ {"role": "user", "content": "What is the answer to 7 multiplied by 6?"},
+ ]
+ tools = [get_multiply_tool()]
+
+ # First turn: collect chunks and find the tool call
+ chunks = list(client.complete_streaming_chat(messages, tools))
+ last_tool_call_chunk = next(
+ (c for c in reversed(chunks)
+ if c.choices and c.choices[0].delta and c.choices[0].delta.tool_calls),
+ None,
+ )
+ assert last_tool_call_chunk is not None
+
+ tool_call_choice = last_tool_call_chunk.choices[0]
+ assert tool_call_choice.finish_reason == "tool_calls"
+
+ tool_call = tool_call_choice.delta.tool_calls[0]
+ assert tool_call.type == "function"
+ assert tool_call.function.name == "multiply_numbers"
+
+ args = json.loads(tool_call.function.arguments)
+ assert args["first"] == 7
+ assert args["second"] == 6
+
+ # Second turn: provide tool result and continue
+ messages.append({"role": "tool", "content": "7 x 6 = 42."})
+ messages.append({"role": "system", "content": "Respond only with the answer generated by the tool."})
+
+ client.settings.tool_choice = {"type": "auto"}
+
+ chunks = list(client.complete_streaming_chat(messages, tools))
+ second_response = "".join(
+ c.choices[0].delta.content
+ for c in chunks
+ if c.choices and c.choices[0].delta and c.choices[0].delta.content
+ )
+ assert "42" in second_response
+ finally:
+ model.unload()
+
+ def test_should_return_generator(self, catalog):
+ """complete_streaming_chat returns a generator that yields chunks."""
+ model = _get_loaded_chat_model(catalog)
+ try:
+ client = model.get_chat_client()
+ client.settings.max_tokens = 50
+ client.settings.temperature = 0.0
+
+ result = client.complete_streaming_chat([{"role": "user", "content": "Say hi."}])
+
+ assert result is not None
+ chunks = list(result)
+ assert len(chunks) > 0
+ finally:
+ model.unload()
\ No newline at end of file
diff --git a/sdk/python/test/test_catalog.py b/sdk/python/test/test_catalog.py
new file mode 100644
index 00000000..aeb39c20
--- /dev/null
+++ b/sdk/python/test/test_catalog.py
@@ -0,0 +1,74 @@
+# -------------------------------------------------------------------------
+# Copyright (c) Microsoft Corporation. All rights reserved.
+# Licensed under the MIT License.
+# --------------------------------------------------------------------------
+"""Tests for Catalog – mirrors catalog.test.ts."""
+
+from __future__ import annotations
+
+from .conftest import TEST_MODEL_ALIAS
+
+
+class TestCatalog:
+ """Catalog Tests."""
+
+ def test_should_initialize_with_catalog_name(self, catalog):
+ """Catalog should expose a non-empty name string."""
+ assert isinstance(catalog.name, str)
+ assert len(catalog.name) > 0
+
+ def test_should_list_models(self, catalog):
+ """list_models() should return a non-empty list containing the test model."""
+ models = catalog.list_models()
+ assert isinstance(models, list)
+ assert len(models) > 0
+
+ # Verify test model is present
+ aliases = {m.alias for m in models}
+ assert TEST_MODEL_ALIAS in aliases
+
+ def test_should_get_model_by_alias(self, catalog):
+ """get_model() should return a Model whose alias matches."""
+ model = catalog.get_model(TEST_MODEL_ALIAS)
+ assert model is not None
+ assert model.alias == TEST_MODEL_ALIAS
+
+ def test_should_return_none_for_empty_alias(self, catalog):
+ """get_model('') should return None (unknown alias)."""
+ result = catalog.get_model("")
+ assert result is None
+
+ def test_should_return_none_for_unknown_alias(self, catalog):
+ """get_model() with a random alias should return None."""
+ result = catalog.get_model("definitely-not-a-real-model-alias-12345")
+ assert result is None
+
+ def test_should_get_cached_models(self, catalog):
+ """get_cached_models() should return a list with at least the test model."""
+ cached = catalog.get_cached_models()
+ assert isinstance(cached, list)
+ assert len(cached) > 0
+
+ # At least the test model should be cached
+ aliases = {m.alias for m in cached}
+ assert TEST_MODEL_ALIAS in aliases
+
+ def test_should_get_model_variant_by_id(self, catalog):
+ """get_model_variant() with a valid ID should return the variant."""
+ cached = catalog.get_cached_models()
+ assert len(cached) > 0
+ variant = cached[0]
+
+ result = catalog.get_model_variant(variant.id)
+ assert result is not None
+ assert result.id == variant.id
+
+ def test_should_return_none_for_empty_variant_id(self, catalog):
+ """get_model_variant('') should return None."""
+ result = catalog.get_model_variant("")
+ assert result is None
+
+ def test_should_return_none_for_unknown_variant_id(self, catalog):
+ """get_model_variant() with a random ID should return None."""
+ result = catalog.get_model_variant("definitely-not-a-real-model-id-12345")
+ assert result is None
diff --git a/sdk/python/test/test_foundry_local_manager.py b/sdk/python/test/test_foundry_local_manager.py
new file mode 100644
index 00000000..b0a9c4e2
--- /dev/null
+++ b/sdk/python/test/test_foundry_local_manager.py
@@ -0,0 +1,22 @@
+# -------------------------------------------------------------------------
+# Copyright (c) Microsoft Corporation. All rights reserved.
+# Licensed under the MIT License.
+# --------------------------------------------------------------------------
+"""Tests for FoundryLocalManager – mirrors foundryLocalManager.test.ts."""
+
+from __future__ import annotations
+
+
+class TestFoundryLocalManager:
+ """Foundry Local Manager Tests."""
+
+ def test_should_initialize_successfully(self, manager):
+ """Manager singleton should be non-None after initialize()."""
+ assert manager is not None
+
+ def test_should_return_catalog(self, manager):
+ """Manager should expose a Catalog with a non-empty name."""
+ catalog = manager.catalog
+ assert catalog is not None
+ assert isinstance(catalog.name, str)
+ assert len(catalog.name) > 0
diff --git a/sdk/python/test/test_model.py b/sdk/python/test/test_model.py
new file mode 100644
index 00000000..54a30ef4
--- /dev/null
+++ b/sdk/python/test/test_model.py
@@ -0,0 +1,58 @@
+# -------------------------------------------------------------------------
+# Copyright (c) Microsoft Corporation. All rights reserved.
+# Licensed under the MIT License.
+# --------------------------------------------------------------------------
+"""Tests for Model – mirrors model.test.ts."""
+
+from __future__ import annotations
+
+from .conftest import TEST_MODEL_ALIAS, AUDIO_MODEL_ALIAS
+
+
+class TestModel:
+ """Model Tests."""
+
+ def test_should_verify_cached_models(self, catalog):
+ """Cached models from test-data-shared should include qwen and whisper."""
+ cached = catalog.get_cached_models()
+ assert isinstance(cached, list)
+ assert len(cached) > 0
+
+ # Check qwen model is cached
+ qwen = next((m for m in cached if m.alias == TEST_MODEL_ALIAS), None)
+ assert qwen is not None, f"{TEST_MODEL_ALIAS} should be cached"
+ assert qwen.is_cached is True
+
+ # Check whisper model is cached
+ whisper = next((m for m in cached if m.alias == AUDIO_MODEL_ALIAS), None)
+ assert whisper is not None, f"{AUDIO_MODEL_ALIAS} should be cached"
+ assert whisper.is_cached is True
+
+ def test_should_load_and_unload_model(self, catalog):
+ """Load/unload cycle should toggle is_loaded on the selected variant."""
+ cached = catalog.get_cached_models()
+ assert len(cached) > 0
+
+ cached_variant = next((m for m in cached if m.alias == TEST_MODEL_ALIAS), None)
+ assert cached_variant is not None
+
+ model = catalog.get_model(TEST_MODEL_ALIAS)
+ assert model is not None
+
+ model.select_variant(cached_variant)
+
+ # Ensure it's not loaded initially (or unload if it is)
+ if model.is_loaded:
+ model.unload()
+ assert model.is_loaded is False
+
+ try:
+ model.load()
+ assert model.is_loaded is True
+
+ model.unload()
+ assert model.is_loaded is False
+ finally:
+ # Safety cleanup
+ if model.is_loaded:
+ model.unload()
From 6ae3337975e06d15b606c1bc1effb3e1de2ce513 Mon Sep 17 00:00:00 2001
From: Copilot <198982749+Copilot@users.noreply.github.com>
Date: Fri, 27 Mar 2026 14:39:53 -0500
Subject: [PATCH 06/21] Update privacy policy link in website footer (#557)
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit
Replaces the hardcoded privacy statement URL in the footer with the
Microsoft short-link redirect.
## Changes
- **`www/src/lib/components/home/footer.svelte`**: Updated `href` from
`https://www.microsoft.com/en-us/privacy/privacystatement` →
`https://go.microsoft.com/fwlink/?LinkId=521839`
Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com>
Co-authored-by: MaanavD <24942306+MaanavD@users.noreply.github.com>
---
www/src/lib/components/home/footer.svelte | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/www/src/lib/components/home/footer.svelte b/www/src/lib/components/home/footer.svelte
index 44bc8df2..03a4df2d 100644
--- a/www/src/lib/components/home/footer.svelte
+++ b/www/src/lib/components/home/footer.svelte
@@ -111,7 +111,7 @@
© {new Date().getFullYear()} Microsoft Corporation. All rights reserved.
Date: Fri, 27 Mar 2026 15:00:18 -0500
Subject: [PATCH 07/21] Add model context capabilities (#554)
SDK: add contextLength, inputModalities, outputModalities, capabilities
- C# ModelInfo: add ContextLength, InputModalities, OutputModalities,
Capabilities
- JS ModelInfo/IModel/Model/ModelVariant: add new fields and convenience
getters
- Rust ModelInfo: add new fields; Model: add accessor methods
---------
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
Co-authored-by: maanavd
---
.github/workflows/build-rust-steps.yml | 2 +-
.../GettingStarted/Directory.Packages.props | 4 +-
sdk/cs/src/FoundryModelInfo.cs | 12 +++
sdk/js/docs/README.md | 84 +++++++++++++++++
sdk/js/docs/classes/AudioClient.md | 2 +-
sdk/js/docs/classes/AudioClientSettings.md | 2 +-
sdk/js/docs/classes/Catalog.md | 2 +-
sdk/js/docs/classes/ChatClient.md | 2 +-
sdk/js/docs/classes/ChatClientSettings.md | 2 +-
sdk/js/docs/classes/FoundryLocalManager.md | 2 +-
sdk/js/docs/classes/Model.md | 92 ++++++++++++++++++-
sdk/js/docs/classes/ModelLoadManager.md | 2 +-
sdk/js/docs/classes/ModelVariant.md | 92 ++++++++++++++++++-
sdk/js/docs/classes/ResponsesClient.md | 2 +-
.../docs/classes/ResponsesClientSettings.md | 2 +-
sdk/js/src/imodel.ts | 6 ++
sdk/js/src/model.ts | 20 ++++
sdk/js/src/modelVariant.ts | 20 ++++
sdk/js/src/types.ts | 4 +
sdk/rust/build.rs | 45 ++-------
sdk/rust/src/model.rs | 25 +++++
sdk/rust/src/types.rs | 8 ++
www/.npmrc | 3 -
www/package.json | 2 +-
24 files changed, 382 insertions(+), 55 deletions(-)
delete mode 100644 www/.npmrc
diff --git a/.github/workflows/build-rust-steps.yml b/.github/workflows/build-rust-steps.yml
index 27c22da8..f007b7ee 100644
--- a/.github/workflows/build-rust-steps.yml
+++ b/.github/workflows/build-rust-steps.yml
@@ -28,7 +28,7 @@ jobs:
working-directory: sdk/rust
env:
- CARGO_FEATURES: ${{ inputs.useWinML && '--features winml' || '' }}
+ CARGO_FEATURES: ${{ inputs.useWinML && '--features winml,nightly' || '--features nightly' }}
steps:
- name: Checkout repository
diff --git a/samples/cs/GettingStarted/Directory.Packages.props b/samples/cs/GettingStarted/Directory.Packages.props
index 2d91a9fe..02984002 100644
--- a/samples/cs/GettingStarted/Directory.Packages.props
+++ b/samples/cs/GettingStarted/Directory.Packages.props
@@ -5,8 +5,8 @@
1.23.2
-
-
+
+
diff --git a/sdk/cs/src/FoundryModelInfo.cs b/sdk/cs/src/FoundryModelInfo.cs
index 1f795d22..2d1327cc 100644
--- a/sdk/cs/src/FoundryModelInfo.cs
+++ b/sdk/cs/src/FoundryModelInfo.cs
@@ -119,4 +119,16 @@ public record ModelInfo
[JsonPropertyName("createdAt")]
public long CreatedAtUnix { get; init; }
+
+ [JsonPropertyName("contextLength")]
+ public long? ContextLength { get; init; }
+
+ [JsonPropertyName("inputModalities")]
+ public string? InputModalities { get; init; }
+
+ [JsonPropertyName("outputModalities")]
+ public string? OutputModalities { get; init; }
+
+ [JsonPropertyName("capabilities")]
+ public string? Capabilities { get; init; }
}
diff --git a/sdk/js/docs/README.md b/sdk/js/docs/README.md
index dd483aa4..5e50e636 100644
--- a/sdk/js/docs/README.md
+++ b/sdk/js/docs/README.md
@@ -462,6 +462,30 @@ get alias(): string;
`string`
+##### capabilities
+
+###### Get Signature
+
+```ts
+get capabilities(): string | null;
+```
+
+###### Returns
+
+`string` \| `null`
+
+##### contextLength
+
+###### Get Signature
+
+```ts
+get contextLength(): number | null;
+```
+
+###### Returns
+
+`number` \| `null`
+
##### id
###### Get Signature
@@ -474,6 +498,18 @@ get id(): string;
`string`
+##### inputModalities
+
+###### Get Signature
+
+```ts
+get inputModalities(): string | null;
+```
+
+###### Returns
+
+`string` \| `null`
+
##### isCached
###### Get Signature
@@ -486,6 +522,18 @@ get isCached(): boolean;
`boolean`
+##### outputModalities
+
+###### Get Signature
+
+```ts
+get outputModalities(): string | null;
+```
+
+###### Returns
+
+`string` \| `null`
+
##### path
###### Get Signature
@@ -498,6 +546,18 @@ get path(): string;
`string`
+##### supportsToolCalling
+
+###### Get Signature
+
+```ts
+get supportsToolCalling(): boolean | null;
+```
+
+###### Returns
+
+`boolean` \| `null`
+
#### Methods
##### createAudioClient()
@@ -740,6 +800,18 @@ alias: string;
cached: boolean;
```
+##### capabilities?
+
+```ts
+optional capabilities?: string | null;
+```
+
+##### contextLength?
+
+```ts
+optional contextLength?: number | null;
+```
+
##### createdAtUnix
```ts
@@ -764,6 +836,12 @@ optional fileSizeMb?: number | null;
id: string;
```
+##### inputModalities?
+
+```ts
+optional inputModalities?: string | null;
+```
+
##### license?
```ts
@@ -806,6 +884,12 @@ modelType: string;
name: string;
```
+##### outputModalities?
+
+```ts
+optional outputModalities?: string | null;
+```
+
##### promptTemplate?
```ts
diff --git a/sdk/js/docs/classes/AudioClient.md b/sdk/js/docs/classes/AudioClient.md
index 12e79de5..e661bad0 100644
--- a/sdk/js/docs/classes/AudioClient.md
+++ b/sdk/js/docs/classes/AudioClient.md
@@ -1,4 +1,4 @@
-[@prathikrao/foundry-local-sdk](../README.md) / AudioClient
+[foundry-local-sdk](../README.md) / AudioClient
# Class: AudioClient
diff --git a/sdk/js/docs/classes/AudioClientSettings.md b/sdk/js/docs/classes/AudioClientSettings.md
index dae7cbbe..49e806dc 100644
--- a/sdk/js/docs/classes/AudioClientSettings.md
+++ b/sdk/js/docs/classes/AudioClientSettings.md
@@ -1,4 +1,4 @@
-[@prathikrao/foundry-local-sdk](../README.md) / AudioClientSettings
+[foundry-local-sdk](../README.md) / AudioClientSettings
# Class: AudioClientSettings
diff --git a/sdk/js/docs/classes/Catalog.md b/sdk/js/docs/classes/Catalog.md
index b77f254f..23f7cff3 100644
--- a/sdk/js/docs/classes/Catalog.md
+++ b/sdk/js/docs/classes/Catalog.md
@@ -1,4 +1,4 @@
-[@prathikrao/foundry-local-sdk](../README.md) / Catalog
+[foundry-local-sdk](../README.md) / Catalog
# Class: Catalog
diff --git a/sdk/js/docs/classes/ChatClient.md b/sdk/js/docs/classes/ChatClient.md
index c3120f0b..26cc6f0c 100644
--- a/sdk/js/docs/classes/ChatClient.md
+++ b/sdk/js/docs/classes/ChatClient.md
@@ -1,4 +1,4 @@
-[@prathikrao/foundry-local-sdk](../README.md) / ChatClient
+[foundry-local-sdk](../README.md) / ChatClient
# Class: ChatClient
diff --git a/sdk/js/docs/classes/ChatClientSettings.md b/sdk/js/docs/classes/ChatClientSettings.md
index 7d48bcca..323bd3ca 100644
--- a/sdk/js/docs/classes/ChatClientSettings.md
+++ b/sdk/js/docs/classes/ChatClientSettings.md
@@ -1,4 +1,4 @@
-[@prathikrao/foundry-local-sdk](../README.md) / ChatClientSettings
+[foundry-local-sdk](../README.md) / ChatClientSettings
# Class: ChatClientSettings
diff --git a/sdk/js/docs/classes/FoundryLocalManager.md b/sdk/js/docs/classes/FoundryLocalManager.md
index fb9a4783..63bb2dd1 100644
--- a/sdk/js/docs/classes/FoundryLocalManager.md
+++ b/sdk/js/docs/classes/FoundryLocalManager.md
@@ -1,4 +1,4 @@
-[@prathikrao/foundry-local-sdk](../README.md) / FoundryLocalManager
+[foundry-local-sdk](../README.md) / FoundryLocalManager
# Class: FoundryLocalManager
diff --git a/sdk/js/docs/classes/Model.md b/sdk/js/docs/classes/Model.md
index 424d673b..0b2dcfa6 100644
--- a/sdk/js/docs/classes/Model.md
+++ b/sdk/js/docs/classes/Model.md
@@ -1,4 +1,4 @@
-[@prathikrao/foundry-local-sdk](../README.md) / Model
+[foundry-local-sdk](../README.md) / Model
# Class: Model
@@ -51,6 +51,42 @@ The model alias.
***
+### capabilities
+
+#### Get Signature
+
+```ts
+get capabilities(): string | null;
+```
+
+##### Returns
+
+`string` \| `null`
+
+#### Implementation of
+
+[`IModel`](../README.md#imodel).[`capabilities`](../README.md#capabilities)
+
+***
+
+### contextLength
+
+#### Get Signature
+
+```ts
+get contextLength(): number | null;
+```
+
+##### Returns
+
+`number` \| `null`
+
+#### Implementation of
+
+[`IModel`](../README.md#imodel).[`contextLength`](../README.md#contextlength)
+
+***
+
### id
#### Get Signature
@@ -73,6 +109,24 @@ The ID of the selected variant.
***
+### inputModalities
+
+#### Get Signature
+
+```ts
+get inputModalities(): string | null;
+```
+
+##### Returns
+
+`string` \| `null`
+
+#### Implementation of
+
+[`IModel`](../README.md#imodel).[`inputModalities`](../README.md#inputmodalities)
+
+***
+
### isCached
#### Get Signature
@@ -95,6 +149,24 @@ True if cached, false otherwise.
***
+### outputModalities
+
+#### Get Signature
+
+```ts
+get outputModalities(): string | null;
+```
+
+##### Returns
+
+`string` \| `null`
+
+#### Implementation of
+
+[`IModel`](../README.md#imodel).[`outputModalities`](../README.md#outputmodalities)
+
+***
+
### path
#### Get Signature
@@ -117,6 +189,24 @@ The local file path.
***
+### supportsToolCalling
+
+#### Get Signature
+
+```ts
+get supportsToolCalling(): boolean | null;
+```
+
+##### Returns
+
+`boolean` \| `null`
+
+#### Implementation of
+
+[`IModel`](../README.md#imodel).[`supportsToolCalling`](../README.md#supportstoolcalling)
+
+***
+
### variants
#### Get Signature
diff --git a/sdk/js/docs/classes/ModelLoadManager.md b/sdk/js/docs/classes/ModelLoadManager.md
index f445659b..564d561f 100644
--- a/sdk/js/docs/classes/ModelLoadManager.md
+++ b/sdk/js/docs/classes/ModelLoadManager.md
@@ -1,4 +1,4 @@
-[@prathikrao/foundry-local-sdk](../README.md) / ModelLoadManager
+[foundry-local-sdk](../README.md) / ModelLoadManager
# Class: ModelLoadManager
diff --git a/sdk/js/docs/classes/ModelVariant.md b/sdk/js/docs/classes/ModelVariant.md
index 837ead70..6f4e5ee8 100644
--- a/sdk/js/docs/classes/ModelVariant.md
+++ b/sdk/js/docs/classes/ModelVariant.md
@@ -1,4 +1,4 @@
-[@prathikrao/foundry-local-sdk](../README.md) / ModelVariant
+[foundry-local-sdk](../README.md) / ModelVariant
# Class: ModelVariant
@@ -56,6 +56,42 @@ The model alias.
***
+### capabilities
+
+#### Get Signature
+
+```ts
+get capabilities(): string | null;
+```
+
+##### Returns
+
+`string` \| `null`
+
+#### Implementation of
+
+[`IModel`](../README.md#imodel).[`capabilities`](../README.md#capabilities)
+
+***
+
+### contextLength
+
+#### Get Signature
+
+```ts
+get contextLength(): number | null;
+```
+
+##### Returns
+
+`number` \| `null`
+
+#### Implementation of
+
+[`IModel`](../README.md#imodel).[`contextLength`](../README.md#contextlength)
+
+***
+
### id
#### Get Signature
@@ -78,6 +114,24 @@ The model ID.
***
+### inputModalities
+
+#### Get Signature
+
+```ts
+get inputModalities(): string | null;
+```
+
+##### Returns
+
+`string` \| `null`
+
+#### Implementation of
+
+[`IModel`](../README.md#imodel).[`inputModalities`](../README.md#inputmodalities)
+
+***
+
### isCached
#### Get Signature
@@ -118,6 +172,24 @@ The ModelInfo object.
***
+### outputModalities
+
+#### Get Signature
+
+```ts
+get outputModalities(): string | null;
+```
+
+##### Returns
+
+`string` \| `null`
+
+#### Implementation of
+
+[`IModel`](../README.md#imodel).[`outputModalities`](../README.md#outputmodalities)
+
+***
+
### path
#### Get Signature
@@ -138,6 +210,24 @@ The local file path.
[`IModel`](../README.md#imodel).[`path`](../README.md#path)
+***
+
+### supportsToolCalling
+
+#### Get Signature
+
+```ts
+get supportsToolCalling(): boolean | null;
+```
+
+##### Returns
+
+`boolean` \| `null`
+
+#### Implementation of
+
+[`IModel`](../README.md#imodel).[`supportsToolCalling`](../README.md#supportstoolcalling)
+
## Methods
### createAudioClient()
diff --git a/sdk/js/docs/classes/ResponsesClient.md b/sdk/js/docs/classes/ResponsesClient.md
index 5ee70c81..0ccd9a60 100644
--- a/sdk/js/docs/classes/ResponsesClient.md
+++ b/sdk/js/docs/classes/ResponsesClient.md
@@ -1,4 +1,4 @@
-[@prathikrao/foundry-local-sdk](../README.md) / ResponsesClient
+[foundry-local-sdk](../README.md) / ResponsesClient
# Class: ResponsesClient
diff --git a/sdk/js/docs/classes/ResponsesClientSettings.md b/sdk/js/docs/classes/ResponsesClientSettings.md
index 8401faf1..47dfc55e 100644
--- a/sdk/js/docs/classes/ResponsesClientSettings.md
+++ b/sdk/js/docs/classes/ResponsesClientSettings.md
@@ -1,4 +1,4 @@
-[@prathikrao/foundry-local-sdk](../README.md) / ResponsesClientSettings
+[foundry-local-sdk](../README.md) / ResponsesClientSettings
# Class: ResponsesClientSettings
diff --git a/sdk/js/src/imodel.ts b/sdk/js/src/imodel.ts
index be0913d6..f5b72622 100644
--- a/sdk/js/src/imodel.ts
+++ b/sdk/js/src/imodel.ts
@@ -8,6 +8,12 @@ export interface IModel {
get isCached(): boolean;
isLoaded(): Promise;
+ get contextLength(): number | null;
+ get inputModalities(): string | null;
+ get outputModalities(): string | null;
+ get capabilities(): string | null;
+ get supportsToolCalling(): boolean | null;
+
download(progressCallback?: (progress: number) => void): Promise;
get path(): string;
load(): Promise;
diff --git a/sdk/js/src/model.ts b/sdk/js/src/model.ts
index e2b37119..155d5dd1 100644
--- a/sdk/js/src/model.ts
+++ b/sdk/js/src/model.ts
@@ -104,6 +104,26 @@ export class Model implements IModel {
return this._variants;
}
+ public get contextLength(): number | null {
+ return this.selectedVariant.contextLength;
+ }
+
+ public get inputModalities(): string | null {
+ return this.selectedVariant.inputModalities;
+ }
+
+ public get outputModalities(): string | null {
+ return this.selectedVariant.outputModalities;
+ }
+
+ public get capabilities(): string | null {
+ return this.selectedVariant.capabilities;
+ }
+
+ public get supportsToolCalling(): boolean | null {
+ return this.selectedVariant.supportsToolCalling;
+ }
+
/**
* Downloads the currently selected variant.
* @param progressCallback - Optional callback to report download progress.
diff --git a/sdk/js/src/modelVariant.ts b/sdk/js/src/modelVariant.ts
index 4d3e2bee..db06033a 100644
--- a/sdk/js/src/modelVariant.ts
+++ b/sdk/js/src/modelVariant.ts
@@ -45,6 +45,26 @@ export class ModelVariant implements IModel {
return this._modelInfo;
}
+ public get contextLength(): number | null {
+ return this._modelInfo.contextLength ?? null;
+ }
+
+ public get inputModalities(): string | null {
+ return this._modelInfo.inputModalities ?? null;
+ }
+
+ public get outputModalities(): string | null {
+ return this._modelInfo.outputModalities ?? null;
+ }
+
+ public get capabilities(): string | null {
+ return this._modelInfo.capabilities ?? null;
+ }
+
+ public get supportsToolCalling(): boolean | null {
+ return this._modelInfo.supportsToolCalling ?? null;
+ }
+
/**
* Checks if the model variant is cached locally.
* @returns True if cached, false otherwise.
diff --git a/sdk/js/src/types.ts b/sdk/js/src/types.ts
index 639676de..40a9110b 100644
--- a/sdk/js/src/types.ts
+++ b/sdk/js/src/types.ts
@@ -50,6 +50,10 @@ export interface ModelInfo {
maxOutputTokens?: number | null;
minFLVersion?: string | null;
createdAtUnix: number;
+ contextLength?: number | null;
+ inputModalities?: string | null;
+ outputModalities?: string | null;
+ capabilities?: string | null;
}
export interface ResponseFormat {
diff --git a/sdk/rust/build.rs b/sdk/rust/build.rs
index 0f9726d5..996eaf2a 100644
--- a/sdk/rust/build.rs
+++ b/sdk/rust/build.rs
@@ -9,7 +9,7 @@ const ORT_NIGHTLY_FEED: &str =
const CORE_VERSION: &str = "0.9.0.8-rc3";
const ORT_VERSION: &str = "1.24.3";
-const GENAI_VERSION: &str = "0.12.2";
+const GENAI_VERSION: &str = "0.13.0-dev-20260319-1131106-439ca0d5";
const WINML_ORT_VERSION: &str = "1.23.2.3";
@@ -42,29 +42,18 @@ fn native_lib_extension() -> &'static str {
fn get_packages(rid: &str) -> Vec {
let winml = env::var("CARGO_FEATURE_WINML").is_ok();
- let nightly = env::var("CARGO_FEATURE_NIGHTLY").is_ok();
let is_linux = rid.starts_with("linux");
- let core_version = if nightly {
- resolve_latest_version("Microsoft.AI.Foundry.Local.Core", ORT_NIGHTLY_FEED)
- .unwrap_or_else(|| CORE_VERSION.to_string())
- } else {
- CORE_VERSION.to_string()
- };
+ // Use pinned versions directly — dynamic resolution via resolve_latest_version
+ // is unreliable (feed returns versions in unexpected order, and some old versions
+ // require authentication).
let mut packages = Vec::new();
if winml {
- let winml_core_version = if nightly {
- resolve_latest_version("Microsoft.AI.Foundry.Local.Core.WinML", ORT_NIGHTLY_FEED)
- .unwrap_or_else(|| CORE_VERSION.to_string())
- } else {
- CORE_VERSION.to_string()
- };
-
packages.push(NuGetPackage {
name: "Microsoft.AI.Foundry.Local.Core.WinML",
- version: winml_core_version,
+ version: CORE_VERSION.to_string(),
feed_url: ORT_NIGHTLY_FEED,
});
packages.push(NuGetPackage {
@@ -75,12 +64,12 @@ fn get_packages(rid: &str) -> Vec {
packages.push(NuGetPackage {
name: "Microsoft.ML.OnnxRuntimeGenAI.WinML",
version: GENAI_VERSION.to_string(),
- feed_url: NUGET_FEED,
+ feed_url: ORT_NIGHTLY_FEED,
});
} else {
packages.push(NuGetPackage {
name: "Microsoft.AI.Foundry.Local.Core",
- version: core_version,
+ version: CORE_VERSION.to_string(),
feed_url: ORT_NIGHTLY_FEED,
});
@@ -101,7 +90,7 @@ fn get_packages(rid: &str) -> Vec {
packages.push(NuGetPackage {
name: "Microsoft.ML.OnnxRuntimeGenAI.Foundry",
version: GENAI_VERSION.to_string(),
- feed_url: NUGET_FEED,
+ feed_url: ORT_NIGHTLY_FEED,
});
}
@@ -143,24 +132,6 @@ fn resolve_base_address(feed_url: &str) -> Result {
))
}
-/// Resolve the latest version of a package from a NuGet feed.
-fn resolve_latest_version(package_name: &str, feed_url: &str) -> Option {
- let base_address = resolve_base_address(feed_url).ok()?;
- let lower_name = package_name.to_lowercase();
- let index_url = format!("{base_address}{lower_name}/index.json");
-
- let body: String = ureq::get(&index_url)
- .call()
- .ok()?
- .body_mut()
- .read_to_string()
- .ok()?;
-
- let index: serde_json::Value = serde_json::from_str(&body).ok()?;
- let versions = index["versions"].as_array()?;
- versions.last()?.as_str().map(|s| s.to_string())
-}
-
/// Download a .nupkg and extract native libraries for the given RID into `out_dir`.
fn download_and_extract(pkg: &NuGetPackage, rid: &str, out_dir: &Path) -> Result<(), String> {
let base_address = resolve_base_address(pkg.feed_url)?;
diff --git a/sdk/rust/src/model.rs b/sdk/rust/src/model.rs
index 4a197e3f..50c1fe1a 100644
--- a/sdk/rust/src/model.rs
+++ b/sdk/rust/src/model.rs
@@ -113,6 +113,31 @@ impl Model {
self.selected_variant().is_loaded().await
}
+ /// Context length (maximum input tokens) of the selected variant.
+ pub fn context_length(&self) -> Option {
+ self.selected_variant().info().context_length
+ }
+
+ /// Input modalities of the selected variant (e.g. "text", "text,image").
+ pub fn input_modalities(&self) -> Option<&str> {
+ self.selected_variant().info().input_modalities.as_deref()
+ }
+
+ /// Output modalities of the selected variant (e.g. "text").
+ pub fn output_modalities(&self) -> Option<&str> {
+ self.selected_variant().info().output_modalities.as_deref()
+ }
+
+ /// Capabilities of the selected variant (e.g. "reasoning", "tool-calling").
+ pub fn capabilities(&self) -> Option<&str> {
+ self.selected_variant().info().capabilities.as_deref()
+ }
+
+ /// Whether the selected variant supports tool calling.
+ pub fn supports_tool_calling(&self) -> Option {
+ self.selected_variant().info().supports_tool_calling
+ }
+
/// Download the selected variant. If `progress` is provided, it receives
/// human-readable progress strings as they arrive from the native core.
pub async fn download(&self, progress: Option) -> Result<()>
diff --git a/sdk/rust/src/types.rs b/sdk/rust/src/types.rs
index d1d1f002..bab2f9c8 100644
--- a/sdk/rust/src/types.rs
+++ b/sdk/rust/src/types.rs
@@ -87,6 +87,14 @@ pub struct ModelInfo {
pub min_fl_version: Option,
#[serde(default)]
pub created_at_unix: u64,
+ #[serde(skip_serializing_if = "Option::is_none")]
+ pub context_length: Option,
+ #[serde(skip_serializing_if = "Option::is_none")]
+ pub input_modalities: Option,
+ #[serde(skip_serializing_if = "Option::is_none")]
+ pub output_modalities: Option,
+ #[serde(skip_serializing_if = "Option::is_none")]
+ pub capabilities: Option,
}
/// Desired response format for chat completions.
diff --git a/www/.npmrc b/www/.npmrc
deleted file mode 100644
index 06fe7275..00000000
--- a/www/.npmrc
+++ /dev/null
@@ -1,3 +0,0 @@
-registry=https://pkgs.dev.azure.com/aiinfra/PublicPackages/_packaging/ORT-Nightly/npm/registry/
-always-auth=true
-engine-strict=true
diff --git a/www/package.json b/www/package.json
index 8a311947..5454236d 100644
--- a/www/package.json
+++ b/www/package.json
@@ -12,7 +12,7 @@
},
"license": "MIT",
"engines": {
- "node": ">=22.0.0",
+ "node": ">=22.0.0 <23.0.0",
"npm": ">=9.0.0"
},
"scripts": {
From 2412c2f6a138b32daad16de649ca98ea687948d6 Mon Sep 17 00:00:00 2001
From: Nenad Banfic <46795300+nenad1002@users.noreply.github.com>
Date: Mon, 30 Mar 2026 04:05:54 -0700
Subject: [PATCH 08/21] Rust bug fixes & changes (#560)
Part 1 of Rust changes (have part 2 but don't have time to test it now).
This is mostly improving perf by reducing cloning and fixing some bugs +
making code more readable (avoiding early returns).
---
sdk/rust/docs/api.md | 2 +-
sdk/rust/src/catalog.rs | 15 ++++----
sdk/rust/src/configuration.rs | 39 ++++++++-----------
sdk/rust/src/detail/core_interop.rs | 47 +++++++++++++++--------
sdk/rust/src/detail/model_load_manager.rs | 34 ++++++++--------
sdk/rust/src/model.rs | 34 ++++++++--------
sdk/rust/src/model_variant.rs | 4 +-
sdk/rust/src/openai/audio_client.rs | 4 +-
sdk/rust/src/openai/chat_client.rs | 4 +-
9 files changed, 98 insertions(+), 85 deletions(-)
diff --git a/sdk/rust/docs/api.md b/sdk/rust/docs/api.md
index bdc86974..278402fb 100644
--- a/sdk/rust/docs/api.md
+++ b/sdk/rust/docs/api.md
@@ -149,7 +149,7 @@ pub struct Model { /* private fields */ }
|--------|-----------|-------------|
| `alias` | `fn alias(&self) -> &str` | Alias shared by all variants. |
| `id` | `fn id(&self) -> &str` | Unique identifier of the selected variant. |
-| `variants` | `fn variants(&self) -> &[ModelVariant]` | All variants in this model. |
+| `variants` | `fn variants(&self) -> &[Arc]` | All variants in this model. |
| `selected_variant` | `fn selected_variant(&self) -> &ModelVariant` | Currently selected variant. |
| `select_variant` | `fn select_variant(&self, id: &str) -> Result<(), FoundryLocalError>` | Select a variant by id. |
| `is_cached` | `async fn is_cached(&self) -> Result` | Whether the selected variant is cached on disk. |
diff --git a/sdk/rust/src/catalog.rs b/sdk/rust/src/catalog.rs
index 78485bff..9e04c943 100644
--- a/sdk/rust/src/catalog.rs
+++ b/sdk/rust/src/catalog.rs
@@ -135,7 +135,7 @@ impl Catalog {
self.update_models().await?;
let s = self.lock_state()?;
s.models_by_alias.get(alias).cloned().ok_or_else(|| {
- let available: Vec<&String> = s.models_by_alias.keys().collect();
+ let available: Vec<&str> = s.models_by_alias.keys().map(|k| k.as_str()).collect();
FoundryLocalError::ModelOperation {
reason: format!("Unknown model alias '{alias}'. Available: {available:?}"),
}
@@ -152,7 +152,7 @@ impl Catalog {
self.update_models().await?;
let s = self.lock_state()?;
s.variants_by_id.get(id).cloned().ok_or_else(|| {
- let available: Vec<&String> = s.variants_by_id.keys().collect();
+ let available: Vec<&str> = s.variants_by_id.keys().map(|k| k.as_str()).collect();
FoundryLocalError::ModelOperation {
reason: format!("Unknown variant id '{id}'. Available: {available:?}"),
}
@@ -216,18 +216,17 @@ impl Catalog {
for info in infos {
let id = info.id.clone();
let alias = info.alias.clone();
- let variant = ModelVariant::new(
+ let variant = Arc::new(ModelVariant::new(
info,
Arc::clone(&self.core),
Arc::clone(&self.model_load_manager),
self.invalidator.clone(),
- );
- let variant_arc = Arc::new(variant.clone());
- id_map.insert(id, variant_arc);
+ ));
+ id_map.insert(id, Arc::clone(&variant));
alias_map_build
- .entry(alias.clone())
- .or_insert_with(|| Model::new(alias, Arc::clone(&self.core)))
+ .entry(alias)
+ .or_insert_with_key(|a| Model::new(a.clone(), Arc::clone(&self.core)))
.add_variant(variant);
}
diff --git a/sdk/rust/src/configuration.rs b/sdk/rust/src/configuration.rs
index d23d5986..c1ec2964 100644
--- a/sdk/rust/src/configuration.rs
+++ b/sdk/rust/src/configuration.rs
@@ -183,31 +183,24 @@ impl Configuration {
let mut params = HashMap::new();
params.insert("AppName".into(), app_name);
- if let Some(v) = config.app_data_dir {
- params.insert("AppDataDir".into(), v);
- }
- if let Some(v) = config.model_cache_dir {
- params.insert("ModelCacheDir".into(), v);
- }
- if let Some(v) = config.logs_dir {
- params.insert("LogsDir".into(), v);
- }
- if let Some(level) = config.log_level {
- params.insert("LogLevel".into(), level.as_core_str().into());
- }
- if let Some(v) = config.web_service_urls {
- params.insert("WebServiceUrls".into(), v);
- }
- if let Some(v) = config.service_endpoint {
- params.insert("WebServiceExternalUrl".into(), v);
- }
- if let Some(v) = config.library_path {
- params.insert("FoundryLocalCorePath".into(), v);
+ let optional_fields = [
+ ("AppDataDir", config.app_data_dir),
+ ("ModelCacheDir", config.model_cache_dir),
+ ("LogsDir", config.logs_dir),
+ ("LogLevel", config.log_level.map(|l| l.as_core_str().into())),
+ ("WebServiceUrls", config.web_service_urls),
+ ("WebServiceExternalUrl", config.service_endpoint),
+ ("FoundryLocalCorePath", config.library_path),
+ ];
+
+ for (key, value) in optional_fields {
+ if let Some(v) = value {
+ params.insert(key.into(), v);
+ }
}
+
if let Some(extra) = config.additional_settings {
- for (k, v) in extra {
- params.insert(k, v);
- }
+ params.extend(extra);
}
Ok((Self { params }, config.logger))
diff --git a/sdk/rust/src/detail/core_interop.rs b/sdk/rust/src/detail/core_interop.rs
index e69a6e98..75146164 100644
--- a/sdk/rust/src/detail/core_interop.rs
+++ b/sdk/rust/src/detail/core_interop.rs
@@ -137,25 +137,42 @@ impl<'a> StreamingCallbackState<'a> {
/// Append raw bytes, decode as much valid UTF-8 as possible, and forward
/// complete text to the callback. Any trailing incomplete multi-byte
- /// sequence is kept in the buffer for the next call.
+ /// sequence is kept in the buffer for the next call. Invalid byte
+ /// sequences are skipped to prevent the buffer from growing unboundedly.
fn push(&mut self, bytes: &[u8]) {
self.buf.extend_from_slice(bytes);
- let valid_up_to = match std::str::from_utf8(&self.buf) {
- Ok(s) => {
- (self.callback)(s);
- s.len()
- }
- Err(e) => {
- let n = e.valid_up_to();
- if n > 0 {
- // SAFETY: `valid_up_to` guarantees this prefix is valid UTF-8.
- let valid = unsafe { std::str::from_utf8_unchecked(&self.buf[..n]) };
- (self.callback)(valid);
+ loop {
+ match std::str::from_utf8(&self.buf) {
+ Ok(s) => {
+ if !s.is_empty() {
+ (self.callback)(s);
+ }
+ self.buf.clear();
+ break;
+ }
+ Err(e) => {
+ let n = e.valid_up_to();
+ if n > 0 {
+ // SAFETY: `valid_up_to` guarantees this prefix is valid UTF-8.
+ let valid = unsafe { std::str::from_utf8_unchecked(&self.buf[..n]) };
+ (self.callback)(valid);
+ }
+ match e.error_len() {
+ Some(err_len) => {
+ // Definite invalid sequence — skip past it and
+ // continue decoding the remainder.
+ self.buf.drain(..n + err_len);
+ }
+ None => {
+ // Incomplete multi-byte sequence at the end —
+ // keep it for the next push.
+ self.buf.drain(..n);
+ break;
+ }
+ }
}
- n
}
- };
- self.buf.drain(..valid_up_to);
+ }
}
/// Flush any remaining bytes as lossy UTF-8 (called once after the native
diff --git a/sdk/rust/src/detail/model_load_manager.rs b/sdk/rust/src/detail/model_load_manager.rs
index 41507cbd..57eb3cfb 100644
--- a/sdk/rust/src/detail/model_load_manager.rs
+++ b/sdk/rust/src/detail/model_load_manager.rs
@@ -34,12 +34,12 @@ impl ModelLoadManager {
let encoded_id = urlencoding::encode(model_id);
self.http_get(&format!("{base_url}/models/load/{encoded_id}"))
.await?;
- return Ok(());
+ } else {
+ let params = json!({ "Params": { "Model": model_id } });
+ self.core
+ .execute_command_async("load_model".into(), Some(params))
+ .await?;
}
- let params = json!({ "Params": { "Model": model_id } });
- self.core
- .execute_command_async("load_model".into(), Some(params))
- .await?;
Ok(())
}
@@ -47,14 +47,14 @@ impl ModelLoadManager {
pub async fn unload(&self, model_id: &str) -> Result {
if let Some(base_url) = &self.external_service_url {
let encoded_id = urlencoding::encode(model_id);
- return self
- .http_get(&format!("{base_url}/models/unload/{encoded_id}"))
- .await;
+ self.http_get(&format!("{base_url}/models/unload/{encoded_id}"))
+ .await
+ } else {
+ let params = json!({ "Params": { "Model": model_id } });
+ self.core
+ .execute_command_async("unload_model".into(), Some(params))
+ .await
}
- let params = json!({ "Params": { "Model": model_id } });
- self.core
- .execute_command_async("unload_model".into(), Some(params))
- .await
}
/// Return the list of currently loaded model identifiers.
@@ -67,11 +67,11 @@ impl ModelLoadManager {
.await?
};
- if raw.trim().is_empty() {
- return Ok(Vec::new());
- }
-
- let ids: Vec = serde_json::from_str(&raw)?;
+ let ids: Vec = if raw.trim().is_empty() {
+ Vec::new()
+ } else {
+ serde_json::from_str(&raw)?
+ };
Ok(ids)
}
diff --git a/sdk/rust/src/model.rs b/sdk/rust/src/model.rs
index 50c1fe1a..9d08f9a5 100644
--- a/sdk/rust/src/model.rs
+++ b/sdk/rust/src/model.rs
@@ -19,7 +19,7 @@ use crate::openai::ChatClient;
pub struct Model {
alias: String,
core: Arc,
- variants: Vec,
+ variants: Vec>,
selected_index: AtomicUsize,
}
@@ -57,7 +57,7 @@ impl Model {
/// Add a variant. If the new variant is cached and the current selection
/// is not, the new variant becomes the selected one.
- pub(crate) fn add_variant(&mut self, variant: ModelVariant) {
+ pub(crate) fn add_variant(&mut self, variant: Arc) {
self.variants.push(variant);
let new_idx = self.variants.len() - 1;
let current = self.selected_index.load(Relaxed);
@@ -70,17 +70,21 @@ impl Model {
/// Select a variant by its unique id.
pub fn select_variant(&self, id: &str) -> Result<()> {
- if let Some(pos) = self.variants.iter().position(|v| v.id() == id) {
- self.selected_index.store(pos, Relaxed);
- return Ok(());
+ match self.variants.iter().position(|v| v.id() == id) {
+ Some(pos) => {
+ self.selected_index.store(pos, Relaxed);
+ Ok(())
+ }
+ None => {
+ let available: Vec<&str> = self.variants.iter().map(|v| v.id()).collect();
+ Err(FoundryLocalError::ModelOperation {
+ reason: format!(
+ "Variant '{id}' not found for model '{}'. Available: {available:?}",
+ self.alias
+ ),
+ })
+ }
}
- let available: Vec = self.variants.iter().map(|v| v.id().to_string()).collect();
- Err(FoundryLocalError::ModelOperation {
- reason: format!(
- "Variant '{id}' not found for model '{}'. Available: {available:?}",
- self.alias
- ),
- })
}
/// Returns a reference to the currently selected variant.
@@ -89,7 +93,7 @@ impl Model {
}
/// Returns all variants that belong to this model.
- pub fn variants(&self) -> &[ModelVariant] {
+ pub fn variants(&self) -> &[Arc] {
&self.variants
}
@@ -169,11 +173,11 @@ impl Model {
/// Create a [`ChatClient`] bound to the selected variant.
pub fn create_chat_client(&self) -> ChatClient {
- ChatClient::new(self.id().to_string(), Arc::clone(&self.core))
+ ChatClient::new(self.id(), Arc::clone(&self.core))
}
/// Create an [`AudioClient`] bound to the selected variant.
pub fn create_audio_client(&self) -> AudioClient {
- AudioClient::new(self.id().to_string(), Arc::clone(&self.core))
+ AudioClient::new(self.id(), Arc::clone(&self.core))
}
}
diff --git a/sdk/rust/src/model_variant.rs b/sdk/rust/src/model_variant.rs
index c4be6822..760306f6 100644
--- a/sdk/rust/src/model_variant.rs
+++ b/sdk/rust/src/model_variant.rs
@@ -143,11 +143,11 @@ impl ModelVariant {
/// Create a [`ChatClient`] bound to this variant.
pub fn create_chat_client(&self) -> ChatClient {
- ChatClient::new(self.info.id.clone(), Arc::clone(&self.core))
+ ChatClient::new(&self.info.id, Arc::clone(&self.core))
}
/// Create an [`AudioClient`] bound to this variant.
pub fn create_audio_client(&self) -> AudioClient {
- AudioClient::new(self.info.id.clone(), Arc::clone(&self.core))
+ AudioClient::new(&self.info.id, Arc::clone(&self.core))
}
}
diff --git a/sdk/rust/src/openai/audio_client.rs b/sdk/rust/src/openai/audio_client.rs
index da0f9f5b..0319da38 100644
--- a/sdk/rust/src/openai/audio_client.rs
+++ b/sdk/rust/src/openai/audio_client.rs
@@ -116,9 +116,9 @@ pub struct AudioClient {
}
impl AudioClient {
- pub(crate) fn new(model_id: String, core: Arc) -> Self {
+ pub(crate) fn new(model_id: &str, core: Arc) -> Self {
Self {
- model_id,
+ model_id: model_id.to_owned(),
core,
settings: AudioClientSettings::default(),
}
diff --git a/sdk/rust/src/openai/chat_client.rs b/sdk/rust/src/openai/chat_client.rs
index 62d0be5b..6597de82 100644
--- a/sdk/rust/src/openai/chat_client.rs
+++ b/sdk/rust/src/openai/chat_client.rs
@@ -132,9 +132,9 @@ pub struct ChatClient {
}
impl ChatClient {
- pub(crate) fn new(model_id: String, core: Arc) -> Self {
+ pub(crate) fn new(model_id: &str, core: Arc) -> Self {
Self {
- model_id,
+ model_id: model_id.to_owned(),
core,
settings: ChatClientSettings::default(),
}
From 21d84091d6521fee31201d3b6eada32513185319 Mon Sep 17 00:00:00 2001
From: bmehta001
Date: Mon, 30 Mar 2026 18:47:39 -0500
Subject: [PATCH 09/21] Add model context capabilities to Python (#564)
Python SDK: add contextLength, inputModalities, outputModalities,
capabilities; also added tests for these fields
---------
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
---
sdk/python/README.md | 25 +++++++++++++++++++
sdk/python/src/detail/model_data_types.py | 4 +++
sdk/python/src/imodel.py | 30 +++++++++++++++++++++++
sdk/python/src/model.py | 25 +++++++++++++++++++
sdk/python/src/model_variant.py | 25 +++++++++++++++++++
sdk/python/test/test_model.py | 30 +++++++++++++++++++++++
6 files changed, 139 insertions(+)
diff --git a/sdk/python/README.md b/sdk/python/README.md
index 7cc8b44c..ace19bac 100644
--- a/sdk/python/README.md
+++ b/sdk/python/README.md
@@ -142,6 +142,31 @@ cached = catalog.get_cached_models()
loaded = catalog.get_loaded_models()
```
+### Inspecting Model Metadata
+
+`Model` exposes metadata properties from the catalog:
+
+```python
+model = catalog.get_model("phi-3.5-mini")
+
+# Identity
+print(model.id) # e.g. "phi-3.5-mini-instruct-generic-gpu:3"
+print(model.alias) # e.g. "phi-3.5-mini"
+
+# Context and token limits
+print(model.context_length) # e.g. 131072 (tokens), or None if unknown
+
+# Modalities and capabilities
+print(model.input_modalities) # e.g. "text" or "text,image"
+print(model.output_modalities) # e.g. "text"
+print(model.capabilities) # e.g. "chat,completion"
+print(model.supports_tool_calling) # True, False, or None
+
+# Cache / load state
+print(model.is_cached)
+print(model.is_loaded)
+```
+
### Loading and Running a Model
```python
diff --git a/sdk/python/src/detail/model_data_types.py b/sdk/python/src/detail/model_data_types.py
index b8b9e8d6..df367b44 100644
--- a/sdk/python/src/detail/model_data_types.py
+++ b/sdk/python/src/detail/model_data_types.py
@@ -74,3 +74,7 @@ class ModelInfo(BaseModel):
max_output_tokens: Optional[int] = Field(alias="maxOutputTokens")
min_fl_version: Optional[str] = Field(alias="minFLVersion")
created_at_unix: int = Field(alias="createdAt")
+ context_length: Optional[int] = Field(alias="contextLength")
+ input_modalities: Optional[str] = Field(alias="inputModalities")
+ output_modalities: Optional[str] = Field(alias="outputModalities")
+ capabilities: Optional[str] = Field(alias="capabilities")
diff --git a/sdk/python/src/imodel.py b/sdk/python/src/imodel.py
index a092b98e..7f83d1cc 100644
--- a/sdk/python/src/imodel.py
+++ b/sdk/python/src/imodel.py
@@ -37,6 +37,36 @@ def is_loaded(self) -> bool:
"""True if the model is loaded into memory."""
pass
+ @property
+ @abstractmethod
+ def context_length(self) -> Optional[int]:
+ """Maximum context length (in tokens) supported by the model, or ``None`` if unknown."""
+ pass
+
+ @property
+ @abstractmethod
+ def input_modalities(self) -> Optional[str]:
+ """Comma-separated input modalities (e.g. ``"text,image"``), or ``None`` if unknown."""
+ pass
+
+ @property
+ @abstractmethod
+ def output_modalities(self) -> Optional[str]:
+ """Comma-separated output modalities (e.g. ``"text"``), or ``None`` if unknown."""
+ pass
+
+ @property
+ @abstractmethod
+ def capabilities(self) -> Optional[str]:
+ """Comma-separated capability tags (e.g. ``"chat,completion"``), or ``None`` if unknown."""
+ pass
+
+ @property
+ @abstractmethod
+ def supports_tool_calling(self) -> Optional[bool]:
+ """Whether the model supports tool/function calling, or ``None`` if unknown."""
+ pass
+
@abstractmethod
def download(self, progress_callback: Callable[[float], None] = None) -> None:
"""
diff --git a/sdk/python/src/model.py b/sdk/python/src/model.py
index 4c8750ca..f964a820 100644
--- a/sdk/python/src/model.py
+++ b/sdk/python/src/model.py
@@ -94,6 +94,31 @@ def alias(self) -> str:
"""Alias of this model."""
return self._alias
+ @property
+ def context_length(self) -> Optional[int]:
+ """Maximum context length (in tokens) of the currently selected variant."""
+ return self._selected_variant.context_length
+
+ @property
+ def input_modalities(self) -> Optional[str]:
+ """Comma-separated input modalities of the currently selected variant."""
+ return self._selected_variant.input_modalities
+
+ @property
+ def output_modalities(self) -> Optional[str]:
+ """Comma-separated output modalities of the currently selected variant."""
+ return self._selected_variant.output_modalities
+
+ @property
+ def capabilities(self) -> Optional[str]:
+ """Comma-separated capability tags of the currently selected variant."""
+ return self._selected_variant.capabilities
+
+ @property
+ def supports_tool_calling(self) -> Optional[bool]:
+ """Whether the currently selected variant supports tool/function calling."""
+ return self._selected_variant.supports_tool_calling
+
@property
def is_cached(self) -> bool:
"""Is the currently selected variant cached locally?"""
diff --git a/sdk/python/src/model_variant.py b/sdk/python/src/model_variant.py
index f0d40109..1c7ad717 100644
--- a/sdk/python/src/model_variant.py
+++ b/sdk/python/src/model_variant.py
@@ -57,6 +57,31 @@ def info(self) -> ModelInfo:
"""Full catalog metadata for this variant."""
return self._model_info
+ @property
+ def context_length(self) -> Optional[int]:
+ """Maximum context length (in tokens) supported by this variant, or ``None`` if unknown."""
+ return self._model_info.context_length
+
+ @property
+ def input_modalities(self) -> Optional[str]:
+ """Comma-separated input modalities (e.g. ``"text,image"``), or ``None`` if unknown."""
+ return self._model_info.input_modalities
+
+ @property
+ def output_modalities(self) -> Optional[str]:
+ """Comma-separated output modalities (e.g. ``"text"``), or ``None`` if unknown."""
+ return self._model_info.output_modalities
+
+ @property
+ def capabilities(self) -> Optional[str]:
+ """Comma-separated capability tags (e.g. ``"chat,completion"``), or ``None`` if unknown."""
+ return self._model_info.capabilities
+
+ @property
+ def supports_tool_calling(self) -> Optional[bool]:
+ """Whether this variant supports tool/function calling, or ``None`` if unknown."""
+ return self._model_info.supports_tool_calling
+
@property
def is_cached(self) -> bool:
"""``True`` if this variant is present in the local model cache."""
diff --git a/sdk/python/test/test_model.py b/sdk/python/test/test_model.py
index 54a30ef4..e2ea1509 100644
--- a/sdk/python/test/test_model.py
+++ b/sdk/python/test/test_model.py
@@ -56,3 +56,33 @@ def test_should_load_and_unload_model(self, catalog):
# Safety cleanup
if model.is_loaded:
model.unload()
+
+ def test_should_expose_context_length(self, catalog):
+ """Model should expose context_length from ModelInfo metadata."""
+ model = catalog.get_model(TEST_MODEL_ALIAS)
+ assert model is not None
+ # context_length should be None or a positive integer
+ ctx = model.context_length
+ assert ctx is None or (isinstance(ctx, int) and ctx > 0)
+
+ def test_should_expose_modalities(self, catalog):
+ """Model should expose input_modalities and output_modalities."""
+ model = catalog.get_model(TEST_MODEL_ALIAS)
+ assert model is not None
+ # Modalities should be None or non-empty strings
+ for val in (model.input_modalities, model.output_modalities):
+ assert val is None or (isinstance(val, str) and len(val) > 0)
+
+ def test_should_expose_capabilities(self, catalog):
+ """Model should expose capabilities metadata."""
+ model = catalog.get_model(TEST_MODEL_ALIAS)
+ assert model is not None
+ caps = model.capabilities
+ assert caps is None or (isinstance(caps, str) and len(caps) > 0)
+
+ def test_should_expose_supports_tool_calling(self, catalog):
+ """Model should expose supports_tool_calling metadata."""
+ model = catalog.get_model(TEST_MODEL_ALIAS)
+ assert model is not None
+ stc = model.supports_tool_calling
+ assert stc is None or isinstance(stc, bool)
From 58780cbc3d79196dd4f6ac200a05216f8f11ab41 Mon Sep 17 00:00:00 2001
From: Scott McKay
Date: Tue, 31 Mar 2026 17:07:26 +1000
Subject: [PATCH 10/21] Use IModel in the public API. (#556)
Use IModel in the public API. Changes allow ICatalog and IModel to be
stubbed for testing as you no longer need a concrete Model or
ModelVariant class.
- Make Model and ModelVariant implementation details
- Add variant info and selection to IModel so it works with either Model
or ModelVariant
- Move GetLatestVersion to Catalog and take IModel as input
- ModelVariant has insufficient info to implement this and intuitively
the catalog should know this information.
- Update tests
- fix usage of test config file for shared test data path
---------
Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com>
Co-authored-by: skottmckay <979079+skottmckay@users.noreply.github.com>
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
---
.../src/ModelManagementExample/Program.cs | 20 ++-
sdk/cs/src/Catalog.cs | 69 ++++++++--
sdk/cs/src/{ => Detail}/Model.cs | 32 ++---
sdk/cs/src/{ => Detail}/ModelVariant.cs | 11 +-
sdk/cs/src/ICatalog.cs | 30 +++--
sdk/cs/src/IModel.cs | 15 +++
.../FoundryLocal.Tests/AudioClientTests.cs | 2 +-
.../test/FoundryLocal.Tests/CatalogTests.cs | 121 ++++++++++++++++++
.../ChatCompletionsTests.cs | 9 +-
sdk/cs/test/FoundryLocal.Tests/EndToEnd.cs | 5 +-
.../FoundryLocalManagerTest.cs | 2 +-
.../FoundryLocal.Tests/LOCAL_MODEL_TESTING.md | 23 +---
.../TestAssemblySetupCleanup.cs | 22 ++--
sdk/cs/test/FoundryLocal.Tests/Utils.cs | 4 +-
.../FoundryLocal.Tests/appsettings.Test.json | 2 +-
15 files changed, 272 insertions(+), 95 deletions(-)
rename sdk/cs/src/{ => Detail}/Model.cs (74%)
rename sdk/cs/src/{ => Detail}/ModelVariant.cs (95%)
create mode 100644 sdk/cs/test/FoundryLocal.Tests/CatalogTests.cs
diff --git a/samples/cs/GettingStarted/src/ModelManagementExample/Program.cs b/samples/cs/GettingStarted/src/ModelManagementExample/Program.cs
index 2b6fe2e8..38dec588 100644
--- a/samples/cs/GettingStarted/src/ModelManagementExample/Program.cs
+++ b/samples/cs/GettingStarted/src/ModelManagementExample/Program.cs
@@ -51,39 +51,35 @@
// Get a model using an alias from the catalog
var model = await catalog.GetModelAsync("qwen2.5-0.5b") ?? throw new Exception("Model not found");
-// `model.SelectedVariant` indicates which variant will be used by default.
-//
// Models in Model.Variants are ordered by priority, with the highest priority first.
// The first downloaded model is selected by default.
// The highest priority is selected if no models have been downloaded.
// If the selected variant is not the highest priority, it means that Foundry Local
// has found a locally cached variant for you to improve performance (remove need to download).
Console.WriteLine("\nThe default selected model variant is: " + model.Id);
-if (model.SelectedVariant != model.Variants.First())
+if (model.Id != model.Variants.First().Id)
{
- Debug.Assert(await model.SelectedVariant.IsCachedAsync());
+ Debug.Assert(await model.IsCachedAsync());
Console.WriteLine("The model variant was selected due to being locally cached.");
}
-// OPTIONAL: `model` can be used directly and `model.SelectedVariant` will be used as the default.
-// You can explicitly select or use a specific ModelVariant if you want more control
-// over the device and/or execution provider used.
-// Model and ModelVariant can be used interchangeably in methods such as
-// DownloadAsync, LoadAsync, UnloadAsync and GetChatClientAsync.
+// OPTIONAL: `model` can be used directly with its currently selected variant.
+// You can explicitly select (`model.SelectVariant`) or use a specific variant from `model.Variants`
+// if you want more control over the device and/or execution provider used.
//
// Choices:
-// - Use a ModelVariant directly from the catalog if you know the variant Id
+// - Use a model variant directly from the catalog if you know the variant Id
// - `var modelVariant = await catalog.GetModelVariantAsync("qwen2.5-0.5b-instruct-generic-gpu:3")`
//
-// - Get the ModelVariant from Model.Variants
+// - Get the model variant from IModel.Variants
// - `var modelVariant = model.Variants.First(v => v.Id == "qwen2.5-0.5b-instruct-generic-cpu:4")`
// - `var modelVariant = model.Variants.First(v => v.Info.Runtime?.DeviceType == DeviceType.GPU)`
// - optional: update selected variant in `model` using `model.SelectVariant(modelVariant);` if you wish to use
// `model` in your code.
// For this example we explicitly select the CPU variant, and call SelectVariant so all the following example code
-// uses the `model` instance.
+// uses the `model` instance. It would be equally valid to use `modelVariant` directly.
Console.WriteLine("Selecting CPU variant of model");
var modelVariant = model.Variants.First(v => v.Info.Runtime?.DeviceType == DeviceType.CPU);
model.SelectVariant(modelVariant);
diff --git a/sdk/cs/src/Catalog.cs b/sdk/cs/src/Catalog.cs
index eb9ba0d7..5cdb050f 100644
--- a/sdk/cs/src/Catalog.cs
+++ b/sdk/cs/src/Catalog.cs
@@ -52,51 +52,59 @@ internal static async Task CreateAsync(IModelLoadManager modelManager,
return catalog;
}
- public async Task> ListModelsAsync(CancellationToken? ct = null)
+ public async Task> ListModelsAsync(CancellationToken? ct = null)
{
return await Utils.CallWithExceptionHandling(() => ListModelsImplAsync(ct),
"Error listing models.", _logger).ConfigureAwait(false);
}
- public async Task> GetCachedModelsAsync(CancellationToken? ct = null)
+ public async Task> GetCachedModelsAsync(CancellationToken? ct = null)
{
return await Utils.CallWithExceptionHandling(() => GetCachedModelsImplAsync(ct),
"Error getting cached models.", _logger).ConfigureAwait(false);
}
- public async Task> GetLoadedModelsAsync(CancellationToken? ct = null)
+ public async Task> GetLoadedModelsAsync(CancellationToken? ct = null)
{
return await Utils.CallWithExceptionHandling(() => GetLoadedModelsImplAsync(ct),
"Error getting loaded models.", _logger).ConfigureAwait(false);
}
- public async Task GetModelAsync(string modelAlias, CancellationToken? ct = null)
+ public async Task GetModelAsync(string modelAlias, CancellationToken? ct = null)
{
return await Utils.CallWithExceptionHandling(() => GetModelImplAsync(modelAlias, ct),
$"Error getting model with alias '{modelAlias}'.", _logger)
.ConfigureAwait(false);
}
- public async Task GetModelVariantAsync(string modelId, CancellationToken? ct = null)
+ public async Task GetModelVariantAsync(string modelId, CancellationToken? ct = null)
{
return await Utils.CallWithExceptionHandling(() => GetModelVariantImplAsync(modelId, ct),
$"Error getting model variant with ID '{modelId}'.", _logger)
.ConfigureAwait(false);
}
- private async Task> ListModelsImplAsync(CancellationToken? ct = null)
+ public async Task GetLatestVersionAsync(IModel modelOrModelVariant, CancellationToken? ct = null)
+ {
+ return await Utils.CallWithExceptionHandling(
+ () => GetLatestVersionImplAsync(modelOrModelVariant, ct),
+ $"Error getting latest version for model with name '{modelOrModelVariant.Info.Name}'.",
+ _logger).ConfigureAwait(false);
+ }
+
+ private async Task> ListModelsImplAsync(CancellationToken? ct = null)
{
await UpdateModels(ct).ConfigureAwait(false);
using var disposable = await _lock.LockAsync().ConfigureAwait(false);
- return _modelAliasToModel.Values.OrderBy(m => m.Alias).ToList();
+ return _modelAliasToModel.Values.OrderBy(m => m.Alias).Cast().ToList();
}
- private async Task> GetCachedModelsImplAsync(CancellationToken? ct = null)
+ private async Task> GetCachedModelsImplAsync(CancellationToken? ct = null)
{
var cachedModelIds = await Utils.GetCachedModelIdsAsync(_coreInterop, ct).ConfigureAwait(false);
- List cachedModels = new();
+ List cachedModels = [];
foreach (var modelId in cachedModelIds)
{
if (_modelIdToModelVariant.TryGetValue(modelId, out ModelVariant? modelVariant))
@@ -108,10 +116,10 @@ private async Task> GetCachedModelsImplAsync(CancellationToke
return cachedModels;
}
- private async Task> GetLoadedModelsImplAsync(CancellationToken? ct = null)
+ private async Task> GetLoadedModelsImplAsync(CancellationToken? ct = null)
{
var loadedModelIds = await _modelLoadManager.ListLoadedModelsAsync(ct).ConfigureAwait(false);
- List loadedModels = new();
+ List loadedModels = [];
foreach (var modelId in loadedModelIds)
{
@@ -143,6 +151,45 @@ private async Task> GetLoadedModelsImplAsync(CancellationToke
return modelVariant;
}
+ private async Task GetLatestVersionImplAsync(IModel modelOrModelVariant, CancellationToken? ct)
+ {
+ Model? model;
+
+ if (modelOrModelVariant is ModelVariant)
+ {
+ // For ModelVariant, resolve the owning Model via alias.
+ model = await GetModelImplAsync(modelOrModelVariant.Alias, ct);
+ }
+ else
+ {
+ // Try to use the concrete Model instance if this is our SDK type.
+ model = modelOrModelVariant as Model;
+
+ // If this is a different IModel implementation (e.g., a test stub),
+ // fall back to resolving the Model via alias.
+ if (model == null)
+ {
+ model = await GetModelImplAsync(modelOrModelVariant.Alias, ct);
+ }
+ }
+
+ if (model == null)
+ {
+ throw new FoundryLocalException($"Model with alias '{modelOrModelVariant.Alias}' not found in catalog.",
+ _logger);
+ }
+
+ // variants are sorted by version, so the first one matching the name is the latest version for that variant.
+ var latest = model!.Variants.FirstOrDefault(v => v.Info.Name == modelOrModelVariant.Info.Name) ??
+ // should not be possible given we internally manage all the state involved
+ throw new FoundryLocalException($"Internal error. Mismatch between model (alias:{model.Alias}) and " +
+ $"model variant (alias:{modelOrModelVariant.Alias}).", _logger);
+
+ // if input was the latest return the input (could be model or model variant)
+ // otherwise return the latest model variant
+ return latest.Id == modelOrModelVariant.Id ? modelOrModelVariant : latest;
+ }
+
private async Task UpdateModels(CancellationToken? ct)
{
// TODO: make this configurable
diff --git a/sdk/cs/src/Model.cs b/sdk/cs/src/Detail/Model.cs
similarity index 74%
rename from sdk/cs/src/Model.cs
rename to sdk/cs/src/Detail/Model.cs
index bbbbcb5b..c4d96057 100644
--- a/sdk/cs/src/Model.cs
+++ b/sdk/cs/src/Detail/Model.cs
@@ -12,11 +12,13 @@ public class Model : IModel
{
private readonly ILogger _logger;
- public List Variants { get; internal set; }
- public ModelVariant SelectedVariant { get; internal set; } = default!;
+ private readonly List _variants;
+ public IReadOnlyList Variants => _variants;
+ internal IModel SelectedVariant { get; set; } = default!;
public string Alias { get; init; }
public string Id => SelectedVariant.Id;
+ public ModelInfo Info => SelectedVariant.Info;
///
/// Is the currently selected variant cached locally?
@@ -33,7 +35,7 @@ internal Model(ModelVariant modelVariant, ILogger logger)
_logger = logger;
Alias = modelVariant.Alias;
- Variants = new() { modelVariant };
+ _variants = [modelVariant];
// variants are sorted by Core, so the first one added is the default
SelectedVariant = modelVariant;
@@ -48,7 +50,7 @@ internal void AddVariant(ModelVariant variant)
_logger);
}
- Variants.Add(variant);
+ _variants.Add(variant);
// prefer the highest priority locally cached variant
if (variant.Info.Cached && !SelectedVariant.Info.Cached)
@@ -62,31 +64,15 @@ internal void AddVariant(ModelVariant variant)
///
/// Model variant to select. Must be one of the variants in .
/// If variant is not valid for this model.
- public void SelectVariant(ModelVariant variant)
+ public void SelectVariant(IModel variant)
{
_ = Variants.FirstOrDefault(v => v == variant) ??
- // user error so don't log
- throw new FoundryLocalException($"Model {Alias} does not have a {variant.Id} variant.");
+ // user error so don't log.
+ throw new FoundryLocalException($"Input variant was not found in Variants.");
SelectedVariant = variant;
}
- ///
- /// Get the latest version of the specified model variant.
- ///
- /// Model variant.
- /// ModelVariant for latest version. Same as `variant` if that is the latest version.
- /// If variant is not valid for this model.
- public ModelVariant GetLatestVersion(ModelVariant variant)
- {
- // variants are sorted by version, so the first one matching the name is the latest version for that variant.
- var latest = Variants.FirstOrDefault(v => v.Info.Name == variant.Info.Name) ??
- // user error so don't log
- throw new FoundryLocalException($"Model {Alias} does not have a {variant.Id} variant.");
-
- return latest;
- }
-
public async Task GetPathAsync(CancellationToken? ct = null)
{
return await SelectedVariant.GetPathAsync(ct).ConfigureAwait(false);
diff --git a/sdk/cs/src/ModelVariant.cs b/sdk/cs/src/Detail/ModelVariant.cs
similarity index 95%
rename from sdk/cs/src/ModelVariant.cs
rename to sdk/cs/src/Detail/ModelVariant.cs
index 6ca7cda7..9f2deaba 100644
--- a/sdk/cs/src/ModelVariant.cs
+++ b/sdk/cs/src/Detail/ModelVariant.cs
@@ -9,7 +9,7 @@ namespace Microsoft.AI.Foundry.Local;
using Microsoft.AI.Foundry.Local.Detail;
using Microsoft.Extensions.Logging;
-public class ModelVariant : IModel
+internal class ModelVariant : IModel
{
private readonly IModelLoadManager _modelLoadManager;
private readonly ICoreInterop _coreInterop;
@@ -22,6 +22,8 @@ public class ModelVariant : IModel
public string Alias => Info.Alias;
public int Version { get; init; } // parsed from Info.Version if possible, else 0
+ public IReadOnlyList Variants => [this];
+
internal ModelVariant(ModelInfo modelInfo, IModelLoadManager modelLoadManager, ICoreInterop coreInterop,
ILogger logger)
{
@@ -190,4 +192,11 @@ private async Task GetAudioClientImplAsync(CancellationToken?
return new OpenAIAudioClient(Id);
}
+
+ public void SelectVariant(IModel variant)
+ {
+ throw new FoundryLocalException(
+ $"SelectVariant is not supported on a ModelVariant. " +
+ $"Call Catalog.GetModelAsync(\"{Alias}\") to get an IModel with all variants available.");
+ }
}
diff --git a/sdk/cs/src/ICatalog.cs b/sdk/cs/src/ICatalog.cs
index 35285736..85851a9c 100644
--- a/sdk/cs/src/ICatalog.cs
+++ b/sdk/cs/src/ICatalog.cs
@@ -18,36 +18,46 @@ public interface ICatalog
/// List the available models in the catalog.
///
/// Optional CancellationToken.
- /// List of Model instances.
- Task> ListModelsAsync(CancellationToken? ct = null);
+ /// List of IModel instances.
+ Task> ListModelsAsync(CancellationToken? ct = null);
///
/// Lookup a model by its alias.
///
/// Model alias.
/// Optional CancellationToken.
- /// The matching Model, or null if no model with the given alias exists.
- Task GetModelAsync(string modelAlias, CancellationToken? ct = null);
+ /// The matching IModel, or null if no model with the given alias exists.
+ Task GetModelAsync(string modelAlias, CancellationToken? ct = null);
///
/// Lookup a model variant by its unique model id.
+ /// NOTE: This will return an IModel with a single variant. Use GetModelAsync to get an IModel with all avaialable
+ /// variants.
///
/// Model id.
/// Optional CancellationToken.
- /// The matching ModelVariant, or null if no variant with the given id exists.
- Task GetModelVariantAsync(string modelId, CancellationToken? ct = null);
+ /// The matching IModel, or null if no variant with the given id exists.
+ Task GetModelVariantAsync(string modelId, CancellationToken? ct = null);
///
/// Get a list of currently downloaded models from the model cache.
///
/// Optional CancellationToken.
- /// List of ModelVariant instances.
- Task> GetCachedModelsAsync(CancellationToken? ct = null);
+ /// List of IModel instances.
+ Task> GetCachedModelsAsync(CancellationToken? ct = null);
///
/// Get a list of the currently loaded models.
///
/// Optional CancellationToken.
- /// List of ModelVariant instances.
- Task> GetLoadedModelsAsync(CancellationToken? ct = null);
+ /// List of IModel instances.
+ Task> GetLoadedModelsAsync(CancellationToken? ct = null);
+
+ ///
+ /// Get the latest version of a model.
+ /// This is used to check if a newer version of a model is available in the catalog for download.
+ ///
+ /// The model to check for the latest version.
+ /// The latest version of the model. Will match the input if it is the latest version.
+ Task GetLatestVersionAsync(IModel model, CancellationToken? ct = null);
}
diff --git a/sdk/cs/src/IModel.cs b/sdk/cs/src/IModel.cs
index c3acba61..a27f3a3d 100644
--- a/sdk/cs/src/IModel.cs
+++ b/sdk/cs/src/IModel.cs
@@ -16,6 +16,8 @@ public interface IModel
Justification = "Alias is a suitable name in this context.")]
string Alias { get; }
+ ModelInfo Info { get; }
+
Task IsCachedAsync(CancellationToken? ct = null);
Task IsLoadedAsync(CancellationToken? ct = null);
@@ -67,4 +69,17 @@ Task DownloadAsync(Action? downloadProgress = null,
/// Optional cancellation token.
/// OpenAI.AudioClient
Task GetAudioClientAsync(CancellationToken? ct = null);
+
+ ///
+ /// Variants of the model that are available. Variants of the model are optimized for different devices.
+ ///
+ IReadOnlyList Variants { get; }
+
+ ///
+ /// Select a model variant from to use for operations.
+ /// An IModel from `Variants` can also be used directly.
+ ///
+ /// Model variant to select. Must be one of the variants in .
+ /// If variant is not valid for this model.
+ void SelectVariant(IModel variant);
}
diff --git a/sdk/cs/test/FoundryLocal.Tests/AudioClientTests.cs b/sdk/cs/test/FoundryLocal.Tests/AudioClientTests.cs
index ec4ab4c9..5c4cc8d6 100644
--- a/sdk/cs/test/FoundryLocal.Tests/AudioClientTests.cs
+++ b/sdk/cs/test/FoundryLocal.Tests/AudioClientTests.cs
@@ -12,7 +12,7 @@ namespace Microsoft.AI.Foundry.Local.Tests;
internal sealed class AudioClientTests
{
- private static Model? model;
+ private static IModel? model;
[Before(Class)]
public static async Task Setup()
diff --git a/sdk/cs/test/FoundryLocal.Tests/CatalogTests.cs b/sdk/cs/test/FoundryLocal.Tests/CatalogTests.cs
new file mode 100644
index 00000000..d270ac15
--- /dev/null
+++ b/sdk/cs/test/FoundryLocal.Tests/CatalogTests.cs
@@ -0,0 +1,121 @@
+// --------------------------------------------------------------------------------------------------------------------
+//
+// Copyright (c) Microsoft. All rights reserved.
+//
+// --------------------------------------------------------------------------------------------------------------------
+
+namespace Microsoft.AI.Foundry.Local.Tests;
+using System.Collections.Generic;
+using System.Linq;
+using System.Text.Json;
+using System.Threading.Tasks;
+
+using Microsoft.AI.Foundry.Local.Detail;
+using Microsoft.Extensions.Logging.Abstractions;
+
+using Moq;
+
+internal sealed class CatalogTests
+{
+ [Test]
+ public async Task GetLatestVersion_Works()
+ {
+ // Create test data with 3 entries for a model with different versions
+ // Sorted by version (descending), so version 3 is first (latest)
+ var testModelInfos = new List
+ {
+ new()
+ {
+ Id = "test-model:3",
+ Name = "test-model",
+ Version = 3,
+ Alias = "test-alias",
+ DisplayName = "Test Model",
+ ProviderType = "test",
+ Uri = "test://model/3",
+ ModelType = "ONNX",
+ Runtime = new Runtime { DeviceType = DeviceType.CPU, ExecutionProvider = "CPUExecutionProvider" },
+ Cached = false
+ },
+ new()
+ {
+ Id = "test-model:2",
+ Name = "test-model",
+ Version = 2,
+ Alias = "test-alias",
+ DisplayName = "Test Model",
+ ProviderType = "test",
+ Uri = "test://model/2",
+ ModelType = "ONNX",
+ Runtime = new Runtime { DeviceType = DeviceType.CPU, ExecutionProvider = "CPUExecutionProvider" },
+ Cached = false
+ },
+ new()
+ {
+ Id = "test-model:1",
+ Name = "test-model",
+ Version = 1,
+ Alias = "test-alias",
+ DisplayName = "Test Model",
+ ProviderType = "test",
+ Uri = "test://model/1",
+ ModelType = "ONNX",
+ Runtime = new Runtime { DeviceType = DeviceType.CPU, ExecutionProvider = "CPUExecutionProvider" },
+ Cached = false
+ }
+ };
+
+ // Serialize the test data
+ var modelListJson = JsonSerializer.Serialize(testModelInfos, JsonSerializationContext.Default.ListModelInfo);
+
+ // Create mock ICoreInterop
+ var mockCoreInterop = new Mock();
+
+ // Mock get_catalog_name
+ mockCoreInterop.Setup(x => x.ExecuteCommand("get_catalog_name", It.IsAny()))
+ .Returns(new ICoreInterop.Response { Data = "TestCatalog", Error = null });
+
+ // Mock get_model_list
+ mockCoreInterop.Setup(x => x.ExecuteCommandAsync("get_model_list", It.IsAny(), It.IsAny()))
+ .ReturnsAsync(new ICoreInterop.Response { Data = modelListJson, Error = null });
+
+ // Create mock IModelLoadManager
+ var mockLoadManager = new Mock();
+
+ // Create Catalog instance directly (internals are visible to test project)
+ var catalog = await Catalog.CreateAsync(mockLoadManager.Object, mockCoreInterop.Object,
+ NullLogger.Instance, null);
+
+ // Get the model
+ var model = await catalog.GetModelAsync("test-alias");
+ await Assert.That(model).IsNotNull();
+
+ // Verify we have 3 variants
+ await Assert.That(model!.Variants).HasCount().EqualTo(3);
+
+ // Get the variants - they should be sorted by version (descending)
+ var variants = model.Variants.ToList();
+ var latestVariant = variants[0]; // version 3
+ var middleVariant = variants[1]; // version 2
+ var oldestVariant = variants[2]; // version 1
+
+ await Assert.That(latestVariant.Id).IsEqualTo("test-model:3");
+ await Assert.That(middleVariant.Id).IsEqualTo("test-model:2");
+ await Assert.That(oldestVariant.Id).IsEqualTo("test-model:1");
+
+ // Test GetLatestVersionAsync with all 3 variants - should always return the first (version 3)
+ var result1 = await catalog.GetLatestVersionAsync(latestVariant);
+ await Assert.That(result1.Id).IsEqualTo("test-model:3");
+
+ var result2 = await catalog.GetLatestVersionAsync(middleVariant);
+ await Assert.That(result2.Id).IsEqualTo("test-model:3");
+
+ var result3 = await catalog.GetLatestVersionAsync(oldestVariant);
+ await Assert.That(result3.Id).IsEqualTo("test-model:3");
+
+ // Test with Model input - when latest is selected, should get Model not ModelVariant back
+ model.SelectVariant(latestVariant);
+ var result4 = await catalog.GetLatestVersionAsync(model);
+ await Assert.That(result4).IsEqualTo(model);
+ }
+}
diff --git a/sdk/cs/test/FoundryLocal.Tests/ChatCompletionsTests.cs b/sdk/cs/test/FoundryLocal.Tests/ChatCompletionsTests.cs
index b7a91190..2624f98a 100644
--- a/sdk/cs/test/FoundryLocal.Tests/ChatCompletionsTests.cs
+++ b/sdk/cs/test/FoundryLocal.Tests/ChatCompletionsTests.cs
@@ -15,7 +15,7 @@ namespace Microsoft.AI.Foundry.Local.Tests;
internal sealed class ChatCompletionsTests
{
- private static Model? model;
+ private static IModel? model;
[Before(Class)]
public static async Task Setup()
@@ -24,11 +24,10 @@ public static async Task Setup()
var catalog = await manager.GetCatalogAsync();
// Load the specific cached model variant directly
- var modelVariant = await catalog.GetModelVariantAsync("qwen2.5-0.5b-instruct-generic-cpu:4").ConfigureAwait(false);
- await Assert.That(modelVariant).IsNotNull();
+ var model = await catalog.GetModelVariantAsync("qwen2.5-0.5b-instruct-generic-cpu:4").ConfigureAwait(false);
+ await Assert.That(model).IsNotNull();
- var model = new Model(modelVariant!, manager.Logger);
- await model.LoadAsync().ConfigureAwait(false);
+ await model!.LoadAsync().ConfigureAwait(false);
await Assert.That(await model.IsLoadedAsync()).IsTrue();
ChatCompletionsTests.model = model;
diff --git a/sdk/cs/test/FoundryLocal.Tests/EndToEnd.cs b/sdk/cs/test/FoundryLocal.Tests/EndToEnd.cs
index 80ab4c0a..56c70769 100644
--- a/sdk/cs/test/FoundryLocal.Tests/EndToEnd.cs
+++ b/sdk/cs/test/FoundryLocal.Tests/EndToEnd.cs
@@ -29,8 +29,9 @@ public async Task EndToEndTest_Succeeds()
await Assert.That(modelVariant).IsNotNull();
await Assert.That(modelVariant!.Alias).IsEqualTo("qwen2.5-0.5b");
- // Create model from the specific variant
- var model = new Model(modelVariant, manager.Logger);
+ // Get Model for variant and select the variant so `model` and `modelVariant` should be equivalent
+ var model = await catalog.GetModelAsync(modelVariant.Alias);
+ model!.SelectVariant(modelVariant);
// uncomment this to remove the model first to test the download progress
// only do this when manually testing as other tests expect the model to be cached
diff --git a/sdk/cs/test/FoundryLocal.Tests/FoundryLocalManagerTest.cs b/sdk/cs/test/FoundryLocal.Tests/FoundryLocalManagerTest.cs
index 5227e062..cd7e7793 100644
--- a/sdk/cs/test/FoundryLocal.Tests/FoundryLocalManagerTest.cs
+++ b/sdk/cs/test/FoundryLocal.Tests/FoundryLocalManagerTest.cs
@@ -26,7 +26,7 @@ public async Task Manager_GetCatalog_Succeeds()
foreach (var model in models)
{
Console.WriteLine($"Model Alias: {model.Alias}, Variants: {model.Variants.Count}");
- Console.WriteLine($"Selected Variant Id: {model.SelectedVariant?.Id ?? "none"}");
+ Console.WriteLine($"Selected Variant Id: {model.Id ?? "none"}");
// variants should be in sorted order
diff --git a/sdk/cs/test/FoundryLocal.Tests/LOCAL_MODEL_TESTING.md b/sdk/cs/test/FoundryLocal.Tests/LOCAL_MODEL_TESTING.md
index 1145cd9d..1b4a71e7 100644
--- a/sdk/cs/test/FoundryLocal.Tests/LOCAL_MODEL_TESTING.md
+++ b/sdk/cs/test/FoundryLocal.Tests/LOCAL_MODEL_TESTING.md
@@ -6,10 +6,14 @@ The test model cache directory name is configured in `sdk/cs/test/FoundryLocal.T
```json
{
- "TestModelCacheDirName": "/path/to/model/cache"
+ "TestModelCacheDirName": "test-data-shared"
}
```
+If the value is a directory name it will be resolved as /../{TestModelCacheDirName}.
+Otherwise the value will be resolved using Path.GetFullPath, which allows for absolute paths or
+relative paths based on the current working directory.
+
## Run the tests
The tests will automatically find the models in the configured test model cache directory.
@@ -17,21 +21,4 @@ The tests will automatically find the models in the configured test model cache
```bash
cd /path/to/parent-dir/foundry-local-sdk/sdk/cs/test/FoundryLocal.Tests
dotnet test Microsoft.AI.Foundry.Local.Tests.csproj --configuration Release# Running Local Model Tests
-
-## Configuration
-
-The test model cache directory name is configured in `sdk/cs/test/FoundryLocal.Tests/appsettings.Test.json`:
-
-```json
-{
- "TestModelCacheDirName": "/path/to/model/cache"
-}
```
-
-## Run the tests
-
-The tests will automatically find the models in the configured test model cache directory.
-
-```bash
-cd /path/to/parent-dir/foundry-local-sdk/sdk/cs/test/FoundryLocal.Tests
-dotnet test Microsoft.AI.Foundry.Local.Tests.csproj --configuration Release
\ No newline at end of file
diff --git a/sdk/cs/test/FoundryLocal.Tests/TestAssemblySetupCleanup.cs b/sdk/cs/test/FoundryLocal.Tests/TestAssemblySetupCleanup.cs
index ac536d12..2136a8eb 100644
--- a/sdk/cs/test/FoundryLocal.Tests/TestAssemblySetupCleanup.cs
+++ b/sdk/cs/test/FoundryLocal.Tests/TestAssemblySetupCleanup.cs
@@ -15,16 +15,20 @@ public static async Task Cleanup(AssemblyHookContext _)
{
try
{
- // ensure any loaded models are unloaded
- var manager = FoundryLocalManager.Instance; // initialized by Utils
- var catalog = await manager.GetCatalogAsync();
- var models = await catalog.GetLoadedModelsAsync().ConfigureAwait(false);
-
- foreach (var model in models)
+ // if running individual test/s they may not have used the Utils class which creates FoundryLocalManager
+ if (FoundryLocalManager.IsInitialized)
{
- await Assert.That(await model.IsLoadedAsync()).IsTrue();
- await model.UnloadAsync().ConfigureAwait(false);
- await Assert.That(await model.IsLoadedAsync()).IsFalse();
+ // ensure any loaded models are unloaded
+ var manager = FoundryLocalManager.Instance; // initialized by Utils
+ var catalog = await manager.GetCatalogAsync();
+ var models = await catalog.GetLoadedModelsAsync().ConfigureAwait(false);
+
+ foreach (var model in models)
+ {
+ await Assert.That(await model.IsLoadedAsync()).IsTrue();
+ await model.UnloadAsync().ConfigureAwait(false);
+ await Assert.That(await model.IsLoadedAsync()).IsFalse();
+ }
}
}
catch (Exception ex)
diff --git a/sdk/cs/test/FoundryLocal.Tests/Utils.cs b/sdk/cs/test/FoundryLocal.Tests/Utils.cs
index 6313b0d5..9611d0d4 100644
--- a/sdk/cs/test/FoundryLocal.Tests/Utils.cs
+++ b/sdk/cs/test/FoundryLocal.Tests/Utils.cs
@@ -55,7 +55,7 @@ public static void AssemblyInit(AssemblyHookContext _)
.AddJsonFile("appsettings.Test.json", optional: true, reloadOnChange: false)
.Build();
- var testModelCacheDirName = "test-data-shared";
+ var testModelCacheDirName = configuration["TestModelCacheDirName"] ?? "test-data-shared";
string testDataSharedPath;
if (Path.IsPathRooted(testModelCacheDirName) ||
testModelCacheDirName.Contains(Path.DirectorySeparatorChar) ||
@@ -74,6 +74,8 @@ public static void AssemblyInit(AssemblyHookContext _)
if (!Directory.Exists(testDataSharedPath))
{
+ // need to ensure there's a user visible error when running in VS.
+ logger.LogCritical($"Test model cache directory does not exist: {testDataSharedPath}");
throw new DirectoryNotFoundException($"Test model cache directory does not exist: {testDataSharedPath}");
}
diff --git a/sdk/cs/test/FoundryLocal.Tests/appsettings.Test.json b/sdk/cs/test/FoundryLocal.Tests/appsettings.Test.json
index 87410c33..d42d8789 100644
--- a/sdk/cs/test/FoundryLocal.Tests/appsettings.Test.json
+++ b/sdk/cs/test/FoundryLocal.Tests/appsettings.Test.json
@@ -1,3 +1,3 @@
{
- "TestModelCacheDirName": "/path/to/test/model/cache"
+ "TestModelCacheDirName": "test-data-shared"
}
From 1b2001e47df4e4c60afed8ae8ef63b5530584497 Mon Sep 17 00:00:00 2001
From: Prathik Rao
Date: Tue, 31 Mar 2026 10:34:59 -0700
Subject: [PATCH 11/21] implement ADO packaging pipeline for FLC & SDK (#552)
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit
## Foundry Local Packaging Pipeline
### Summary
This PR introduces the **Foundry Local Packaging Pipeline**, a unified
ADO pipeline that builds, signs, and tests Foundry Local Core (FLC) for
all platforms, packages it as NuGet and Python wheels, then builds,
signs, and tests the C#, JS, Python, and Rust SDKs — for both standard
and WinML variants.
**Pipeline stages:**
1. **Build FLC** — Native AOT binaries for win-x64, win-arm64,
linux-x64, osx-arm64
2. **Package FLC** — Multi-platform NuGet package + Python wheels from
the built binaries
3. **Build SDKs** — C#, JS, Python, Rust using the packaged FLC
4. **Test SDKs** — Validate each SDK against the pipeline-built FLC
**Produced artifacts:** `flc-nuget`, `flc-nuget-winml`, `flc-wheels`,
`flc-wheels-winml`, `cs-sdk`, `cs-sdk-winml`, `js-sdk`, `js-sdk-winml`,
`python-sdk`, `python-sdk-winml`, `rust-sdk`, `rust-sdk-winml`
**SDK Changes:**
1. Adds ability for python sdk to skip installing native depenencies and
use pre-installed binaries like `foundry-local-core`, `onnxruntime`,
`onnxruntime-genai`
2. Adjusts APIs to leverage new download_and_register_eps native interop
call for manually downloading and registering EPs
3. Adds temporary nuget.config to github actions c# pipeline to allow
ORT-Nightly to auto-fetch missing dependencies from upstream nuget.org
### Test coverage
All SDK tests currently run on **win-x64 only**. Additional platform
test jobs are blocked on infrastructure:
- **Windows ARM64** — waiting on a 1ES-hosted win-arm64 pool
- **macOS ARM64** — waiting on a 1ES-hosted macOS ARM64 pool
- **Linux x64** — waiting on the Linux onnxruntime dependency to be
stabilized
TODOs are tracked in the pipeline YAML for each.
### Build strategy
All FLC builds (including win-arm64 and osx-arm64) run on **x64
machines** because .NET Native AOT supports cross-compilation. The
win-arm64 build cross-compiles from x64 Windows — see [Cross-compilation
docs](https://learn.microsoft.com/en-us/dotnet/core/deploying/native-aot/cross-compile#windows).
Linux builds run on its own respective x64 hosted image.
### Origin
- **Foundry Local Core build steps** were lifted from
`neutron-server/.pipelines/FoundryLocalCore/`
- **SDK build/test steps** were lifted from `Foundry-Local/.github/`
---------
Co-authored-by: Prathik Rao
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
---
.github/workflows/build-cs-steps.yml | 35 +-
.github/workflows/foundry-local-sdk-build.yml | 56 +-
.pipelines/foundry-local-packaging.yml | 812 +++++++++++++++++-
.pipelines/templates/build-core-steps.yml | 194 +++++
.pipelines/templates/build-cs-steps.yml | 191 ++++
.pipelines/templates/build-js-steps.yml | 156 ++++
.pipelines/templates/build-python-steps.yml | 146 ++++
.pipelines/templates/build-rust-steps.yml | 207 +++++
.pipelines/templates/package-core-steps.yml | 256 ++++++
.pipelines/templates/test-cs-steps.yml | 116 +++
.pipelines/templates/test-js-steps.yml | 121 +++
.pipelines/templates/test-python-steps.yml | 133 +++
.pipelines/templates/test-rust-steps.yml | 159 ++++
sdk/cs/README.md | 4 +-
...ft.ai.foundry.local.foundrylocalmanager.md | 8 +-
sdk/cs/src/Detail/CoreInterop.cs | 9 +
sdk/cs/src/FoundryLocalManager.cs | 16 +-
sdk/cs/src/Microsoft.AI.Foundry.Local.csproj | 5 +-
.../Microsoft.AI.Foundry.Local.Tests.csproj | 5 +-
sdk/js/docs/classes/FoundryLocalManager.md | 23 +
sdk/js/src/foundryLocalManager.ts | 18 +
sdk/python/build_backend.py | 115 ++-
sdk/python/src/detail/core_interop.py | 3 +
sdk/python/src/foundry_local_manager.py | 10 +-
sdk/rust/src/foundry_local_manager.rs | 14 +
25 files changed, 2693 insertions(+), 119 deletions(-)
create mode 100644 .pipelines/templates/build-core-steps.yml
create mode 100644 .pipelines/templates/build-cs-steps.yml
create mode 100644 .pipelines/templates/build-js-steps.yml
create mode 100644 .pipelines/templates/build-python-steps.yml
create mode 100644 .pipelines/templates/build-rust-steps.yml
create mode 100644 .pipelines/templates/package-core-steps.yml
create mode 100644 .pipelines/templates/test-cs-steps.yml
create mode 100644 .pipelines/templates/test-js-steps.yml
create mode 100644 .pipelines/templates/test-python-steps.yml
create mode 100644 .pipelines/templates/test-rust-steps.yml
diff --git a/.github/workflows/build-cs-steps.yml b/.github/workflows/build-cs-steps.yml
index dcfed979..cf680d49 100644
--- a/.github/workflows/build-cs-steps.yml
+++ b/.github/workflows/build-cs-steps.yml
@@ -41,19 +41,41 @@ jobs:
env:
NUGET_AUTH_TOKEN: ${{ secrets.AZURE_DEVOPS_PAT }}
+ - name: Generate temporary NuGet.config
+ run: |
+ # The repo-level NuGet.config cleared all sources and only included ORT-Nightly.
+ # We generate a temporary one with both nuget.org and ORT-Nightly.
+ # We provide credentials to allow the ORT-Nightly feed to pull from its upstreams.
+ $xml = @"
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+ "@
+ Set-Content -Path sdk/cs/NuGet.temp.config -Value $xml
+ shell: pwsh
+
# TODO: once the nightly packaging is fixed, add back the commented out lines with /p:FoundryLocalCoreVersion="*-*"
# /p:FoundryLocalCoreVersion="*-*" to always use nightly version of Foundry Local Core
- - name: Authenticate to Azure Artifacts NuGet feed
- run: dotnet nuget update source ORT-Nightly --username az --password ${{ secrets.AZURE_DEVOPS_PAT }} --store-password-in-clear-text --configfile sdk/cs/NuGet.config
-
- name: Restore dependencies
run: |
- # dotnet restore sdk/cs/src/Microsoft.AI.Foundry.Local.csproj /p:UseWinML=${{ inputs.useWinML }} /p:FoundryLocalCoreVersion="*-*" --configfile sdk/cs/NuGet.config
- dotnet restore sdk/cs/src/Microsoft.AI.Foundry.Local.csproj /p:UseWinML=${{ inputs.useWinML }} --configfile sdk/cs/NuGet.config
+ # Clear the local NuGet cache to avoid bad metadata or corrupted package states.
+ dotnet nuget locals all --clear
+ # Restore using the temporary config file with credentials.
+ dotnet restore sdk/cs/src/Microsoft.AI.Foundry.Local.csproj /p:UseWinML=${{ inputs.useWinML }} --configfile sdk/cs/NuGet.temp.config
- name: Build solution
run: |
- # dotnet build sdk/cs/src/Microsoft.AI.Foundry.Local.csproj --no-restore --configuration ${{ inputs.buildConfiguration }} /p:UseWinML=${{ inputs.useWinML }} /p:FoundryLocalCoreVersion="*-*"
dotnet build sdk/cs/src/Microsoft.AI.Foundry.Local.csproj --no-restore --configuration ${{ inputs.buildConfiguration }} /p:UseWinML=${{ inputs.useWinML }}
# need to use direct git commands to clone from Azure DevOps instead of actions/checkout
@@ -89,6 +111,7 @@ jobs:
- name: Run Foundry Local Core tests
run: |
# dotnet test sdk/cs/test/FoundryLocal.Tests/Microsoft.AI.Foundry.Local.Tests.csproj --verbosity normal /p:UseWinML=${{ inputs.useWinML }} /p:FoundryLocalCoreVersion="*-*"
+ # Use the temporary config file for test restore as well.
dotnet test sdk/cs/test/FoundryLocal.Tests/Microsoft.AI.Foundry.Local.Tests.csproj --verbosity normal /p:UseWinML=${{ inputs.useWinML }}
- name: Pack NuGet package
diff --git a/.github/workflows/foundry-local-sdk-build.yml b/.github/workflows/foundry-local-sdk-build.yml
index 13eddf6d..07ae4d68 100644
--- a/.github/workflows/foundry-local-sdk-build.yml
+++ b/.github/workflows/foundry-local-sdk-build.yml
@@ -17,60 +17,8 @@ permissions:
contents: read
jobs:
- build-cs-windows:
- uses: ./.github/workflows/build-cs-steps.yml
- with:
- version: '0.9.0.${{ github.run_number }}'
- platform: 'windows'
- secrets: inherit
- build-js-windows:
- uses: ./.github/workflows/build-js-steps.yml
- with:
- version: '0.9.0.${{ github.run_number }}'
- platform: 'windows'
- secrets: inherit
- build-python-windows:
- uses: ./.github/workflows/build-python-steps.yml
- with:
- version: '0.9.0.${{ github.run_number }}'
- platform: 'windows'
- secrets: inherit
- build-rust-windows:
- uses: ./.github/workflows/build-rust-steps.yml
- with:
- platform: 'windows'
- run-integration-tests: true
- secrets: inherit
-
- build-cs-windows-WinML:
- uses: ./.github/workflows/build-cs-steps.yml
- with:
- version: '0.9.0.${{ github.run_number }}'
- platform: 'windows'
- useWinML: true
- secrets: inherit
- build-js-windows-WinML:
- uses: ./.github/workflows/build-js-steps.yml
- with:
- version: '0.9.0.${{ github.run_number }}'
- platform: 'windows'
- useWinML: true
- secrets: inherit
- build-python-windows-WinML:
- uses: ./.github/workflows/build-python-steps.yml
- with:
- version: '0.9.0.${{ github.run_number }}'
- platform: 'windows'
- useWinML: true
- secrets: inherit
- build-rust-windows-WinML:
- uses: ./.github/workflows/build-rust-steps.yml
- with:
- platform: 'windows'
- useWinML: true
- run-integration-tests: true
- secrets: inherit
-
+ # Windows build/test moved to .pipelines/foundry-local-packaging.yml and runs in ADO
+ # MacOS ARM64 not supported in ADO, need to use GitHub Actions
build-cs-macos:
uses: ./.github/workflows/build-cs-steps.yml
with:
diff --git a/.pipelines/foundry-local-packaging.yml b/.pipelines/foundry-local-packaging.yml
index b87eb70e..2cb9ee2a 100644
--- a/.pipelines/foundry-local-packaging.yml
+++ b/.pipelines/foundry-local-packaging.yml
@@ -1,9 +1,807 @@
-# Foundry Local SDK Packaging Pipeline (placeholder)
-trigger: none
+# Foundry Local Packaging Pipeline
+#
+# Builds Foundry Local Core from neutron-server (windows.ai.toolkit project),
+# then packages the C# and JS SDKs from this repo using the built Core.
+#
+# Produces artifacts: flc-nuget, flc-nuget-winml, flc-wheels, flc-wheels-winml,
+# cs-sdk, cs-sdk-winml, js-sdk, js-sdk-winml, python-sdk, python-sdk-winml,
+# rust-sdk, rust-sdk-winml
-pool:
- vmImage: 'windows-latest'
+pr:
+- main
+- releases/*
+
+name: $(Date:yyyyMMdd).$(Rev:r)
+
+parameters:
+- name: version
+ displayName: 'Package version'
+ type: string
+ default: '0.9.0'
+- name: prereleaseId
+ displayName: 'Pre-release identifier (e.g. rc1, beta).'
+ type: string
+ default: 'none'
+- name: isRelease
+ displayName: 'Release build'
+ type: boolean
+ default: false
+- name: neutronServerBranch
+ displayName: 'Foundry Local Core branch (windows.ai.toolkit/neutron-server)'
+ type: string
+ default: 'dev/FoundryLocalCore/main'
+
+variables:
+- group: FoundryLocal-ESRP-Signing
+
+resources:
+ repositories:
+ - repository: neutron-server
+ type: git
+ name: windows.ai.toolkit/neutron-server
+ endpoint: AIFoundryLocal-WindowsAIToolkit-SC
+ ref: refs/heads/${{ parameters.neutronServerBranch }}
+ - repository: test-data-shared
+ type: git
+ name: windows.ai.toolkit/test-data-shared
+ endpoint: AIFoundryLocal-WindowsAIToolkit-SC
+ lfs: true
+ ref: refs/heads/main
+ - repository: 1ESPipelineTemplates
+ type: git
+ name: 1ESPipelineTemplates/1ESPipelineTemplates
+ ref: refs/tags/release
+
+extends:
+ template: v1/1ES.Official.PipelineTemplate.yml@1ESPipelineTemplates
+ parameters:
+ settings:
+ networkIsolationPolicy: Permissive
+ pool:
+ # default all windows jobs, individual jobs override
+ name: onnxruntime-Win-CPU-2022
+ os: windows
+ sdl:
+ binskim:
+ break: false
+ scanOutputDirectoryOnly: true
+ sourceRepositoriesToScan:
+ include:
+ - repository: neutron-server
+ - repository: test-data-shared
+ stages:
+ # ── Build & Test FLC ──
+ - stage: build_core
+ displayName: 'Build & Test FLC'
+ jobs:
+ - job: flc_win_x64
+ displayName: 'FLC win-x64'
+ pool:
+ name: onnxruntime-Win-CPU-2022
+ os: windows
+ templateContext:
+ outputs:
+ - output: pipelineArtifact
+ artifactName: 'flc-win-x64'
+ targetPath: '$(Build.ArtifactStagingDirectory)/native'
+ steps:
+ - checkout: neutron-server
+ clean: true
+ - checkout: test-data-shared
+ lfs: true
+ - template: .pipelines/templates/build-core-steps.yml@self
+ parameters:
+ flavor: win-x64
+ platform: x64
+
+ - job: flc_win_arm64
+ displayName: 'FLC win-arm64'
+ pool:
+ name: onnxruntime-Win-CPU-2022
+ os: windows
+ templateContext:
+ outputs:
+ - output: pipelineArtifact
+ artifactName: 'flc-win-arm64'
+ targetPath: '$(Build.ArtifactStagingDirectory)/native'
+ steps:
+ - checkout: neutron-server
+ clean: true
+ - template: .pipelines/templates/build-core-steps.yml@self
+ parameters:
+ flavor: win-arm64
+ platform: arm64
+
+ - job: flc_linux_x64
+ displayName: 'FLC linux-x64'
+ pool:
+ name: onnxruntime-Ubuntu2404-AMD-CPU
+ os: linux
+ templateContext:
+ outputs:
+ - output: pipelineArtifact
+ artifactName: 'flc-linux-x64'
+ targetPath: '$(Build.ArtifactStagingDirectory)/native'
+ steps:
+ - checkout: neutron-server
+ clean: true
+ - template: .pipelines/templates/build-core-steps.yml@self
+ parameters:
+ flavor: linux-x64
+ platform: x64
+
+ - job: flc_osx_arm64
+ displayName: 'FLC osx-arm64'
+ pool:
+ name: Azure Pipelines
+ vmImage: 'macOS-14'
+ os: macOS
+ templateContext:
+ outputs:
+ - output: pipelineArtifact
+ artifactName: 'flc-osx-arm64'
+ targetPath: '$(Build.ArtifactStagingDirectory)/native'
+ steps:
+ - checkout: neutron-server
+ clean: true
+ - template: .pipelines/templates/build-core-steps.yml@self
+ parameters:
+ flavor: osx-arm64
+ platform: arm64
+
+ # ── Package FLC ──
+ - stage: package_core
+ displayName: 'Package FLC'
+ dependsOn: build_core
+ jobs:
+ - job: package_flc
+ displayName: 'Package FLC'
+ pool:
+ name: onnxruntime-Win-CPU-2022
+ os: windows
+ templateContext:
+ outputs:
+ - output: pipelineArtifact
+ artifactName: 'flc-nuget'
+ targetPath: '$(Build.ArtifactStagingDirectory)/flc-nuget'
+ - output: pipelineArtifact
+ artifactName: 'flc-wheels'
+ targetPath: '$(Build.ArtifactStagingDirectory)/flc-wheels'
+ steps:
+ - checkout: neutron-server
+ clean: true
+ - task: DownloadPipelineArtifact@2
+ inputs:
+ buildType: current
+ artifactName: 'flc-win-x64'
+ targetPath: '$(Pipeline.Workspace)/flc-win-x64'
+ - task: DownloadPipelineArtifact@2
+ inputs:
+ buildType: current
+ artifactName: 'flc-win-arm64'
+ targetPath: '$(Pipeline.Workspace)/flc-win-arm64'
+ - task: DownloadPipelineArtifact@2
+ inputs:
+ buildType: current
+ artifactName: 'flc-linux-x64'
+ targetPath: '$(Pipeline.Workspace)/flc-linux-x64'
+ - task: DownloadPipelineArtifact@2
+ inputs:
+ buildType: current
+ artifactName: 'flc-osx-arm64'
+ targetPath: '$(Pipeline.Workspace)/flc-osx-arm64'
+ - task: PowerShell@2
+ displayName: 'List downloaded platform artifacts'
+ inputs:
+ targetType: inline
+ script: |
+ foreach ($name in @('flc-win-x64','flc-win-arm64','flc-linux-x64','flc-osx-arm64')) {
+ $dir = "$(Pipeline.Workspace)/$name"
+ Write-Host "Contents of ${dir}:"
+ if (Test-Path $dir) { Get-ChildItem $dir -Recurse | ForEach-Object { Write-Host $_.FullName } }
+ else { Write-Host " (directory not found)" }
+ }
+ - template: .pipelines/templates/package-core-steps.yml@self
+ parameters:
+ version: ${{ parameters.version }}
+ isRelease: ${{ parameters.isRelease }}
+ prereleaseId: ${{ parameters.prereleaseId }}
+ isWinML: false
+ platforms:
+ - name: win-x64
+ artifactName: flc-win-x64
+ - name: win-arm64
+ artifactName: flc-win-arm64
+ - name: linux-x64
+ artifactName: flc-linux-x64
+ - name: osx-arm64
+ artifactName: flc-osx-arm64
+
+ # ── Build C# SDK ──
+ - stage: build_cs
+ displayName: 'Build C# SDK'
+ dependsOn: package_core
+ jobs:
+ - job: cs_sdk
+ displayName: 'Build'
+ pool:
+ name: onnxruntime-Win-CPU-2022
+ os: windows
+ templateContext:
+ inputs:
+ - input: pipelineArtifact
+ artifactName: 'flc-nuget'
+ targetPath: '$(Pipeline.Workspace)/flc-nuget'
+ outputs:
+ - output: pipelineArtifact
+ artifactName: 'cs-sdk'
+ targetPath: '$(Build.ArtifactStagingDirectory)/cs-sdk'
+ steps:
+ - checkout: self
+ clean: true
+ - checkout: test-data-shared
+ lfs: true
+ - template: .pipelines/templates/build-cs-steps.yml@self
+ parameters:
+ version: ${{ parameters.version }}
+ isRelease: ${{ parameters.isRelease }}
+ prereleaseId: ${{ parameters.prereleaseId }}
+ isWinML: false
+ flcNugetDir: '$(Pipeline.Workspace)/flc-nuget'
+
+ # ── Build JS SDK ──
+ - stage: build_js
+ displayName: 'Build JS SDK'
+ dependsOn: package_core
+ jobs:
+ - job: js_sdk
+ displayName: 'Build'
+ pool:
+ name: onnxruntime-Win-CPU-2022
+ os: windows
+ templateContext:
+ inputs:
+ - input: pipelineArtifact
+ artifactName: 'flc-nuget'
+ targetPath: '$(Pipeline.Workspace)/flc-nuget'
+ outputs:
+ - output: pipelineArtifact
+ artifactName: 'js-sdk'
+ targetPath: '$(Build.ArtifactStagingDirectory)/js-sdk'
+ steps:
+ - checkout: self
+ clean: true
+ - checkout: test-data-shared
+ lfs: true
+ - template: .pipelines/templates/build-js-steps.yml@self
+ parameters:
+ version: ${{ parameters.version }}
+ isRelease: ${{ parameters.isRelease }}
+ prereleaseId: ${{ parameters.prereleaseId }}
+ isWinML: false
+ flcNugetDir: '$(Pipeline.Workspace)/flc-nuget'
+
+ # ── Build Python SDK ──
+ - stage: build_python
+ displayName: 'Build Python SDK'
+ dependsOn: package_core
+ jobs:
+ - job: python_sdk
+ displayName: 'Build'
+ pool:
+ name: onnxruntime-Win-CPU-2022
+ os: windows
+ templateContext:
+ inputs:
+ - input: pipelineArtifact
+ artifactName: 'flc-wheels'
+ targetPath: '$(Pipeline.Workspace)/flc-wheels'
+ outputs:
+ - output: pipelineArtifact
+ artifactName: 'python-sdk'
+ targetPath: '$(Build.ArtifactStagingDirectory)/python-sdk'
+ steps:
+ - checkout: self
+ clean: true
+ - checkout: test-data-shared
+ lfs: true
+ - template: .pipelines/templates/build-python-steps.yml@self
+ parameters:
+ version: ${{ parameters.version }}
+ isRelease: ${{ parameters.isRelease }}
+ prereleaseId: ${{ parameters.prereleaseId }}
+ isWinML: false
+ flcWheelsDir: '$(Pipeline.Workspace)/flc-wheels'
+
+ # ── Build Rust SDK ──
+ - stage: build_rust
+ displayName: 'Build Rust SDK'
+ dependsOn: package_core
+ jobs:
+ - job: rust_sdk
+ displayName: 'Build'
+ pool:
+ name: onnxruntime-Win-CPU-2022
+ os: windows
+ templateContext:
+ inputs:
+ - input: pipelineArtifact
+ artifactName: 'flc-nuget'
+ targetPath: '$(Pipeline.Workspace)/flc-nuget'
+ outputs:
+ - output: pipelineArtifact
+ artifactName: 'rust-sdk'
+ targetPath: '$(Build.ArtifactStagingDirectory)/rust-sdk'
+ steps:
+ - checkout: self
+ clean: true
+ - checkout: test-data-shared
+ lfs: true
+ - template: .pipelines/templates/build-rust-steps.yml@self
+ parameters:
+ version: ${{ parameters.version }}
+ isRelease: ${{ parameters.isRelease }}
+ prereleaseId: ${{ parameters.prereleaseId }}
+ isWinML: false
+ flcNugetDir: '$(Pipeline.Workspace)/flc-nuget'
+
+ # ── Test C# SDK (win-x64) ──
+ - stage: test_cs
+ displayName: 'Test C# SDK'
+ dependsOn: build_cs
+ jobs:
+ - job: test_cs_win_x64
+ displayName: 'Test C# (win-x64)'
+ pool:
+ name: onnxruntime-Win-CPU-2022
+ os: windows
+ templateContext:
+ inputs:
+ - input: pipelineArtifact
+ artifactName: 'flc-nuget'
+ targetPath: '$(Pipeline.Workspace)/flc-nuget'
+ steps:
+ - checkout: self
+ clean: true
+ - checkout: test-data-shared
+ lfs: true
+ - template: .pipelines/templates/test-cs-steps.yml@self
+ parameters:
+ version: ${{ parameters.version }}
+ isWinML: false
+ flcNugetDir: '$(Pipeline.Workspace)/flc-nuget'
+
+ # TODO: Add macOS (osx-arm64) test job when a macOS ARM64 pool is available.
+ # TODO: Add Linux (linux-x64) test job when Linux onnxruntime dependency is stabilized.
+ # TODO: Add Windows ARM64 (win-arm64) test job when a Windows ARM64 pool is available.
+
+ # ── Test JS SDK (win-x64) ──
+ - stage: test_js
+ displayName: 'Test JS SDK'
+ dependsOn: build_js
+ jobs:
+ - job: test_js_win_x64
+ displayName: 'Test JS (win-x64)'
+ pool:
+ name: onnxruntime-Win-CPU-2022
+ os: windows
+ templateContext:
+ inputs:
+ - input: pipelineArtifact
+ artifactName: 'flc-nuget'
+ targetPath: '$(Pipeline.Workspace)/flc-nuget'
+ steps:
+ - checkout: self
+ clean: true
+ - checkout: test-data-shared
+ lfs: true
+ - template: .pipelines/templates/test-js-steps.yml@self
+ parameters:
+ version: ${{ parameters.version }}
+ isWinML: false
+ flcNugetDir: '$(Pipeline.Workspace)/flc-nuget'
+
+ # TODO: Add macOS (osx-arm64) test job when a macOS ARM64 pool is available.
+ # TODO: Add Linux (linux-x64) test job when Linux onnxruntime dependency is stabilized.
+ # TODO: Add Windows ARM64 (win-arm64) test job when a Windows ARM64 pool is available.
+
+ # ── Test Python SDK (win-x64) ──
+ - stage: test_python
+ displayName: 'Test Python SDK'
+ dependsOn: build_python
+ jobs:
+ - job: test_python_win_x64
+ displayName: 'Test Python (win-x64)'
+ pool:
+ name: onnxruntime-Win-CPU-2022
+ os: windows
+ templateContext:
+ inputs:
+ - input: pipelineArtifact
+ artifactName: 'flc-wheels'
+ targetPath: '$(Pipeline.Workspace)/flc-wheels'
+ steps:
+ - checkout: self
+ clean: true
+ - checkout: test-data-shared
+ lfs: true
+ - template: .pipelines/templates/test-python-steps.yml@self
+ parameters:
+ version: ${{ parameters.version }}
+ isWinML: false
+ flcWheelsDir: '$(Pipeline.Workspace)/flc-wheels'
+
+ # TODO: Add macOS (osx-arm64) test job when a macOS ARM64 pool is available.
+ # TODO: Add Linux (linux-x64) test job when Linux onnxruntime dependency is stabilized.
+ # TODO: Add Windows ARM64 (win-arm64) test job when a Windows ARM64 pool is available.
+
+ # ── Test Rust SDK (win-x64) ──
+ - stage: test_rust
+ displayName: 'Test Rust SDK'
+ dependsOn: build_rust
+ jobs:
+ - job: test_rust_win_x64
+ displayName: 'Test Rust (win-x64)'
+ pool:
+ name: onnxruntime-Win-CPU-2022
+ os: windows
+ templateContext:
+ inputs:
+ - input: pipelineArtifact
+ artifactName: 'flc-nuget'
+ targetPath: '$(Pipeline.Workspace)/flc-nuget'
+ steps:
+ - checkout: self
+ clean: true
+ - checkout: test-data-shared
+ lfs: true
+ - template: .pipelines/templates/test-rust-steps.yml@self
+ parameters:
+ isWinML: false
+ flcNugetDir: '$(Pipeline.Workspace)/flc-nuget'
+
+ # TODO: Add macOS (osx-arm64) test job when a macOS ARM64 pool is available.
+ # TODO: Add Linux (linux-x64) test job when Linux onnxruntime dependency is stabilized.
+ # TODO: Add Windows ARM64 (win-arm64) test job when a Windows ARM64 pool is available.
+
+ # ── Build & Test FLC (WinML) ──
+ - stage: build_core_winml
+ displayName: 'Build & Test FLC WinML'
+ dependsOn: []
+ jobs:
+ - job: flc_winml_win_x64
+ displayName: 'FLC win-x64 (WinML)'
+ pool:
+ name: onnxruntime-Win-CPU-2022
+ os: windows
+ templateContext:
+ outputs:
+ - output: pipelineArtifact
+ artifactName: 'flc-winml-win-x64'
+ targetPath: '$(Build.ArtifactStagingDirectory)/native'
+ steps:
+ - checkout: neutron-server
+ clean: true
+ - checkout: test-data-shared
+ lfs: true
+ - template: .pipelines/templates/build-core-steps.yml@self
+ parameters:
+ flavor: win-x64
+ platform: x64
+ isWinML: true
+
+ - job: flc_winml_win_arm64
+ displayName: 'FLC win-arm64 (WinML)'
+ pool:
+ name: onnxruntime-Win-CPU-2022
+ os: windows
+ templateContext:
+ outputs:
+ - output: pipelineArtifact
+ artifactName: 'flc-winml-win-arm64'
+ targetPath: '$(Build.ArtifactStagingDirectory)/native'
+ steps:
+ - checkout: neutron-server
+ clean: true
+ - template: .pipelines/templates/build-core-steps.yml@self
+ parameters:
+ flavor: win-arm64
+ platform: arm64
+ isWinML: true
+
+ # ── Package FLC (WinML) ──
+ - stage: package_core_winml
+ displayName: 'Package FLC WinML'
+ dependsOn: build_core_winml
+ jobs:
+ - job: package_flc_winml
+ displayName: 'Package FLC (WinML)'
+ pool:
+ name: onnxruntime-Win-CPU-2022
+ os: windows
+ templateContext:
+ outputs:
+ - output: pipelineArtifact
+ artifactName: 'flc-nuget-winml'
+ targetPath: '$(Build.ArtifactStagingDirectory)/flc-nuget'
+ - output: pipelineArtifact
+ artifactName: 'flc-wheels-winml'
+ targetPath: '$(Build.ArtifactStagingDirectory)/flc-wheels'
+ steps:
+ - checkout: neutron-server
+ clean: true
+ - task: DownloadPipelineArtifact@2
+ inputs:
+ buildType: current
+ artifactName: 'flc-winml-win-x64'
+ targetPath: '$(Pipeline.Workspace)/flc-winml-win-x64'
+ - task: DownloadPipelineArtifact@2
+ inputs:
+ buildType: current
+ artifactName: 'flc-winml-win-arm64'
+ targetPath: '$(Pipeline.Workspace)/flc-winml-win-arm64'
+ - task: PowerShell@2
+ displayName: 'List downloaded WinML platform artifacts'
+ inputs:
+ targetType: inline
+ script: |
+ foreach ($name in @('flc-winml-win-x64','flc-winml-win-arm64')) {
+ $dir = "$(Pipeline.Workspace)/$name"
+ Write-Host "Contents of ${dir}:"
+ if (Test-Path $dir) { Get-ChildItem $dir -Recurse | ForEach-Object { Write-Host $_.FullName } }
+ else { Write-Host " (directory not found)" }
+ }
+ - template: .pipelines/templates/package-core-steps.yml@self
+ parameters:
+ version: ${{ parameters.version }}
+ isRelease: ${{ parameters.isRelease }}
+ prereleaseId: ${{ parameters.prereleaseId }}
+ isWinML: true
+ platforms:
+ - name: win-x64
+ artifactName: flc-winml-win-x64
+ - name: win-arm64
+ artifactName: flc-winml-win-arm64
+
+ # ── Build C# SDK (WinML) ──
+ - stage: build_cs_winml
+ displayName: 'Build C# SDK WinML'
+ dependsOn: package_core_winml
+ jobs:
+ - job: cs_sdk_winml
+ displayName: 'Build'
+ pool:
+ name: onnxruntime-Win-CPU-2022
+ os: windows
+ templateContext:
+ inputs:
+ - input: pipelineArtifact
+ artifactName: 'flc-nuget-winml'
+ targetPath: '$(Pipeline.Workspace)/flc-nuget-winml'
+ outputs:
+ - output: pipelineArtifact
+ artifactName: 'cs-sdk-winml'
+ targetPath: '$(Build.ArtifactStagingDirectory)/cs-sdk-winml'
+ steps:
+ - checkout: self
+ clean: true
+ - checkout: test-data-shared
+ lfs: true
+ - template: .pipelines/templates/build-cs-steps.yml@self
+ parameters:
+ version: ${{ parameters.version }}
+ isRelease: ${{ parameters.isRelease }}
+ prereleaseId: ${{ parameters.prereleaseId }}
+ isWinML: true
+ flcNugetDir: '$(Pipeline.Workspace)/flc-nuget-winml'
+ outputDir: '$(Build.ArtifactStagingDirectory)/cs-sdk-winml'
+
+ # ── Build JS SDK (WinML) ──
+ - stage: build_js_winml
+ displayName: 'Build JS SDK WinML'
+ dependsOn: package_core_winml
+ jobs:
+ - job: js_sdk_winml
+ displayName: 'Build'
+ pool:
+ name: onnxruntime-Win-CPU-2022
+ os: windows
+ templateContext:
+ inputs:
+ - input: pipelineArtifact
+ artifactName: 'flc-nuget-winml'
+ targetPath: '$(Pipeline.Workspace)/flc-nuget-winml'
+ outputs:
+ - output: pipelineArtifact
+ artifactName: 'js-sdk-winml'
+ targetPath: '$(Build.ArtifactStagingDirectory)/js-sdk'
+ steps:
+ - checkout: self
+ clean: true
+ - checkout: test-data-shared
+ lfs: true
+ - template: .pipelines/templates/build-js-steps.yml@self
+ parameters:
+ version: ${{ parameters.version }}
+ isRelease: ${{ parameters.isRelease }}
+ prereleaseId: ${{ parameters.prereleaseId }}
+ isWinML: true
+ flcNugetDir: '$(Pipeline.Workspace)/flc-nuget-winml'
+
+ # ── Build Python SDK (WinML) ──
+ - stage: build_python_winml
+ displayName: 'Build Python SDK WinML'
+ dependsOn: package_core_winml
+ jobs:
+ - job: python_sdk_winml
+ displayName: 'Build'
+ pool:
+ name: onnxruntime-Win-CPU-2022
+ os: windows
+ templateContext:
+ inputs:
+ - input: pipelineArtifact
+ artifactName: 'flc-wheels-winml'
+ targetPath: '$(Pipeline.Workspace)/flc-wheels-winml'
+ outputs:
+ - output: pipelineArtifact
+ artifactName: 'python-sdk-winml'
+ targetPath: '$(Build.ArtifactStagingDirectory)/python-sdk-winml'
+ steps:
+ - checkout: self
+ clean: true
+ - checkout: test-data-shared
+ lfs: true
+ - template: .pipelines/templates/build-python-steps.yml@self
+ parameters:
+ version: ${{ parameters.version }}
+ isRelease: ${{ parameters.isRelease }}
+ prereleaseId: ${{ parameters.prereleaseId }}
+ isWinML: true
+ flcWheelsDir: '$(Pipeline.Workspace)/flc-wheels-winml'
+ outputDir: '$(Build.ArtifactStagingDirectory)/python-sdk-winml'
+
+ # ── Build Rust SDK (WinML) ──
+ - stage: build_rust_winml
+ displayName: 'Build Rust SDK WinML'
+ dependsOn: package_core_winml
+ jobs:
+ - job: rust_sdk_winml
+ displayName: 'Build'
+ pool:
+ name: onnxruntime-Win-CPU-2022
+ os: windows
+ templateContext:
+ inputs:
+ - input: pipelineArtifact
+ artifactName: 'flc-nuget-winml'
+ targetPath: '$(Pipeline.Workspace)/flc-nuget-winml'
+ outputs:
+ - output: pipelineArtifact
+ artifactName: 'rust-sdk-winml'
+ targetPath: '$(Build.ArtifactStagingDirectory)/rust-sdk-winml'
+ steps:
+ - checkout: self
+ clean: true
+ - checkout: test-data-shared
+ lfs: true
+ - template: .pipelines/templates/build-rust-steps.yml@self
+ parameters:
+ version: ${{ parameters.version }}
+ isRelease: ${{ parameters.isRelease }}
+ prereleaseId: ${{ parameters.prereleaseId }}
+ isWinML: true
+ flcNugetDir: '$(Pipeline.Workspace)/flc-nuget-winml'
+ outputDir: '$(Build.ArtifactStagingDirectory)/rust-sdk-winml'
+
+ # ── Test C# SDK WinML (win-x64) ──
+ - stage: test_cs_winml
+ displayName: 'Test C# SDK WinML'
+ dependsOn: build_cs_winml
+ jobs:
+ - job: test_cs_winml_win_x64
+ displayName: 'Test C# WinML (win-x64)'
+ pool:
+ name: onnxruntime-Win-CPU-2022
+ os: windows
+ templateContext:
+ inputs:
+ - input: pipelineArtifact
+ artifactName: 'flc-nuget-winml'
+ targetPath: '$(Pipeline.Workspace)/flc-nuget-winml'
+ steps:
+ - checkout: self
+ clean: true
+ - checkout: test-data-shared
+ lfs: true
+ - template: .pipelines/templates/test-cs-steps.yml@self
+ parameters:
+ version: ${{ parameters.version }}
+ isWinML: true
+ flcNugetDir: '$(Pipeline.Workspace)/flc-nuget-winml'
+
+ # TODO: Add Windows ARM64 (win-arm64) test job when a Windows ARM64 pool is available.
+
+ # ── Test JS SDK WinML (win-x64) ──
+ - stage: test_js_winml
+ displayName: 'Test JS SDK WinML'
+ dependsOn: build_js_winml
+ jobs:
+ - job: test_js_winml_win_x64
+ displayName: 'Test JS WinML (win-x64)'
+ pool:
+ name: onnxruntime-Win-CPU-2022
+ os: windows
+ templateContext:
+ inputs:
+ - input: pipelineArtifact
+ artifactName: 'flc-nuget-winml'
+ targetPath: '$(Pipeline.Workspace)/flc-nuget-winml'
+ steps:
+ - checkout: self
+ clean: true
+ - checkout: test-data-shared
+ lfs: true
+ - template: .pipelines/templates/test-js-steps.yml@self
+ parameters:
+ version: ${{ parameters.version }}
+ isWinML: true
+ flcNugetDir: '$(Pipeline.Workspace)/flc-nuget-winml'
+
+ # TODO: Add Windows ARM64 (win-arm64) test job when a Windows ARM64 pool is available.
+
+ # ── Test Python SDK WinML (win-x64) ──
+ - stage: test_python_winml
+ displayName: 'Test Python SDK WinML'
+ dependsOn: build_python_winml
+ jobs:
+ - job: test_python_winml_win_x64
+ displayName: 'Test Python WinML (win-x64)'
+ pool:
+ name: onnxruntime-Win-CPU-2022
+ os: windows
+ templateContext:
+ inputs:
+ - input: pipelineArtifact
+ artifactName: 'flc-wheels-winml'
+ targetPath: '$(Pipeline.Workspace)/flc-wheels-winml'
+ steps:
+ - checkout: self
+ clean: true
+ - checkout: test-data-shared
+ lfs: true
+ - template: .pipelines/templates/test-python-steps.yml@self
+ parameters:
+ version: ${{ parameters.version }}
+ isWinML: true
+ flcWheelsDir: '$(Pipeline.Workspace)/flc-wheels-winml'
+
+ # TODO: Add Windows ARM64 (win-arm64) test job when a Windows ARM64 pool is available.
+
+ # ── Test Rust SDK WinML (win-x64) ──
+ - stage: test_rust_winml
+ displayName: 'Test Rust SDK WinML'
+ dependsOn: build_rust_winml
+ jobs:
+ - job: test_rust_winml_win_x64
+ displayName: 'Test Rust WinML (win-x64)'
+ pool:
+ name: onnxruntime-Win-CPU-2022
+ os: windows
+ templateContext:
+ inputs:
+ - input: pipelineArtifact
+ artifactName: 'flc-nuget-winml'
+ targetPath: '$(Pipeline.Workspace)/flc-nuget-winml'
+ steps:
+ - checkout: self
+ clean: true
+ - checkout: test-data-shared
+ lfs: true
+ - template: .pipelines/templates/test-rust-steps.yml@self
+ parameters:
+ isWinML: true
+ flcNugetDir: '$(Pipeline.Workspace)/flc-nuget-winml'
+
+ # TODO: Add Windows ARM64 (win-arm64) test job when a Windows ARM64 pool is available.
-steps:
-- script: echo "Foundry Local packaging pipeline - placeholder"
- displayName: 'Placeholder'
\ No newline at end of file
diff --git a/.pipelines/templates/build-core-steps.yml b/.pipelines/templates/build-core-steps.yml
new file mode 100644
index 00000000..9f024c42
--- /dev/null
+++ b/.pipelines/templates/build-core-steps.yml
@@ -0,0 +1,194 @@
+# Steps to build a single Foundry Local Core native AOT binary.
+# Parameterized by flavor (RID) and platform (arch).
+# The parent job must checkout 'neutron-server'.
+parameters:
+- name: flavor
+ type: string # e.g. win-x64, linux-x64, osx-arm64
+- name: platform
+ type: string # e.g. x64, arm64
+- name: isWinML
+ type: boolean
+ default: false
+
+steps:
+- task: PowerShell@2
+ displayName: 'Set source paths'
+ inputs:
+ targetType: inline
+ script: |
+ # Multi-checkout places repos in subdirectories; single checkout places contents at root
+ $multiCheckout = "$(Build.SourcesDirectory)/neutron-server"
+ if (Test-Path $multiCheckout) {
+ $nsRoot = $multiCheckout
+ } else {
+ $nsRoot = "$(Build.SourcesDirectory)"
+ }
+ Write-Host "##vso[task.setvariable variable=nsRoot]$nsRoot"
+ Write-Host "neutron-server root: $nsRoot"
+
+- task: UseDotNet@2
+ displayName: 'Use .NET SDK from global.json'
+ inputs:
+ packageType: sdk
+ useGlobalJson: true
+ workingDirectory: '$(nsRoot)'
+
+- task: PowerShell@2
+ displayName: 'Override nuget.config'
+ inputs:
+ targetType: inline
+ script: |
+ $nugetConfig = @"
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+ "@
+ Set-Content -Path "$(nsRoot)/nuget.config" -Value $nugetConfig
+ Write-Host "Updated nuget.config to use nuget.org, ORT-Nightly, and Neutron with mappings"
+
+- ${{ if eq(parameters.isWinML, true) }}:
+ - task: DotNetCoreCLI@2
+ displayName: 'Restore FLC Core ${{ parameters.flavor }} (WinML)'
+ inputs:
+ command: restore
+ projects: '$(nsRoot)/src/FoundryLocalCore/Core/Core.csproj'
+ restoreArguments: '-r ${{ parameters.flavor }} /p:Platform=${{ parameters.platform }} /p:IncludeWebService=true /p:Configuration=Release /p:NetTargetFramework=net9.0-windows10.0.26100.0 /p:UseWinML=true'
+ feedsToUse: config
+ nugetConfigPath: '$(nsRoot)/nuget.config'
+
+ - task: DotNetCoreCLI@2
+ displayName: 'Build FLC Core ${{ parameters.flavor }} (WinML)'
+ inputs:
+ command: build
+ projects: '$(nsRoot)/src/FoundryLocalCore/Core/Core.csproj'
+ arguments: '--no-restore -r ${{ parameters.flavor }} -f net9.0-windows10.0.26100.0 /p:Platform=${{ parameters.platform }} /p:IncludeWebService=true /p:Configuration=Release /p:NetTargetFramework=net9.0-windows10.0.26100.0 /p:UseWinML=true'
+
+ - task: DotNetCoreCLI@2
+ displayName: 'Publish FLC AOT ${{ parameters.flavor }} (WinML)'
+ inputs:
+ command: publish
+ projects: '$(nsRoot)/src/FoundryLocalCore/Core/Core.csproj'
+ arguments: '--no-restore --no-build -r ${{ parameters.flavor }} -f net9.0-windows10.0.26100.0 /p:Platform=${{ parameters.platform }} /p:Configuration=Release /p:PublishAot=true /p:NetTargetFramework=net9.0-windows10.0.26100.0 /p:UseWinML=true'
+ publishWebProjects: false
+ zipAfterPublish: false
+
+ - ${{ if eq(parameters.flavor, 'win-x64') }}:
+ - task: DotNetCoreCLI@2
+ displayName: 'Restore FLC Tests ${{ parameters.flavor }} (WinML)'
+ inputs:
+ command: restore
+ projects: '$(nsRoot)/test/FoundryLocalCore/Core/FoundryLocalCore.Tests.csproj'
+ restoreArguments: '-r ${{ parameters.flavor }} /p:Platform=${{ parameters.platform }} /p:IncludeWebService=true /p:Configuration=Release /p:NetTargetFramework=net9.0-windows10.0.26100.0 /p:UseWinML=true'
+ feedsToUse: config
+ nugetConfigPath: '$(nsRoot)/nuget.config'
+
+ - task: DotNetCoreCLI@2
+ displayName: 'Build FLC Tests ${{ parameters.flavor }} (WinML)'
+ inputs:
+ command: build
+ projects: '$(nsRoot)/test/FoundryLocalCore/Core/FoundryLocalCore.Tests.csproj'
+ arguments: '--no-restore -r ${{ parameters.flavor }} /p:Platform=${{ parameters.platform }} /p:IncludeWebService=true /p:Configuration=Release /p:NetTargetFramework=net9.0-windows10.0.26100.0 /p:UseWinML=true'
+
+ - task: DotNetCoreCLI@2
+ displayName: 'Test FLC ${{ parameters.flavor }} (WinML)'
+ inputs:
+ command: test
+ projects: '$(nsRoot)/test/FoundryLocalCore/Core/FoundryLocalCore.Tests.csproj'
+ arguments: '--no-build --configuration Release -r ${{ parameters.flavor }} /p:Platform=${{ parameters.platform }}'
+
+- ${{ if eq(parameters.isWinML, false) }}:
+ - task: DotNetCoreCLI@2
+ displayName: 'Restore FLC Core ${{ parameters.flavor }}'
+ inputs:
+ command: restore
+ projects: '$(nsRoot)/src/FoundryLocalCore/Core/Core.csproj'
+ restoreArguments: '-r ${{ parameters.flavor }} /p:Platform=${{ parameters.platform }} /p:IncludeWebService=true /p:Configuration=Release /p:TargetFramework=net9.0'
+ feedsToUse: config
+ nugetConfigPath: '$(nsRoot)/nuget.config'
+
+ - task: DotNetCoreCLI@2
+ displayName: 'Build FLC Core ${{ parameters.flavor }}'
+ inputs:
+ command: build
+ projects: '$(nsRoot)/src/FoundryLocalCore/Core/Core.csproj'
+ arguments: '--no-restore -r ${{ parameters.flavor }} /p:Platform=${{ parameters.platform }} /p:IncludeWebService=true /p:Configuration=Release'
+
+ - ${{ if eq(parameters.flavor, 'win-x64') }}:
+ - task: DotNetCoreCLI@2
+ displayName: 'Restore FLC Tests ${{ parameters.flavor }}'
+ inputs:
+ command: restore
+ projects: '$(nsRoot)/test/FoundryLocalCore/Core/FoundryLocalCore.Tests.csproj'
+ restoreArguments: '-r ${{ parameters.flavor }} /p:Platform=${{ parameters.platform }} /p:IncludeWebService=true /p:Configuration=Release /p:TargetFramework=net9.0'
+ feedsToUse: config
+ nugetConfigPath: '$(nsRoot)/nuget.config'
+
+ - task: DotNetCoreCLI@2
+ displayName: 'Build FLC Tests ${{ parameters.flavor }}'
+ inputs:
+ command: build
+ projects: '$(nsRoot)/test/FoundryLocalCore/Core/FoundryLocalCore.Tests.csproj'
+ arguments: '--no-restore -r ${{ parameters.flavor }} /p:Platform=${{ parameters.platform }} /p:IncludeWebService=true /p:Configuration=Release'
+
+ - task: DotNetCoreCLI@2
+ displayName: 'Test FLC ${{ parameters.flavor }}'
+ inputs:
+ command: test
+ projects: '$(nsRoot)/test/FoundryLocalCore/Core/FoundryLocalCore.Tests.csproj'
+ arguments: '--no-build --configuration Release -r ${{ parameters.flavor }} /p:Platform=${{ parameters.platform }}'
+
+ - task: DotNetCoreCLI@2
+ displayName: 'Publish FLC AOT ${{ parameters.flavor }}'
+ inputs:
+ command: publish
+ projects: '$(nsRoot)/src/FoundryLocalCore/Core/Core.csproj'
+ arguments: '--no-restore --no-build -r ${{ parameters.flavor }} /p:Platform=${{ parameters.platform }} /p:Configuration=Release /p:PublishAot=true /p:TargetFramework=net9.0'
+ publishWebProjects: false
+ zipAfterPublish: false
+
+# Cleanup non-binary files
+- task: PowerShell@2
+ displayName: 'Cleanup publish artifacts'
+ inputs:
+ targetType: inline
+ script: |
+ Get-ChildItem "$(nsRoot)/artifacts/publish" -Recurse -Include "*.json", "*.xml" |
+ Remove-Item -Force
+
+# Stage the native binary for the artifact
+- task: PowerShell@2
+ displayName: 'Stage ${{ parameters.flavor }} binary'
+ inputs:
+ targetType: inline
+ script: |
+ $destDir = "$(Build.ArtifactStagingDirectory)/native"
+ New-Item -ItemType Directory -Path $destDir -Force | Out-Null
+ # WinML publishes additional files (e.g. WindowsAppRuntime Bootstrapper DLLs)
+ # beyond Microsoft.AI.Foundry.Local.Core.*.
+ $isWinML = "${{ parameters.isWinML }}" -eq "True"
+ if ($isWinML) {
+ Get-ChildItem "$(nsRoot)/artifacts/publish" -Recurse -File |
+ Where-Object { $_.Name -like "Microsoft.AI.Foundry.Local.Core.*" -or $_.Name -eq "Microsoft.WindowsAppRuntime.Bootstrap.dll" } |
+ Copy-Item -Destination $destDir -Force
+ } else {
+ Get-ChildItem "$(nsRoot)/artifacts/publish" -Recurse -File |
+ Where-Object { $_.Name -like "Microsoft.AI.Foundry.Local.Core.*" } |
+ Copy-Item -Destination $destDir -Force
+ }
+ Write-Host "Staged binaries:"
+ Get-ChildItem $destDir | ForEach-Object { Write-Host " $($_.Name)" }
+
diff --git a/.pipelines/templates/build-cs-steps.yml b/.pipelines/templates/build-cs-steps.yml
new file mode 100644
index 00000000..978c2fff
--- /dev/null
+++ b/.pipelines/templates/build-cs-steps.yml
@@ -0,0 +1,191 @@
+# Steps to build, sign, and pack the C# SDK NuGet package.
+# When test-data-shared is checked out alongside self, ADO places repos under
+# $(Build.SourcesDirectory)/. The self repo is 'Foundry-Local'.
+parameters:
+- name: version
+ type: string
+- name: isRelease
+ type: boolean
+ default: false
+- name: isWinML
+ type: boolean
+ default: false
+- name: flcNugetDir
+ type: string
+ displayName: 'Path to directory containing the FLC .nupkg'
+- name: outputDir
+ type: string
+ default: '$(Build.ArtifactStagingDirectory)/cs-sdk'
+ displayName: 'Path to directory for the packed SDK'
+- name: prereleaseId
+ type: string
+ default: ''
+steps:
+# Set paths for multi-repo checkout
+- task: PowerShell@2
+ displayName: 'Set source paths'
+ inputs:
+ targetType: inline
+ script: |
+ $repoRoot = "$(Build.SourcesDirectory)/Foundry-Local"
+ $testDataDir = "$(Build.SourcesDirectory)/test-data-shared"
+ Write-Host "##vso[task.setvariable variable=repoRoot]$repoRoot"
+ Write-Host "##vso[task.setvariable variable=testDataDir]$testDataDir"
+
+- task: UseDotNet@2
+ displayName: 'Use .NET 9 SDK'
+ inputs:
+ packageType: sdk
+ version: '9.0.x'
+
+# Compute package version
+- task: PowerShell@2
+ displayName: 'Set package version'
+ inputs:
+ targetType: inline
+ script: |
+ $v = "${{ parameters.version }}"
+ $preId = "${{ parameters.prereleaseId }}"
+ if ($preId -ne '' -and $preId -ne 'none') {
+ $v = "$v-$preId"
+ } elseif ("${{ parameters.isRelease }}" -ne "True") {
+ $ts = Get-Date -Format "yyyyMMddHHmm"
+ $v = "$v-dev.$ts"
+ }
+ Write-Host "##vso[task.setvariable variable=packageVersion]$v"
+ Write-Host "Package version: $v"
+
+# List downloaded artifact for debugging
+- task: PowerShell@2
+ displayName: 'List downloaded FLC artifact'
+ inputs:
+ targetType: inline
+ script: |
+ Write-Host "Contents of ${{ parameters.flcNugetDir }}:"
+ Get-ChildItem "${{ parameters.flcNugetDir }}" -Recurse | ForEach-Object { Write-Host $_.FullName }
+
+# Create a temporary NuGet.config that includes the local FLC feed
+- task: PowerShell@2
+ displayName: 'Create NuGet.config with local FLC feed'
+ inputs:
+ targetType: inline
+ script: |
+ $nugetConfig = @"
+
+
+
+
+
+
+
+
+
+ "@
+ # Determine the FLC version from the .nupkg filename
+ $nupkg = Get-ChildItem "${{ parameters.flcNugetDir }}" -Recurse -Filter "Microsoft.AI.Foundry.Local.Core*.nupkg" -Exclude "*.snupkg" | Select-Object -First 1
+ if (-not $nupkg) { throw "No FLC .nupkg found in ${{ parameters.flcNugetDir }}" }
+ $flcVer = $nupkg.BaseName -replace '^Microsoft\.AI\.Foundry\.Local\.Core(\.WinML)?\.', ''
+ Write-Host "##vso[task.setvariable variable=resolvedFlcVersion]$flcVer"
+ Write-Host "Resolved FLC version: $flcVer"
+
+ # Point the local NuGet feed at the directory that actually contains the .nupkg
+ $flcFeedDir = $nupkg.DirectoryName
+ $nugetConfig = $nugetConfig -replace [regex]::Escape("${{ parameters.flcNugetDir }}"), $flcFeedDir
+ $configPath = "$(Build.ArtifactStagingDirectory)/NuGet.config"
+ Set-Content -Path $configPath -Value $nugetConfig
+ Write-Host "##vso[task.setvariable variable=customNugetConfig]$configPath"
+ Write-Host "Local FLC feed directory: $flcFeedDir"
+
+- task: NuGetAuthenticate@1
+ displayName: 'Authenticate NuGet feeds'
+
+- task: PowerShell@2
+ displayName: 'Restore SDK'
+ inputs:
+ targetType: inline
+ script: |
+ $proj = "$(repoRoot)/sdk/cs/src/Microsoft.AI.Foundry.Local.csproj"
+ if (-not (Test-Path $proj)) { throw "Project not found: $proj" }
+ dotnet restore $proj `
+ --configfile "$(customNugetConfig)" `
+ /p:UseWinML=${{ parameters.isWinML }}
+ if ($LASTEXITCODE -ne 0) { exit $LASTEXITCODE }
+
+- task: PowerShell@2
+ displayName: 'Build SDK'
+ inputs:
+ targetType: inline
+ script: |
+ dotnet build "$(repoRoot)/sdk/cs/src/Microsoft.AI.Foundry.Local.csproj" `
+ --no-restore --configuration Release `
+ /p:UseWinML=${{ parameters.isWinML }}
+ if ($LASTEXITCODE -ne 0) { exit $LASTEXITCODE }
+
+# Discover target framework directory
+- task: PowerShell@2
+ displayName: 'Find target framework'
+ inputs:
+ targetType: inline
+ script: |
+ $base = "$(repoRoot)/sdk/cs/src/bin/Release"
+ # The SDK targets net9.0 (standard) or net9.0-windows10.0.26100.0 (WinML).
+ # Find whichever TFM directory was produced by the build.
+ $tfmDir = Get-ChildItem $base -Directory | Select-Object -First 1
+ if (-not $tfmDir) { throw "No target framework directory found under $base" }
+ Write-Host "##vso[task.setvariable variable=TargetFramework]$($tfmDir.Name)"
+ Write-Host "Target framework: $($tfmDir.Name)"
+
+# Sign DLLs
+- task: SFP.build-tasks.custom-build-task-1.EsrpCodeSigning@5
+ displayName: 'Sign SDK DLLs'
+ inputs:
+ ConnectedServiceName: 'OnnxrunTimeCodeSign_20240611'
+ UseMSIAuthentication: true
+ AppRegistrationClientId: '$(esrpClientId)'
+ AppRegistrationTenantId: '$(esrpTenantId)'
+ EsrpClientId: '$(esrpClientId)'
+ AuthAKVName: '$(esrpAkvName)'
+ AuthSignCertName: '$(esrpSignCertName)'
+ FolderPath: '$(repoRoot)/sdk/cs/src/bin/Release/$(TargetFramework)'
+ Pattern: '*.dll'
+ SessionTimeout: 90
+ ServiceEndpointUrl: 'https://api.esrp.microsoft.com/api/v2'
+ MaxConcurrency: 25
+ signConfigType: inlineSignParams
+ inlineOperation: |
+ [{"keyCode":"CP-230012","operationSetCode":"SigntoolSign","parameters":[{"parameterName":"OpusName","parameterValue":"Microsoft"},{"parameterName":"OpusInfo","parameterValue":"http://www.microsoft.com"},{"parameterName":"PageHash","parameterValue":"/NPH"},{"parameterName":"FileDigest","parameterValue":"/fd sha256"},{"parameterName":"TimeStamp","parameterValue":"/tr \"http://rfc3161.gtm.corp.microsoft.com/TSS/HttpTspServer\" /td sha256"}],"toolName":"signtool.exe","toolVersion":"6.2.9304.0"}]
+
+# Pack NuGet
+- task: PowerShell@2
+ displayName: 'Pack NuGet'
+ inputs:
+ targetType: inline
+ script: |
+ dotnet pack "$(repoRoot)/sdk/cs/src/Microsoft.AI.Foundry.Local.csproj" `
+ --no-build --no-restore --configuration Release `
+ --output "${{ parameters.outputDir }}" `
+ /p:PackageVersion=$(packageVersion) `
+ /p:UseWinML=${{ parameters.isWinML }} `
+ /p:IncludeSymbols=true `
+ /p:SymbolPackageFormat=snupkg
+ if ($LASTEXITCODE -ne 0) { exit $LASTEXITCODE }
+
+# Sign NuGet package
+- task: SFP.build-tasks.custom-build-task-1.EsrpCodeSigning@5
+ displayName: 'Sign SDK NuGet package'
+ inputs:
+ ConnectedServiceName: 'OnnxrunTimeCodeSign_20240611'
+ UseMSIAuthentication: true
+ AppRegistrationClientId: '$(esrpClientId)'
+ AppRegistrationTenantId: '$(esrpTenantId)'
+ EsrpClientId: '$(esrpClientId)'
+ AuthAKVName: '$(esrpAkvName)'
+ AuthSignCertName: '$(esrpSignCertName)'
+ FolderPath: '${{ parameters.outputDir }}'
+ Pattern: '*.nupkg'
+ SessionTimeout: 90
+ ServiceEndpointUrl: 'https://api.esrp.microsoft.com/api/v2'
+ MaxConcurrency: 25
+ signConfigType: inlineSignParams
+ inlineOperation: |
+ [{"keyCode":"CP-401405","operationSetCode":"NuGetSign","parameters":[],"toolName":"sign","toolVersion":"6.2.9304.0"},{"keyCode":"CP-401405","operationSetCode":"NuGetVerify","parameters":[],"toolName":"sign","toolVersion":"6.2.9304.0"}]
diff --git a/.pipelines/templates/build-js-steps.yml b/.pipelines/templates/build-js-steps.yml
new file mode 100644
index 00000000..e288bbce
--- /dev/null
+++ b/.pipelines/templates/build-js-steps.yml
@@ -0,0 +1,156 @@
+# Steps to build and pack the JS SDK.
+# When test-data-shared is checked out alongside self, ADO places repos under
+# $(Build.SourcesDirectory)/. The self repo is 'Foundry-Local'.
+parameters:
+- name: version
+ type: string
+- name: isRelease
+ type: boolean
+ default: false
+- name: isWinML
+ type: boolean
+ default: false
+- name: flcNugetDir
+ type: string
+ default: ''
+ displayName: 'Path to directory containing the FLC .nupkg (for tests)'
+- name: prereleaseId
+ type: string
+ default: ''
+steps:
+# Set paths for multi-repo checkout
+- task: PowerShell@2
+ displayName: 'Set source paths'
+ inputs:
+ targetType: inline
+ script: |
+ $repoRoot = "$(Build.SourcesDirectory)/Foundry-Local"
+ $testDataDir = "$(Build.SourcesDirectory)/test-data-shared"
+ Write-Host "##vso[task.setvariable variable=repoRoot]$repoRoot"
+ Write-Host "##vso[task.setvariable variable=testDataDir]$testDataDir"
+ Write-Host "Repo root: $repoRoot"
+ Write-Host "Test data: $testDataDir"
+
+- task: PowerShell@2
+ displayName: 'List downloaded FLC artifact'
+ condition: and(succeeded(), ne('${{ parameters.flcNugetDir }}', ''))
+ inputs:
+ targetType: inline
+ script: |
+ Write-Host "Contents of ${{ parameters.flcNugetDir }}:"
+ Get-ChildItem "${{ parameters.flcNugetDir }}" -Recurse | ForEach-Object { Write-Host $_.FullName }
+
+- task: NodeTool@0
+ displayName: 'Use Node.js 20'
+ inputs:
+ versionSpec: '20.x'
+
+# Compute version
+- task: PowerShell@2
+ displayName: 'Set package version'
+ inputs:
+ targetType: inline
+ script: |
+ $v = "${{ parameters.version }}"
+ $preId = "${{ parameters.prereleaseId }}"
+ if ($preId -ne '' -and $preId -ne 'none') {
+ $v = "$v-$preId"
+ } elseif ("${{ parameters.isRelease }}" -ne "True") {
+ $ts = Get-Date -Format "yyyyMMddHHmm"
+ $v = "$v-dev.$ts"
+ }
+ Write-Host "##vso[task.setvariable variable=packageVersion]$v"
+
+# Install dependencies including native binaries (FLC, ORT, GenAI) from NuGet feeds
+- task: Npm@1
+ displayName: 'npm install'
+ inputs:
+ command: custom
+ workingDir: $(repoRoot)/sdk/js
+ customCommand: 'install'
+
+# Overwrite the FLC native binary with the one we just built
+- task: PowerShell@2
+ displayName: 'Overwrite FLC with pipeline-built binary'
+ condition: and(succeeded(), ne('${{ parameters.flcNugetDir }}', ''))
+ inputs:
+ targetType: inline
+ script: |
+ $os = 'win32'
+ $arch = if ([System.Runtime.InteropServices.RuntimeInformation]::OSArchitecture -eq 'Arm64') { 'arm64' } else { 'x64' }
+ $platformKey = "$os-$arch"
+ $rid = if ($arch -eq 'arm64') { 'win-arm64' } else { 'win-x64' }
+
+ # Detect macOS/Linux
+ if ($IsLinux) {
+ $os = 'linux'
+ $platformKey = "$os-$arch"
+ $rid = "linux-$arch"
+ } elseif ($IsMacOS) {
+ $os = 'darwin'
+ $platformKey = "$os-$arch"
+ $rid = "osx-$arch"
+ }
+
+ $nupkg = Get-ChildItem "${{ parameters.flcNugetDir }}" -Recurse -Filter "Microsoft.AI.Foundry.Local.Core*.nupkg" -Exclude "*.snupkg" | Select-Object -First 1
+ if (-not $nupkg) { throw "No FLC .nupkg found in ${{ parameters.flcNugetDir }}" }
+
+ # Extract the NuGet package (it's a zip)
+ $extractDir = "$(Build.ArtifactStagingDirectory)/flc-extract"
+ $zip = [System.IO.Path]::ChangeExtension($nupkg.FullName, ".zip")
+ Copy-Item $nupkg.FullName $zip -Force
+ Expand-Archive -Path $zip -DestinationPath $extractDir -Force
+
+ # Overwrite FLC binary in the npm-installed location
+ $destDir = "$(repoRoot)/sdk/js/packages/@foundry-local-core/$platformKey"
+ $nativeDir = "$extractDir/runtimes/$rid/native"
+ if (Test-Path $nativeDir) {
+ Get-ChildItem $nativeDir -File | ForEach-Object {
+ Copy-Item $_.FullName -Destination "$destDir/$($_.Name)" -Force
+ Write-Host "Overwrote $($_.Name) with pipeline-built version"
+ }
+ } else {
+ Write-Warning "No native binaries found at $nativeDir for RID $rid"
+ }
+
+ Write-Host "Final binaries in $destDir`:"
+ Get-ChildItem $destDir | ForEach-Object { Write-Host " $($_.Name)" }
+
+- task: Npm@1
+ displayName: 'npm version'
+ inputs:
+ command: custom
+ workingDir: $(repoRoot)/sdk/js
+ customCommand: 'version $(packageVersion) --no-git-tag-version --allow-same-version'
+
+- task: Npm@1
+ displayName: 'npm build'
+ inputs:
+ command: custom
+ workingDir: $(repoRoot)/sdk/js
+ customCommand: 'run build'
+
+- ${{ if eq(parameters.isWinML, true) }}:
+ - task: Npm@1
+ displayName: 'npm run pack:winml'
+ inputs:
+ command: custom
+ workingDir: $(repoRoot)/sdk/js
+ customCommand: 'run pack:winml'
+
+- ${{ else }}:
+ - task: Npm@1
+ displayName: 'npm run pack'
+ inputs:
+ command: custom
+ workingDir: $(repoRoot)/sdk/js
+ customCommand: 'run pack'
+
+- task: PowerShell@2
+ displayName: 'Stage artifact'
+ inputs:
+ targetType: inline
+ script: |
+ $destDir = "$(Build.ArtifactStagingDirectory)/js-sdk"
+ New-Item -ItemType Directory -Path $destDir -Force | Out-Null
+ Copy-Item "$(repoRoot)/sdk/js/*.tgz" "$destDir/"
diff --git a/.pipelines/templates/build-python-steps.yml b/.pipelines/templates/build-python-steps.yml
new file mode 100644
index 00000000..6fd0cd34
--- /dev/null
+++ b/.pipelines/templates/build-python-steps.yml
@@ -0,0 +1,146 @@
+# Steps to build and pack the Python SDK wheel.
+# When test-data-shared is checked out alongside self, ADO places repos under
+# $(Build.SourcesDirectory)/. The self repo is 'Foundry-Local'.
+parameters:
+- name: version
+ type: string
+- name: isRelease
+ type: boolean
+ default: false
+- name: isWinML
+ type: boolean
+ default: false
+- name: flcWheelsDir
+ type: string
+ default: ''
+ displayName: 'Path to directory containing the FLC wheels (for overriding foundry-local-core)'
+- name: outputDir
+ type: string
+ default: '$(Build.ArtifactStagingDirectory)/python-sdk'
+ displayName: 'Path to directory for the built wheel'
+- name: prereleaseId
+ type: string
+ default: ''
+steps:
+# Set paths for multi-repo checkout
+- task: PowerShell@2
+ displayName: 'Set source paths'
+ inputs:
+ targetType: inline
+ script: |
+ $repoRoot = "$(Build.SourcesDirectory)/Foundry-Local"
+ $testDataDir = "$(Build.SourcesDirectory)/test-data-shared"
+ Write-Host "##vso[task.setvariable variable=repoRoot]$repoRoot"
+ Write-Host "##vso[task.setvariable variable=testDataDir]$testDataDir"
+
+- task: UsePythonVersion@0
+ displayName: 'Use Python 3.12'
+ inputs:
+ versionSpec: '3.12'
+
+# List downloaded FLC wheels for debugging
+- task: PowerShell@2
+ displayName: 'List downloaded FLC wheels'
+ condition: and(succeeded(), ne('${{ parameters.flcWheelsDir }}', ''))
+ inputs:
+ targetType: inline
+ script: |
+ Write-Host "Contents of ${{ parameters.flcWheelsDir }}:"
+ Get-ChildItem "${{ parameters.flcWheelsDir }}" -Recurse | ForEach-Object { Write-Host $_.FullName }
+
+# Compute package version
+- task: PowerShell@2
+ displayName: 'Set package version'
+ inputs:
+ targetType: inline
+ script: |
+ $v = "${{ parameters.version }}"
+ $preId = "${{ parameters.prereleaseId }}"
+ if ($preId -ne '' -and $preId -ne 'none') {
+ $v = "$v-$preId"
+ } elseif ("${{ parameters.isRelease }}" -ne "True") {
+ $ts = Get-Date -Format "yyyyMMddHHmm"
+ $v = "$v-dev.$ts"
+ }
+ Write-Host "##vso[task.setvariable variable=packageVersion]$v"
+ Write-Host "Package version: $v"
+
+# Configure pip to use ORT-Nightly feed (plus PyPI as fallback)
+- task: PowerShell@2
+ displayName: 'Configure pip for Azure Artifacts'
+ inputs:
+ targetType: inline
+ script: |
+ pip config set global.index-url https://pkgs.dev.azure.com/aiinfra/PublicPackages/_packaging/ORT-Nightly/pypi/simple/
+ pip config set global.extra-index-url https://pypi.org/simple/
+ pip config set global.pre true
+
+# Install the build tool
+- script: python -m pip install build
+ displayName: 'Install build tool'
+
+# Write version file
+- task: PowerShell@2
+ displayName: 'Set SDK version'
+ inputs:
+ targetType: inline
+ script: |
+ Set-Content -Path "$(repoRoot)/sdk/python/src/version.py" -Value '__version__ = "$(packageVersion)"'
+
+# Install the FLC wheels from the pipeline if provided, so the build
+# backend picks up the freshly-built foundry-local-core instead of
+# pulling a stale one from the feed.
+- task: PowerShell@2
+ displayName: 'Pre-install pipeline-built FLC wheel'
+ condition: and(succeeded(), ne('${{ parameters.flcWheelsDir }}', ''))
+ inputs:
+ targetType: inline
+ script: |
+ # Determine platform wheel tag for the current machine
+ $arch = if ([System.Runtime.InteropServices.RuntimeInformation]::OSArchitecture -eq 'Arm64') { 'arm64' } else { 'amd64' }
+ if ($IsLinux) { $platTag = "manylinux*x86_64" }
+ elseif ($IsMacOS) { $platTag = "macosx*$arch" }
+ else { $platTag = "win_$arch" }
+
+ $filter = if ("${{ parameters.isWinML }}" -eq "True") { "foundry_local_core_winml*$platTag.whl" } else { "foundry_local_core-*$platTag.whl" }
+ $wheel = Get-ChildItem "${{ parameters.flcWheelsDir }}" -Recurse -Filter $filter | Select-Object -First 1
+ if ($wheel) {
+ Write-Host "Installing pipeline-built FLC wheel: $($wheel.FullName)"
+ pip install $($wheel.FullName)
+ } else {
+ Write-Warning "No FLC wheel found matching $filter in ${{ parameters.flcWheelsDir }}"
+ }
+
+# Build wheel — standard or WinML variant
+# skip-native-deps=true omits foundry-local-core/onnxruntime pinned versions
+# from the wheel metadata, since the pipeline pre-installs its own builds.
+- ${{ if eq(parameters.isWinML, true) }}:
+ - script: python -m build --wheel -C winml=true -C skip-native-deps=true --outdir dist/
+ displayName: 'Build wheel (WinML)'
+ workingDirectory: $(repoRoot)/sdk/python
+
+- ${{ else }}:
+ - script: python -m build --wheel -C skip-native-deps=true --outdir dist/
+ displayName: 'Build wheel'
+ workingDirectory: $(repoRoot)/sdk/python
+
+# Install the built wheel
+- task: PowerShell@2
+ displayName: 'Install built wheel'
+ inputs:
+ targetType: inline
+ script: |
+ $wheel = (Get-ChildItem "$(repoRoot)/sdk/python/dist/*.whl" | Select-Object -First 1).FullName
+ pip install $wheel
+
+# Stage output
+- task: PowerShell@2
+ displayName: 'Stage wheel artifact'
+ inputs:
+ targetType: inline
+ script: |
+ $destDir = "${{ parameters.outputDir }}"
+ New-Item -ItemType Directory -Path $destDir -Force | Out-Null
+ Copy-Item "$(repoRoot)/sdk/python/dist/*" "$destDir/"
+ Write-Host "Staged wheels:"
+ Get-ChildItem $destDir | ForEach-Object { Write-Host " $($_.Name)" }
diff --git a/.pipelines/templates/build-rust-steps.yml b/.pipelines/templates/build-rust-steps.yml
new file mode 100644
index 00000000..efccfaa4
--- /dev/null
+++ b/.pipelines/templates/build-rust-steps.yml
@@ -0,0 +1,207 @@
+# Steps to build and package the Rust SDK crate.
+# When test-data-shared is checked out alongside self, ADO places repos under
+# $(Build.SourcesDirectory)/. The self repo is 'Foundry-Local'.
+parameters:
+- name: version
+ type: string
+- name: isRelease
+ type: boolean
+ default: false
+- name: prereleaseId
+ type: string
+ default: ''
+- name: isWinML
+ type: boolean
+ default: false
+- name: flcNugetDir
+ type: string
+ displayName: 'Path to directory containing the FLC .nupkg'
+- name: outputDir
+ type: string
+ default: '$(Build.ArtifactStagingDirectory)/rust-sdk'
+ displayName: 'Path to directory for the packaged crate'
+steps:
+# Set paths for multi-repo checkout
+- task: PowerShell@2
+ displayName: 'Set source paths'
+ inputs:
+ targetType: inline
+ script: |
+ $repoRoot = "$(Build.SourcesDirectory)/Foundry-Local"
+ $testDataDir = "$(Build.SourcesDirectory)/test-data-shared"
+ Write-Host "##vso[task.setvariable variable=repoRoot]$repoRoot"
+ Write-Host "##vso[task.setvariable variable=testDataDir]$testDataDir"
+
+# Compute package version and patch Cargo.toml
+- task: PowerShell@2
+ displayName: 'Set crate version'
+ inputs:
+ targetType: inline
+ script: |
+ $v = "${{ parameters.version }}"
+ $preId = "${{ parameters.prereleaseId }}"
+ if ($preId -ne '' -and $preId -ne 'none') {
+ $v = "$v-$preId"
+ } elseif ("${{ parameters.isRelease }}" -ne "True") {
+ $ts = Get-Date -Format "yyyyMMddHHmm"
+ $v = "$v-dev.$ts"
+ }
+ Write-Host "Crate version: $v"
+
+ # Patch Cargo.toml version field
+ $cargoPath = "$(repoRoot)/sdk/rust/Cargo.toml"
+ $content = Get-Content $cargoPath -Raw
+ $content = $content -replace '(?m)^version\s*=\s*"[^"]+"', "version = `"$v`""
+ Set-Content -Path $cargoPath -Value $content
+ Write-Host "Patched Cargo.toml with version $v"
+
+# List downloaded FLC artifact for debugging
+- task: PowerShell@2
+ displayName: 'List downloaded FLC artifact'
+ inputs:
+ targetType: inline
+ script: |
+ Write-Host "Contents of ${{ parameters.flcNugetDir }}:"
+ Get-ChildItem "${{ parameters.flcNugetDir }}" -Recurse | ForEach-Object { Write-Host $_.FullName }
+
+# Extract FLC native binaries from the pipeline-built .nupkg so that
+# build.rs finds them already present and skips downloading from the feed.
+- task: PowerShell@2
+ displayName: 'Extract FLC native binaries for Rust build'
+ inputs:
+ targetType: inline
+ script: |
+ $nupkg = Get-ChildItem "${{ parameters.flcNugetDir }}" -Recurse -Filter "Microsoft.AI.Foundry.Local.Core*.nupkg" -Exclude "*.snupkg" | Select-Object -First 1
+ if (-not $nupkg) { throw "No FLC .nupkg found in ${{ parameters.flcNugetDir }}" }
+ Write-Host "Found NuGet package: $($nupkg.FullName)"
+
+ $extractDir = "$(Build.ArtifactStagingDirectory)/flc-extract-rust"
+ $zip = [System.IO.Path]::ChangeExtension($nupkg.FullName, ".zip")
+ Copy-Item $nupkg.FullName $zip -Force
+ Expand-Archive -Path $zip -DestinationPath $extractDir -Force
+
+ # Determine RID for this agent
+ $arch = if ([System.Runtime.InteropServices.RuntimeInformation]::OSArchitecture -eq 'Arm64') { 'arm64' } else { 'x64' }
+ if ($IsLinux) {
+ $rid = "linux-$arch"
+ } elseif ($IsMacOS) {
+ $rid = "osx-$arch"
+ } else {
+ $rid = "win-$arch"
+ }
+
+ $nativeDir = "$extractDir/runtimes/$rid/native"
+ if (-not (Test-Path $nativeDir)) { throw "No native binaries found at $nativeDir for RID $rid" }
+
+ # Stage them where build.rs can discover them
+ $flcNativeDir = "$(Build.ArtifactStagingDirectory)/flc-native-rust"
+ New-Item -ItemType Directory -Path $flcNativeDir -Force | Out-Null
+ Get-ChildItem $nativeDir -File | Copy-Item -Destination $flcNativeDir -Force
+ Write-Host "##vso[task.setvariable variable=flcNativeDir]$flcNativeDir"
+ Write-Host "Extracted FLC native binaries to $flcNativeDir`:"
+ Get-ChildItem $flcNativeDir | ForEach-Object { Write-Host " $($_.Name)" }
+
+# Install Rust toolchain
+- task: PowerShell@2
+ displayName: 'Install Rust toolchain'
+ inputs:
+ targetType: inline
+ script: |
+ if ($IsWindows -or (-not $IsLinux -and -not $IsMacOS)) {
+ Invoke-WebRequest -Uri https://win.rustup.rs/x86_64 -OutFile rustup-init.exe
+ .\rustup-init.exe -y --default-toolchain stable --profile minimal -c clippy,rustfmt
+ Remove-Item rustup-init.exe
+ $cargoPath = "$env:USERPROFILE\.cargo\bin"
+ } else {
+ curl --proto '=https' --tlsv1.2 -sSf https://sh.rustup.rs | sh -s -- -y --default-toolchain stable --profile minimal -c clippy,rustfmt
+ $cargoPath = "$env:HOME/.cargo/bin"
+ }
+ Write-Host "##vso[task.prependpath]$cargoPath"
+
+# The .cargo/config.toml redirects crates-io to an Azure Artifacts feed
+# for CFS compliance. Remove the redirect in CI so cargo can fetch from
+# crates.io directly without Azure DevOps auth.
+- task: PowerShell@2
+ displayName: 'Use crates.io directly'
+ inputs:
+ targetType: inline
+ script: |
+ $configPath = "$(repoRoot)/sdk/rust/.cargo/config.toml"
+ if (Test-Path $configPath) {
+ Remove-Item $configPath
+ Write-Host "Removed .cargo/config.toml crates-io redirect"
+ }
+
+- task: PowerShell@2
+ displayName: 'Check formatting'
+ inputs:
+ targetType: inline
+ script: |
+ Set-Location "$(repoRoot)/sdk/rust"
+ cargo fmt --all -- --check
+ if ($LASTEXITCODE -ne 0) { exit $LASTEXITCODE }
+
+- task: PowerShell@2
+ displayName: 'Run clippy'
+ inputs:
+ targetType: inline
+ script: |
+ Set-Location "$(repoRoot)/sdk/rust"
+ $features = if ("${{ parameters.isWinML }}" -eq "True") { "--features winml" } else { "" }
+ Invoke-Expression "cargo clippy --all-targets $features -- -D warnings"
+ if ($LASTEXITCODE -ne 0) { exit $LASTEXITCODE }
+
+- task: PowerShell@2
+ displayName: 'Build'
+ inputs:
+ targetType: inline
+ script: |
+ Set-Location "$(repoRoot)/sdk/rust"
+ $features = if ("${{ parameters.isWinML }}" -eq "True") { "--features winml" } else { "" }
+ Invoke-Expression "cargo build $features"
+ if ($LASTEXITCODE -ne 0) { exit $LASTEXITCODE }
+
+# Overwrite the FLC core binary in cargo's OUT_DIR with the pipeline-built
+# version so that integration tests use the freshly-built FLC. build.rs
+# sets FOUNDRY_NATIVE_DIR to OUT_DIR, which the SDK checks at runtime.
+- task: PowerShell@2
+ displayName: 'Overwrite FLC binary with pipeline-built version'
+ inputs:
+ targetType: inline
+ script: |
+ # Find cargo's OUT_DIR for the foundry-local-sdk build script
+ $outDir = Get-ChildItem "$(repoRoot)/sdk/rust/target/debug/build" -Directory -Filter "foundry-local-sdk-*" -Recurse |
+ Where-Object { Test-Path "$($_.FullName)/out" } |
+ ForEach-Object { "$($_.FullName)/out" } |
+ Select-Object -First 1
+ if (-not $outDir) { throw "Could not find cargo OUT_DIR for foundry-local-sdk" }
+ Write-Host "Cargo OUT_DIR: $outDir"
+
+ # Copy pipeline-built FLC native binaries over the downloaded ones
+ Get-ChildItem "$(flcNativeDir)" -File -Filter "Microsoft.AI.Foundry.Local.Core.*" | ForEach-Object {
+ Copy-Item $_.FullName -Destination "$outDir/$($_.Name)" -Force
+ Write-Host "Overwrote $($_.Name) with pipeline-built version"
+ }
+
+# --allow-dirty allows packaging with uncommitted changes (build.rs modifies generated files)
+- task: PowerShell@2
+ displayName: 'Package crate'
+ inputs:
+ targetType: inline
+ script: |
+ Set-Location "$(repoRoot)/sdk/rust"
+ $features = if ("${{ parameters.isWinML }}" -eq "True") { "--features winml" } else { "" }
+ Invoke-Expression "cargo package $features --allow-dirty"
+ if ($LASTEXITCODE -ne 0) { exit $LASTEXITCODE }
+
+# Stage output
+- task: PowerShell@2
+ displayName: 'Stage crate artifact'
+ inputs:
+ targetType: inline
+ script: |
+ $destDir = "${{ parameters.outputDir }}"
+ New-Item -ItemType Directory -Path $destDir -Force | Out-Null
+ Copy-Item "$(repoRoot)/sdk/rust/target/package/*.crate" "$destDir/"
+ Write-Host "Staged crates:"
+ Get-ChildItem $destDir | ForEach-Object { Write-Host " $($_.Name)" }
diff --git a/.pipelines/templates/package-core-steps.yml b/.pipelines/templates/package-core-steps.yml
new file mode 100644
index 00000000..e5755a21
--- /dev/null
+++ b/.pipelines/templates/package-core-steps.yml
@@ -0,0 +1,256 @@
+# Steps to collect per-platform FLC native binaries, organize into NuGet layout,
+# pack + sign the NuGet package, and build Python wheels (wheel package name and
+# platforms depend on the isWinML parameter). The parent job must download all
+# platform artifacts and checkout neutron-server.
+parameters:
+- name: version
+ type: string
+- name: isRelease
+ type: boolean
+ default: false
+- name: isWinML
+ type: boolean
+ default: false
+- name: prereleaseId
+ type: string
+ default: ''
+- name: platforms
+ type: object # list of { name, artifactName }
+
+steps:
+- task: PowerShell@2
+ displayName: 'Set source paths'
+ inputs:
+ targetType: inline
+ script: |
+ $nsRoot = "$(Build.SourcesDirectory)"
+ Write-Host "##vso[task.setvariable variable=nsRoot]$nsRoot"
+
+- task: PowerShell@2
+ displayName: 'Organize native binaries'
+ inputs:
+ targetType: inline
+ script: |
+ $unifiedPath = "$(Build.ArtifactStagingDirectory)/unified"
+ New-Item -ItemType Directory -Path $unifiedPath -Force | Out-Null
+
+ $platformsJson = @'
+ ${{ convertToJson(parameters.platforms) }}
+ '@
+ $platforms = $platformsJson | ConvertFrom-Json
+
+ foreach ($p in $platforms) {
+ $srcDir = "$(Pipeline.Workspace)/$($p.artifactName)"
+ Write-Host "Looking for artifacts at: $srcDir"
+ if (-not (Test-Path $srcDir)) {
+ throw "Artifact directory $srcDir does not exist. All platform artifacts must be present to produce a complete NuGet package."
+ }
+ $destDir = "$unifiedPath/runtimes/$($p.name)/native"
+ New-Item -ItemType Directory -Path $destDir -Force | Out-Null
+ # WinML artifacts include WindowsAppRuntime Bootstrapper DLLs in addition
+ # to Microsoft.AI.Foundry.Local.Core.*.
+ $isWinML = "${{ parameters.isWinML }}" -eq "True"
+ if ($isWinML) {
+ Get-ChildItem $srcDir -File |
+ Where-Object { $_.Name -like "Microsoft.AI.Foundry.Local.Core.*" -or $_.Name -eq "Microsoft.WindowsAppRuntime.Bootstrap.dll" } |
+ Copy-Item -Destination $destDir -Force
+ } else {
+ Get-ChildItem $srcDir -File | Where-Object { $_.Name -like "Microsoft.AI.Foundry.Local.Core.*" } |
+ Copy-Item -Destination $destDir -Force
+ }
+ Write-Host "Copied $($p.name) binaries to $destDir"
+ }
+
+ # Copy build integration files from neutron-server
+ $nsRoot = "$(nsRoot)"
+ foreach ($dir in @("build", "buildTransitive")) {
+ $src = "$nsRoot/src/FoundryLocalCore/Core/$dir"
+ if (Test-Path $src) {
+ Copy-Item -Path $src -Destination "$unifiedPath/$dir" -Recurse -Force
+ }
+ }
+ $license = "$nsRoot/src/FoundryLocalCore/Core/LICENSE.txt"
+ if (Test-Path $license) {
+ Copy-Item $license "$unifiedPath/LICENSE.txt" -Force
+ }
+
+# Compute version
+- task: PowerShell@2
+ displayName: 'Set FLC package version'
+ inputs:
+ targetType: inline
+ script: |
+ $v = "${{ parameters.version }}"
+ $preId = "${{ parameters.prereleaseId }}"
+ if ($preId -ne '' -and $preId -ne 'none') {
+ $v = "$v-$preId"
+ } elseif ("${{ parameters.isRelease }}" -ne "True") {
+ $ts = Get-Date -Format "yyyyMMddHHmm"
+ $commitId = "$(Build.SourceVersion)".Substring(0, 8)
+ $v = "$v-dev-$ts-$commitId"
+ }
+ Write-Host "##vso[task.setvariable variable=flcVersion]$v"
+ Write-Host "FLC version: $v"
+
+# Pack NuGet
+- task: PowerShell@2
+ displayName: 'Pack FLC NuGet'
+ inputs:
+ targetType: inline
+ script: |
+ $nsRoot = "$(nsRoot)"
+ [xml]$propsXml = Get-Content "$nsRoot/Directory.Packages.props"
+ $pg = $propsXml.Project.PropertyGroup
+
+ $outDir = "$(Build.ArtifactStagingDirectory)/flc-nuget"
+ New-Item -ItemType Directory -Path $outDir -Force | Out-Null
+
+ if ("${{ parameters.isWinML }}" -eq "True") {
+ $nuspec = "$nsRoot/src/FoundryLocalCore/Core/WinMLNuget.nuspec"
+ $id = "Microsoft.AI.Foundry.Local.Core.WinML"
+ $ortVer = $pg.OnnxRuntimeFoundryVersionForWinML
+ $genaiVer = $pg.OnnxRuntimeGenAIWinML
+ $winAppSdkVer = $pg.WinAppSdkVersion
+ $props = "id=$id;version=$(flcVersion);commitId=$(Build.SourceVersion);OnnxRuntimeFoundryVersion=$ortVer;OnnxRuntimeGenAIWinML=$genaiVer;WinAppSdkVersion=$winAppSdkVer"
+ } else {
+ $nuspec = "$nsRoot/src/FoundryLocalCore/Core/NativeNuget.nuspec"
+ $id = "Microsoft.AI.Foundry.Local.Core"
+ $ortVer = $pg.OnnxRuntimeFoundryVersion
+ $genaiVer = $pg.OnnxRuntimeGenAIFoundryVersion
+ $props = "id=$id;version=$(flcVersion);commitId=$(Build.SourceVersion);OnnxRuntimeFoundryVersion=$ortVer;OnnxRuntimeGenAIFoundryVersion=$genaiVer"
+ }
+
+ $nugetArgs = @(
+ 'pack', $nuspec,
+ '-OutputDirectory', $outDir,
+ '-BasePath', "$(Build.ArtifactStagingDirectory)/unified",
+ '-Properties', $props,
+ '-Symbols', '-SymbolPackageFormat', 'snupkg'
+ )
+ Write-Host "Running: nuget $($nugetArgs -join ' ')"
+ & nuget $nugetArgs
+ if ($LASTEXITCODE -ne 0) { throw "NuGet pack failed" }
+
+# Sign NuGet package
+- task: SFP.build-tasks.custom-build-task-1.EsrpCodeSigning@5
+ displayName: 'Sign FLC NuGet package'
+ inputs:
+ ConnectedServiceName: 'OnnxrunTimeCodeSign_20240611'
+ UseMSIAuthentication: true
+ AppRegistrationClientId: '$(esrpClientId)'
+ AppRegistrationTenantId: '$(esrpTenantId)'
+ EsrpClientId: '$(esrpClientId)'
+ AuthAKVName: '$(esrpAkvName)'
+ AuthSignCertName: '$(esrpSignCertName)'
+ FolderPath: '$(Build.ArtifactStagingDirectory)/flc-nuget'
+ Pattern: '*.nupkg'
+ SessionTimeout: 90
+ ServiceEndpointUrl: 'https://api.esrp.microsoft.com/api/v2'
+ MaxConcurrency: 25
+ signConfigType: inlineSignParams
+ inlineOperation: |
+ [{"keyCode":"CP-401405","operationSetCode":"NuGetSign","parameters":[],"toolName":"sign","toolVersion":"6.2.9304.0"},{"keyCode":"CP-401405","operationSetCode":"NuGetVerify","parameters":[],"toolName":"sign","toolVersion":"6.2.9304.0"}]
+
+# Build Python wheels from the NuGet package
+- task: PowerShell@2
+ displayName: 'Build foundry_local_core Python Wheels'
+ inputs:
+ targetType: inline
+ script: |
+ $stagingDir = "$(Build.ArtifactStagingDirectory)/flc-wheels"
+ New-Item -ItemType Directory -Path $stagingDir -Force | Out-Null
+
+ $isWinML = "${{ parameters.isWinML }}" -eq "True"
+
+ # Find and extract the NuGet package (.nupkg is a zip archive)
+ $nupkgFilter = if ($isWinML) { "Microsoft.AI.Foundry.Local.Core.WinML*.nupkg" } else { "Microsoft.AI.Foundry.Local.Core*.nupkg" }
+ $nupkg = Get-ChildItem "$(Build.ArtifactStagingDirectory)/flc-nuget" -Filter $nupkgFilter | Where-Object { $_.Name -notlike "*.snupkg" } | Select-Object -First 1
+ if (-not $nupkg) { throw "No FLC .nupkg found matching $nupkgFilter" }
+ Write-Host "Found NuGet package: $($nupkg.Name)"
+
+ $extractDir = "$(Build.ArtifactStagingDirectory)/flc-extracted"
+ $nupkgZip = [System.IO.Path]::ChangeExtension($nupkg.FullName, ".zip")
+ Copy-Item -Path $nupkg.FullName -Destination $nupkgZip -Force
+ Expand-Archive -Path $nupkgZip -DestinationPath $extractDir -Force
+
+ # Convert NuGet version to PEP 440
+ # NuGet: 0.9.0-dev-202603271723-bb400310 → PEP 440: 0.9.0.dev202603271723
+ # The commit hash is dropped because .devN requires N to be a pure integer.
+ $nupkgVersion = $nupkg.BaseName -replace '^Microsoft\.AI\.Foundry\.Local\.Core(\.WinML)?\.', ''
+ $parts = $nupkgVersion -split '-'
+ $pyVersion = if ($parts.Count -ge 3 -and $parts[1] -eq 'dev') { "$($parts[0]).dev$($parts[2])" }
+ elseif ($parts.Count -eq 2) { "$($parts[0])$($parts[1])" }
+ else { $parts[0] }
+ Write-Host "Python package version: $pyVersion"
+
+ $packageName = if ($isWinML) { "foundry_local_core_winml" } else { "foundry_local_core" }
+
+ if ($isWinML) {
+ $platforms = @(
+ @{rid="win-x64"; pyKey="bin"; tag="win_amd64"},
+ @{rid="win-arm64"; pyKey="bin"; tag="win_arm64"}
+ )
+ } else {
+ $platforms = @(
+ @{rid="win-x64"; pyKey="bin"; tag="win_amd64"},
+ @{rid="win-arm64"; pyKey="bin"; tag="win_arm64"},
+ @{rid="linux-x64"; pyKey="bin"; tag="manylinux_2_28_x86_64"},
+ @{rid="osx-arm64"; pyKey="bin"; tag="macosx_11_0_arm64"}
+ )
+ }
+
+ foreach ($p in $platforms) {
+ $nativeSrc = "$extractDir/runtimes/$($p.rid)/native"
+ if (-not (Test-Path $nativeSrc)) {
+ Write-Warning "No native binaries found for $($p.rid) — skipping."
+ continue
+ }
+
+ $wheelRoot = "$(Build.ArtifactStagingDirectory)/wheels-build/flc_wheel_$($p.tag)"
+ $pkgDir = "$wheelRoot/$packageName"
+ New-Item -ItemType Directory -Path "$pkgDir/$($p.pyKey)" -Force | Out-Null
+ "" | Set-Content -Encoding ascii "$pkgDir/__init__.py"
+ Get-ChildItem $nativeSrc -File | Copy-Item -Destination "$pkgDir/$($p.pyKey)"
+
+ $normalizedName = $packageName.Replace('_', '-')
+ $wheelTag = "py3-none-$($p.tag)"
+ $distInfoName = "$packageName-$pyVersion"
+ $wheelName = "$distInfoName-$wheelTag.whl"
+ $distInfoDir = "$wheelRoot/$distInfoName.dist-info"
+ New-Item -ItemType Directory -Path $distInfoDir -Force | Out-Null
+
+ $utf8NoBom = [System.Text.UTF8Encoding]::new($false)
+
+ [System.IO.File]::WriteAllText("$distInfoDir/WHEEL",
+ "Wheel-Version: 1.0`nGenerator: custom`nRoot-Is-Purelib: false`nTag: $wheelTag`n", $utf8NoBom)
+
+ [System.IO.File]::WriteAllText("$distInfoDir/METADATA",
+ "Metadata-Version: 2.1`nName: $normalizedName`nVersion: $pyVersion`n", $utf8NoBom)
+
+ $recordLines = Get-ChildItem $wheelRoot -Recurse -File | ForEach-Object {
+ $rel = $_.FullName.Substring($wheelRoot.Length + 1).Replace('\', '/')
+ $raw = (Get-FileHash $_.FullName -Algorithm SHA256).Hash
+ $bytes = [byte[]]::new($raw.Length / 2)
+ for ($i = 0; $i -lt $raw.Length; $i += 2) { $bytes[$i/2] = [Convert]::ToByte($raw.Substring($i, 2), 16) }
+ $b64 = [Convert]::ToBase64String($bytes) -replace '\+','-' -replace '/','_' -replace '=',''
+ "$rel,sha256=$b64,$($_.Length)"
+ }
+ $recordContent = ($recordLines + "$distInfoName.dist-info/RECORD,,") -join "`n"
+ [System.IO.File]::WriteAllText("$distInfoDir/RECORD", $recordContent, $utf8NoBom)
+
+ $wheelPath = "$stagingDir/$wheelName"
+ Add-Type -AssemblyName System.IO.Compression.FileSystem
+ $zip = [System.IO.Compression.ZipFile]::Open($wheelPath, 'Create')
+ try {
+ Get-ChildItem $wheelRoot -Recurse -File | ForEach-Object {
+ $rel = $_.FullName.Substring($wheelRoot.Length + 1).Replace('\', '/')
+ [System.IO.Compression.ZipFileExtensions]::CreateEntryFromFile($zip, $_.FullName, $rel) | Out-Null
+ }
+ } finally {
+ $zip.Dispose()
+ }
+ Write-Host "Created wheel: $wheelName"
+ }
+
+ Write-Host "`nAll wheels:"
+ Get-ChildItem $stagingDir -Filter "*.whl" | ForEach-Object { Write-Host " $($_.Name)" }
diff --git a/.pipelines/templates/test-cs-steps.yml b/.pipelines/templates/test-cs-steps.yml
new file mode 100644
index 00000000..f7dc1aff
--- /dev/null
+++ b/.pipelines/templates/test-cs-steps.yml
@@ -0,0 +1,116 @@
+# Lightweight test-only steps for the C# SDK.
+# Builds from source and runs tests — no signing or NuGet packing.
+parameters:
+- name: version
+ type: string
+- name: isWinML
+ type: boolean
+ default: false
+- name: flcNugetDir
+ type: string
+ displayName: 'Path to directory containing the FLC .nupkg'
+
+steps:
+- task: PowerShell@2
+ displayName: 'Set source paths'
+ inputs:
+ targetType: inline
+ script: |
+ $repoRoot = "$(Build.SourcesDirectory)/Foundry-Local"
+ $testDataDir = "$(Build.SourcesDirectory)/test-data-shared"
+ Write-Host "##vso[task.setvariable variable=repoRoot]$repoRoot"
+ Write-Host "##vso[task.setvariable variable=testDataDir]$testDataDir"
+
+- task: UseDotNet@2
+ displayName: 'Use .NET 9 SDK'
+ inputs:
+ packageType: sdk
+ version: '9.0.x'
+
+- task: PowerShell@2
+ displayName: 'List downloaded FLC artifact'
+ inputs:
+ targetType: inline
+ script: |
+ Write-Host "Contents of ${{ parameters.flcNugetDir }}:"
+ Get-ChildItem "${{ parameters.flcNugetDir }}" -Recurse | ForEach-Object { Write-Host $_.FullName }
+
+- ${{ if eq(parameters.isWinML, true) }}:
+ - task: PowerShell@2
+ displayName: 'Install Windows App SDK Runtime'
+ inputs:
+ targetType: 'inline'
+ script: |
+ $installerUrl = "https://aka.ms/windowsappsdk/1.8/latest/windowsappruntimeinstall-x64.exe"
+ $installerPath = "$env:TEMP\windowsappruntimeinstall.exe"
+
+ Write-Host "Downloading Windows App SDK Runtime installer from $installerUrl..."
+ Invoke-WebRequest -Uri $installerUrl -OutFile $installerPath
+
+ Write-Host "Installing Windows App SDK Runtime..."
+ & $installerPath --quiet --force
+
+ if ($LASTEXITCODE -ne 0) {
+ Write-Error "Installation failed with exit code $LASTEXITCODE"
+ exit 1
+ }
+
+ Write-Host "Windows App SDK Runtime installed successfully."
+ errorActionPreference: 'stop'
+
+- task: PowerShell@2
+ displayName: 'Create NuGet.config with local FLC feed'
+ inputs:
+ targetType: inline
+ script: |
+ $nugetConfig = @"
+
+
+
+
+
+
+
+
+ "@
+ $nupkg = Get-ChildItem "${{ parameters.flcNugetDir }}" -Recurse -Filter "Microsoft.AI.Foundry.Local.Core*.nupkg" -Exclude "*.snupkg" | Select-Object -First 1
+ if (-not $nupkg) { throw "No FLC .nupkg found in ${{ parameters.flcNugetDir }}" }
+ $flcVer = $nupkg.BaseName -replace '^Microsoft\.AI\.Foundry\.Local\.Core(\.WinML)?\.', ''
+ Write-Host "##vso[task.setvariable variable=resolvedFlcVersion]$flcVer"
+
+ $flcFeedDir = $nupkg.DirectoryName
+ $nugetConfig = $nugetConfig -replace [regex]::Escape("${{ parameters.flcNugetDir }}"), $flcFeedDir
+ $configPath = "$(Build.ArtifactStagingDirectory)/NuGet.config"
+ Set-Content -Path $configPath -Value $nugetConfig
+ Write-Host "##vso[task.setvariable variable=customNugetConfig]$configPath"
+
+- task: NuGetAuthenticate@1
+ displayName: 'Authenticate NuGet feeds'
+
+- task: PowerShell@2
+ displayName: 'Restore & build tests'
+ inputs:
+ targetType: inline
+ script: |
+ dotnet restore "$(repoRoot)/sdk/cs/test/FoundryLocal.Tests/Microsoft.AI.Foundry.Local.Tests.csproj" `
+ --configfile "$(customNugetConfig)" `
+ /p:UseWinML=${{ parameters.isWinML }} `
+ /p:FoundryLocalCoreVersion=$(resolvedFlcVersion)
+ if ($LASTEXITCODE -ne 0) { exit $LASTEXITCODE }
+
+ dotnet build "$(repoRoot)/sdk/cs/test/FoundryLocal.Tests/Microsoft.AI.Foundry.Local.Tests.csproj" `
+ --no-restore --configuration Release `
+ /p:UseWinML=${{ parameters.isWinML }}
+ if ($LASTEXITCODE -ne 0) { exit $LASTEXITCODE }
+
+- task: PowerShell@2
+ displayName: 'Run SDK tests'
+ inputs:
+ targetType: inline
+ script: |
+ dotnet test "$(repoRoot)/sdk/cs/test/FoundryLocal.Tests/Microsoft.AI.Foundry.Local.Tests.csproj" `
+ --no-build --configuration Release `
+ /p:UseWinML=${{ parameters.isWinML }}
+ if ($LASTEXITCODE -ne 0) { exit $LASTEXITCODE }
+ env:
+ TF_BUILD: 'true'
diff --git a/.pipelines/templates/test-js-steps.yml b/.pipelines/templates/test-js-steps.yml
new file mode 100644
index 00000000..41ef7f62
--- /dev/null
+++ b/.pipelines/templates/test-js-steps.yml
@@ -0,0 +1,121 @@
+# Lightweight test-only steps for the JS SDK.
+# Builds from source and runs tests — no npm pack or artifact staging.
+parameters:
+- name: version
+ type: string
+- name: isWinML
+ type: boolean
+ default: false
+- name: flcNugetDir
+ type: string
+ displayName: 'Path to directory containing the FLC .nupkg'
+
+steps:
+- task: PowerShell@2
+ displayName: 'Set source paths'
+ inputs:
+ targetType: inline
+ script: |
+ $repoRoot = "$(Build.SourcesDirectory)/Foundry-Local"
+ $testDataDir = "$(Build.SourcesDirectory)/test-data-shared"
+ Write-Host "##vso[task.setvariable variable=repoRoot]$repoRoot"
+ Write-Host "##vso[task.setvariable variable=testDataDir]$testDataDir"
+
+- ${{ if eq(parameters.isWinML, true) }}:
+ - task: PowerShell@2
+ displayName: 'Install Windows App SDK Runtime'
+ inputs:
+ targetType: 'inline'
+ script: |
+ $installerUrl = "https://aka.ms/windowsappsdk/1.8/latest/windowsappruntimeinstall-x64.exe"
+ $installerPath = "$env:TEMP\windowsappruntimeinstall.exe"
+
+ Write-Host "Downloading Windows App SDK Runtime installer from $installerUrl..."
+ Invoke-WebRequest -Uri $installerUrl -OutFile $installerPath
+
+ Write-Host "Installing Windows App SDK Runtime..."
+ & $installerPath --quiet --force
+
+ if ($LASTEXITCODE -ne 0) {
+ Write-Error "Installation failed with exit code $LASTEXITCODE"
+ exit 1
+ }
+
+ Write-Host "Windows App SDK Runtime installed successfully."
+ errorActionPreference: 'stop'
+
+- task: PowerShell@2
+ displayName: 'List downloaded FLC artifact'
+ inputs:
+ targetType: inline
+ script: |
+ Write-Host "Contents of ${{ parameters.flcNugetDir }}:"
+ Get-ChildItem "${{ parameters.flcNugetDir }}" -Recurse | ForEach-Object { Write-Host $_.FullName }
+
+- task: NodeTool@0
+ displayName: 'Use Node.js 20'
+ inputs:
+ versionSpec: '20.x'
+
+- task: Npm@1
+ displayName: 'npm install'
+ inputs:
+ command: custom
+ workingDir: $(repoRoot)/sdk/js
+ customCommand: 'install'
+
+# Overwrite the FLC native binary with the pipeline-built one
+- task: PowerShell@2
+ displayName: 'Overwrite FLC with pipeline-built binary'
+ inputs:
+ targetType: inline
+ script: |
+ $os = 'win32'
+ $arch = if ([System.Runtime.InteropServices.RuntimeInformation]::OSArchitecture -eq 'Arm64') { 'arm64' } else { 'x64' }
+ $platformKey = "$os-$arch"
+ $rid = if ($arch -eq 'arm64') { 'win-arm64' } else { 'win-x64' }
+
+ if ($IsLinux) {
+ $os = 'linux'
+ $platformKey = "$os-$arch"
+ $rid = "linux-$arch"
+ } elseif ($IsMacOS) {
+ $os = 'darwin'
+ $platformKey = "$os-$arch"
+ $rid = "osx-$arch"
+ }
+
+ $nupkg = Get-ChildItem "${{ parameters.flcNugetDir }}" -Recurse -Filter "Microsoft.AI.Foundry.Local.Core*.nupkg" -Exclude "*.snupkg" | Select-Object -First 1
+ if (-not $nupkg) { throw "No FLC .nupkg found in ${{ parameters.flcNugetDir }}" }
+
+ $extractDir = "$(Build.ArtifactStagingDirectory)/flc-extract"
+ $zip = [System.IO.Path]::ChangeExtension($nupkg.FullName, ".zip")
+ Copy-Item $nupkg.FullName $zip -Force
+ Expand-Archive -Path $zip -DestinationPath $extractDir -Force
+
+ $destDir = "$(repoRoot)/sdk/js/packages/@foundry-local-core/$platformKey"
+ $nativeDir = "$extractDir/runtimes/$rid/native"
+ if (Test-Path $nativeDir) {
+ Get-ChildItem $nativeDir -File | ForEach-Object {
+ Copy-Item $_.FullName -Destination "$destDir/$($_.Name)" -Force
+ Write-Host "Overwrote $($_.Name) with pipeline-built version"
+ }
+ } else {
+ Write-Warning "No native binaries found at $nativeDir for RID $rid"
+ }
+
+- task: Npm@1
+ displayName: 'npm build'
+ inputs:
+ command: custom
+ workingDir: $(repoRoot)/sdk/js
+ customCommand: 'run build'
+
+- task: Npm@1
+ displayName: 'npm test'
+ inputs:
+ command: custom
+ workingDir: $(repoRoot)/sdk/js
+ customCommand: 'test'
+ env:
+ TF_BUILD: 'true'
diff --git a/.pipelines/templates/test-python-steps.yml b/.pipelines/templates/test-python-steps.yml
new file mode 100644
index 00000000..f54a9464
--- /dev/null
+++ b/.pipelines/templates/test-python-steps.yml
@@ -0,0 +1,133 @@
+# Lightweight test-only steps for the Python SDK.
+# Builds from source and runs tests — no artifact staging.
+parameters:
+- name: version
+ type: string
+- name: isWinML
+ type: boolean
+ default: false
+- name: flcWheelsDir
+ type: string
+ default: ''
+ displayName: 'Path to directory containing the FLC wheels'
+
+steps:
+- task: PowerShell@2
+ displayName: 'Set source paths'
+ inputs:
+ targetType: inline
+ script: |
+ $repoRoot = "$(Build.SourcesDirectory)/Foundry-Local"
+ $testDataDir = "$(Build.SourcesDirectory)/test-data-shared"
+ Write-Host "##vso[task.setvariable variable=repoRoot]$repoRoot"
+ Write-Host "##vso[task.setvariable variable=testDataDir]$testDataDir"
+
+- ${{ if eq(parameters.isWinML, true) }}:
+ - task: PowerShell@2
+ displayName: 'Install Windows App SDK Runtime'
+ inputs:
+ targetType: 'inline'
+ script: |
+ $installerUrl = "https://aka.ms/windowsappsdk/1.8/latest/windowsappruntimeinstall-x64.exe"
+ $installerPath = "$env:TEMP\windowsappruntimeinstall.exe"
+
+ Write-Host "Downloading Windows App SDK Runtime installer from $installerUrl..."
+ Invoke-WebRequest -Uri $installerUrl -OutFile $installerPath
+
+ Write-Host "Installing Windows App SDK Runtime..."
+ & $installerPath --quiet --force
+
+ if ($LASTEXITCODE -ne 0) {
+ Write-Error "Installation failed with exit code $LASTEXITCODE"
+ exit 1
+ }
+
+ Write-Host "Windows App SDK Runtime installed successfully."
+ errorActionPreference: 'stop'
+
+- task: UsePythonVersion@0
+ displayName: 'Use Python 3.12'
+ inputs:
+ versionSpec: '3.12'
+
+- task: PowerShell@2
+ displayName: 'List downloaded FLC wheels'
+ condition: and(succeeded(), ne('${{ parameters.flcWheelsDir }}', ''))
+ inputs:
+ targetType: inline
+ script: |
+ Write-Host "Contents of ${{ parameters.flcWheelsDir }}:"
+ Get-ChildItem "${{ parameters.flcWheelsDir }}" -Recurse | ForEach-Object { Write-Host $_.FullName }
+
+- task: PowerShell@2
+ displayName: 'Configure pip for Azure Artifacts'
+ inputs:
+ targetType: inline
+ script: |
+ pip config set global.index-url https://pkgs.dev.azure.com/aiinfra/PublicPackages/_packaging/ORT-Nightly/pypi/simple/
+ pip config set global.extra-index-url https://pypi.org/simple/
+ pip config set global.pre true
+
+- script: python -m pip install build
+ displayName: 'Install build tool'
+
+- task: PowerShell@2
+ displayName: 'Set SDK version'
+ inputs:
+ targetType: inline
+ script: |
+ Set-Content -Path "$(repoRoot)/sdk/python/src/version.py" -Value '__version__ = "${{ parameters.version }}"'
+
+- task: PowerShell@2
+ displayName: 'Pre-install pipeline-built FLC wheel'
+ condition: and(succeeded(), ne('${{ parameters.flcWheelsDir }}', ''))
+ inputs:
+ targetType: inline
+ script: |
+ # Determine platform wheel tag for the current machine
+ $arch = if ([System.Runtime.InteropServices.RuntimeInformation]::OSArchitecture -eq 'Arm64') { 'arm64' } else { 'amd64' }
+ if ($IsLinux) { $platTag = "manylinux*x86_64" }
+ elseif ($IsMacOS) { $platTag = "macosx*$arch" }
+ else { $platTag = "win_$arch" }
+
+ $filter = if ("${{ parameters.isWinML }}" -eq "True") { "foundry_local_core_winml*$platTag.whl" } else { "foundry_local_core-*$platTag.whl" }
+ $wheel = Get-ChildItem "${{ parameters.flcWheelsDir }}" -Recurse -Filter $filter | Select-Object -First 1
+ if ($wheel) {
+ Write-Host "Installing pipeline-built FLC wheel: $($wheel.FullName)"
+ pip install $($wheel.FullName)
+ } else {
+ Write-Warning "No FLC wheel found matching $filter"
+ }
+
+# Install ORT native packages from the ORT-Nightly feed.
+# skip-native-deps strips these from the SDK wheel metadata, so they
+# must be installed explicitly for tests to locate the native binaries.
+- script: pip install onnxruntime-core onnxruntime-genai-core
+ displayName: 'Install ORT native packages'
+
+- ${{ if not(parameters.isWinML) }}:
+ - script: python -m build --wheel -C skip-native-deps=true --outdir dist/
+ displayName: 'Build wheel'
+ workingDirectory: $(repoRoot)/sdk/python
+
+- ${{ if parameters.isWinML }}:
+ - script: python -m build --wheel -C winml=true -C skip-native-deps=true --outdir dist/
+ displayName: 'Build wheel (WinML)'
+ workingDirectory: $(repoRoot)/sdk/python
+
+- task: PowerShell@2
+ displayName: 'Install built wheel'
+ inputs:
+ targetType: inline
+ script: |
+ $wheel = (Get-ChildItem "$(repoRoot)/sdk/python/dist/*.whl" | Select-Object -First 1).FullName
+ pip install $wheel
+
+- script: pip install coverage pytest>=7.0.0 pytest-timeout>=2.1.0
+ displayName: 'Install test dependencies'
+
+- script: python -m pytest test/ -v
+ displayName: 'Run tests'
+ workingDirectory: $(repoRoot)/sdk/python
+ env:
+ TF_BUILD: 'true'
diff --git a/.pipelines/templates/test-rust-steps.yml b/.pipelines/templates/test-rust-steps.yml
new file mode 100644
index 00000000..31bfd75e
--- /dev/null
+++ b/.pipelines/templates/test-rust-steps.yml
@@ -0,0 +1,159 @@
+# Lightweight test-only steps for the Rust SDK.
+# Builds from source and runs tests — no cargo package or artifact staging.
+parameters:
+- name: isWinML
+ type: boolean
+ default: false
+- name: flcNugetDir
+ type: string
+ displayName: 'Path to directory containing the FLC .nupkg'
+
+steps:
+- task: PowerShell@2
+ displayName: 'Set source paths'
+ inputs:
+ targetType: inline
+ script: |
+ $repoRoot = "$(Build.SourcesDirectory)/Foundry-Local"
+ $testDataDir = "$(Build.SourcesDirectory)/test-data-shared"
+ Write-Host "##vso[task.setvariable variable=repoRoot]$repoRoot"
+ Write-Host "##vso[task.setvariable variable=testDataDir]$testDataDir"
+
+- ${{ if eq(parameters.isWinML, true) }}:
+ - task: PowerShell@2
+ displayName: 'Install Windows App SDK Runtime'
+ inputs:
+ targetType: 'inline'
+ script: |
+ $installerUrl = "https://aka.ms/windowsappsdk/1.8/latest/windowsappruntimeinstall-x64.exe"
+ $installerPath = "$env:TEMP\windowsappruntimeinstall.exe"
+
+ Write-Host "Downloading Windows App SDK Runtime installer from $installerUrl..."
+ Invoke-WebRequest -Uri $installerUrl -OutFile $installerPath
+
+ Write-Host "Installing Windows App SDK Runtime..."
+ & $installerPath --quiet --force
+
+ if ($LASTEXITCODE -ne 0) {
+ Write-Error "Installation failed with exit code $LASTEXITCODE"
+ exit 1
+ }
+
+ Write-Host "Windows App SDK Runtime installed successfully."
+ errorActionPreference: 'stop'
+
+- task: PowerShell@2
+ displayName: 'List downloaded FLC artifact'
+ inputs:
+ targetType: inline
+ script: |
+ Write-Host "Contents of ${{ parameters.flcNugetDir }}:"
+ Get-ChildItem "${{ parameters.flcNugetDir }}" -Recurse | ForEach-Object { Write-Host $_.FullName }
+
+# Extract FLC native binaries from the pipeline-built .nupkg
+- task: PowerShell@2
+ displayName: 'Extract FLC native binaries'
+ inputs:
+ targetType: inline
+ script: |
+ $nupkg = Get-ChildItem "${{ parameters.flcNugetDir }}" -Recurse -Filter "Microsoft.AI.Foundry.Local.Core*.nupkg" -Exclude "*.snupkg" | Select-Object -First 1
+ if (-not $nupkg) { throw "No FLC .nupkg found in ${{ parameters.flcNugetDir }}" }
+
+ $extractDir = "$(Build.ArtifactStagingDirectory)/flc-extract-rust"
+ $zip = [System.IO.Path]::ChangeExtension($nupkg.FullName, ".zip")
+ Copy-Item $nupkg.FullName $zip -Force
+ Expand-Archive -Path $zip -DestinationPath $extractDir -Force
+
+ $arch = if ([System.Runtime.InteropServices.RuntimeInformation]::OSArchitecture -eq 'Arm64') { 'arm64' } else { 'x64' }
+ if ($IsLinux) {
+ $rid = "linux-$arch"
+ } elseif ($IsMacOS) {
+ $rid = "osx-$arch"
+ } else {
+ $rid = "win-$arch"
+ }
+
+ $nativeDir = "$extractDir/runtimes/$rid/native"
+ if (-not (Test-Path $nativeDir)) { throw "No native binaries found at $nativeDir for RID $rid" }
+
+ $flcNativeDir = "$(Build.ArtifactStagingDirectory)/flc-native-rust"
+ New-Item -ItemType Directory -Path $flcNativeDir -Force | Out-Null
+ Get-ChildItem $nativeDir -File | Copy-Item -Destination $flcNativeDir -Force
+ Write-Host "##vso[task.setvariable variable=flcNativeDir]$flcNativeDir"
+ Write-Host "Extracted FLC native binaries for $rid"
+
+- task: PowerShell@2
+ displayName: 'Install Rust toolchain'
+ inputs:
+ targetType: inline
+ script: |
+ if ($IsWindows -or (-not $IsLinux -and -not $IsMacOS)) {
+ Invoke-WebRequest -Uri https://win.rustup.rs/x86_64 -OutFile rustup-init.exe
+ .\rustup-init.exe -y --default-toolchain stable --profile minimal -c clippy,rustfmt
+ Remove-Item rustup-init.exe
+ $cargoPath = "$env:USERPROFILE\.cargo\bin"
+ } else {
+ curl --proto '=https' --tlsv1.2 -sSf https://sh.rustup.rs | sh -s -- -y --default-toolchain stable --profile minimal -c clippy,rustfmt
+ $cargoPath = "$env:HOME/.cargo/bin"
+ }
+ Write-Host "##vso[task.prependpath]$cargoPath"
+
+- task: PowerShell@2
+ displayName: 'Use crates.io directly'
+ inputs:
+ targetType: inline
+ script: |
+ $configPath = "$(repoRoot)/sdk/rust/.cargo/config.toml"
+ if (Test-Path $configPath) {
+ Remove-Item $configPath
+ Write-Host "Removed .cargo/config.toml crates-io redirect"
+ }
+
+- task: PowerShell@2
+ displayName: 'Build'
+ inputs:
+ targetType: inline
+ script: |
+ Set-Location "$(repoRoot)/sdk/rust"
+ $features = if ("${{ parameters.isWinML }}" -eq "True") { "--features winml" } else { "" }
+ Invoke-Expression "cargo build $features"
+ if ($LASTEXITCODE -ne 0) { exit $LASTEXITCODE }
+
+# Overwrite FLC binary with pipeline-built version
+- task: PowerShell@2
+ displayName: 'Overwrite FLC binary with pipeline-built version'
+ inputs:
+ targetType: inline
+ script: |
+ $outDir = Get-ChildItem "$(repoRoot)/sdk/rust/target/debug/build" -Directory -Filter "foundry-local-sdk-*" -Recurse |
+ Where-Object { Test-Path "$($_.FullName)/out" } |
+ ForEach-Object { "$($_.FullName)/out" } |
+ Select-Object -First 1
+ if (-not $outDir) { throw "Could not find cargo OUT_DIR for foundry-local-sdk" }
+
+ Get-ChildItem "$(flcNativeDir)" -File -Filter "Microsoft.AI.Foundry.Local.Core.*" | ForEach-Object {
+ Copy-Item $_.FullName -Destination "$outDir/$($_.Name)" -Force
+ Write-Host "Overwrote $($_.Name) with pipeline-built version"
+ }
+
+- task: PowerShell@2
+ displayName: 'Run unit tests'
+ inputs:
+ targetType: inline
+ script: |
+ Set-Location "$(repoRoot)/sdk/rust"
+ $features = if ("${{ parameters.isWinML }}" -eq "True") { "--features winml" } else { "" }
+ Invoke-Expression "cargo test --lib $features"
+ if ($LASTEXITCODE -ne 0) { exit $LASTEXITCODE }
+
+- task: PowerShell@2
+ displayName: 'Run integration tests'
+ inputs:
+ targetType: inline
+ script: |
+ Set-Location "$(repoRoot)/sdk/rust"
+ $features = if ("${{ parameters.isWinML }}" -eq "True") { "--features winml" } else { "" }
+ Invoke-Expression "cargo test --tests $features -- --include-ignored --test-threads=1 --nocapture"
+ if ($LASTEXITCODE -ne 0) { exit $LASTEXITCODE }
+ env:
+ TF_BUILD: 'true'
diff --git a/sdk/cs/README.md b/sdk/cs/README.md
index f58e41e0..2b574325 100644
--- a/sdk/cs/README.md
+++ b/sdk/cs/README.md
@@ -48,7 +48,7 @@ dotnet build src/Microsoft.AI.Foundry.Local.csproj /p:UseWinML=true
### Triggering EP download
-EP download can be time-consuming. Call `EnsureEpsDownloadedAsync` early (after initialization) to separate the download step from catalog access:
+EP download can be time-consuming. Call `DownloadAndRegisterEpsAsync` early (after initialization) to separate the download step from catalog access:
```csharp
// Initialize the manager first (see Quick Start)
@@ -56,7 +56,7 @@ await FoundryLocalManager.CreateAsync(
new Configuration { AppName = "my-app" },
NullLogger.Instance);
-await FoundryLocalManager.Instance.EnsureEpsDownloadedAsync();
+await FoundryLocalManager.Instance.DownloadAndRegisterEpsAsync();
// Now catalog access won't trigger an EP download
var catalog = await FoundryLocalManager.Instance.GetCatalogAsync();
diff --git a/sdk/cs/docs/api/microsoft.ai.foundry.local.foundrylocalmanager.md b/sdk/cs/docs/api/microsoft.ai.foundry.local.foundrylocalmanager.md
index 93f162b7..9e5be8aa 100644
--- a/sdk/cs/docs/api/microsoft.ai.foundry.local.foundrylocalmanager.md
+++ b/sdk/cs/docs/api/microsoft.ai.foundry.local.foundrylocalmanager.md
@@ -98,7 +98,7 @@ The model catalog.
The catalog is populated on first use.
If you are using a WinML build this will trigger a one-off execution provider download if not already done.
- It is recommended to call [FoundryLocalManager.EnsureEpsDownloadedAsync(Nullable<CancellationToken>)](./microsoft.ai.foundry.local.foundrylocalmanager.md#ensureepsdownloadedasyncnullablecancellationtoken) first to separate out the two steps.
+ It is recommended to call [FoundryLocalManager.DownloadAndRegisterEpsAsync(Nullable<CancellationToken>)](./microsoft.ai.foundry.local.foundrylocalmanager.md#downloadandregisterepsasyncnullablecancellationtoken) first to separate out the two steps.
### **StartWebServiceAsync(Nullable<CancellationToken>)**
@@ -141,9 +141,9 @@ Optional cancellation token.
[Task](https://docs.microsoft.com/en-us/dotnet/api/system.threading.tasks.task)
Task stopping the web service.
-### **EnsureEpsDownloadedAsync(Nullable<CancellationToken>)**
+### **DownloadAndRegisterEpsAsync(Nullable<CancellationToken>)**
-Ensure execution providers are downloaded and registered.
+Download and register execution providers.
Only relevant when using WinML.
Execution provider download can be time consuming due to the size of the packages.
@@ -151,7 +151,7 @@ Ensure execution providers are downloaded and registered.
on subsequent calls.
```csharp
-public Task EnsureEpsDownloadedAsync(Nullable ct)
+public Task DownloadAndRegisterEpsAsync(Nullable ct)
```
#### Parameters
diff --git a/sdk/cs/src/Detail/CoreInterop.cs b/sdk/cs/src/Detail/CoreInterop.cs
index 8411473b..d7867cad 100644
--- a/sdk/cs/src/Detail/CoreInterop.cs
+++ b/sdk/cs/src/Detail/CoreInterop.cs
@@ -124,6 +124,15 @@ internal CoreInterop(Configuration config, ILogger logger)
_logger = logger ?? throw new ArgumentNullException(nameof(logger));
var request = new CoreInteropRequest { Params = config.AsDictionary() };
+
+#if IS_WINML
+ // WinML builds require bootstrapping the Windows App Runtime
+ if (!request.Params.ContainsKey("Bootstrap"))
+ {
+ request.Params["Bootstrap"] = "true";
+ }
+#endif
+
var response = ExecuteCommand("initialize", request);
if (response.Error != null)
diff --git a/sdk/cs/src/FoundryLocalManager.cs b/sdk/cs/src/FoundryLocalManager.cs
index 639be3a2..d3e4fb79 100644
--- a/sdk/cs/src/FoundryLocalManager.cs
+++ b/sdk/cs/src/FoundryLocalManager.cs
@@ -99,7 +99,7 @@ public static async Task CreateAsync(Configuration configuration, ILogger logger
///
/// The catalog is populated on first use.
/// If you are using a WinML build this will trigger a one-off execution provider download if not already done.
- /// It is recommended to call first to separate out the two steps.
+ /// It is recommended to call first to separate out the two steps.
///
public async Task GetCatalogAsync(CancellationToken? ct = null)
{
@@ -135,7 +135,7 @@ await Utils.CallWithExceptionHandling(() => StopWebServiceImplAsync(ct),
}
///
- /// Ensure execution providers are downloaded and registered.
+ /// Download and register execution providers.
/// Only relevant when using WinML.
///
/// Execution provider download can be time consuming due to the size of the packages.
@@ -143,10 +143,10 @@ await Utils.CallWithExceptionHandling(() => StopWebServiceImplAsync(ct),
/// on subsequent calls.
///
/// Optional cancellation token.
- public async Task EnsureEpsDownloadedAsync(CancellationToken? ct = null)
+ public async Task DownloadAndRegisterEpsAsync(CancellationToken? ct = null)
{
- await Utils.CallWithExceptionHandling(() => EnsureEpsDownloadedImplAsync(ct),
- "Error ensuring execution providers downloaded.", _logger)
+ await Utils.CallWithExceptionHandling(() => DownloadAndRegisterEpsImplAsync(ct),
+ "Error downloading and registering execution providers.", _logger)
.ConfigureAwait(false);
}
@@ -259,16 +259,16 @@ private async Task StopWebServiceImplAsync(CancellationToken? ct = null)
Urls = null;
}
- private async Task EnsureEpsDownloadedImplAsync(CancellationToken? ct = null)
+ private async Task DownloadAndRegisterEpsImplAsync(CancellationToken? ct = null)
{
using var disposable = await asyncLock.LockAsync().ConfigureAwait(false);
CoreInteropRequest? input = null;
- var result = await _coreInterop!.ExecuteCommandAsync("ensure_eps_downloaded", input, ct);
+ var result = await _coreInterop!.ExecuteCommandAsync("download_and_register_eps", input, ct).ConfigureAwait(false);
if (result.Error != null)
{
- throw new FoundryLocalException($"Error ensuring execution providers downloaded: {result.Error}", _logger);
+ throw new FoundryLocalException($"Error downloading and registering execution providers: {result.Error}", _logger);
}
}
diff --git a/sdk/cs/src/Microsoft.AI.Foundry.Local.csproj b/sdk/cs/src/Microsoft.AI.Foundry.Local.csproj
index 905f9652..936f3a93 100644
--- a/sdk/cs/src/Microsoft.AI.Foundry.Local.csproj
+++ b/sdk/cs/src/Microsoft.AI.Foundry.Local.csproj
@@ -13,7 +13,7 @@
https://github.com/microsoft/Foundry-Local
git
- net8.0
+ net9.0
win-x64;win-arm64;linux-x64;linux-arm64;osx-arm64
true
@@ -87,7 +87,8 @@
Microsoft Foundry Local SDK for WinML
Microsoft.AI.Foundry.Local.WinML
Microsoft.AI.Foundry.Local.WinML
- net8.0-windows10.0.26100.0
+ $(DefineConstants);IS_WINML
+ net9.0-windows10.0.26100.0
win-x64;win-arm64
10.0.17763.0
diff --git a/sdk/cs/test/FoundryLocal.Tests/Microsoft.AI.Foundry.Local.Tests.csproj b/sdk/cs/test/FoundryLocal.Tests/Microsoft.AI.Foundry.Local.Tests.csproj
index 5f0c7cf2..fe0dfcd2 100644
--- a/sdk/cs/test/FoundryLocal.Tests/Microsoft.AI.Foundry.Local.Tests.csproj
+++ b/sdk/cs/test/FoundryLocal.Tests/Microsoft.AI.Foundry.Local.Tests.csproj
@@ -1,7 +1,7 @@
- net10.0
+ net9.0
enable
enable
false
@@ -19,10 +19,9 @@
- net10.0-windows10.0.26100.0
+ net9.0-windows10.0.26100.0
10.0.17763.0
None
- true
diff --git a/sdk/js/docs/classes/FoundryLocalManager.md b/sdk/js/docs/classes/FoundryLocalManager.md
index 63bb2dd1..dc4908a6 100644
--- a/sdk/js/docs/classes/FoundryLocalManager.md
+++ b/sdk/js/docs/classes/FoundryLocalManager.md
@@ -87,6 +87,29 @@ Error - If the web service is not running.
***
+### downloadAndRegisterEps()
+
+```ts
+downloadAndRegisterEps(): void;
+```
+
+Download and register execution providers.
+Only relevant when using the WinML variant. On non-WinML builds this is a no-op.
+
+Call this after initialization to trigger EP download before accessing the catalog,
+so that hardware-accelerated execution providers (e.g. QNN for NPU) are available
+when listing and loading models.
+
+#### Returns
+
+`void`
+
+#### Throws
+
+Error - If execution provider download or registration fails.
+
+***
+
### startWebService()
```ts
diff --git a/sdk/js/src/foundryLocalManager.ts b/sdk/js/src/foundryLocalManager.ts
index bc408f78..6da0bcc7 100644
--- a/sdk/js/src/foundryLocalManager.ts
+++ b/sdk/js/src/foundryLocalManager.ts
@@ -61,6 +61,24 @@ export class FoundryLocalManager {
return this._urls;
}
+ /**
+ * Download and register execution providers.
+ * Only relevant when using the WinML variant. On non-WinML builds this is a no-op.
+ *
+ * Call this after initialization to trigger EP download before accessing the catalog,
+ * so that hardware-accelerated execution providers (e.g. QNN for NPU) are available
+ * when listing and loading models.
+ *
+ * @throws Error - If execution provider download or registration fails.
+ */
+ public downloadAndRegisterEps(): void {
+ try {
+ this.coreInterop.executeCommand("download_and_register_eps");
+ } catch (error) {
+ throw new Error(`Error downloading and registering execution providers: ${error}`);
+ }
+ }
+
/**
* Starts the local web service.
* Use the `urls` property to retrieve the bound addresses after the service has started.
diff --git a/sdk/python/build_backend.py b/sdk/python/build_backend.py
index b4b91a1b..3789501b 100644
--- a/sdk/python/build_backend.py
+++ b/sdk/python/build_backend.py
@@ -18,9 +18,14 @@
python -m build --wheel -C winml=true
+Skip native deps (use pre-installed foundry-local-core / ORT / GenAI)::
+
+ python -m build --wheel -C skip-native-deps=true
+
Environment variable fallback (useful in CI pipelines)::
FOUNDRY_VARIANT=winml python -m build --wheel
+ FOUNDRY_SKIP_NATIVE_DEPS=1 python -m build --wheel
"""
from __future__ import annotations
@@ -46,6 +51,13 @@
_STANDARD_NAME = 'name = "foundry-local-sdk"'
_WINML_NAME = 'name = "foundry-local-sdk-winml"'
+# Native binary package prefixes to strip when skip-native-deps is active.
+_NATIVE_DEP_PREFIXES = (
+ "foundry-local-core",
+ "onnxruntime-core",
+ "onnxruntime-genai-core",
+)
+
# ---------------------------------------------------------------------------
# Variant detection
@@ -63,6 +75,23 @@ def _is_winml(config_settings: dict | None) -> bool:
return os.environ.get("FOUNDRY_VARIANT", "").lower() == "winml"
+def _is_skip_native_deps(config_settings: dict | None) -> bool:
+ """Return True when native binary dependencies should be omitted.
+
+ When set, ``foundry-local-core``, ``onnxruntime-core``, and
+ ``onnxruntime-genai-core`` are stripped from requirements.txt so the
+ wheel is built against whatever versions are already installed.
+ Useful in CI pipelines that pre-install pipeline-built native wheels.
+
+ Checks ``config_settings["skip-native-deps"]`` first
+ (set via ``-C skip-native-deps=true``), then falls back to the
+ ``FOUNDRY_SKIP_NATIVE_DEPS`` environment variable.
+ """
+ if config_settings and str(config_settings.get("skip-native-deps", "")).lower() == "true":
+ return True
+ return os.environ.get("FOUNDRY_SKIP_NATIVE_DEPS", "").lower() in ("1", "true")
+
+
# ---------------------------------------------------------------------------
# In-place patching context manager
# ---------------------------------------------------------------------------
@@ -96,58 +125,88 @@ def _patch_for_winml() -> Generator[None, None, None]:
_REQUIREMENTS.write_text(requirements_original, encoding="utf-8")
+@contextlib.contextmanager
+def _strip_native_deps() -> Generator[None, None, None]:
+ """Temporarily remove native binary deps from requirements.txt.
+
+ Lines starting with any prefix in ``_NATIVE_DEP_PREFIXES`` (case-
+ insensitive) are removed. The file is restored in the ``finally``
+ block.
+ """
+ requirements_original = _REQUIREMENTS.read_text(encoding="utf-8")
+ try:
+ filtered = [
+ line for line in requirements_original.splitlines(keepends=True)
+ if not any(line.lstrip().lower().startswith(p) for p in _NATIVE_DEP_PREFIXES)
+ ]
+ _REQUIREMENTS.write_text("".join(filtered), encoding="utf-8")
+ yield
+ finally:
+ _REQUIREMENTS.write_text(requirements_original, encoding="utf-8")
+
+
+def _apply_patches(config_settings: dict | None):
+ """Return a context manager that applies the appropriate patches."""
+ winml = _is_winml(config_settings)
+ skip_native = _is_skip_native_deps(config_settings)
+
+ @contextlib.contextmanager
+ def _combined():
+ # Stack contexts: WinML swaps requirements first, then strip_native
+ # removes native deps from whatever requirements are active.
+ if winml and skip_native:
+ with _patch_for_winml(), _strip_native_deps():
+ yield
+ elif winml:
+ with _patch_for_winml():
+ yield
+ elif skip_native:
+ with _strip_native_deps():
+ yield
+ else:
+ yield
+
+ return _combined()
+
+
# ---------------------------------------------------------------------------
# PEP 517 hook delegation
# ---------------------------------------------------------------------------
def get_requires_for_build_wheel(config_settings=None):
- if _is_winml(config_settings):
- with _patch_for_winml():
- return _sb.get_requires_for_build_wheel(config_settings)
- return _sb.get_requires_for_build_wheel(config_settings)
+ with _apply_patches(config_settings):
+ return _sb.get_requires_for_build_wheel(config_settings)
def prepare_metadata_for_build_wheel(metadata_directory, config_settings=None):
- if _is_winml(config_settings):
- with _patch_for_winml():
- return _sb.prepare_metadata_for_build_wheel(metadata_directory, config_settings)
- return _sb.prepare_metadata_for_build_wheel(metadata_directory, config_settings)
+ with _apply_patches(config_settings):
+ return _sb.prepare_metadata_for_build_wheel(metadata_directory, config_settings)
def build_wheel(wheel_directory, config_settings=None, metadata_directory=None):
- if _is_winml(config_settings):
- with _patch_for_winml():
- return _sb.build_wheel(wheel_directory, config_settings, metadata_directory)
- return _sb.build_wheel(wheel_directory, config_settings, metadata_directory)
+ with _apply_patches(config_settings):
+ return _sb.build_wheel(wheel_directory, config_settings, metadata_directory)
def get_requires_for_build_editable(config_settings=None):
- if _is_winml(config_settings):
- with _patch_for_winml():
- return _sb.get_requires_for_build_editable(config_settings)
- return _sb.get_requires_for_build_editable(config_settings)
+ with _apply_patches(config_settings):
+ return _sb.get_requires_for_build_editable(config_settings)
def prepare_metadata_for_build_editable(metadata_directory, config_settings=None):
- if _is_winml(config_settings):
- with _patch_for_winml():
- return _sb.prepare_metadata_for_build_editable(metadata_directory, config_settings)
- return _sb.prepare_metadata_for_build_editable(metadata_directory, config_settings)
+ with _apply_patches(config_settings):
+ return _sb.prepare_metadata_for_build_editable(metadata_directory, config_settings)
def build_editable(wheel_directory, config_settings=None, metadata_directory=None):
- if _is_winml(config_settings):
- with _patch_for_winml():
- return _sb.build_editable(wheel_directory, config_settings, metadata_directory)
- return _sb.build_editable(wheel_directory, config_settings, metadata_directory)
+ with _apply_patches(config_settings):
+ return _sb.build_editable(wheel_directory, config_settings, metadata_directory)
def get_requires_for_build_sdist(config_settings=None):
- if _is_winml(config_settings):
- with _patch_for_winml():
- return _sb.get_requires_for_build_sdist(config_settings)
- return _sb.get_requires_for_build_sdist(config_settings)
+ with _apply_patches(config_settings):
+ return _sb.get_requires_for_build_sdist(config_settings)
def build_sdist(sdist_directory, config_settings=None):
diff --git a/sdk/python/src/detail/core_interop.py b/sdk/python/src/detail/core_interop.py
index 7a6bb08c..4f4ddb67 100644
--- a/sdk/python/src/detail/core_interop.py
+++ b/sdk/python/src/detail/core_interop.py
@@ -205,6 +205,9 @@ def __init__(self, config: Configuration):
if sys.platform.startswith("win"):
bootstrap_dll = paths.core_dir / "Microsoft.WindowsAppRuntime.Bootstrap.dll"
if bootstrap_dll.exists():
+ # Pre-load so the DLL is already in the process when
+ # C# P/Invoke resolves it during Bootstrap.Initialize().
+ ctypes.CDLL(str(bootstrap_dll))
if config.additional_settings is None:
config.additional_settings = {}
if "Bootstrap" not in config.additional_settings:
diff --git a/sdk/python/src/foundry_local_manager.py b/sdk/python/src/foundry_local_manager.py
index 4486eaf1..4c02a127 100644
--- a/sdk/python/src/foundry_local_manager.py
+++ b/sdk/python/src/foundry_local_manager.py
@@ -71,17 +71,17 @@ def _initialize(self):
self._model_load_manager = ModelLoadManager(self._core_interop, external_service_url)
self.catalog = Catalog(self._model_load_manager, self._core_interop)
- def ensure_eps_downloaded(self) -> None:
- """Ensure execution providers are downloaded and registered (synchronous).
+ def download_and_register_eps(self) -> None:
+ """Download and register execution providers.
Only relevant when using WinML.
Raises:
- FoundryLocalException: If execution provider download fails.
+ FoundryLocalException: If execution provider download or registration fails.
"""
- result = self._core_interop.execute_command("ensure_eps_downloaded")
+ result = self._core_interop.execute_command("download_and_register_eps")
if result.error is not None:
- raise FoundryLocalException(f"Error ensuring execution providers downloaded: {result.error}")
+ raise FoundryLocalException(f"Error downloading and registering execution providers: {result.error}")
def start_web_service(self):
"""Start the optional web service.
diff --git a/sdk/rust/src/foundry_local_manager.rs b/sdk/rust/src/foundry_local_manager.rs
index f80a7176..9cf2477f 100644
--- a/sdk/rust/src/foundry_local_manager.rs
+++ b/sdk/rust/src/foundry_local_manager.rs
@@ -133,4 +133,18 @@ impl FoundryLocalManager {
.clear();
Ok(())
}
+
+ /// Download and register execution providers.
+ ///
+ /// Only relevant when using the WinML variant. On non-WinML builds this
+ /// is a no-op. Call this after initialisation to trigger EP download
+ /// before accessing the catalog, so that hardware-accelerated execution
+ /// providers (e.g. QNN for NPU) are available when listing and loading
+ /// models.
+ pub async fn download_and_register_eps(&self) -> Result<()> {
+ self.core
+ .execute_command_async("download_and_register_eps".into(), None)
+ .await?;
+ Ok(())
+ }
}
From 6c7af0fef72f69953c45dd10c82edeb82f274359 Mon Sep 17 00:00:00 2001
From: Samuel Kemp
Date: Tue, 31 Mar 2026 20:34:26 +0100
Subject: [PATCH 12/21] Add named regions and tutorial samples for docs
externalization (#563)
To ensure that our docs on MS Learn have accurate code samples, we will
update the docs so they consume the code from this repo. In this repo,
we will run a test to ensure that the samples work - if there is a break
in the samples then this should be fix before a PR can be merged in.
- Add named regions to 15 existing sample files (CS, JS, Python, Rust)
- Create 3 missing Python samples (audio-transcription, web-server,
langchain-integration)
- Create 16 tutorial sample projects (4 tutorials x 4 languages)
- Add samples-integration-test.yml CI workflow
---------
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
---
.../workflows/samples-integration-test.yml | 260 ++++
.gitignore | 3 +
.../Directory.Packages.props | 3 +
samples/cs/GettingStarted/README.md | 61 -
.../AudioTranscriptionExample.csproj | 39 -
.../FoundryLocalWebServer.csproj | 33 -
.../FoundrySamplesXPlatform.sln | 53 -
.../HelloFoundryLocalSdk.csproj | 32 -
.../ModelManagementExample.csproj | 33 -
.../ToolCallingFoundryLocalSdk.csproj | 31 -
.../ToolCallingFoundryLocalWebServer.csproj | 32 -
samples/cs/GettingStarted/nuget.config | 7 -
.../AudioTranscriptionExample.csproj | 36 -
.../FoundryLocalWebServer.csproj | 30 -
.../windows/FoundrySamplesWinML.sln | 71 --
.../HelloFoundryLocalSdk.csproj | 30 -
.../ModelManagementExample.csproj | 30 -
.../ToolCallingFoundryLocalSdk.csproj | 30 -
.../ToolCallingFoundryLocalWebServer.csproj | 30 -
samples/cs/README.md | 43 +
.../{GettingStarted/src => }/Shared/Utils.cs | 0
.../AudioTranscriptionExample.csproj | 55 +
.../AudioTranscriptionExample.sln | 34 +
.../Program.cs | 23 +-
.../Recording.mp3 | Bin
.../FoundryLocalWebServer.csproj | 52 +
.../FoundryLocalWebServer.sln | 34 +
.../Program.cs | 14 +-
.../LiveAudioTranscriptionExample.csproj | 55 +
.../LiveAudioTranscriptionExample.sln | 34 +
.../Program.cs | 106 ++
.../ModelManagementExample.csproj | 48 +
.../ModelManagementExample.sln | 34 +
.../Program.cs | 0
.../NativeChatCompletions.csproj | 48 +
.../NativeChatCompletions.sln | 34 +
.../Program.cs | 16 +-
samples/cs/nuget.config | 22 +
.../Program.cs | 18 +-
.../ToolCallingFoundryLocalSdk.csproj | 48 +
.../ToolCallingFoundryLocalSdk.sln | 34 +
.../Program.cs | 6 +-
.../ToolCallingFoundryLocalWebServer.csproj | 52 +
.../ToolCallingFoundryLocalWebServer.sln | 34 +
samples/cs/tutorial-chat-assistant/Program.cs | 101 ++
.../TutorialChatAssistant.csproj | 50 +
.../TutorialChatAssistant.sln | 34 +
.../tutorial-document-summarizer/Program.cs | 109 ++
.../TutorialDocumentSummarizer.csproj | 50 +
.../TutorialDocumentSummarizer.sln | 34 +
samples/cs/tutorial-tool-calling/Program.cs | 228 ++++
.../TutorialToolCalling.csproj | 50 +
.../TutorialToolCalling.sln | 34 +
samples/cs/tutorial-voice-to-text/Program.cs | 104 ++
.../TutorialVoiceToText.csproj | 50 +
.../TutorialVoiceToText.sln | 34 +
samples/js/audio-transcription-example/app.js | 19 +-
.../chat-and-audio-foundry-local/src/app.js | 2 +-
.../js/langchain-integration-example/app.js | 12 +-
samples/js/native-chat-completions/app.js | 14 +
.../js/tool-calling-foundry-local/src/app.js | 16 +-
samples/js/tutorial-chat-assistant/app.js | 84 ++
.../js/tutorial-chat-assistant/package.json | 9 +
.../js/tutorial-document-summarizer/app.js | 84 ++
.../tutorial-document-summarizer/package.json | 9 +
samples/js/tutorial-tool-calling/app.js | 186 +++
samples/js/tutorial-tool-calling/package.json | 9 +
samples/js/tutorial-voice-to-text/app.js | 78 ++
.../js/tutorial-voice-to-text/package.json | 9 +
samples/js/web-server-example/app.js | 10 +
.../python/audio-transcription/Recording.mp3 | Bin 0 -> 329760 bytes
.../audio-transcription/requirements.txt | 1 +
samples/python/audio-transcription/src/app.py | 39 +
samples/python/functioncalling/README.md | 53 -
samples/python/functioncalling/fl_tools.ipynb | 362 ------
samples/python/hello-foundry-local/README.md | 18 -
samples/python/hello-foundry-local/src/app.py | 33 -
.../langchain-integration/requirements.txt | 4 +
.../python/langchain-integration/src/app.py | 59 +
.../native-chat-completions/requirements.txt | 1 +
.../python/native-chat-completions/src/app.py | 54 +
samples/python/summarize/.vscode/launch.json | 14 -
samples/python/summarize/README.md | 38 -
samples/python/summarize/requirements.txt | 3 -
samples/python/summarize/summarize.py | 86 --
samples/python/tool-calling/requirements.txt | 1 +
samples/python/tool-calling/src/app.py | 182 +++
.../tutorial-chat-assistant/requirements.txt | 1 +
.../python/tutorial-chat-assistant/src/app.py | 71 ++
.../requirements.txt | 1 +
.../tutorial-document-summarizer/src/app.py | 78 ++
.../tutorial-tool-calling/requirements.txt | 1 +
.../python/tutorial-tool-calling/src/app.py | 187 +++
.../tutorial-voice-to-text/requirements.txt | 1 +
.../python/tutorial-voice-to-text/src/app.py | 78 ++
samples/python/web-server/requirements.txt | 2 +
samples/python/web-server/src/app.py | 59 +
samples/rag/README.md | 206 ----
samples/rag/foundry-local-architecture.md | 116 --
samples/rag/rag_foundrylocal_demo.ipynb | 1042 -----------------
samples/rust/Cargo.toml | 4 +
.../audio-transcription-example/Recording.mp3 | Bin 0 -> 329760 bytes
.../audio-transcription-example/src/main.rs | 25 +-
.../rust/foundry-local-webserver/src/main.rs | 12 +-
.../rust/native-chat-completions/src/main.rs | 26 +-
.../tool-calling-foundry-local/src/main.rs | 20 +-
.../rust/tutorial-chat-assistant/Cargo.toml | 11 +
.../rust/tutorial-chat-assistant/src/main.rs | 102 ++
.../tutorial-document-summarizer/Cargo.toml | 10 +
.../tutorial-document-summarizer/src/main.rs | 157 +++
samples/rust/tutorial-tool-calling/Cargo.toml | 11 +
.../rust/tutorial-tool-calling/src/main.rs | 330 ++++++
.../rust/tutorial-voice-to-text/Cargo.toml | 10 +
.../rust/tutorial-voice-to-text/src/main.rs | 110 ++
114 files changed, 4142 insertions(+), 2584 deletions(-)
create mode 100644 .github/workflows/samples-integration-test.yml
rename samples/cs/{GettingStarted => }/Directory.Packages.props (73%)
delete mode 100644 samples/cs/GettingStarted/README.md
delete mode 100644 samples/cs/GettingStarted/cross-platform/AudioTranscriptionExample/AudioTranscriptionExample.csproj
delete mode 100644 samples/cs/GettingStarted/cross-platform/FoundryLocalWebServer/FoundryLocalWebServer.csproj
delete mode 100644 samples/cs/GettingStarted/cross-platform/FoundrySamplesXPlatform.sln
delete mode 100644 samples/cs/GettingStarted/cross-platform/HelloFoundryLocalSdk/HelloFoundryLocalSdk.csproj
delete mode 100644 samples/cs/GettingStarted/cross-platform/ModelManagementExample/ModelManagementExample.csproj
delete mode 100644 samples/cs/GettingStarted/cross-platform/ToolCallingFoundryLocalSdk/ToolCallingFoundryLocalSdk.csproj
delete mode 100644 samples/cs/GettingStarted/cross-platform/ToolCallingFoundryLocalWebServer/ToolCallingFoundryLocalWebServer.csproj
delete mode 100644 samples/cs/GettingStarted/nuget.config
delete mode 100644 samples/cs/GettingStarted/windows/AudioTranscriptionExample/AudioTranscriptionExample.csproj
delete mode 100644 samples/cs/GettingStarted/windows/FoundryLocalWebServer/FoundryLocalWebServer.csproj
delete mode 100644 samples/cs/GettingStarted/windows/FoundrySamplesWinML.sln
delete mode 100644 samples/cs/GettingStarted/windows/HelloFoundryLocalSdk/HelloFoundryLocalSdk.csproj
delete mode 100644 samples/cs/GettingStarted/windows/ModelManagementExample/ModelManagementExample.csproj
delete mode 100644 samples/cs/GettingStarted/windows/ToolCallingFoundryLocalSdk/ToolCallingFoundryLocalSdk.csproj
delete mode 100644 samples/cs/GettingStarted/windows/ToolCallingFoundryLocalWebServer/ToolCallingFoundryLocalWebServer.csproj
create mode 100644 samples/cs/README.md
rename samples/cs/{GettingStarted/src => }/Shared/Utils.cs (100%)
create mode 100644 samples/cs/audio-transcription-example/AudioTranscriptionExample.csproj
create mode 100644 samples/cs/audio-transcription-example/AudioTranscriptionExample.sln
rename samples/cs/{GettingStarted/src/AudioTranscriptionExample => audio-transcription-example}/Program.cs (78%)
rename samples/cs/{GettingStarted/src/AudioTranscriptionExample => audio-transcription-example}/Recording.mp3 (100%)
create mode 100644 samples/cs/foundry-local-web-server/FoundryLocalWebServer.csproj
create mode 100644 samples/cs/foundry-local-web-server/FoundryLocalWebServer.sln
rename samples/cs/{GettingStarted/src/FoundryLocalWebServer => foundry-local-web-server}/Program.cs (91%)
create mode 100644 samples/cs/live-audio-transcription-example/LiveAudioTranscriptionExample.csproj
create mode 100644 samples/cs/live-audio-transcription-example/LiveAudioTranscriptionExample.sln
create mode 100644 samples/cs/live-audio-transcription-example/Program.cs
create mode 100644 samples/cs/model-management-example/ModelManagementExample.csproj
create mode 100644 samples/cs/model-management-example/ModelManagementExample.sln
rename samples/cs/{GettingStarted/src/ModelManagementExample => model-management-example}/Program.cs (100%)
create mode 100644 samples/cs/native-chat-completions/NativeChatCompletions.csproj
create mode 100644 samples/cs/native-chat-completions/NativeChatCompletions.sln
rename samples/cs/{GettingStarted/src/HelloFoundryLocalSdk => native-chat-completions}/Program.cs (88%)
create mode 100644 samples/cs/nuget.config
rename samples/cs/{GettingStarted/src/ToolCallingFoundryLocalSdk => tool-calling-foundry-local-sdk}/Program.cs (94%)
create mode 100644 samples/cs/tool-calling-foundry-local-sdk/ToolCallingFoundryLocalSdk.csproj
create mode 100644 samples/cs/tool-calling-foundry-local-sdk/ToolCallingFoundryLocalSdk.sln
rename samples/cs/{GettingStarted/src/ToolCallingFoundryLocalWebServer => tool-calling-foundry-local-web-server}/Program.cs (98%)
create mode 100644 samples/cs/tool-calling-foundry-local-web-server/ToolCallingFoundryLocalWebServer.csproj
create mode 100644 samples/cs/tool-calling-foundry-local-web-server/ToolCallingFoundryLocalWebServer.sln
create mode 100644 samples/cs/tutorial-chat-assistant/Program.cs
create mode 100644 samples/cs/tutorial-chat-assistant/TutorialChatAssistant.csproj
create mode 100644 samples/cs/tutorial-chat-assistant/TutorialChatAssistant.sln
create mode 100644 samples/cs/tutorial-document-summarizer/Program.cs
create mode 100644 samples/cs/tutorial-document-summarizer/TutorialDocumentSummarizer.csproj
create mode 100644 samples/cs/tutorial-document-summarizer/TutorialDocumentSummarizer.sln
create mode 100644 samples/cs/tutorial-tool-calling/Program.cs
create mode 100644 samples/cs/tutorial-tool-calling/TutorialToolCalling.csproj
create mode 100644 samples/cs/tutorial-tool-calling/TutorialToolCalling.sln
create mode 100644 samples/cs/tutorial-voice-to-text/Program.cs
create mode 100644 samples/cs/tutorial-voice-to-text/TutorialVoiceToText.csproj
create mode 100644 samples/cs/tutorial-voice-to-text/TutorialVoiceToText.sln
create mode 100644 samples/js/tutorial-chat-assistant/app.js
create mode 100644 samples/js/tutorial-chat-assistant/package.json
create mode 100644 samples/js/tutorial-document-summarizer/app.js
create mode 100644 samples/js/tutorial-document-summarizer/package.json
create mode 100644 samples/js/tutorial-tool-calling/app.js
create mode 100644 samples/js/tutorial-tool-calling/package.json
create mode 100644 samples/js/tutorial-voice-to-text/app.js
create mode 100644 samples/js/tutorial-voice-to-text/package.json
create mode 100644 samples/python/audio-transcription/Recording.mp3
create mode 100644 samples/python/audio-transcription/requirements.txt
create mode 100644 samples/python/audio-transcription/src/app.py
delete mode 100644 samples/python/functioncalling/README.md
delete mode 100644 samples/python/functioncalling/fl_tools.ipynb
delete mode 100644 samples/python/hello-foundry-local/README.md
delete mode 100644 samples/python/hello-foundry-local/src/app.py
create mode 100644 samples/python/langchain-integration/requirements.txt
create mode 100644 samples/python/langchain-integration/src/app.py
create mode 100644 samples/python/native-chat-completions/requirements.txt
create mode 100644 samples/python/native-chat-completions/src/app.py
delete mode 100644 samples/python/summarize/.vscode/launch.json
delete mode 100644 samples/python/summarize/README.md
delete mode 100644 samples/python/summarize/requirements.txt
delete mode 100644 samples/python/summarize/summarize.py
create mode 100644 samples/python/tool-calling/requirements.txt
create mode 100644 samples/python/tool-calling/src/app.py
create mode 100644 samples/python/tutorial-chat-assistant/requirements.txt
create mode 100644 samples/python/tutorial-chat-assistant/src/app.py
create mode 100644 samples/python/tutorial-document-summarizer/requirements.txt
create mode 100644 samples/python/tutorial-document-summarizer/src/app.py
create mode 100644 samples/python/tutorial-tool-calling/requirements.txt
create mode 100644 samples/python/tutorial-tool-calling/src/app.py
create mode 100644 samples/python/tutorial-voice-to-text/requirements.txt
create mode 100644 samples/python/tutorial-voice-to-text/src/app.py
create mode 100644 samples/python/web-server/requirements.txt
create mode 100644 samples/python/web-server/src/app.py
delete mode 100644 samples/rag/README.md
delete mode 100644 samples/rag/foundry-local-architecture.md
delete mode 100644 samples/rag/rag_foundrylocal_demo.ipynb
create mode 100644 samples/rust/audio-transcription-example/Recording.mp3
create mode 100644 samples/rust/tutorial-chat-assistant/Cargo.toml
create mode 100644 samples/rust/tutorial-chat-assistant/src/main.rs
create mode 100644 samples/rust/tutorial-document-summarizer/Cargo.toml
create mode 100644 samples/rust/tutorial-document-summarizer/src/main.rs
create mode 100644 samples/rust/tutorial-tool-calling/Cargo.toml
create mode 100644 samples/rust/tutorial-tool-calling/src/main.rs
create mode 100644 samples/rust/tutorial-voice-to-text/Cargo.toml
create mode 100644 samples/rust/tutorial-voice-to-text/src/main.rs
diff --git a/.github/workflows/samples-integration-test.yml b/.github/workflows/samples-integration-test.yml
new file mode 100644
index 00000000..c844ca12
--- /dev/null
+++ b/.github/workflows/samples-integration-test.yml
@@ -0,0 +1,260 @@
+name: Samples Build Check
+
+on:
+ pull_request:
+ paths:
+ - 'samples/**'
+ - '.github/workflows/samples-integration-test.yml'
+ push:
+ paths:
+ - 'samples/**'
+ - '.github/workflows/samples-integration-test.yml'
+ branches:
+ - main
+ workflow_dispatch:
+
+permissions:
+ contents: read
+
+jobs:
+ # ── Python Samples ──────────────────────────────────────────────────
+ python-samples:
+ runs-on: ${{ matrix.platform }}-latest
+ strategy:
+ fail-fast: false
+ matrix:
+ platform: [windows, macos]
+
+ steps:
+ - name: Checkout repository
+ uses: actions/checkout@v4
+ with:
+ clean: true
+
+ - name: Setup Python
+ uses: actions/setup-python@v5
+ with:
+ python-version: '3.12'
+
+ - name: Configure pip for Azure Artifacts
+ run: |
+ pip config set global.index-url https://pkgs.dev.azure.com/aiinfra/PublicPackages/_packaging/ORT-Nightly/pypi/simple/
+ pip config set global.extra-index-url https://pypi.org/simple/
+ pip config set global.pre true
+
+ - name: Build and install SDK from source
+ working-directory: sdk/python
+ shell: pwsh
+ run: |
+ python -m pip install build
+ echo '__version__ = "0.0.0-dev"' > src/version.py
+ python -m build --wheel --outdir dist/
+ $wheel = (Get-ChildItem dist/*.whl | Select-Object -First 1).FullName
+ pip install $wheel
+
+ - name: Install sample dependencies
+ shell: pwsh
+ run: |
+ Get-ChildItem samples/python/*/requirements.txt -ErrorAction SilentlyContinue | ForEach-Object {
+ Write-Host "Installing dependencies for $($_.Directory.Name)..."
+ pip install -r $_.FullName
+ }
+
+ - name: Syntax check Python samples
+ shell: pwsh
+ run: |
+ $failed = @()
+ $samples = Get-ChildItem samples/python/*/src/app.py -ErrorAction SilentlyContinue
+ foreach ($sample in $samples) {
+ $name = $sample.Directory.Parent.Name
+ Write-Host "=== Checking: $name ==="
+ python -m py_compile $sample.FullName
+ if ($LASTEXITCODE -ne 0) {
+ Write-Host "FAILED: $name"
+ $failed += $name
+ } else {
+ Write-Host "OK: $name"
+ }
+ }
+ if ($failed.Count -gt 0) {
+ Write-Error "Failed syntax checks: $($failed -join ', ')"
+ exit 1
+ }
+
+ # ── JavaScript Samples ──────────────────────────────────────────────
+ js-samples:
+ runs-on: ${{ matrix.platform }}-latest
+ strategy:
+ fail-fast: false
+ matrix:
+ platform: [windows, macos]
+
+ steps:
+ - name: Checkout repository
+ uses: actions/checkout@v4
+ with:
+ clean: true
+
+ - name: Setup Node.js
+ uses: actions/setup-node@v4
+ with:
+ node-version: '20.x'
+
+ - name: Setup .NET SDK for NuGet authentication
+ uses: actions/setup-dotnet@v5
+ with:
+ dotnet-version: '10.0.x'
+
+ - name: Build SDK from source
+ working-directory: sdk/js
+ run: |
+ npm install
+ npm run build
+ npm link
+
+ - name: Syntax check JS samples
+ shell: pwsh
+ run: |
+ $failed = @()
+ # Find all sample app.js files (either in root or src/)
+ $samples = @()
+ $samples += Get-ChildItem samples/js/*/app.js -ErrorAction SilentlyContinue
+ $samples += Get-ChildItem samples/js/*/src/app.js -ErrorAction SilentlyContinue
+ foreach ($sample in $samples) {
+ $dir = if ($sample.Directory.Name -eq 'src') { $sample.Directory.Parent } else { $sample.Directory }
+ $name = $dir.Name
+ Write-Host "=== Checking: $name ==="
+ # Link SDK and install dependencies
+ Push-Location $dir.FullName
+ npm link foundry-local-sdk 2>$null
+ if (Test-Path "package.json") { npm install 2>$null }
+ Pop-Location
+ # Syntax check
+ node --check $sample.FullName 2>&1
+ if ($LASTEXITCODE -ne 0) {
+ Write-Host "FAILED: $name"
+ $failed += $name
+ } else {
+ Write-Host "OK: $name"
+ }
+ }
+ if ($failed.Count -gt 0) {
+ Write-Error "Failed syntax checks: $($failed -join ', ')"
+ exit 1
+ }
+
+ # ── C# Samples ─────────────────────────────────────────────────────
+ cs-samples:
+ runs-on: ${{ matrix.platform }}-latest
+ strategy:
+ fail-fast: false
+ matrix:
+ platform: [windows, macos]
+
+ steps:
+ - name: Checkout repository
+ uses: actions/checkout@v4
+ with:
+ clean: true
+
+ - name: Setup .NET SDK
+ uses: actions/setup-dotnet@v5
+ with:
+ dotnet-version: |
+ 8.0.x
+ 10.0.x
+
+ - name: Build SDK from source
+ shell: pwsh
+ run: |
+ # Build cross-platform SDK package
+ # Note: /p:TreatWarningsAsErrors=false avoids failing on SDK doc warnings
+ dotnet pack sdk/cs/src/Microsoft.AI.Foundry.Local.csproj `
+ -o local-packages `
+ /p:Version=0.9.0-dev `
+ /p:IsPacking=true `
+ /p:TreatWarningsAsErrors=false `
+ --configuration Release
+
+ # Build WinML SDK package (Windows only)
+ if ($IsWindows) {
+ dotnet pack sdk/cs/src/Microsoft.AI.Foundry.Local.csproj `
+ -o local-packages `
+ /p:Version=0.9.0-dev-20260324 `
+ /p:UseWinML=true `
+ /p:IsPacking=true `
+ /p:TreatWarningsAsErrors=false `
+ --configuration Release
+ }
+
+ Write-Host "Local packages:"
+ Get-ChildItem local-packages/*.nupkg | ForEach-Object { Write-Host " $($_.Name)" }
+
+ - name: Build C# samples
+ shell: pwsh
+ run: |
+ $failed = @()
+ $projects = Get-ChildItem samples/cs -Recurse -Filter "*.csproj"
+ foreach ($proj in $projects) {
+ $name = $proj.BaseName
+ Write-Host "`n=== Building: $name ==="
+ dotnet build $proj.FullName --configuration Debug 2>&1
+ if ($LASTEXITCODE -ne 0) {
+ Write-Host "BUILD FAILED: $name"
+ $failed += $name
+ } else {
+ Write-Host "BUILD PASSED: $name"
+ }
+ }
+ if ($failed.Count -gt 0) {
+ Write-Error "Failed builds: $($failed -join ', ')"
+ exit 1
+ }
+
+ # ── Rust Samples ────────────────────────────────────────────────────
+ rust-samples:
+ runs-on: ${{ matrix.platform }}-latest
+ strategy:
+ fail-fast: false
+ matrix:
+ platform: [windows, macos]
+
+ steps:
+ - name: Checkout repository
+ uses: actions/checkout@v4
+ with:
+ clean: true
+
+ - name: Install Rust toolchain
+ uses: dtolnay/rust-toolchain@stable
+ with:
+ components: clippy
+
+ - name: Cache cargo dependencies
+ uses: Swatinem/rust-cache@v2
+ with:
+ workspaces: samples/rust -> target
+
+ - name: Use crates.io directly
+ shell: pwsh
+ run: |
+ # Remove crates-io redirect in SDK (points to Azure Artifacts)
+ $configPath = "sdk/rust/.cargo/config.toml"
+ if (Test-Path $configPath) {
+ Remove-Item $configPath
+ Write-Host "Removed sdk/rust/.cargo/config.toml"
+ }
+ # Remove crates-io redirect in samples
+ $configPath = "samples/rust/.cargo/config.toml"
+ if (Test-Path $configPath) {
+ Remove-Item $configPath
+ Write-Host "Removed samples/rust/.cargo/config.toml"
+ }
+
+ - name: Build Rust samples workspace
+ working-directory: samples/rust
+ run: cargo build --workspace
+
+ - name: Clippy check
+ working-directory: samples/rust
+ run: cargo clippy --workspace -- -D warnings
diff --git a/.gitignore b/.gitignore
index 8d088525..552012ec 100644
--- a/.gitignore
+++ b/.gitignore
@@ -33,3 +33,6 @@ bin/
obj/
/src/cs/samples/ConsoleClient/test.http
logs/
+
+# Local NuGet packages built from source
+local-packages/
diff --git a/samples/cs/GettingStarted/Directory.Packages.props b/samples/cs/Directory.Packages.props
similarity index 73%
rename from samples/cs/GettingStarted/Directory.Packages.props
rename to samples/cs/Directory.Packages.props
index 02984002..6f7b9eef 100644
--- a/samples/cs/GettingStarted/Directory.Packages.props
+++ b/samples/cs/Directory.Packages.props
@@ -7,7 +7,10 @@
+
+
+
diff --git a/samples/cs/GettingStarted/README.md b/samples/cs/GettingStarted/README.md
deleted file mode 100644
index afe6e88d..00000000
--- a/samples/cs/GettingStarted/README.md
+++ /dev/null
@@ -1,61 +0,0 @@
-# 🚀 Getting started with the Foundry Local C# SDK
-
-There are two NuGet packages for the Foundry Local SDK - a WinML and a cross-platform package - that have *exactly* the same API surface but are optimised for different platforms:
-
-- **Windows**: Uses the `Microsoft.AI.Foundry.Local.WinML` package that is specific to Windows applications. The WinML package uses Windows Machine Learning to deliver optimal performance and user experience on Windows devices.
-- **Cross-Platform**: Use the `Microsoft.AI.Foundry.Local` package that can be used for cross-platform applications (Windows, Linux, macOS).
-
-> [!TIP]
-> Whilst you can use either package on Windows, we recommend using the WinML package for Windows applications to take advantage of the Windows ML framework for optimal performance and user experience. Your end users will benefit with:
-> - a wider range of hardware acceleration options that are automatically managed by Windows ML.
-> - a smaller application package size because downloading hardware-specific libraries occurs at application runtime rather than bundled with your application.
-
-Both the WinML and cross-platform packages provide the same APIs, so you can easily switch between the two packages if you need to target multiple platforms. The samples include the following projects:
-
-- **HelloFoundryLocalSdk**: A simple console application that initializes the Foundry Local SDK, downloads a model, loads it and does chat completions.
-- **FoundryLocalWebServer**: A simple console application that shows how to set up a local OpenAI-compliant web server using the Foundry Local SDK.
-- **AudioTranscriptionExample**: A simple console application that demonstrates how to use the Foundry Local SDK for audio transcription tasks.
-- **ModelManagementExample**: A simple console application that demonstrates how to manage models - such as variant selection and updates - using the Foundry Local SDK.
-- **ToolCallingFoundryLocalSdk**: A simple console application that initializes the Foundry Local SDK, downloads a model, loads it and does tool calling with chat completions.
-- **ToolCallingFoundryLocalWebServer**: A simple console application that shows how to set up a local OpenAI-compliant web server with tool calling using the Foundry Local SDK.
-
-## Running the samples
-
-1. Clone the Foundry Local repository from GitHub.
- ```bash
- git clone https://github.com/microsoft/Foundry-Local.git
- ```
-2. Open and run the samples.
-
- **Windows:**
- 1. Open the `Foundry-Local/samples/cs/GettingStarted/windows/FoundrySamplesWinML.sln` solution in Visual Studio or your preferred IDE.
- 1. If you're using Visual Studio, run any of the sample projects (e.g., `HelloFoundryLocalSdk`) by selecting the project in the Solution Explorer and selecting the **Start** button (or pressing **F5**).
-
- Alternatively, you can run the projects using the .NET CLI. For x64 (update the `` as needed):
- ```bash
- cd Foundry-Local/samples/cs/GettingStarted/windows
- dotnet run --project /.csproj -r:win-x64
- ```
- or for ARM64:
- ```bash
- ```bash
- cd Foundry-Local/samples/cs/GettingStarted/windows
- dotnet run --project /.csproj -r:win-arm64
- ```
-
-
- **macOS or Linux:**
- 1. Open the `Foundry-Local/samples/cs/GettingStarted/cross-platform/FoundrySamplesXPlatform.sln` solution in Visual Studio Code or your preferred IDE.
- 1. Run the project using the .NET CLI (update the `` and `` as needed):
- ```bash
- cd Foundry-Local/samples/cs/GettingStarted/cross-platform
- dotnet run --project /.csproj -r:
- ```
- For example, to run the `HelloFoundryLocalSdk` project on macOS (Apple Silicon), use the following command:
-
- ```bash
- cd Foundry-Local/samples/cs/GettingStarted/cross-platform
- dotnet run --project HelloFoundryLocalSdk/HelloFoundryLocalSdk.csproj -r:osx-arm64
- ```
-
-
diff --git a/samples/cs/GettingStarted/cross-platform/AudioTranscriptionExample/AudioTranscriptionExample.csproj b/samples/cs/GettingStarted/cross-platform/AudioTranscriptionExample/AudioTranscriptionExample.csproj
deleted file mode 100644
index 02eefb31..00000000
--- a/samples/cs/GettingStarted/cross-platform/AudioTranscriptionExample/AudioTranscriptionExample.csproj
+++ /dev/null
@@ -1,39 +0,0 @@
-
-
-
- Exe
- net9.0
- enable
- enable
-
-
-
- $(NETCoreSdkRuntimeIdentifier)
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
- PreserveNewest
-
-
-
-
-
diff --git a/samples/cs/GettingStarted/cross-platform/FoundryLocalWebServer/FoundryLocalWebServer.csproj b/samples/cs/GettingStarted/cross-platform/FoundryLocalWebServer/FoundryLocalWebServer.csproj
deleted file mode 100644
index 672e8726..00000000
--- a/samples/cs/GettingStarted/cross-platform/FoundryLocalWebServer/FoundryLocalWebServer.csproj
+++ /dev/null
@@ -1,33 +0,0 @@
-
-
-
- Exe
- net9.0
- enable
- enable
-
-
-
- $(NETCoreSdkRuntimeIdentifier)
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
diff --git a/samples/cs/GettingStarted/cross-platform/FoundrySamplesXPlatform.sln b/samples/cs/GettingStarted/cross-platform/FoundrySamplesXPlatform.sln
deleted file mode 100644
index a51c62d6..00000000
--- a/samples/cs/GettingStarted/cross-platform/FoundrySamplesXPlatform.sln
+++ /dev/null
@@ -1,53 +0,0 @@
-
-Microsoft Visual Studio Solution File, Format Version 12.00
-# Visual Studio Version 17
-VisualStudioVersion = 17.14.36705.20
-MinimumVisualStudioVersion = 10.0.40219.1
-Project("{FAE04EC0-301F-11D3-BF4B-00C04F79EFBC}") = "HelloFoundryLocalSdk", "HelloFoundryLocalSdk\HelloFoundryLocalSdk.csproj", "{785AAE8A-8CD6-4916-B858-29B8A7EF8FF2}"
-EndProject
-Project("{FAE04EC0-301F-11D3-BF4B-00C04F79EFBC}") = "ToolCallingFoundryLocalSdk", "ToolCallingFoundryLocalSdk\ToolCallingFoundryLocalSdk.csproj", "{2F99B88E-BE58-4ED6-A71E-60B6EE955D1B}"
-EndProject
-Project("{2150E333-8FDC-42A3-9474-1A3956D46DE8}") = "build", "build", "{8EC462FD-D22E-90A8-E5CE-7E832BA40C5D}"
- ProjectSection(SolutionItems) = preProject
- ..\Directory.Packages.props = ..\Directory.Packages.props
- ..\nuget.config = ..\nuget.config
- EndProjectSection
-EndProject
-Project("{FAE04EC0-301F-11D3-BF4B-00C04F79EFBC}") = "FoundryLocalWebServer", "FoundryLocalWebServer\FoundryLocalWebServer.csproj", "{D1D6C453-3088-4D8D-B320-24D718601C26}"
-EndProject
-Project("{FAE04EC0-301F-11D3-BF4B-00C04F79EFBC}") = "ToolCallingFoundryLocalWebServer", "ToolCallingFoundryLocalWebServer\ToolCallingFoundryLocalWebServer.csproj", "{B59762E0-B699-4F80-B2B6-8BC5751A4620}"
-EndProject
-Project("{FAE04EC0-301F-11D3-BF4B-00C04F79EFBC}") = "AudioTranscriptionExample", "AudioTranscriptionExample\AudioTranscriptionExample.csproj", "{2FAD8210-8AEB-4063-9C61-57B7AD26772D}"
-EndProject
-Project("{FAE04EC0-301F-11D3-BF4B-00C04F79EFBC}") = "ModelManagementExample", "ModelManagementExample\ModelManagementExample.csproj", "{AAD0233C-9FDD-46A7-9428-2F72BC76D38E}"
-EndProject
-Global
- GlobalSection(SolutionConfigurationPlatforms) = preSolution
- Debug|Any CPU = Debug|Any CPU
- Release|Any CPU = Release|Any CPU
- EndGlobalSection
- GlobalSection(ProjectConfigurationPlatforms) = postSolution
- {785AAE8A-8CD6-4916-B858-29B8A7EF8FF2}.Debug|Any CPU.ActiveCfg = Debug|Any CPU
- {785AAE8A-8CD6-4916-B858-29B8A7EF8FF2}.Debug|Any CPU.Build.0 = Debug|Any CPU
- {785AAE8A-8CD6-4916-B858-29B8A7EF8FF2}.Release|Any CPU.ActiveCfg = Release|Any CPU
- {785AAE8A-8CD6-4916-B858-29B8A7EF8FF2}.Release|Any CPU.Build.0 = Release|Any CPU
- {D1D6C453-3088-4D8D-B320-24D718601C26}.Debug|Any CPU.ActiveCfg = Debug|Any CPU
- {D1D6C453-3088-4D8D-B320-24D718601C26}.Debug|Any CPU.Build.0 = Debug|Any CPU
- {D1D6C453-3088-4D8D-B320-24D718601C26}.Release|Any CPU.ActiveCfg = Release|Any CPU
- {D1D6C453-3088-4D8D-B320-24D718601C26}.Release|Any CPU.Build.0 = Release|Any CPU
- {2FAD8210-8AEB-4063-9C61-57B7AD26772D}.Debug|Any CPU.ActiveCfg = Debug|Any CPU
- {2FAD8210-8AEB-4063-9C61-57B7AD26772D}.Debug|Any CPU.Build.0 = Debug|Any CPU
- {2FAD8210-8AEB-4063-9C61-57B7AD26772D}.Release|Any CPU.ActiveCfg = Release|Any CPU
- {2FAD8210-8AEB-4063-9C61-57B7AD26772D}.Release|Any CPU.Build.0 = Release|Any CPU
- {AAD0233C-9FDD-46A7-9428-2F72BC76D38E}.Debug|Any CPU.ActiveCfg = Debug|Any CPU
- {AAD0233C-9FDD-46A7-9428-2F72BC76D38E}.Debug|Any CPU.Build.0 = Debug|Any CPU
- {AAD0233C-9FDD-46A7-9428-2F72BC76D38E}.Release|Any CPU.ActiveCfg = Release|Any CPU
- {AAD0233C-9FDD-46A7-9428-2F72BC76D38E}.Release|Any CPU.Build.0 = Release|Any CPU
- EndGlobalSection
- GlobalSection(SolutionProperties) = preSolution
- HideSolutionNode = FALSE
- EndGlobalSection
- GlobalSection(ExtensibilityGlobals) = postSolution
- SolutionGuid = {9FC1F302-B28C-4CAB-8ABA-24FA9EBBED6F}
- EndGlobalSection
-EndGlobal
diff --git a/samples/cs/GettingStarted/cross-platform/HelloFoundryLocalSdk/HelloFoundryLocalSdk.csproj b/samples/cs/GettingStarted/cross-platform/HelloFoundryLocalSdk/HelloFoundryLocalSdk.csproj
deleted file mode 100644
index bb8df514..00000000
--- a/samples/cs/GettingStarted/cross-platform/HelloFoundryLocalSdk/HelloFoundryLocalSdk.csproj
+++ /dev/null
@@ -1,32 +0,0 @@
-
-
-
- Exe
- net9.0
- enable
- enable
-
-
-
- $(NETCoreSdkRuntimeIdentifier)
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
diff --git a/samples/cs/GettingStarted/cross-platform/ModelManagementExample/ModelManagementExample.csproj b/samples/cs/GettingStarted/cross-platform/ModelManagementExample/ModelManagementExample.csproj
deleted file mode 100644
index 70af7023..00000000
--- a/samples/cs/GettingStarted/cross-platform/ModelManagementExample/ModelManagementExample.csproj
+++ /dev/null
@@ -1,33 +0,0 @@
-
-
-
- Exe
- net9.0
- enable
- enable
-
-
-
- $(NETCoreSdkRuntimeIdentifier)
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
diff --git a/samples/cs/GettingStarted/cross-platform/ToolCallingFoundryLocalSdk/ToolCallingFoundryLocalSdk.csproj b/samples/cs/GettingStarted/cross-platform/ToolCallingFoundryLocalSdk/ToolCallingFoundryLocalSdk.csproj
deleted file mode 100644
index aa2b5400..00000000
--- a/samples/cs/GettingStarted/cross-platform/ToolCallingFoundryLocalSdk/ToolCallingFoundryLocalSdk.csproj
+++ /dev/null
@@ -1,31 +0,0 @@
-
-
-
- Exe
- net9.0
- enable
- enable
-
-
-
- $(NETCoreSdkRuntimeIdentifier)
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
diff --git a/samples/cs/GettingStarted/cross-platform/ToolCallingFoundryLocalWebServer/ToolCallingFoundryLocalWebServer.csproj b/samples/cs/GettingStarted/cross-platform/ToolCallingFoundryLocalWebServer/ToolCallingFoundryLocalWebServer.csproj
deleted file mode 100644
index dcaeb80d..00000000
--- a/samples/cs/GettingStarted/cross-platform/ToolCallingFoundryLocalWebServer/ToolCallingFoundryLocalWebServer.csproj
+++ /dev/null
@@ -1,32 +0,0 @@
-
-
-
- Exe
- net9.0
- enable
- enable
-
-
-
- $(NETCoreSdkRuntimeIdentifier)
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
diff --git a/samples/cs/GettingStarted/nuget.config b/samples/cs/GettingStarted/nuget.config
deleted file mode 100644
index b5c4e511..00000000
--- a/samples/cs/GettingStarted/nuget.config
+++ /dev/null
@@ -1,7 +0,0 @@
-
-
-
-
-
-
-
\ No newline at end of file
diff --git a/samples/cs/GettingStarted/windows/AudioTranscriptionExample/AudioTranscriptionExample.csproj b/samples/cs/GettingStarted/windows/AudioTranscriptionExample/AudioTranscriptionExample.csproj
deleted file mode 100644
index 98219697..00000000
--- a/samples/cs/GettingStarted/windows/AudioTranscriptionExample/AudioTranscriptionExample.csproj
+++ /dev/null
@@ -1,36 +0,0 @@
-
-
-
- Exe
- enable
- enable
-
- net9.0-windows10.0.26100
- false
- ARM64;x64
- None
- false
-
-
-
- $(NETCoreSdkRuntimeIdentifier)
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
- PreserveNewest
-
-
-
-
\ No newline at end of file
diff --git a/samples/cs/GettingStarted/windows/FoundryLocalWebServer/FoundryLocalWebServer.csproj b/samples/cs/GettingStarted/windows/FoundryLocalWebServer/FoundryLocalWebServer.csproj
deleted file mode 100644
index f08a2b4a..00000000
--- a/samples/cs/GettingStarted/windows/FoundryLocalWebServer/FoundryLocalWebServer.csproj
+++ /dev/null
@@ -1,30 +0,0 @@
-
-
-
- Exe
- enable
- enable
-
- net9.0-windows10.0.26100
- false
- ARM64;x64
- None
- false
-
-
-
- $(NETCoreSdkRuntimeIdentifier)
-
-
-
-
-
-
-
-
-
-
-
-
-
-
\ No newline at end of file
diff --git a/samples/cs/GettingStarted/windows/FoundrySamplesWinML.sln b/samples/cs/GettingStarted/windows/FoundrySamplesWinML.sln
deleted file mode 100644
index 10a0d851..00000000
--- a/samples/cs/GettingStarted/windows/FoundrySamplesWinML.sln
+++ /dev/null
@@ -1,71 +0,0 @@
-
-Microsoft Visual Studio Solution File, Format Version 12.00
-# Visual Studio Version 17
-VisualStudioVersion = 17.14.36705.20
-MinimumVisualStudioVersion = 10.0.40219.1
-Project("{FAE04EC0-301F-11D3-BF4B-00C04F79EFBC}") = "HelloFoundryLocalSdk", "HelloFoundryLocalSdk\HelloFoundryLocalSdk.csproj", "{72ABF21E-2BFD-412A-9039-A594B392F00C}"
-EndProject
-Project("{FAE04EC0-301F-11D3-BF4B-00C04F79EFBC}") = "ToolCallingFoundryLocalSdk", "ToolCallingFoundryLocalSdk\ToolCallingFoundryLocalSdk.csproj", "{93C21DF0-17D5-4927-9507-C10A79359E7D}"
-EndProject
-Project("{FAE04EC0-301F-11D3-BF4B-00C04F79EFBC}") = "FoundryLocalWebServer", "FoundryLocalWebServer\FoundryLocalWebServer.csproj", "{77026F3A-25E0-40AB-B941-2A6252E13A35}"
-EndProject
-Project("{FAE04EC0-301F-11D3-BF4B-00C04F79EFBC}") = "ToolCallingFoundryLocalWebServer", "ToolCallingFoundryLocalWebServer\ToolCallingFoundryLocalWebServer.csproj", "{5A8536E2-04B6-4F06-80B1-1018069DF73F}"
-EndProject
-Project("{FAE04EC0-301F-11D3-BF4B-00C04F79EFBC}") = "AudioTranscriptionExample", "AudioTranscriptionExample\AudioTranscriptionExample.csproj", "{80F60523-40E1-4743-A256-974B21A9C6AB}"
-EndProject
-Project("{2150E333-8FDC-42A3-9474-1A3956D46DE8}") = "build", "build", "{8EC462FD-D22E-90A8-E5CE-7E832BA40C5D}"
- ProjectSection(SolutionItems) = preProject
- ..\Directory.Packages.props = ..\Directory.Packages.props
- ..\nuget.config = ..\nuget.config
- EndProjectSection
-EndProject
-Project("{FAE04EC0-301F-11D3-BF4B-00C04F79EFBC}") = "ModelManagementExample", "ModelManagementExample\ModelManagementExample.csproj", "{6BBA4217-6798-4629-AF27-6526FCC5FA5B}"
-EndProject
-Global
- GlobalSection(SolutionConfigurationPlatforms) = preSolution
- Debug|ARM64 = Debug|ARM64
- Debug|x64 = Debug|x64
- Release|ARM64 = Release|ARM64
- Release|x64 = Release|x64
- EndGlobalSection
- GlobalSection(ProjectConfigurationPlatforms) = postSolution
- {72ABF21E-2BFD-412A-9039-A594B392F00C}.Debug|ARM64.ActiveCfg = Debug|ARM64
- {72ABF21E-2BFD-412A-9039-A594B392F00C}.Debug|ARM64.Build.0 = Debug|ARM64
- {72ABF21E-2BFD-412A-9039-A594B392F00C}.Debug|x64.ActiveCfg = Debug|x64
- {72ABF21E-2BFD-412A-9039-A594B392F00C}.Debug|x64.Build.0 = Debug|x64
- {72ABF21E-2BFD-412A-9039-A594B392F00C}.Release|ARM64.ActiveCfg = Release|ARM64
- {72ABF21E-2BFD-412A-9039-A594B392F00C}.Release|ARM64.Build.0 = Release|ARM64
- {72ABF21E-2BFD-412A-9039-A594B392F00C}.Release|x64.ActiveCfg = Release|x64
- {72ABF21E-2BFD-412A-9039-A594B392F00C}.Release|x64.Build.0 = Release|x64
- {77026F3A-25E0-40AB-B941-2A6252E13A35}.Debug|ARM64.ActiveCfg = Debug|ARM64
- {77026F3A-25E0-40AB-B941-2A6252E13A35}.Debug|ARM64.Build.0 = Debug|ARM64
- {77026F3A-25E0-40AB-B941-2A6252E13A35}.Debug|x64.ActiveCfg = Debug|x64
- {77026F3A-25E0-40AB-B941-2A6252E13A35}.Debug|x64.Build.0 = Debug|x64
- {77026F3A-25E0-40AB-B941-2A6252E13A35}.Release|ARM64.ActiveCfg = Release|ARM64
- {77026F3A-25E0-40AB-B941-2A6252E13A35}.Release|ARM64.Build.0 = Release|ARM64
- {77026F3A-25E0-40AB-B941-2A6252E13A35}.Release|x64.ActiveCfg = Release|x64
- {77026F3A-25E0-40AB-B941-2A6252E13A35}.Release|x64.Build.0 = Release|x64
- {80F60523-40E1-4743-A256-974B21A9C6AB}.Debug|ARM64.ActiveCfg = Debug|ARM64
- {80F60523-40E1-4743-A256-974B21A9C6AB}.Debug|ARM64.Build.0 = Debug|ARM64
- {80F60523-40E1-4743-A256-974B21A9C6AB}.Debug|x64.ActiveCfg = Debug|x64
- {80F60523-40E1-4743-A256-974B21A9C6AB}.Debug|x64.Build.0 = Debug|x64
- {80F60523-40E1-4743-A256-974B21A9C6AB}.Release|ARM64.ActiveCfg = Release|ARM64
- {80F60523-40E1-4743-A256-974B21A9C6AB}.Release|ARM64.Build.0 = Release|ARM64
- {80F60523-40E1-4743-A256-974B21A9C6AB}.Release|x64.ActiveCfg = Release|x64
- {80F60523-40E1-4743-A256-974B21A9C6AB}.Release|x64.Build.0 = Release|x64
- {6BBA4217-6798-4629-AF27-6526FCC5FA5B}.Debug|ARM64.ActiveCfg = Debug|Any CPU
- {6BBA4217-6798-4629-AF27-6526FCC5FA5B}.Debug|ARM64.Build.0 = Debug|Any CPU
- {6BBA4217-6798-4629-AF27-6526FCC5FA5B}.Debug|x64.ActiveCfg = Debug|x64
- {6BBA4217-6798-4629-AF27-6526FCC5FA5B}.Debug|x64.Build.0 = Debug|x64
- {6BBA4217-6798-4629-AF27-6526FCC5FA5B}.Release|ARM64.ActiveCfg = Release|Any CPU
- {6BBA4217-6798-4629-AF27-6526FCC5FA5B}.Release|ARM64.Build.0 = Release|Any CPU
- {6BBA4217-6798-4629-AF27-6526FCC5FA5B}.Release|x64.ActiveCfg = Release|x64
- {6BBA4217-6798-4629-AF27-6526FCC5FA5B}.Release|x64.Build.0 = Release|x64
- EndGlobalSection
- GlobalSection(SolutionProperties) = preSolution
- HideSolutionNode = FALSE
- EndGlobalSection
- GlobalSection(ExtensibilityGlobals) = postSolution
- SolutionGuid = {17462B72-2BD9-446A-8E57-E313251686D9}
- EndGlobalSection
-EndGlobal
diff --git a/samples/cs/GettingStarted/windows/HelloFoundryLocalSdk/HelloFoundryLocalSdk.csproj b/samples/cs/GettingStarted/windows/HelloFoundryLocalSdk/HelloFoundryLocalSdk.csproj
deleted file mode 100644
index 23d2ee91..00000000
--- a/samples/cs/GettingStarted/windows/HelloFoundryLocalSdk/HelloFoundryLocalSdk.csproj
+++ /dev/null
@@ -1,30 +0,0 @@
-
-
-
- Exe
- enable
- enable
-
- net9.0-windows10.0.26100
- false
- ARM64;x64
- None
- false
-
-
-
- $(NETCoreSdkRuntimeIdentifier)
-
-
-
-
-
-
-
-
-
-
-
-
-
-
\ No newline at end of file
diff --git a/samples/cs/GettingStarted/windows/ModelManagementExample/ModelManagementExample.csproj b/samples/cs/GettingStarted/windows/ModelManagementExample/ModelManagementExample.csproj
deleted file mode 100644
index bc4afe67..00000000
--- a/samples/cs/GettingStarted/windows/ModelManagementExample/ModelManagementExample.csproj
+++ /dev/null
@@ -1,30 +0,0 @@
-
-
-
- Exe
- enable
- enable
-
- net9.0-windows10.0.26100
- false
- ARM64;x64
- None
- false
-
-
-
- $(NETCoreSdkRuntimeIdentifier)
-
-
-
-
-
-
-
-
-
-
-
-
-
-
\ No newline at end of file
diff --git a/samples/cs/GettingStarted/windows/ToolCallingFoundryLocalSdk/ToolCallingFoundryLocalSdk.csproj b/samples/cs/GettingStarted/windows/ToolCallingFoundryLocalSdk/ToolCallingFoundryLocalSdk.csproj
deleted file mode 100644
index de209c13..00000000
--- a/samples/cs/GettingStarted/windows/ToolCallingFoundryLocalSdk/ToolCallingFoundryLocalSdk.csproj
+++ /dev/null
@@ -1,30 +0,0 @@
-
-
-
- Exe
- enable
- enable
-
- net9.0-windows10.0.26100
- false
- ARM64;x64
- None
- false
-
-
-
- $(NETCoreSdkRuntimeIdentifier)
-
-
-
-
-
-
-
-
-
-
-
-
-
-
\ No newline at end of file
diff --git a/samples/cs/GettingStarted/windows/ToolCallingFoundryLocalWebServer/ToolCallingFoundryLocalWebServer.csproj b/samples/cs/GettingStarted/windows/ToolCallingFoundryLocalWebServer/ToolCallingFoundryLocalWebServer.csproj
deleted file mode 100644
index 9101d778..00000000
--- a/samples/cs/GettingStarted/windows/ToolCallingFoundryLocalWebServer/ToolCallingFoundryLocalWebServer.csproj
+++ /dev/null
@@ -1,30 +0,0 @@
-
-
-
- Exe
- enable
- enable
-
- net9.0-windows10.0.26100
- false
- ARM64;x64
- None
- false
-
-
-
- $(NETCoreSdkRuntimeIdentifier)
-
-
-
-
-
-
-
-
-
-
-
-
-
-
\ No newline at end of file
diff --git a/samples/cs/README.md b/samples/cs/README.md
new file mode 100644
index 00000000..1847bb8e
--- /dev/null
+++ b/samples/cs/README.md
@@ -0,0 +1,43 @@
+# 🚀 Foundry Local C# Samples
+
+These samples demonstrate how to use the Foundry Local C# SDK. Each sample uses a **unified project file** that automatically detects your operating system and selects the optimal NuGet package:
+
+- **Windows**: Uses `Microsoft.AI.Foundry.Local.WinML` for hardware acceleration via Windows ML.
+- **macOS / Linux**: Uses `Microsoft.AI.Foundry.Local` for cross-platform support.
+
+Both packages provide the same APIs, so the same source code works on all platforms.
+
+## Samples
+
+| Sample | Description |
+|---|---|
+| [native-chat-completions](native-chat-completions/) | Initialize the SDK, download a model, and run chat completions. |
+| [audio-transcription-example](audio-transcription-example/) | Transcribe audio files using the Foundry Local SDK. |
+| [foundry-local-web-server](foundry-local-web-server/) | Set up a local OpenAI-compliant web server. |
+| [tool-calling-foundry-local-sdk](tool-calling-foundry-local-sdk/) | Use tool calling with native chat completions. |
+| [tool-calling-foundry-local-web-server](tool-calling-foundry-local-web-server/) | Use tool calling with the local web server. |
+| [model-management-example](model-management-example/) | Manage models, variant selection, and updates. |
+| [tutorial-chat-assistant](tutorial-chat-assistant/) | Build an interactive chat assistant (tutorial). |
+| [tutorial-document-summarizer](tutorial-document-summarizer/) | Summarize documents with AI (tutorial). |
+| [tutorial-tool-calling](tutorial-tool-calling/) | Create a tool-calling assistant (tutorial). |
+| [tutorial-voice-to-text](tutorial-voice-to-text/) | Transcribe and summarize audio (tutorial). |
+
+## Running a sample
+
+1. Clone the repository:
+ ```bash
+ git clone https://github.com/microsoft/Foundry-Local.git
+ cd Foundry-Local/samples/cs
+ ```
+
+2. Open and run a sample:
+ ```bash
+ cd native-chat-completions
+ dotnet run
+ ```
+
+ The unified project file automatically selects the correct SDK package for your platform.
+
+> [!TIP]
+> On Windows, we recommend using the WinML package (selected automatically) for optimal performance. Your users benefit from a wider range of hardware acceleration options and a smaller application package size.
+
diff --git a/samples/cs/GettingStarted/src/Shared/Utils.cs b/samples/cs/Shared/Utils.cs
similarity index 100%
rename from samples/cs/GettingStarted/src/Shared/Utils.cs
rename to samples/cs/Shared/Utils.cs
diff --git a/samples/cs/audio-transcription-example/AudioTranscriptionExample.csproj b/samples/cs/audio-transcription-example/AudioTranscriptionExample.csproj
new file mode 100644
index 00000000..bd42e38b
--- /dev/null
+++ b/samples/cs/audio-transcription-example/AudioTranscriptionExample.csproj
@@ -0,0 +1,55 @@
+
+
+
+ Exe
+ enable
+ enable
+
+
+
+
+ net9.0-windows10.0.26100
+ false
+ ARM64;x64
+ None
+ false
+
+
+
+
+ net9.0
+
+
+
+ $(NETCoreSdkRuntimeIdentifier)
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+ PreserveNewest
+
+
+
+
+
+
+
+
+
diff --git a/samples/cs/audio-transcription-example/AudioTranscriptionExample.sln b/samples/cs/audio-transcription-example/AudioTranscriptionExample.sln
new file mode 100644
index 00000000..46fb73d9
--- /dev/null
+++ b/samples/cs/audio-transcription-example/AudioTranscriptionExample.sln
@@ -0,0 +1,34 @@
+
+Microsoft Visual Studio Solution File, Format Version 12.00
+# Visual Studio Version 17
+VisualStudioVersion = 17.0.31903.59
+MinimumVisualStudioVersion = 10.0.40219.1
+Project("{FAE04EC0-301F-11D3-BF4B-00C04F79EFBC}") = "AudioTranscriptionExample", "AudioTranscriptionExample.csproj", "{11616852-BB4F-4B60-9FAC-D94E2688BB30}"
+EndProject
+Global
+ GlobalSection(SolutionConfigurationPlatforms) = preSolution
+ Debug|Any CPU = Debug|Any CPU
+ Debug|x64 = Debug|x64
+ Debug|x86 = Debug|x86
+ Release|Any CPU = Release|Any CPU
+ Release|x64 = Release|x64
+ Release|x86 = Release|x86
+ EndGlobalSection
+ GlobalSection(ProjectConfigurationPlatforms) = postSolution
+ {11616852-BB4F-4B60-9FAC-D94E2688BB30}.Debug|Any CPU.ActiveCfg = Debug|ARM64
+ {11616852-BB4F-4B60-9FAC-D94E2688BB30}.Debug|Any CPU.Build.0 = Debug|ARM64
+ {11616852-BB4F-4B60-9FAC-D94E2688BB30}.Debug|x64.ActiveCfg = Debug|x64
+ {11616852-BB4F-4B60-9FAC-D94E2688BB30}.Debug|x64.Build.0 = Debug|x64
+ {11616852-BB4F-4B60-9FAC-D94E2688BB30}.Debug|x86.ActiveCfg = Debug|ARM64
+ {11616852-BB4F-4B60-9FAC-D94E2688BB30}.Debug|x86.Build.0 = Debug|ARM64
+ {11616852-BB4F-4B60-9FAC-D94E2688BB30}.Release|Any CPU.ActiveCfg = Release|ARM64
+ {11616852-BB4F-4B60-9FAC-D94E2688BB30}.Release|Any CPU.Build.0 = Release|ARM64
+ {11616852-BB4F-4B60-9FAC-D94E2688BB30}.Release|x64.ActiveCfg = Release|x64
+ {11616852-BB4F-4B60-9FAC-D94E2688BB30}.Release|x64.Build.0 = Release|x64
+ {11616852-BB4F-4B60-9FAC-D94E2688BB30}.Release|x86.ActiveCfg = Release|ARM64
+ {11616852-BB4F-4B60-9FAC-D94E2688BB30}.Release|x86.Build.0 = Release|ARM64
+ EndGlobalSection
+ GlobalSection(SolutionProperties) = preSolution
+ HideSolutionNode = FALSE
+ EndGlobalSection
+EndGlobal
diff --git a/samples/cs/GettingStarted/src/AudioTranscriptionExample/Program.cs b/samples/cs/audio-transcription-example/Program.cs
similarity index 78%
rename from samples/cs/GettingStarted/src/AudioTranscriptionExample/Program.cs
rename to samples/cs/audio-transcription-example/Program.cs
index be1db5db..b78e13d2 100644
--- a/samples/cs/GettingStarted/src/AudioTranscriptionExample/Program.cs
+++ b/samples/cs/audio-transcription-example/Program.cs
@@ -1,5 +1,9 @@
-using Microsoft.AI.Foundry.Local;
+//
+//
+using Microsoft.AI.Foundry.Local;
+//
+//
var config = new Configuration
{
AppName = "foundry_local_samples",
@@ -17,8 +21,10 @@
// Download is only required again if a new version of the EP is released.
// For cross platform builds there is no dynamic EP download and this will return immediately.
await Utils.RunWithSpinner("Registering execution providers", mgr.EnsureEpsDownloadedAsync());
+//
+//
// Get the model catalog
var catalog = await mgr.GetCatalogAsync();
@@ -44,15 +50,16 @@ await model.DownloadAsync(progress =>
Console.Write($"Loading model {model.Id}...");
await model.LoadAsync();
Console.WriteLine("done.");
+//
-// Get a chat client
+//
+// Get an audio client
var audioClient = await model.GetAudioClientAsync();
-
// Get a transcription with streaming outputs
-Console.WriteLine("Transcribing audio with streaming output:");
-var audioFile = Path.Combine(AppContext.BaseDirectory, "Recording.mp3");
+var audioFile = args.Length > 0 ? args[0] : Path.Combine(AppContext.BaseDirectory, "Recording.mp3");
+Console.WriteLine($"Transcribing audio with streaming output: {Path.GetFileName(audioFile)}");
var response = audioClient.TranscribeAudioStreamingAsync(audioFile, CancellationToken.None);
await foreach (var chunk in response)
{
@@ -61,7 +68,11 @@ await model.DownloadAsync(progress =>
}
Console.WriteLine();
+//
+//
// Tidy up - unload the model
-await model.UnloadAsync();
\ No newline at end of file
+await model.UnloadAsync();
+//
+//
\ No newline at end of file
diff --git a/samples/cs/GettingStarted/src/AudioTranscriptionExample/Recording.mp3 b/samples/cs/audio-transcription-example/Recording.mp3
similarity index 100%
rename from samples/cs/GettingStarted/src/AudioTranscriptionExample/Recording.mp3
rename to samples/cs/audio-transcription-example/Recording.mp3
diff --git a/samples/cs/foundry-local-web-server/FoundryLocalWebServer.csproj b/samples/cs/foundry-local-web-server/FoundryLocalWebServer.csproj
new file mode 100644
index 00000000..fe890be2
--- /dev/null
+++ b/samples/cs/foundry-local-web-server/FoundryLocalWebServer.csproj
@@ -0,0 +1,52 @@
+
+
+
+ Exe
+ enable
+ enable
+
+
+
+
+ net9.0-windows10.0.26100
+ false
+ ARM64;x64
+ None
+ false
+
+
+
+
+ net9.0
+
+
+
+ $(NETCoreSdkRuntimeIdentifier)
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
diff --git a/samples/cs/foundry-local-web-server/FoundryLocalWebServer.sln b/samples/cs/foundry-local-web-server/FoundryLocalWebServer.sln
new file mode 100644
index 00000000..91d7e953
--- /dev/null
+++ b/samples/cs/foundry-local-web-server/FoundryLocalWebServer.sln
@@ -0,0 +1,34 @@
+
+Microsoft Visual Studio Solution File, Format Version 12.00
+# Visual Studio Version 17
+VisualStudioVersion = 17.0.31903.59
+MinimumVisualStudioVersion = 10.0.40219.1
+Project("{FAE04EC0-301F-11D3-BF4B-00C04F79EFBC}") = "FoundryLocalWebServer", "FoundryLocalWebServer.csproj", "{2DEC84E5-8530-45AF-B26D-EC78A6A7D6E7}"
+EndProject
+Global
+ GlobalSection(SolutionConfigurationPlatforms) = preSolution
+ Debug|Any CPU = Debug|Any CPU
+ Debug|x64 = Debug|x64
+ Debug|x86 = Debug|x86
+ Release|Any CPU = Release|Any CPU
+ Release|x64 = Release|x64
+ Release|x86 = Release|x86
+ EndGlobalSection
+ GlobalSection(ProjectConfigurationPlatforms) = postSolution
+ {2DEC84E5-8530-45AF-B26D-EC78A6A7D6E7}.Debug|Any CPU.ActiveCfg = Debug|ARM64
+ {2DEC84E5-8530-45AF-B26D-EC78A6A7D6E7}.Debug|Any CPU.Build.0 = Debug|ARM64
+ {2DEC84E5-8530-45AF-B26D-EC78A6A7D6E7}.Debug|x64.ActiveCfg = Debug|x64
+ {2DEC84E5-8530-45AF-B26D-EC78A6A7D6E7}.Debug|x64.Build.0 = Debug|x64
+ {2DEC84E5-8530-45AF-B26D-EC78A6A7D6E7}.Debug|x86.ActiveCfg = Debug|ARM64
+ {2DEC84E5-8530-45AF-B26D-EC78A6A7D6E7}.Debug|x86.Build.0 = Debug|ARM64
+ {2DEC84E5-8530-45AF-B26D-EC78A6A7D6E7}.Release|Any CPU.ActiveCfg = Release|ARM64
+ {2DEC84E5-8530-45AF-B26D-EC78A6A7D6E7}.Release|Any CPU.Build.0 = Release|ARM64
+ {2DEC84E5-8530-45AF-B26D-EC78A6A7D6E7}.Release|x64.ActiveCfg = Release|x64
+ {2DEC84E5-8530-45AF-B26D-EC78A6A7D6E7}.Release|x64.Build.0 = Release|x64
+ {2DEC84E5-8530-45AF-B26D-EC78A6A7D6E7}.Release|x86.ActiveCfg = Release|ARM64
+ {2DEC84E5-8530-45AF-B26D-EC78A6A7D6E7}.Release|x86.Build.0 = Release|ARM64
+ EndGlobalSection
+ GlobalSection(SolutionProperties) = preSolution
+ HideSolutionNode = FALSE
+ EndGlobalSection
+EndGlobal
diff --git a/samples/cs/GettingStarted/src/FoundryLocalWebServer/Program.cs b/samples/cs/foundry-local-web-server/Program.cs
similarity index 91%
rename from samples/cs/GettingStarted/src/FoundryLocalWebServer/Program.cs
rename to samples/cs/foundry-local-web-server/Program.cs
index f50ac1b0..3ca68854 100644
--- a/samples/cs/GettingStarted/src/FoundryLocalWebServer/Program.cs
+++ b/samples/cs/foundry-local-web-server/Program.cs
@@ -1,7 +1,11 @@
-using Microsoft.AI.Foundry.Local;
+//
+//
+using Microsoft.AI.Foundry.Local;
using OpenAI;
using System.ClientModel;
+//
+//
var config = new Configuration
{
AppName = "foundry_local_samples",
@@ -23,8 +27,10 @@
// Download is only required again if a new version of the EP is released.
// For cross platform builds there is no dynamic EP download and this will return immediately.
await Utils.RunWithSpinner("Registering execution providers", mgr.EnsureEpsDownloadedAsync());
+//
+//
// Get the model catalog
var catalog = await mgr.GetCatalogAsync();
@@ -46,8 +52,10 @@ await model.DownloadAsync(progress =>
Console.Write($"Loading model {model.Id}...");
await model.LoadAsync();
Console.WriteLine("done.");
+//
+//
// Start the web service
Console.Write($"Starting web service on {config.Web.Urls}...");
await mgr.StartWebServiceAsync();
@@ -79,4 +87,6 @@ await model.DownloadAsync(progress =>
// Tidy up
// Stop the web service and unload model
await mgr.StopWebServiceAsync();
-await model.UnloadAsync();
\ No newline at end of file
+await model.UnloadAsync();
+//
+//
\ No newline at end of file
diff --git a/samples/cs/live-audio-transcription-example/LiveAudioTranscriptionExample.csproj b/samples/cs/live-audio-transcription-example/LiveAudioTranscriptionExample.csproj
new file mode 100644
index 00000000..3d91b677
--- /dev/null
+++ b/samples/cs/live-audio-transcription-example/LiveAudioTranscriptionExample.csproj
@@ -0,0 +1,55 @@
+
+
+
+ Exe
+ enable
+ enable
+
+
+
+
+ net9.0-windows10.0.26100
+ false
+ ARM64;x64
+ None
+ false
+
+
+
+
+ net9.0
+
+
+
+ $(NETCoreSdkRuntimeIdentifier)
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
diff --git a/samples/cs/live-audio-transcription-example/LiveAudioTranscriptionExample.sln b/samples/cs/live-audio-transcription-example/LiveAudioTranscriptionExample.sln
new file mode 100644
index 00000000..65ba7510
--- /dev/null
+++ b/samples/cs/live-audio-transcription-example/LiveAudioTranscriptionExample.sln
@@ -0,0 +1,34 @@
+
+Microsoft Visual Studio Solution File, Format Version 12.00
+# Visual Studio Version 17
+VisualStudioVersion = 17.0.31903.59
+MinimumVisualStudioVersion = 10.0.40219.1
+Project("{FAE04EC0-301F-11D3-BF4B-00C04F79EFBC}") = "LiveAudioTranscriptionExample", "LiveAudioTranscriptionExample.csproj", "{A2B3C4D5-E6F7-4A8B-9C0D-1E2F3A4B5C6D}"
+EndProject
+Global
+ GlobalSection(SolutionConfigurationPlatforms) = preSolution
+ Debug|Any CPU = Debug|Any CPU
+ Debug|x64 = Debug|x64
+ Debug|x86 = Debug|x86
+ Release|Any CPU = Release|Any CPU
+ Release|x64 = Release|x64
+ Release|x86 = Release|x86
+ EndGlobalSection
+ GlobalSection(ProjectConfigurationPlatforms) = postSolution
+ {A2B3C4D5-E6F7-4A8B-9C0D-1E2F3A4B5C6D}.Debug|Any CPU.ActiveCfg = Debug|ARM64
+ {A2B3C4D5-E6F7-4A8B-9C0D-1E2F3A4B5C6D}.Debug|Any CPU.Build.0 = Debug|ARM64
+ {A2B3C4D5-E6F7-4A8B-9C0D-1E2F3A4B5C6D}.Debug|x64.ActiveCfg = Debug|x64
+ {A2B3C4D5-E6F7-4A8B-9C0D-1E2F3A4B5C6D}.Debug|x64.Build.0 = Debug|x64
+ {A2B3C4D5-E6F7-4A8B-9C0D-1E2F3A4B5C6D}.Debug|x86.ActiveCfg = Debug|ARM64
+ {A2B3C4D5-E6F7-4A8B-9C0D-1E2F3A4B5C6D}.Debug|x86.Build.0 = Debug|ARM64
+ {A2B3C4D5-E6F7-4A8B-9C0D-1E2F3A4B5C6D}.Release|Any CPU.ActiveCfg = Release|ARM64
+ {A2B3C4D5-E6F7-4A8B-9C0D-1E2F3A4B5C6D}.Release|Any CPU.Build.0 = Release|ARM64
+ {A2B3C4D5-E6F7-4A8B-9C0D-1E2F3A4B5C6D}.Release|x64.ActiveCfg = Release|x64
+ {A2B3C4D5-E6F7-4A8B-9C0D-1E2F3A4B5C6D}.Release|x64.Build.0 = Release|x64
+ {A2B3C4D5-E6F7-4A8B-9C0D-1E2F3A4B5C6D}.Release|x86.ActiveCfg = Release|ARM64
+ {A2B3C4D5-E6F7-4A8B-9C0D-1E2F3A4B5C6D}.Release|x86.Build.0 = Release|ARM64
+ EndGlobalSection
+ GlobalSection(SolutionProperties) = preSolution
+ HideSolutionNode = FALSE
+ EndGlobalSection
+EndGlobal
diff --git a/samples/cs/live-audio-transcription-example/Program.cs b/samples/cs/live-audio-transcription-example/Program.cs
new file mode 100644
index 00000000..68bba83f
--- /dev/null
+++ b/samples/cs/live-audio-transcription-example/Program.cs
@@ -0,0 +1,106 @@
+// Live Audio Transcription — Foundry Local SDK Example
+//
+// Demonstrates real-time microphone-to-text using:
+// SDK (FoundryLocalManager) → Core (NativeAOT DLL) → onnxruntime-genai (StreamingProcessor)
+
+using Microsoft.AI.Foundry.Local;
+using NAudio.Wave;
+
+Console.WriteLine("===========================================================");
+Console.WriteLine(" Foundry Local -- Live Audio Transcription Demo");
+Console.WriteLine("===========================================================");
+Console.WriteLine();
+
+var config = new Configuration
+{
+ AppName = "foundry_local_samples",
+ LogLevel = Microsoft.AI.Foundry.Local.LogLevel.Information
+};
+
+await FoundryLocalManager.CreateAsync(config, Utils.GetAppLogger());
+var mgr = FoundryLocalManager.Instance;
+
+await Utils.RunWithSpinner("Registering execution providers", mgr.EnsureEpsDownloadedAsync());
+
+var catalog = await mgr.GetCatalogAsync();
+
+var model = await catalog.GetModelAsync("nemotron") ?? throw new Exception("Model \"nemotron\" not found in catalog");
+
+await model.DownloadAsync(progress =>
+{
+ Console.Write($"\rDownloading model: {progress:F2}%");
+ if (progress >= 100f)
+ {
+ Console.WriteLine();
+ }
+});
+
+Console.Write($"Loading model {model.Id}...");
+await model.LoadAsync();
+Console.WriteLine("done.");
+
+var audioClient = await model.GetAudioClientAsync();
+var session = audioClient.CreateLiveTranscriptionSession();
+session.Settings.SampleRate = 16000; // Default is 16000; shown here to match the NAudio WaveFormat below
+session.Settings.Channels = 1;
+session.Settings.Language = "en";
+
+await session.StartAsync();
+Console.WriteLine(" Session started");
+
+var readTask = Task.Run(async () =>
+{
+ try
+ {
+ await foreach (var result in session.GetTranscriptionStream())
+ {
+ var text = result.Content?[0]?.Text;
+ if (result.IsFinal)
+ {
+ Console.WriteLine();
+ Console.WriteLine($" [FINAL] {text}");
+ Console.Out.Flush();
+ }
+ else if (!string.IsNullOrEmpty(text))
+ {
+ Console.ForegroundColor = ConsoleColor.Cyan;
+ Console.Write(text);
+ Console.ResetColor();
+ Console.Out.Flush();
+ }
+ }
+ }
+ catch (OperationCanceledException) { }
+});
+
+using var waveIn = new WaveInEvent
+{
+ WaveFormat = new WaveFormat(rate: 16000, bits: 16, channels: 1),
+ BufferMilliseconds = 100
+};
+
+waveIn.DataAvailable += (sender, e) =>
+{
+ if (e.BytesRecorded > 0)
+ {
+ _ = session.AppendAsync(new ReadOnlyMemory(e.Buffer, 0, e.BytesRecorded));
+ }
+};
+
+Console.WriteLine();
+Console.WriteLine("===========================================================");
+Console.WriteLine(" LIVE TRANSCRIPTION ACTIVE");
+Console.WriteLine(" Speak into your microphone.");
+Console.WriteLine(" Transcription appears in real-time (cyan text).");
+Console.WriteLine(" Press ENTER to stop recording.");
+Console.WriteLine("===========================================================");
+Console.WriteLine();
+
+waveIn.StartRecording();
+Console.ReadLine();
+waveIn.StopRecording();
+
+await session.StopAsync();
+await readTask;
+
+await model.UnloadAsync();
diff --git a/samples/cs/model-management-example/ModelManagementExample.csproj b/samples/cs/model-management-example/ModelManagementExample.csproj
new file mode 100644
index 00000000..4d948c56
--- /dev/null
+++ b/samples/cs/model-management-example/ModelManagementExample.csproj
@@ -0,0 +1,48 @@
+
+
+
+ Exe
+ enable
+ enable
+
+
+
+
+ net9.0-windows10.0.26100
+ false
+ ARM64;x64
+ None
+ false
+
+
+
+
+ net9.0
+
+
+
+ $(NETCoreSdkRuntimeIdentifier)
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
diff --git a/samples/cs/model-management-example/ModelManagementExample.sln b/samples/cs/model-management-example/ModelManagementExample.sln
new file mode 100644
index 00000000..f255391b
--- /dev/null
+++ b/samples/cs/model-management-example/ModelManagementExample.sln
@@ -0,0 +1,34 @@
+
+Microsoft Visual Studio Solution File, Format Version 12.00
+# Visual Studio Version 17
+VisualStudioVersion = 17.0.31903.59
+MinimumVisualStudioVersion = 10.0.40219.1
+Project("{FAE04EC0-301F-11D3-BF4B-00C04F79EFBC}") = "ModelManagementExample", "ModelManagementExample.csproj", "{9316B939-946C-4956-A4E7-9410017FD319}"
+EndProject
+Global
+ GlobalSection(SolutionConfigurationPlatforms) = preSolution
+ Debug|Any CPU = Debug|Any CPU
+ Debug|x64 = Debug|x64
+ Debug|x86 = Debug|x86
+ Release|Any CPU = Release|Any CPU
+ Release|x64 = Release|x64
+ Release|x86 = Release|x86
+ EndGlobalSection
+ GlobalSection(ProjectConfigurationPlatforms) = postSolution
+ {9316B939-946C-4956-A4E7-9410017FD319}.Debug|Any CPU.ActiveCfg = Debug|ARM64
+ {9316B939-946C-4956-A4E7-9410017FD319}.Debug|Any CPU.Build.0 = Debug|ARM64
+ {9316B939-946C-4956-A4E7-9410017FD319}.Debug|x64.ActiveCfg = Debug|x64
+ {9316B939-946C-4956-A4E7-9410017FD319}.Debug|x64.Build.0 = Debug|x64
+ {9316B939-946C-4956-A4E7-9410017FD319}.Debug|x86.ActiveCfg = Debug|ARM64
+ {9316B939-946C-4956-A4E7-9410017FD319}.Debug|x86.Build.0 = Debug|ARM64
+ {9316B939-946C-4956-A4E7-9410017FD319}.Release|Any CPU.ActiveCfg = Release|ARM64
+ {9316B939-946C-4956-A4E7-9410017FD319}.Release|Any CPU.Build.0 = Release|ARM64
+ {9316B939-946C-4956-A4E7-9410017FD319}.Release|x64.ActiveCfg = Release|x64
+ {9316B939-946C-4956-A4E7-9410017FD319}.Release|x64.Build.0 = Release|x64
+ {9316B939-946C-4956-A4E7-9410017FD319}.Release|x86.ActiveCfg = Release|ARM64
+ {9316B939-946C-4956-A4E7-9410017FD319}.Release|x86.Build.0 = Release|ARM64
+ EndGlobalSection
+ GlobalSection(SolutionProperties) = preSolution
+ HideSolutionNode = FALSE
+ EndGlobalSection
+EndGlobal
diff --git a/samples/cs/GettingStarted/src/ModelManagementExample/Program.cs b/samples/cs/model-management-example/Program.cs
similarity index 100%
rename from samples/cs/GettingStarted/src/ModelManagementExample/Program.cs
rename to samples/cs/model-management-example/Program.cs
diff --git a/samples/cs/native-chat-completions/NativeChatCompletions.csproj b/samples/cs/native-chat-completions/NativeChatCompletions.csproj
new file mode 100644
index 00000000..4d948c56
--- /dev/null
+++ b/samples/cs/native-chat-completions/NativeChatCompletions.csproj
@@ -0,0 +1,48 @@
+
+
+
+ Exe
+ enable
+ enable
+
+
+
+
+ net9.0-windows10.0.26100
+ false
+ ARM64;x64
+ None
+ false
+
+
+
+
+ net9.0
+
+
+
+ $(NETCoreSdkRuntimeIdentifier)
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
diff --git a/samples/cs/native-chat-completions/NativeChatCompletions.sln b/samples/cs/native-chat-completions/NativeChatCompletions.sln
new file mode 100644
index 00000000..a127bfba
--- /dev/null
+++ b/samples/cs/native-chat-completions/NativeChatCompletions.sln
@@ -0,0 +1,34 @@
+
+Microsoft Visual Studio Solution File, Format Version 12.00
+# Visual Studio Version 17
+VisualStudioVersion = 17.0.31903.59
+MinimumVisualStudioVersion = 10.0.40219.1
+Project("{FAE04EC0-301F-11D3-BF4B-00C04F79EFBC}") = "NativeChatCompletions", "NativeChatCompletions.csproj", "{A53372CE-F7E1-4F09-B186-77F76E388659}"
+EndProject
+Global
+ GlobalSection(SolutionConfigurationPlatforms) = preSolution
+ Debug|Any CPU = Debug|Any CPU
+ Debug|x64 = Debug|x64
+ Debug|x86 = Debug|x86
+ Release|Any CPU = Release|Any CPU
+ Release|x64 = Release|x64
+ Release|x86 = Release|x86
+ EndGlobalSection
+ GlobalSection(ProjectConfigurationPlatforms) = postSolution
+ {A53372CE-F7E1-4F09-B186-77F76E388659}.Debug|Any CPU.ActiveCfg = Debug|ARM64
+ {A53372CE-F7E1-4F09-B186-77F76E388659}.Debug|Any CPU.Build.0 = Debug|ARM64
+ {A53372CE-F7E1-4F09-B186-77F76E388659}.Debug|x64.ActiveCfg = Debug|x64
+ {A53372CE-F7E1-4F09-B186-77F76E388659}.Debug|x64.Build.0 = Debug|x64
+ {A53372CE-F7E1-4F09-B186-77F76E388659}.Debug|x86.ActiveCfg = Debug|ARM64
+ {A53372CE-F7E1-4F09-B186-77F76E388659}.Debug|x86.Build.0 = Debug|ARM64
+ {A53372CE-F7E1-4F09-B186-77F76E388659}.Release|Any CPU.ActiveCfg = Release|ARM64
+ {A53372CE-F7E1-4F09-B186-77F76E388659}.Release|Any CPU.Build.0 = Release|ARM64
+ {A53372CE-F7E1-4F09-B186-77F76E388659}.Release|x64.ActiveCfg = Release|x64
+ {A53372CE-F7E1-4F09-B186-77F76E388659}.Release|x64.Build.0 = Release|x64
+ {A53372CE-F7E1-4F09-B186-77F76E388659}.Release|x86.ActiveCfg = Release|ARM64
+ {A53372CE-F7E1-4F09-B186-77F76E388659}.Release|x86.Build.0 = Release|ARM64
+ EndGlobalSection
+ GlobalSection(SolutionProperties) = preSolution
+ HideSolutionNode = FALSE
+ EndGlobalSection
+EndGlobal
diff --git a/samples/cs/GettingStarted/src/HelloFoundryLocalSdk/Program.cs b/samples/cs/native-chat-completions/Program.cs
similarity index 88%
rename from samples/cs/GettingStarted/src/HelloFoundryLocalSdk/Program.cs
rename to samples/cs/native-chat-completions/Program.cs
index 52efe410..082a19f5 100644
--- a/samples/cs/GettingStarted/src/HelloFoundryLocalSdk/Program.cs
+++ b/samples/cs/native-chat-completions/Program.cs
@@ -1,6 +1,10 @@
-using Microsoft.AI.Foundry.Local;
+//
+//
+using Microsoft.AI.Foundry.Local;
using Betalgo.Ranul.OpenAI.ObjectModels.RequestModels;
+//
+//
CancellationToken ct = new CancellationToken();
var config = new Configuration
@@ -20,8 +24,10 @@
// Download is only required again if a new version of the EP is released.
// For cross platform builds there is no dynamic EP download and this will return immediately.
await Utils.RunWithSpinner("Registering execution providers", mgr.EnsureEpsDownloadedAsync());
+//
+//
// Get the model catalog
var catalog = await mgr.GetCatalogAsync();
@@ -43,7 +49,9 @@ await model.DownloadAsync(progress =>
Console.Write($"Loading model {model.Id}...");
await model.LoadAsync();
Console.WriteLine("done.");
+//
+//
// Get a chat client
var chatClient = await model.GetChatClientAsync();
@@ -62,6 +70,10 @@ await model.DownloadAsync(progress =>
Console.Out.Flush();
}
Console.WriteLine();
+//
+//
// Tidy up - unload the model
-await model.UnloadAsync();
\ No newline at end of file
+await model.UnloadAsync();
+//
+//
\ No newline at end of file
diff --git a/samples/cs/nuget.config b/samples/cs/nuget.config
new file mode 100644
index 00000000..9913c715
--- /dev/null
+++ b/samples/cs/nuget.config
@@ -0,0 +1,22 @@
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
\ No newline at end of file
diff --git a/samples/cs/GettingStarted/src/ToolCallingFoundryLocalSdk/Program.cs b/samples/cs/tool-calling-foundry-local-sdk/Program.cs
similarity index 94%
rename from samples/cs/GettingStarted/src/ToolCallingFoundryLocalSdk/Program.cs
rename to samples/cs/tool-calling-foundry-local-sdk/Program.cs
index 3cdf3d38..bbb050c0 100644
--- a/samples/cs/GettingStarted/src/ToolCallingFoundryLocalSdk/Program.cs
+++ b/samples/cs/tool-calling-foundry-local-sdk/Program.cs
@@ -1,9 +1,13 @@
-using Microsoft.AI.Foundry.Local;
+//
+//
+using Microsoft.AI.Foundry.Local;
using Betalgo.Ranul.OpenAI.ObjectModels.RequestModels;
using Betalgo.Ranul.OpenAI.ObjectModels.ResponseModels;
using Betalgo.Ranul.OpenAI.ObjectModels.SharedModels;
using System.Text.Json;
+//
+//
CancellationToken ct = new CancellationToken();
var config = new Configuration
@@ -23,8 +27,10 @@
// Download is only required again if a new version of the EP is released.
// For cross platform builds there is no dynamic EP download and this will return immediately.
await Utils.RunWithSpinner("Registering execution providers", mgr.EnsureEpsDownloadedAsync());
+//
+//
// Get the model catalog
var catalog = await mgr.GetCatalogAsync();
@@ -48,6 +54,7 @@ await model.DownloadAsync(progress =>
Console.Write($"Loading model {model.Id}...");
await model.LoadAsync();
Console.WriteLine("done.");
+//
// Get a chat client
@@ -63,6 +70,7 @@ await model.DownloadAsync(progress =>
];
+//
// Prepare tools
List tools =
[
@@ -86,8 +94,10 @@ await model.DownloadAsync(progress =>
}
}
];
+//
+//
// Get a streaming chat completion response
var toolCallResponses = new List();
Console.WriteLine("Chat completion response:");
@@ -150,7 +160,11 @@ await model.DownloadAsync(progress =>
Console.Out.Flush();
}
Console.WriteLine();
+//
+//
// Tidy up - unload the model
-await model.UnloadAsync();
\ No newline at end of file
+await model.UnloadAsync();
+//
+//
\ No newline at end of file
diff --git a/samples/cs/tool-calling-foundry-local-sdk/ToolCallingFoundryLocalSdk.csproj b/samples/cs/tool-calling-foundry-local-sdk/ToolCallingFoundryLocalSdk.csproj
new file mode 100644
index 00000000..4d948c56
--- /dev/null
+++ b/samples/cs/tool-calling-foundry-local-sdk/ToolCallingFoundryLocalSdk.csproj
@@ -0,0 +1,48 @@
+
+
+
+ Exe
+ enable
+ enable
+
+
+
+
+ net9.0-windows10.0.26100
+ false
+ ARM64;x64
+ None
+ false
+
+
+
+
+ net9.0
+
+
+
+ $(NETCoreSdkRuntimeIdentifier)
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
diff --git a/samples/cs/tool-calling-foundry-local-sdk/ToolCallingFoundryLocalSdk.sln b/samples/cs/tool-calling-foundry-local-sdk/ToolCallingFoundryLocalSdk.sln
new file mode 100644
index 00000000..adbf5ea2
--- /dev/null
+++ b/samples/cs/tool-calling-foundry-local-sdk/ToolCallingFoundryLocalSdk.sln
@@ -0,0 +1,34 @@
+
+Microsoft Visual Studio Solution File, Format Version 12.00
+# Visual Studio Version 17
+VisualStudioVersion = 17.0.31903.59
+MinimumVisualStudioVersion = 10.0.40219.1
+Project("{FAE04EC0-301F-11D3-BF4B-00C04F79EFBC}") = "ToolCallingFoundryLocalSdk", "ToolCallingFoundryLocalSdk.csproj", "{7B40637D-D7E3-4A95-9B57-8D0EF84C8532}"
+EndProject
+Global
+ GlobalSection(SolutionConfigurationPlatforms) = preSolution
+ Debug|Any CPU = Debug|Any CPU
+ Debug|x64 = Debug|x64
+ Debug|x86 = Debug|x86
+ Release|Any CPU = Release|Any CPU
+ Release|x64 = Release|x64
+ Release|x86 = Release|x86
+ EndGlobalSection
+ GlobalSection(ProjectConfigurationPlatforms) = postSolution
+ {7B40637D-D7E3-4A95-9B57-8D0EF84C8532}.Debug|Any CPU.ActiveCfg = Debug|ARM64
+ {7B40637D-D7E3-4A95-9B57-8D0EF84C8532}.Debug|Any CPU.Build.0 = Debug|ARM64
+ {7B40637D-D7E3-4A95-9B57-8D0EF84C8532}.Debug|x64.ActiveCfg = Debug|x64
+ {7B40637D-D7E3-4A95-9B57-8D0EF84C8532}.Debug|x64.Build.0 = Debug|x64
+ {7B40637D-D7E3-4A95-9B57-8D0EF84C8532}.Debug|x86.ActiveCfg = Debug|ARM64
+ {7B40637D-D7E3-4A95-9B57-8D0EF84C8532}.Debug|x86.Build.0 = Debug|ARM64
+ {7B40637D-D7E3-4A95-9B57-8D0EF84C8532}.Release|Any CPU.ActiveCfg = Release|ARM64
+ {7B40637D-D7E3-4A95-9B57-8D0EF84C8532}.Release|Any CPU.Build.0 = Release|ARM64
+ {7B40637D-D7E3-4A95-9B57-8D0EF84C8532}.Release|x64.ActiveCfg = Release|x64
+ {7B40637D-D7E3-4A95-9B57-8D0EF84C8532}.Release|x64.Build.0 = Release|x64
+ {7B40637D-D7E3-4A95-9B57-8D0EF84C8532}.Release|x86.ActiveCfg = Release|ARM64
+ {7B40637D-D7E3-4A95-9B57-8D0EF84C8532}.Release|x86.Build.0 = Release|ARM64
+ EndGlobalSection
+ GlobalSection(SolutionProperties) = preSolution
+ HideSolutionNode = FALSE
+ EndGlobalSection
+EndGlobal
diff --git a/samples/cs/GettingStarted/src/ToolCallingFoundryLocalWebServer/Program.cs b/samples/cs/tool-calling-foundry-local-web-server/Program.cs
similarity index 98%
rename from samples/cs/GettingStarted/src/ToolCallingFoundryLocalWebServer/Program.cs
rename to samples/cs/tool-calling-foundry-local-web-server/Program.cs
index 6d6937fd..4c283cd4 100644
--- a/samples/cs/GettingStarted/src/ToolCallingFoundryLocalWebServer/Program.cs
+++ b/samples/cs/tool-calling-foundry-local-web-server/Program.cs
@@ -1,4 +1,5 @@
-using Microsoft.AI.Foundry.Local;
+//
+using Microsoft.AI.Foundry.Local;
using OpenAI;
using OpenAI.Chat;
using System.ClientModel;
@@ -178,4 +179,5 @@ await model.DownloadAsync(progress =>
// Tidy up
// Stop the web service and unload model
await mgr.StopWebServiceAsync();
-await model.UnloadAsync();
\ No newline at end of file
+await model.UnloadAsync();
+//
\ No newline at end of file
diff --git a/samples/cs/tool-calling-foundry-local-web-server/ToolCallingFoundryLocalWebServer.csproj b/samples/cs/tool-calling-foundry-local-web-server/ToolCallingFoundryLocalWebServer.csproj
new file mode 100644
index 00000000..fe890be2
--- /dev/null
+++ b/samples/cs/tool-calling-foundry-local-web-server/ToolCallingFoundryLocalWebServer.csproj
@@ -0,0 +1,52 @@
+
+
+
+ Exe
+ enable
+ enable
+
+
+
+
+ net9.0-windows10.0.26100
+ false
+ ARM64;x64
+ None
+ false
+
+
+
+
+ net9.0
+
+
+
+ $(NETCoreSdkRuntimeIdentifier)
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
diff --git a/samples/cs/tool-calling-foundry-local-web-server/ToolCallingFoundryLocalWebServer.sln b/samples/cs/tool-calling-foundry-local-web-server/ToolCallingFoundryLocalWebServer.sln
new file mode 100644
index 00000000..7d1568e1
--- /dev/null
+++ b/samples/cs/tool-calling-foundry-local-web-server/ToolCallingFoundryLocalWebServer.sln
@@ -0,0 +1,34 @@
+
+Microsoft Visual Studio Solution File, Format Version 12.00
+# Visual Studio Version 17
+VisualStudioVersion = 17.0.31903.59
+MinimumVisualStudioVersion = 10.0.40219.1
+Project("{FAE04EC0-301F-11D3-BF4B-00C04F79EFBC}") = "ToolCallingFoundryLocalWebServer", "ToolCallingFoundryLocalWebServer.csproj", "{F9BD2479-A235-4BBF-A722-DF180A076143}"
+EndProject
+Global
+ GlobalSection(SolutionConfigurationPlatforms) = preSolution
+ Debug|Any CPU = Debug|Any CPU
+ Debug|x64 = Debug|x64
+ Debug|x86 = Debug|x86
+ Release|Any CPU = Release|Any CPU
+ Release|x64 = Release|x64
+ Release|x86 = Release|x86
+ EndGlobalSection
+ GlobalSection(ProjectConfigurationPlatforms) = postSolution
+ {F9BD2479-A235-4BBF-A722-DF180A076143}.Debug|Any CPU.ActiveCfg = Debug|ARM64
+ {F9BD2479-A235-4BBF-A722-DF180A076143}.Debug|Any CPU.Build.0 = Debug|ARM64
+ {F9BD2479-A235-4BBF-A722-DF180A076143}.Debug|x64.ActiveCfg = Debug|x64
+ {F9BD2479-A235-4BBF-A722-DF180A076143}.Debug|x64.Build.0 = Debug|x64
+ {F9BD2479-A235-4BBF-A722-DF180A076143}.Debug|x86.ActiveCfg = Debug|ARM64
+ {F9BD2479-A235-4BBF-A722-DF180A076143}.Debug|x86.Build.0 = Debug|ARM64
+ {F9BD2479-A235-4BBF-A722-DF180A076143}.Release|Any CPU.ActiveCfg = Release|ARM64
+ {F9BD2479-A235-4BBF-A722-DF180A076143}.Release|Any CPU.Build.0 = Release|ARM64
+ {F9BD2479-A235-4BBF-A722-DF180A076143}.Release|x64.ActiveCfg = Release|x64
+ {F9BD2479-A235-4BBF-A722-DF180A076143}.Release|x64.Build.0 = Release|x64
+ {F9BD2479-A235-4BBF-A722-DF180A076143}.Release|x86.ActiveCfg = Release|ARM64
+ {F9BD2479-A235-4BBF-A722-DF180A076143}.Release|x86.Build.0 = Release|ARM64
+ EndGlobalSection
+ GlobalSection(SolutionProperties) = preSolution
+ HideSolutionNode = FALSE
+ EndGlobalSection
+EndGlobal
diff --git a/samples/cs/tutorial-chat-assistant/Program.cs b/samples/cs/tutorial-chat-assistant/Program.cs
new file mode 100644
index 00000000..10e9a63b
--- /dev/null
+++ b/samples/cs/tutorial-chat-assistant/Program.cs
@@ -0,0 +1,101 @@
+//
+//
+using Microsoft.AI.Foundry.Local;
+using Betalgo.Ranul.OpenAI.ObjectModels.RequestModels;
+using Microsoft.Extensions.Logging;
+//
+
+//
+CancellationToken ct = CancellationToken.None;
+
+var config = new Configuration
+{
+ AppName = "foundry_local_samples",
+ LogLevel = Microsoft.AI.Foundry.Local.LogLevel.Information
+};
+
+using var loggerFactory = LoggerFactory.Create(builder =>
+{
+ builder.SetMinimumLevel(Microsoft.Extensions.Logging.LogLevel.Information);
+});
+var logger = loggerFactory.CreateLogger();
+
+// Initialize the singleton instance
+await FoundryLocalManager.CreateAsync(config, logger);
+var mgr = FoundryLocalManager.Instance;
+
+// Select and load a model from the catalog
+var catalog = await mgr.GetCatalogAsync();
+var model = await catalog.GetModelAsync("qwen2.5-0.5b")
+ ?? throw new Exception("Model not found");
+
+await model.DownloadAsync(progress =>
+{
+ Console.Write($"\rDownloading model: {progress:F2}%");
+ if (progress >= 100f) Console.WriteLine();
+});
+
+await model.LoadAsync();
+Console.WriteLine("Model loaded and ready.");
+
+// Get a chat client
+var chatClient = await model.GetChatClientAsync();
+//
+
+//
+// Start the conversation with a system prompt
+var messages = new List
+{
+ new ChatMessage
+ {
+ Role = "system",
+ Content = "You are a helpful, friendly assistant. Keep your responses " +
+ "concise and conversational. If you don't know something, say so."
+ }
+};
+//
+
+Console.WriteLine("\nChat assistant ready! Type 'quit' to exit.\n");
+
+//
+while (true)
+{
+ Console.Write("You: ");
+ var userInput = Console.ReadLine();
+ if (string.IsNullOrWhiteSpace(userInput) ||
+ userInput.Equals("quit", StringComparison.OrdinalIgnoreCase) ||
+ userInput.Equals("exit", StringComparison.OrdinalIgnoreCase))
+ {
+ break;
+ }
+
+ // Add the user's message to conversation history
+ messages.Add(new ChatMessage { Role = "user", Content = userInput });
+
+ //
+ // Stream the response token by token
+ Console.Write("Assistant: ");
+ var fullResponse = string.Empty;
+ var streamingResponse = chatClient.CompleteChatStreamingAsync(messages, ct);
+ await foreach (var chunk in streamingResponse)
+ {
+ var content = chunk.Choices[0].Message.Content;
+ if (!string.IsNullOrEmpty(content))
+ {
+ Console.Write(content);
+ Console.Out.Flush();
+ fullResponse += content;
+ }
+ }
+ Console.WriteLine("\n");
+ //
+
+ // Add the complete response to conversation history
+ messages.Add(new ChatMessage { Role = "assistant", Content = fullResponse });
+}
+//
+
+// Clean up - unload the model
+await model.UnloadAsync();
+Console.WriteLine("Model unloaded. Goodbye!");
+//
diff --git a/samples/cs/tutorial-chat-assistant/TutorialChatAssistant.csproj b/samples/cs/tutorial-chat-assistant/TutorialChatAssistant.csproj
new file mode 100644
index 00000000..a3533047
--- /dev/null
+++ b/samples/cs/tutorial-chat-assistant/TutorialChatAssistant.csproj
@@ -0,0 +1,50 @@
+
+
+
+ Exe
+ enable
+ enable
+
+
+
+
+ net9.0-windows10.0.26100
+ false
+ ARM64;x64
+ None
+ false
+
+
+
+
+ net9.0
+
+
+
+ $(NETCoreSdkRuntimeIdentifier)
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
diff --git a/samples/cs/tutorial-chat-assistant/TutorialChatAssistant.sln b/samples/cs/tutorial-chat-assistant/TutorialChatAssistant.sln
new file mode 100644
index 00000000..a9c77e16
--- /dev/null
+++ b/samples/cs/tutorial-chat-assistant/TutorialChatAssistant.sln
@@ -0,0 +1,34 @@
+
+Microsoft Visual Studio Solution File, Format Version 12.00
+# Visual Studio Version 17
+VisualStudioVersion = 17.0.31903.59
+MinimumVisualStudioVersion = 10.0.40219.1
+Project("{FAE04EC0-301F-11D3-BF4B-00C04F79EFBC}") = "TutorialChatAssistant", "TutorialChatAssistant.csproj", "{5D5778BD-B40A-4D9E-BC2F-65AD50EE6F94}"
+EndProject
+Global
+ GlobalSection(SolutionConfigurationPlatforms) = preSolution
+ Debug|Any CPU = Debug|Any CPU
+ Debug|x64 = Debug|x64
+ Debug|x86 = Debug|x86
+ Release|Any CPU = Release|Any CPU
+ Release|x64 = Release|x64
+ Release|x86 = Release|x86
+ EndGlobalSection
+ GlobalSection(ProjectConfigurationPlatforms) = postSolution
+ {5D5778BD-B40A-4D9E-BC2F-65AD50EE6F94}.Debug|Any CPU.ActiveCfg = Debug|ARM64
+ {5D5778BD-B40A-4D9E-BC2F-65AD50EE6F94}.Debug|Any CPU.Build.0 = Debug|ARM64
+ {5D5778BD-B40A-4D9E-BC2F-65AD50EE6F94}.Debug|x64.ActiveCfg = Debug|x64
+ {5D5778BD-B40A-4D9E-BC2F-65AD50EE6F94}.Debug|x64.Build.0 = Debug|x64
+ {5D5778BD-B40A-4D9E-BC2F-65AD50EE6F94}.Debug|x86.ActiveCfg = Debug|ARM64
+ {5D5778BD-B40A-4D9E-BC2F-65AD50EE6F94}.Debug|x86.Build.0 = Debug|ARM64
+ {5D5778BD-B40A-4D9E-BC2F-65AD50EE6F94}.Release|Any CPU.ActiveCfg = Release|ARM64
+ {5D5778BD-B40A-4D9E-BC2F-65AD50EE6F94}.Release|Any CPU.Build.0 = Release|ARM64
+ {5D5778BD-B40A-4D9E-BC2F-65AD50EE6F94}.Release|x64.ActiveCfg = Release|x64
+ {5D5778BD-B40A-4D9E-BC2F-65AD50EE6F94}.Release|x64.Build.0 = Release|x64
+ {5D5778BD-B40A-4D9E-BC2F-65AD50EE6F94}.Release|x86.ActiveCfg = Release|ARM64
+ {5D5778BD-B40A-4D9E-BC2F-65AD50EE6F94}.Release|x86.Build.0 = Release|ARM64
+ EndGlobalSection
+ GlobalSection(SolutionProperties) = preSolution
+ HideSolutionNode = FALSE
+ EndGlobalSection
+EndGlobal
diff --git a/samples/cs/tutorial-document-summarizer/Program.cs b/samples/cs/tutorial-document-summarizer/Program.cs
new file mode 100644
index 00000000..bc5546f6
--- /dev/null
+++ b/samples/cs/tutorial-document-summarizer/Program.cs
@@ -0,0 +1,109 @@
+//
+//
+using Microsoft.AI.Foundry.Local;
+using Betalgo.Ranul.OpenAI.ObjectModels.RequestModels;
+using Microsoft.Extensions.Logging;
+//
+
+//
+CancellationToken ct = CancellationToken.None;
+
+var config = new Configuration
+{
+ AppName = "foundry_local_samples",
+ LogLevel = Microsoft.AI.Foundry.Local.LogLevel.Information
+};
+
+using var loggerFactory = LoggerFactory.Create(builder =>
+{
+ builder.SetMinimumLevel(Microsoft.Extensions.Logging.LogLevel.Information);
+});
+var logger = loggerFactory.CreateLogger();
+
+// Initialize the singleton instance
+await FoundryLocalManager.CreateAsync(config, logger);
+var mgr = FoundryLocalManager.Instance;
+
+// Select and load a model from the catalog
+var catalog = await mgr.GetCatalogAsync();
+var model = await catalog.GetModelAsync("qwen2.5-0.5b")
+ ?? throw new Exception("Model not found");
+
+await model.DownloadAsync(progress =>
+{
+ Console.Write($"\rDownloading model: {progress:F2}%");
+ if (progress >= 100f) Console.WriteLine();
+});
+
+await model.LoadAsync();
+Console.WriteLine("Model loaded and ready.\n");
+
+// Get a chat client
+var chatClient = await model.GetChatClientAsync();
+//
+
+//
+var systemPrompt =
+ "Summarize the following document into concise bullet points. " +
+ "Focus on the key points and main ideas.";
+
+//
+var target = args.Length > 0 ? args[0] : "document.txt";
+//
+
+if (Directory.Exists(target))
+{
+ await SummarizeDirectoryAsync(chatClient, target, systemPrompt, ct);
+}
+else
+{
+ Console.WriteLine($"--- {Path.GetFileName(target)} ---");
+ await SummarizeFileAsync(chatClient, target, systemPrompt, ct);
+}
+//
+
+// Clean up
+await model.UnloadAsync();
+Console.WriteLine("\nModel unloaded. Done!");
+
+async Task SummarizeFileAsync(
+ dynamic client,
+ string filePath,
+ string prompt,
+ CancellationToken token)
+{
+ var fileContent = await File.ReadAllTextAsync(filePath, token);
+ var messages = new List
+ {
+ new ChatMessage { Role = "system", Content = prompt },
+ new ChatMessage { Role = "user", Content = fileContent }
+ };
+
+ var response = await client.CompleteChatAsync(messages, token);
+ Console.WriteLine(response.Choices[0].Message.Content);
+}
+
+async Task SummarizeDirectoryAsync(
+ dynamic client,
+ string directory,
+ string prompt,
+ CancellationToken token)
+{
+ var txtFiles = Directory.GetFiles(directory, "*.txt")
+ .OrderBy(f => f)
+ .ToArray();
+
+ if (txtFiles.Length == 0)
+ {
+ Console.WriteLine($"No .txt files found in {directory}");
+ return;
+ }
+
+ foreach (var txtFile in txtFiles)
+ {
+ Console.WriteLine($"--- {Path.GetFileName(txtFile)} ---");
+ await SummarizeFileAsync(client, txtFile, prompt, token);
+ Console.WriteLine();
+ }
+}
+//
diff --git a/samples/cs/tutorial-document-summarizer/TutorialDocumentSummarizer.csproj b/samples/cs/tutorial-document-summarizer/TutorialDocumentSummarizer.csproj
new file mode 100644
index 00000000..a3533047
--- /dev/null
+++ b/samples/cs/tutorial-document-summarizer/TutorialDocumentSummarizer.csproj
@@ -0,0 +1,50 @@
+
+
+
+ Exe
+ enable
+ enable
+
+
+
+
+ net9.0-windows10.0.26100
+ false
+ ARM64;x64
+ None
+ false
+
+
+
+
+ net9.0
+
+
+
+ $(NETCoreSdkRuntimeIdentifier)
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
diff --git a/samples/cs/tutorial-document-summarizer/TutorialDocumentSummarizer.sln b/samples/cs/tutorial-document-summarizer/TutorialDocumentSummarizer.sln
new file mode 100644
index 00000000..7d7a0fc9
--- /dev/null
+++ b/samples/cs/tutorial-document-summarizer/TutorialDocumentSummarizer.sln
@@ -0,0 +1,34 @@
+
+Microsoft Visual Studio Solution File, Format Version 12.00
+# Visual Studio Version 17
+VisualStudioVersion = 17.0.31903.59
+MinimumVisualStudioVersion = 10.0.40219.1
+Project("{FAE04EC0-301F-11D3-BF4B-00C04F79EFBC}") = "TutorialDocumentSummarizer", "TutorialDocumentSummarizer.csproj", "{6868D03F-BD8E-46ED-9A5B-95346A3810A4}"
+EndProject
+Global
+ GlobalSection(SolutionConfigurationPlatforms) = preSolution
+ Debug|Any CPU = Debug|Any CPU
+ Debug|x64 = Debug|x64
+ Debug|x86 = Debug|x86
+ Release|Any CPU = Release|Any CPU
+ Release|x64 = Release|x64
+ Release|x86 = Release|x86
+ EndGlobalSection
+ GlobalSection(ProjectConfigurationPlatforms) = postSolution
+ {6868D03F-BD8E-46ED-9A5B-95346A3810A4}.Debug|Any CPU.ActiveCfg = Debug|ARM64
+ {6868D03F-BD8E-46ED-9A5B-95346A3810A4}.Debug|Any CPU.Build.0 = Debug|ARM64
+ {6868D03F-BD8E-46ED-9A5B-95346A3810A4}.Debug|x64.ActiveCfg = Debug|x64
+ {6868D03F-BD8E-46ED-9A5B-95346A3810A4}.Debug|x64.Build.0 = Debug|x64
+ {6868D03F-BD8E-46ED-9A5B-95346A3810A4}.Debug|x86.ActiveCfg = Debug|ARM64
+ {6868D03F-BD8E-46ED-9A5B-95346A3810A4}.Debug|x86.Build.0 = Debug|ARM64
+ {6868D03F-BD8E-46ED-9A5B-95346A3810A4}.Release|Any CPU.ActiveCfg = Release|ARM64
+ {6868D03F-BD8E-46ED-9A5B-95346A3810A4}.Release|Any CPU.Build.0 = Release|ARM64
+ {6868D03F-BD8E-46ED-9A5B-95346A3810A4}.Release|x64.ActiveCfg = Release|x64
+ {6868D03F-BD8E-46ED-9A5B-95346A3810A4}.Release|x64.Build.0 = Release|x64
+ {6868D03F-BD8E-46ED-9A5B-95346A3810A4}.Release|x86.ActiveCfg = Release|ARM64
+ {6868D03F-BD8E-46ED-9A5B-95346A3810A4}.Release|x86.Build.0 = Release|ARM64
+ EndGlobalSection
+ GlobalSection(SolutionProperties) = preSolution
+ HideSolutionNode = FALSE
+ EndGlobalSection
+EndGlobal
diff --git a/samples/cs/tutorial-tool-calling/Program.cs b/samples/cs/tutorial-tool-calling/Program.cs
new file mode 100644
index 00000000..74f137db
--- /dev/null
+++ b/samples/cs/tutorial-tool-calling/Program.cs
@@ -0,0 +1,228 @@
+//
+//
+using System.Text.Json;
+using Microsoft.AI.Foundry.Local;
+using Betalgo.Ranul.OpenAI.ObjectModels.RequestModels;
+using Betalgo.Ranul.OpenAI.ObjectModels.ResponseModels;
+using Betalgo.Ranul.OpenAI.ObjectModels.SharedModels;
+using Microsoft.Extensions.Logging;
+//
+
+CancellationToken ct = CancellationToken.None;
+
+//
+// --- Tool definitions ---
+List tools =
+[
+ new ToolDefinition
+ {
+ Type = "function",
+ Function = new FunctionDefinition()
+ {
+ Name = "get_weather",
+ Description = "Get the current weather for a location",
+ Parameters = new PropertyDefinition()
+ {
+ Type = "object",
+ Properties = new Dictionary()
+ {
+ { "location", new PropertyDefinition() { Type = "string", Description = "The city or location" } },
+ { "unit", new PropertyDefinition() { Type = "string", Description = "Temperature unit (celsius or fahrenheit)" } }
+ },
+ Required = ["location"]
+ }
+ }
+ },
+ new ToolDefinition
+ {
+ Type = "function",
+ Function = new FunctionDefinition()
+ {
+ Name = "calculate",
+ Description = "Perform a math calculation",
+ Parameters = new PropertyDefinition()
+ {
+ Type = "object",
+ Properties = new Dictionary()
+ {
+ { "expression", new PropertyDefinition() { Type = "string", Description = "The math expression to evaluate" } }
+ },
+ Required = ["expression"]
+ }
+ }
+ }
+];
+
+// --- Tool implementations ---
+string ExecuteTool(string functionName, JsonElement arguments)
+{
+ switch (functionName)
+ {
+ case "get_weather":
+ var location = arguments.GetProperty("location")
+ .GetString() ?? "unknown";
+ var unit = arguments.TryGetProperty("unit", out var u)
+ ? u.GetString() ?? "celsius"
+ : "celsius";
+ var temp = unit == "celsius" ? 22 : 72;
+ return JsonSerializer.Serialize(new
+ {
+ location,
+ temperature = temp,
+ unit,
+ condition = "Sunny"
+ });
+
+ case "calculate":
+ var expression = arguments.GetProperty("expression")
+ .GetString() ?? "";
+ try
+ {
+ var result = new System.Data.DataTable()
+ .Compute(expression, null);
+ return JsonSerializer.Serialize(new
+ {
+ expression,
+ result = result?.ToString()
+ });
+ }
+ catch (Exception ex)
+ {
+ return JsonSerializer.Serialize(new
+ {
+ error = ex.Message
+ });
+ }
+
+ default:
+ return JsonSerializer.Serialize(new
+ {
+ error = $"Unknown function: {functionName}"
+ });
+ }
+}
+//
+
+//
+// --- Main application ---
+var config = new Configuration
+{
+ AppName = "foundry_local_samples",
+ LogLevel = Microsoft.AI.Foundry.Local.LogLevel.Information
+};
+
+using var loggerFactory = LoggerFactory.Create(builder =>
+{
+ builder.SetMinimumLevel(
+ Microsoft.Extensions.Logging.LogLevel.Information
+ );
+});
+var logger = loggerFactory.CreateLogger();
+
+await FoundryLocalManager.CreateAsync(config, logger);
+var mgr = FoundryLocalManager.Instance;
+
+var catalog = await mgr.GetCatalogAsync();
+var model = await catalog.GetModelAsync("qwen2.5-0.5b")
+ ?? throw new Exception("Model not found");
+
+await model.DownloadAsync(progress =>
+{
+ Console.Write($"\rDownloading model: {progress:F2}%");
+ if (progress >= 100f) Console.WriteLine();
+});
+
+await model.LoadAsync();
+Console.WriteLine("Model loaded and ready.");
+
+var chatClient = await model.GetChatClientAsync();
+chatClient.Settings.ToolChoice = ToolChoice.Auto;
+
+var messages = new List
+{
+ new ChatMessage
+ {
+ Role = "system",
+ Content = "You are a helpful assistant with access to tools. " +
+ "Use them when needed to answer questions accurately."
+ }
+};
+//
+
+//
+Console.WriteLine("\nTool-calling assistant ready! Type 'quit' to exit.\n");
+
+while (true)
+{
+ Console.Write("You: ");
+ var userInput = Console.ReadLine();
+ if (string.IsNullOrWhiteSpace(userInput) ||
+ userInput.Equals("quit", StringComparison.OrdinalIgnoreCase) ||
+ userInput.Equals("exit", StringComparison.OrdinalIgnoreCase))
+ {
+ break;
+ }
+
+ messages.Add(new ChatMessage
+ {
+ Role = "user",
+ Content = userInput
+ });
+
+ var response = await chatClient.CompleteChatAsync(
+ messages, tools, ct
+ );
+
+ var choice = response.Choices[0].Message;
+
+ if (choice.ToolCalls is { Count: > 0 })
+ {
+ messages.Add(choice);
+
+ foreach (var toolCall in choice.ToolCalls)
+ {
+ var toolArgs = JsonDocument.Parse(
+ toolCall.FunctionCall.Arguments
+ ).RootElement;
+ Console.WriteLine(
+ $" Tool call: {toolCall.FunctionCall.Name}({toolArgs})"
+ );
+
+ var result = ExecuteTool(
+ toolCall.FunctionCall.Name, toolArgs
+ );
+ messages.Add(new ChatMessage
+ {
+ Role = "tool",
+ ToolCallId = toolCall.Id,
+ Content = result
+ });
+ }
+
+ var finalResponse = await chatClient.CompleteChatAsync(
+ messages, tools, ct
+ );
+ var answer = finalResponse.Choices[0].Message.Content ?? "";
+ messages.Add(new ChatMessage
+ {
+ Role = "assistant",
+ Content = answer
+ });
+ Console.WriteLine($"Assistant: {answer}\n");
+ }
+ else
+ {
+ var answer = choice.Content ?? "";
+ messages.Add(new ChatMessage
+ {
+ Role = "assistant",
+ Content = answer
+ });
+ Console.WriteLine($"Assistant: {answer}\n");
+ }
+}
+
+await model.UnloadAsync();
+Console.WriteLine("Model unloaded. Goodbye!");
+//
+//
diff --git a/samples/cs/tutorial-tool-calling/TutorialToolCalling.csproj b/samples/cs/tutorial-tool-calling/TutorialToolCalling.csproj
new file mode 100644
index 00000000..a3533047
--- /dev/null
+++ b/samples/cs/tutorial-tool-calling/TutorialToolCalling.csproj
@@ -0,0 +1,50 @@
+
+
+
+ Exe
+ enable
+ enable
+
+
+
+
+ net9.0-windows10.0.26100
+ false
+ ARM64;x64
+ None
+ false
+
+
+
+
+ net9.0
+
+
+
+ $(NETCoreSdkRuntimeIdentifier)
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
diff --git a/samples/cs/tutorial-tool-calling/TutorialToolCalling.sln b/samples/cs/tutorial-tool-calling/TutorialToolCalling.sln
new file mode 100644
index 00000000..6a86331b
--- /dev/null
+++ b/samples/cs/tutorial-tool-calling/TutorialToolCalling.sln
@@ -0,0 +1,34 @@
+
+Microsoft Visual Studio Solution File, Format Version 12.00
+# Visual Studio Version 17
+VisualStudioVersion = 17.0.31903.59
+MinimumVisualStudioVersion = 10.0.40219.1
+Project("{FAE04EC0-301F-11D3-BF4B-00C04F79EFBC}") = "TutorialToolCalling", "TutorialToolCalling.csproj", "{155923AB-A0C6-447D-A46A-7C8318D31596}"
+EndProject
+Global
+ GlobalSection(SolutionConfigurationPlatforms) = preSolution
+ Debug|Any CPU = Debug|Any CPU
+ Debug|x64 = Debug|x64
+ Debug|x86 = Debug|x86
+ Release|Any CPU = Release|Any CPU
+ Release|x64 = Release|x64
+ Release|x86 = Release|x86
+ EndGlobalSection
+ GlobalSection(ProjectConfigurationPlatforms) = postSolution
+ {155923AB-A0C6-447D-A46A-7C8318D31596}.Debug|Any CPU.ActiveCfg = Debug|ARM64
+ {155923AB-A0C6-447D-A46A-7C8318D31596}.Debug|Any CPU.Build.0 = Debug|ARM64
+ {155923AB-A0C6-447D-A46A-7C8318D31596}.Debug|x64.ActiveCfg = Debug|x64
+ {155923AB-A0C6-447D-A46A-7C8318D31596}.Debug|x64.Build.0 = Debug|x64
+ {155923AB-A0C6-447D-A46A-7C8318D31596}.Debug|x86.ActiveCfg = Debug|ARM64
+ {155923AB-A0C6-447D-A46A-7C8318D31596}.Debug|x86.Build.0 = Debug|ARM64
+ {155923AB-A0C6-447D-A46A-7C8318D31596}.Release|Any CPU.ActiveCfg = Release|ARM64
+ {155923AB-A0C6-447D-A46A-7C8318D31596}.Release|Any CPU.Build.0 = Release|ARM64
+ {155923AB-A0C6-447D-A46A-7C8318D31596}.Release|x64.ActiveCfg = Release|x64
+ {155923AB-A0C6-447D-A46A-7C8318D31596}.Release|x64.Build.0 = Release|x64
+ {155923AB-A0C6-447D-A46A-7C8318D31596}.Release|x86.ActiveCfg = Release|ARM64
+ {155923AB-A0C6-447D-A46A-7C8318D31596}.Release|x86.Build.0 = Release|ARM64
+ EndGlobalSection
+ GlobalSection(SolutionProperties) = preSolution
+ HideSolutionNode = FALSE
+ EndGlobalSection
+EndGlobal
diff --git a/samples/cs/tutorial-voice-to-text/Program.cs b/samples/cs/tutorial-voice-to-text/Program.cs
new file mode 100644
index 00000000..976b44e4
--- /dev/null
+++ b/samples/cs/tutorial-voice-to-text/Program.cs
@@ -0,0 +1,104 @@
+//
+//
+using Microsoft.AI.Foundry.Local;
+using Betalgo.Ranul.OpenAI.ObjectModels.RequestModels;
+using Microsoft.Extensions.Logging;
+using System.Text;
+//
+
+//
+CancellationToken ct = CancellationToken.None;
+
+var config = new Configuration
+{
+ AppName = "foundry_local_samples",
+ LogLevel = Microsoft.AI.Foundry.Local.LogLevel.Information
+};
+
+using var loggerFactory = LoggerFactory.Create(builder =>
+{
+ builder.SetMinimumLevel(
+ Microsoft.Extensions.Logging.LogLevel.Information
+ );
+});
+var logger = loggerFactory.CreateLogger();
+
+// Initialize the singleton instance
+await FoundryLocalManager.CreateAsync(config, logger);
+var mgr = FoundryLocalManager.Instance;
+var catalog = await mgr.GetCatalogAsync();
+//
+
+//
+// Load the speech-to-text model
+var speechModel = await catalog.GetModelAsync("whisper-tiny")
+ ?? throw new Exception("Speech model not found");
+
+await speechModel.DownloadAsync(progress =>
+{
+ Console.Write($"\rDownloading speech model: {progress:F2}%");
+ if (progress >= 100f) Console.WriteLine();
+});
+
+await speechModel.LoadAsync();
+Console.WriteLine("Speech model loaded.");
+
+// Transcribe the audio file
+var audioClient = await speechModel.GetAudioClientAsync();
+var transcriptionText = new StringBuilder();
+
+Console.WriteLine("\nTranscription:");
+var audioResponse = audioClient
+ .TranscribeAudioStreamingAsync("meeting-notes.wav", ct);
+await foreach (var chunk in audioResponse)
+{
+ Console.Write(chunk.Text);
+ transcriptionText.Append(chunk.Text);
+}
+Console.WriteLine();
+
+// Unload the speech model to free memory
+await speechModel.UnloadAsync();
+//
+
+//
+// Load the chat model for summarization
+var chatModel = await catalog.GetModelAsync("qwen2.5-0.5b")
+ ?? throw new Exception("Chat model not found");
+
+await chatModel.DownloadAsync(progress =>
+{
+ Console.Write($"\rDownloading chat model: {progress:F2}%");
+ if (progress >= 100f) Console.WriteLine();
+});
+
+await chatModel.LoadAsync();
+Console.WriteLine("Chat model loaded.");
+
+// Summarize the transcription into organized notes
+var chatClient = await chatModel.GetChatClientAsync();
+var messages = new List
+{
+ new ChatMessage
+ {
+ Role = "system",
+ Content = "You are a note-taking assistant. Summarize " +
+ "the following transcription into organized, " +
+ "concise notes with bullet points."
+ },
+ new ChatMessage
+ {
+ Role = "user",
+ Content = transcriptionText.ToString()
+ }
+};
+
+var chatResponse = await chatClient.CompleteChatAsync(messages, ct);
+var summary = chatResponse.Choices[0].Message.Content;
+Console.WriteLine($"\nSummary:\n{summary}");
+
+// Clean up
+await chatModel.UnloadAsync();
+Console.WriteLine("\nDone. Models unloaded.");
+//
+//
diff --git a/samples/cs/tutorial-voice-to-text/TutorialVoiceToText.csproj b/samples/cs/tutorial-voice-to-text/TutorialVoiceToText.csproj
new file mode 100644
index 00000000..a3533047
--- /dev/null
+++ b/samples/cs/tutorial-voice-to-text/TutorialVoiceToText.csproj
@@ -0,0 +1,50 @@
+
+
+
+ Exe
+ enable
+ enable
+
+
+
+
+ net9.0-windows10.0.26100
+ false
+ ARM64;x64
+ None
+ false
+
+
+
+
+ net9.0
+
+
+
+ $(NETCoreSdkRuntimeIdentifier)
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
diff --git a/samples/cs/tutorial-voice-to-text/TutorialVoiceToText.sln b/samples/cs/tutorial-voice-to-text/TutorialVoiceToText.sln
new file mode 100644
index 00000000..ae2a2b39
--- /dev/null
+++ b/samples/cs/tutorial-voice-to-text/TutorialVoiceToText.sln
@@ -0,0 +1,34 @@
+
+Microsoft Visual Studio Solution File, Format Version 12.00
+# Visual Studio Version 17
+VisualStudioVersion = 17.0.31903.59
+MinimumVisualStudioVersion = 10.0.40219.1
+Project("{FAE04EC0-301F-11D3-BF4B-00C04F79EFBC}") = "TutorialVoiceToText", "TutorialVoiceToText.csproj", "{C12663C3-AB3F-4652-BC43-A92E43602ACC}"
+EndProject
+Global
+ GlobalSection(SolutionConfigurationPlatforms) = preSolution
+ Debug|Any CPU = Debug|Any CPU
+ Debug|x64 = Debug|x64
+ Debug|x86 = Debug|x86
+ Release|Any CPU = Release|Any CPU
+ Release|x64 = Release|x64
+ Release|x86 = Release|x86
+ EndGlobalSection
+ GlobalSection(ProjectConfigurationPlatforms) = postSolution
+ {C12663C3-AB3F-4652-BC43-A92E43602ACC}.Debug|Any CPU.ActiveCfg = Debug|ARM64
+ {C12663C3-AB3F-4652-BC43-A92E43602ACC}.Debug|Any CPU.Build.0 = Debug|ARM64
+ {C12663C3-AB3F-4652-BC43-A92E43602ACC}.Debug|x64.ActiveCfg = Debug|x64
+ {C12663C3-AB3F-4652-BC43-A92E43602ACC}.Debug|x64.Build.0 = Debug|x64
+ {C12663C3-AB3F-4652-BC43-A92E43602ACC}.Debug|x86.ActiveCfg = Debug|ARM64
+ {C12663C3-AB3F-4652-BC43-A92E43602ACC}.Debug|x86.Build.0 = Debug|ARM64
+ {C12663C3-AB3F-4652-BC43-A92E43602ACC}.Release|Any CPU.ActiveCfg = Release|ARM64
+ {C12663C3-AB3F-4652-BC43-A92E43602ACC}.Release|Any CPU.Build.0 = Release|ARM64
+ {C12663C3-AB3F-4652-BC43-A92E43602ACC}.Release|x64.ActiveCfg = Release|x64
+ {C12663C3-AB3F-4652-BC43-A92E43602ACC}.Release|x64.Build.0 = Release|x64
+ {C12663C3-AB3F-4652-BC43-A92E43602ACC}.Release|x86.ActiveCfg = Release|ARM64
+ {C12663C3-AB3F-4652-BC43-A92E43602ACC}.Release|x86.Build.0 = Release|ARM64
+ EndGlobalSection
+ GlobalSection(SolutionProperties) = preSolution
+ HideSolutionNode = FALSE
+ EndGlobalSection
+EndGlobal
diff --git a/samples/js/audio-transcription-example/app.js b/samples/js/audio-transcription-example/app.js
index 78efc8af..c2517ec7 100644
--- a/samples/js/audio-transcription-example/app.js
+++ b/samples/js/audio-transcription-example/app.js
@@ -1,14 +1,20 @@
+//
+//
import { FoundryLocalManager } from 'foundry-local-sdk';
+//
// Initialize the Foundry Local SDK
console.log('Initializing Foundry Local SDK...');
+//
const manager = FoundryLocalManager.create({
appName: 'foundry_local_samples',
logLevel: 'info'
});
+//
console.log('✓ SDK initialized successfully');
+//
// Get the model object
const modelAlias = 'whisper-tiny'; // Using an available model from the list above
let model = await manager.catalog.getModel(modelAlias);
@@ -25,15 +31,18 @@ console.log('\n✓ Model downloaded');
console.log(`\nLoading model ${modelAlias}...`);
await model.load();
console.log('✓ Model loaded');
+//
+//
// Create audio client
console.log('\nCreating audio client...');
const audioClient = model.createAudioClient();
console.log('✓ Audio client created');
// Example audio transcription
-console.log('\nTesting audio transcription...');
-const transcription = await audioClient.transcribe('./Recording.mp3');
+const audioFile = process.argv[2] || './Recording.mp3';
+console.log(`\nTranscribing ${audioFile}...`);
+const transcription = await audioClient.transcribe(audioFile);
console.log('\nAudio transcription result:');
console.log(transcription.text);
@@ -41,13 +50,17 @@ console.log('✓ Audio transcription completed');
// Same example but with streaming transcription using async iteration
console.log('\nTesting streaming audio transcription...');
-for await (const result of audioClient.transcribeStreaming('./Recording.mp3')) {
+for await (const result of audioClient.transcribeStreaming(audioFile)) {
// Output the intermediate transcription results as they are received without line ending
process.stdout.write(result.text);
}
console.log('\n✓ Streaming transcription completed');
+//
+//
// Unload the model
console.log('Unloading model...');
await model.unload();
console.log(`✓ Model unloaded`);
+//
+//
diff --git a/samples/js/chat-and-audio-foundry-local/src/app.js b/samples/js/chat-and-audio-foundry-local/src/app.js
index 49ce199c..50bc195f 100644
--- a/samples/js/chat-and-audio-foundry-local/src/app.js
+++ b/samples/js/chat-and-audio-foundry-local/src/app.js
@@ -11,7 +11,7 @@ const WHISPER_MODEL = "whisper-tiny";
async function main() {
console.log("Initializing Foundry Local SDK...");
const manager = FoundryLocalManager.create({
- appName: "ChatAndAudioSample",
+ appName: "foundry_local_samples",
logLevel: "info",
});
diff --git a/samples/js/langchain-integration-example/app.js b/samples/js/langchain-integration-example/app.js
index 94e0afdc..9e4b7b60 100644
--- a/samples/js/langchain-integration-example/app.js
+++ b/samples/js/langchain-integration-example/app.js
@@ -1,17 +1,22 @@
+//
+//
import { ChatOpenAI } from "@langchain/openai";
import { ChatPromptTemplate } from "@langchain/core/prompts";
import { FoundryLocalManager } from 'foundry-local-sdk';
+//
// Initialize the Foundry Local SDK
console.log('Initializing Foundry Local SDK...');
const endpointUrl = 'http://localhost:5764';
+//
const manager = FoundryLocalManager.create({
appName: 'foundry_local_samples',
logLevel: 'info',
webServiceUrls: endpointUrl
});
+//
console.log('✓ SDK initialized successfully');
// Get the model object
@@ -35,6 +40,7 @@ console.log('\nStarting web service...');
manager.startWebService();
console.log('✓ Web service started');
+//
// Configure ChatOpenAI to use your locally-running model
const llm = new ChatOpenAI({
@@ -61,7 +67,9 @@ const prompt = ChatPromptTemplate.fromMessages([
// Build a simple chain by connecting the prompt to the language model
const chain = prompt.pipe(llm);
+//
+//
const input = "I love to code.";
console.log(`Translating '${input}' to French...`);
@@ -76,9 +84,11 @@ await chain.invoke({
}).catch(err => {
console.error("Error:", err);
});
+//
// Tidy up
console.log('Unloading model and stopping web service...');
await model.unload();
manager.stopWebService();
-console.log(`✓ Model unloaded and web service stopped`);
\ No newline at end of file
+console.log(`✓ Model unloaded and web service stopped`);
+//
\ No newline at end of file
diff --git a/samples/js/native-chat-completions/app.js b/samples/js/native-chat-completions/app.js
index 67348e8c..399fd634 100644
--- a/samples/js/native-chat-completions/app.js
+++ b/samples/js/native-chat-completions/app.js
@@ -1,14 +1,20 @@
+//
+//
import { FoundryLocalManager } from 'foundry-local-sdk';
+//
// Initialize the Foundry Local SDK
console.log('Initializing Foundry Local SDK...');
+//
const manager = FoundryLocalManager.create({
appName: 'foundry_local_samples',
logLevel: 'info'
});
+//
console.log('✓ SDK initialized successfully');
+//
// Get the model object
const modelAlias = 'qwen2.5-0.5b'; // Using an available model from the list above
const model = await manager.catalog.getModel(modelAlias);
@@ -24,7 +30,9 @@ console.log('\n✓ Model downloaded');
console.log(`\nLoading model ${modelAlias}...`);
await model.load();
console.log('✓ Model loaded');
+//
+//
// Create chat client
console.log('\nCreating chat client...');
const chatClient = model.createChatClient();
@@ -38,7 +46,9 @@ const completion = await chatClient.completeChat([
console.log('\nChat completion result:');
console.log(completion.choices[0]?.message?.content);
+//
+//
// Example streaming completion
console.log('\nTesting streaming completion...');
for await (const chunk of chatClient.completeStreamingChat(
@@ -50,9 +60,13 @@ for await (const chunk of chatClient.completeStreamingChat(
}
}
console.log('\n');
+//
+//
// Unload the model
console.log('Unloading model...');
await model.unload();
console.log(`✓ Model unloaded`);
+//
+//
\ No newline at end of file
diff --git a/samples/js/tool-calling-foundry-local/src/app.js b/samples/js/tool-calling-foundry-local/src/app.js
index f11eacdd..f92464ee 100644
--- a/samples/js/tool-calling-foundry-local/src/app.js
+++ b/samples/js/tool-calling-foundry-local/src/app.js
@@ -1,8 +1,11 @@
+//
// Copyright (c) Microsoft Corporation. All rights reserved.
// Licensed under the MIT License.
+//
import { OpenAI } from "openai";
import { FoundryLocalManager } from "foundry-local-sdk";
+//
// By using an alias, the most suitable model will be downloaded
// to your end-user's device.
@@ -10,22 +13,27 @@ import { FoundryLocalManager } from "foundry-local-sdk";
// following command in your terminal: `foundry model list`.
const alias = "qwen2.5-0.5b";
+//
function multiplyNumbers(first, second) {
return first * second;
}
+//
async function runToolCallingExample() {
let manager = null;
let model = null;
try {
+ //
console.log("Initializing Foundry Local SDK...");
manager = FoundryLocalManager.create({
- appName: "FoundryLocalSample",
+ appName: "foundry_local_samples",
serviceEndpoint: "http://localhost:5000",
logLevel: "info"
});
+ //
+ //
const catalog = manager.catalog;
model = await catalog.getModel(alias);
if (!model) {
@@ -47,7 +55,9 @@ async function runToolCallingExample() {
baseURL: `${endpoint.replace(/\/$/, "")}/v1`,
apiKey: "local"
});
+ //
+ //
// Prepare messages
const messages = [
{
@@ -154,7 +164,9 @@ async function runToolCallingExample() {
}
console.log();
+ //
} finally {
+ //
if (model) {
try {
if (await model.isLoaded()) {
@@ -172,6 +184,7 @@ async function runToolCallingExample() {
console.warn("Cleanup warning while stopping service:", cleanupError);
}
}
+ //
}
}
@@ -179,3 +192,4 @@ await runToolCallingExample().catch((error) => {
console.error("Error running sample:", error);
process.exitCode = 1;
});
+//
diff --git a/samples/js/tutorial-chat-assistant/app.js b/samples/js/tutorial-chat-assistant/app.js
new file mode 100644
index 00000000..9a5a430c
--- /dev/null
+++ b/samples/js/tutorial-chat-assistant/app.js
@@ -0,0 +1,84 @@
+//
+//
+import { FoundryLocalManager } from 'foundry-local-sdk';
+import * as readline from 'readline';
+//
+
+//
+// Initialize the Foundry Local SDK
+const manager = FoundryLocalManager.create({
+ appName: 'foundry_local_samples',
+ logLevel: 'info'
+});
+
+// Select and load a model from the catalog
+const model = await manager.catalog.getModel('qwen2.5-0.5b');
+
+await model.download((progress) => {
+ process.stdout.write(`\rDownloading model: ${progress.toFixed(2)}%`);
+});
+console.log('\nModel downloaded.');
+
+await model.load();
+console.log('Model loaded and ready.');
+
+// Create a chat client
+const chatClient = model.createChatClient();
+//
+
+//
+// Start the conversation with a system prompt
+const messages = [
+ {
+ role: 'system',
+ content: 'You are a helpful, friendly assistant. Keep your responses ' +
+ 'concise and conversational. If you don\'t know something, say so.'
+ }
+];
+//
+
+// Set up readline for console input
+const rl = readline.createInterface({
+ input: process.stdin,
+ output: process.stdout
+});
+
+const askQuestion = (prompt) => new Promise((resolve) => rl.question(prompt, resolve));
+
+console.log('\nChat assistant ready! Type \'quit\' to exit.\n');
+
+//
+while (true) {
+ const userInput = await askQuestion('You: ');
+ if (userInput.trim().toLowerCase() === 'quit' ||
+ userInput.trim().toLowerCase() === 'exit') {
+ break;
+ }
+
+ // Add the user's message to conversation history
+ messages.push({ role: 'user', content: userInput });
+
+ //
+ // Stream the response token by token
+ process.stdout.write('Assistant: ');
+ let fullResponse = '';
+ await chatClient.completeStreamingChat(messages, (chunk) => {
+ const content = chunk.choices?.[0]?.message?.content;
+ if (content) {
+ process.stdout.write(content);
+ fullResponse += content;
+ }
+ });
+ console.log('\n');
+ //
+
+ // Add the complete response to conversation history
+ messages.push({ role: 'assistant', content: fullResponse });
+}
+//
+
+// Clean up - unload the model
+await model.unload();
+console.log('Model unloaded. Goodbye!');
+rl.close();
+//
diff --git a/samples/js/tutorial-chat-assistant/package.json b/samples/js/tutorial-chat-assistant/package.json
new file mode 100644
index 00000000..3e2393ce
--- /dev/null
+++ b/samples/js/tutorial-chat-assistant/package.json
@@ -0,0 +1,9 @@
+{
+ "name": "tutorial-chat-assistant",
+ "version": "1.0.0",
+ "type": "module",
+ "main": "app.js",
+ "dependencies": {
+ "foundry-local-sdk": "*"
+ }
+}
diff --git a/samples/js/tutorial-document-summarizer/app.js b/samples/js/tutorial-document-summarizer/app.js
new file mode 100644
index 00000000..f43e204d
--- /dev/null
+++ b/samples/js/tutorial-document-summarizer/app.js
@@ -0,0 +1,84 @@
+//
+//
+import { FoundryLocalManager } from 'foundry-local-sdk';
+import { readFileSync, readdirSync, statSync } from 'fs';
+import { join, basename } from 'path';
+//
+
+async function summarizeFile(chatClient, filePath, systemPrompt) {
+ const content = readFileSync(filePath, 'utf-8');
+ const messages = [
+ { role: 'system', content: systemPrompt },
+ { role: 'user', content: content }
+ ];
+
+ const response = await chatClient.completeChat(messages);
+ console.log(response.choices[0]?.message?.content);
+}
+
+async function summarizeDirectory(chatClient, directory, systemPrompt) {
+ const txtFiles = readdirSync(directory)
+ .filter(f => f.endsWith('.txt'))
+ .sort();
+
+ if (txtFiles.length === 0) {
+ console.log(`No .txt files found in ${directory}`);
+ return;
+ }
+
+ for (const fileName of txtFiles) {
+ console.log(`--- ${fileName} ---`);
+ await summarizeFile(chatClient, join(directory, fileName), systemPrompt);
+ console.log();
+ }
+}
+
+//
+// Initialize the Foundry Local SDK
+const manager = FoundryLocalManager.create({
+ appName: 'foundry_local_samples',
+ logLevel: 'info'
+});
+
+// Select and load a model from the catalog
+const model = await manager.catalog.getModel('qwen2.5-0.5b');
+
+await model.download((progress) => {
+ process.stdout.write(`\rDownloading model: ${progress.toFixed(2)}%`);
+});
+console.log('\nModel downloaded.');
+
+await model.load();
+console.log('Model loaded and ready.\n');
+
+// Create a chat client
+const chatClient = model.createChatClient();
+//
+
+//
+const systemPrompt =
+ 'Summarize the following document into concise bullet points. ' +
+ 'Focus on the key points and main ideas.';
+
+//
+const target = process.argv[2] || 'document.txt';
+//
+
+try {
+ const stats = statSync(target);
+ if (stats.isDirectory()) {
+ await summarizeDirectory(chatClient, target, systemPrompt);
+ } else {
+ console.log(`--- ${basename(target)} ---`);
+ await summarizeFile(chatClient, target, systemPrompt);
+ }
+} catch {
+ console.log(`--- ${basename(target)} ---`);
+ await summarizeFile(chatClient, target, systemPrompt);
+}
+//
+
+// Clean up
+await model.unload();
+console.log('\nModel unloaded. Done!');
+//
diff --git a/samples/js/tutorial-document-summarizer/package.json b/samples/js/tutorial-document-summarizer/package.json
new file mode 100644
index 00000000..c3c62321
--- /dev/null
+++ b/samples/js/tutorial-document-summarizer/package.json
@@ -0,0 +1,9 @@
+{
+ "name": "tutorial-document-summarizer",
+ "version": "1.0.0",
+ "type": "module",
+ "main": "app.js",
+ "dependencies": {
+ "foundry-local-sdk": "*"
+ }
+}
diff --git a/samples/js/tutorial-tool-calling/app.js b/samples/js/tutorial-tool-calling/app.js
new file mode 100644
index 00000000..efdd710c
--- /dev/null
+++ b/samples/js/tutorial-tool-calling/app.js
@@ -0,0 +1,186 @@
+//
+//
+import { FoundryLocalManager } from 'foundry-local-sdk';
+import * as readline from 'readline';
+//
+
+//
+// --- Tool definitions ---
+const tools = [
+ {
+ type: 'function',
+ function: {
+ name: 'get_weather',
+ description: 'Get the current weather for a location',
+ parameters: {
+ type: 'object',
+ properties: {
+ location: {
+ type: 'string',
+ description: 'The city or location'
+ },
+ unit: {
+ type: 'string',
+ enum: ['celsius', 'fahrenheit'],
+ description: 'Temperature unit'
+ }
+ },
+ required: ['location']
+ }
+ }
+ },
+ {
+ type: 'function',
+ function: {
+ name: 'calculate',
+ description: 'Perform a math calculation',
+ parameters: {
+ type: 'object',
+ properties: {
+ expression: {
+ type: 'string',
+ description:
+ 'The math expression to evaluate'
+ }
+ },
+ required: ['expression']
+ }
+ }
+ }
+];
+
+// --- Tool implementations ---
+function getWeather(location, unit = 'celsius') {
+ return {
+ location,
+ temperature: unit === 'celsius' ? 22 : 72,
+ unit,
+ condition: 'Sunny'
+ };
+}
+
+function calculate(expression) {
+ // Input is validated against a strict allowlist of numeric/math characters,
+ // making this safe from code injection in this tutorial context.
+ const allowed = /^[0-9+\-*/(). ]+$/;
+ if (!allowed.test(expression)) {
+ return { error: 'Invalid expression' };
+ }
+ try {
+ const result = Function(
+ `"use strict"; return (${expression})`
+ )();
+ return { expression, result };
+ } catch (err) {
+ return { error: err.message };
+ }
+}
+
+const toolFunctions = {
+ get_weather: (args) => getWeather(args.location, args.unit),
+ calculate: (args) => calculate(args.expression)
+};
+//
+
+//
+async function processToolCalls(messages, response, chatClient) {
+ let choice = response.choices[0]?.message;
+
+ while (choice?.tool_calls?.length > 0) {
+ messages.push(choice);
+
+ for (const toolCall of choice.tool_calls) {
+ const functionName = toolCall.function.name;
+ const args = JSON.parse(toolCall.function.arguments);
+ console.log(
+ ` Tool call: ${functionName}` +
+ `(${JSON.stringify(args)})`
+ );
+
+ const result = toolFunctions[functionName](args);
+ messages.push({
+ role: 'tool',
+ tool_call_id: toolCall.id,
+ content: JSON.stringify(result)
+ });
+ }
+
+ response = await chatClient.completeChat(
+ messages, { tools }
+ );
+ choice = response.choices[0]?.message;
+ }
+
+ return choice?.content ?? '';
+}
+//
+
+//
+// --- Main application ---
+const manager = FoundryLocalManager.create({
+ appName: 'foundry_local_samples',
+ logLevel: 'info'
+});
+
+const model = await manager.catalog.getModel('qwen2.5-0.5b');
+
+await model.download((progress) => {
+ process.stdout.write(
+ `\rDownloading model: ${progress.toFixed(2)}%`
+ );
+});
+console.log('\nModel downloaded.');
+
+await model.load();
+console.log('Model loaded and ready.');
+
+const chatClient = model.createChatClient();
+
+const messages = [
+ {
+ role: 'system',
+ content:
+ 'You are a helpful assistant with access to tools. ' +
+ 'Use them when needed to answer questions accurately.'
+ }
+];
+
+const rl = readline.createInterface({
+ input: process.stdin,
+ output: process.stdout
+});
+
+const askQuestion = (prompt) =>
+ new Promise((resolve) => rl.question(prompt, resolve));
+
+console.log(
+ '\nTool-calling assistant ready! Type \'quit\' to exit.\n'
+);
+
+while (true) {
+ const userInput = await askQuestion('You: ');
+ if (
+ userInput.trim().toLowerCase() === 'quit' ||
+ userInput.trim().toLowerCase() === 'exit'
+ ) {
+ break;
+ }
+
+ messages.push({ role: 'user', content: userInput });
+
+ const response = await chatClient.completeChat(
+ messages, { tools }
+ );
+ const answer = await processToolCalls(
+ messages, response, chatClient
+ );
+
+ messages.push({ role: 'assistant', content: answer });
+ console.log(`Assistant: ${answer}\n`);
+}
+
+await model.unload();
+console.log('Model unloaded. Goodbye!');
+rl.close();
+//
+//
diff --git a/samples/js/tutorial-tool-calling/package.json b/samples/js/tutorial-tool-calling/package.json
new file mode 100644
index 00000000..07337434
--- /dev/null
+++ b/samples/js/tutorial-tool-calling/package.json
@@ -0,0 +1,9 @@
+{
+ "name": "tutorial-tool-calling",
+ "version": "1.0.0",
+ "type": "module",
+ "main": "app.js",
+ "dependencies": {
+ "foundry-local-sdk": "*"
+ }
+}
diff --git a/samples/js/tutorial-voice-to-text/app.js b/samples/js/tutorial-voice-to-text/app.js
new file mode 100644
index 00000000..08074100
--- /dev/null
+++ b/samples/js/tutorial-voice-to-text/app.js
@@ -0,0 +1,78 @@
+//
+//
+import { FoundryLocalManager } from 'foundry-local-sdk';
+import { fileURLToPath } from 'url';
+import path from 'path';
+//
+
+const __dirname = path.dirname(fileURLToPath(import.meta.url));
+
+//
+// Initialize the Foundry Local SDK
+const manager = FoundryLocalManager.create({
+ appName: 'foundry_local_samples',
+ logLevel: 'info'
+});
+//
+
+//
+// Load the speech-to-text model
+const speechModel = await manager.catalog.getModel('whisper-tiny');
+await speechModel.download((progress) => {
+ process.stdout.write(
+ `\rDownloading speech model: ${progress.toFixed(2)}%`
+ );
+});
+console.log('\nSpeech model downloaded.');
+
+await speechModel.load();
+console.log('Speech model loaded.');
+
+// Transcribe the audio file
+const audioClient = speechModel.createAudioClient();
+const transcription = await audioClient.transcribe(
+ path.join(__dirname, 'meeting-notes.wav')
+);
+console.log(`\nTranscription:\n${transcription.text}`);
+
+// Unload the speech model to free memory
+await speechModel.unload();
+//
+
+//
+// Load the chat model for summarization
+const chatModel = await manager.catalog.getModel('qwen2.5-0.5b');
+await chatModel.download((progress) => {
+ process.stdout.write(
+ `\rDownloading chat model: ${progress.toFixed(2)}%`
+ );
+});
+console.log('\nChat model downloaded.');
+
+await chatModel.load();
+console.log('Chat model loaded.');
+
+// Summarize the transcription into organized notes
+const chatClient = chatModel.createChatClient();
+const messages = [
+ {
+ role: 'system',
+ content: 'You are a note-taking assistant. Summarize ' +
+ 'the following transcription into organized, ' +
+ 'concise notes with bullet points.'
+ },
+ {
+ role: 'user',
+ content: transcription.text
+ }
+];
+
+const response = await chatClient.completeChat(messages);
+const summary = response.choices[0]?.message?.content;
+console.log(`\nSummary:\n${summary}`);
+
+// Clean up
+await chatModel.unload();
+console.log('\nDone. Models unloaded.');
+//
+//
diff --git a/samples/js/tutorial-voice-to-text/package.json b/samples/js/tutorial-voice-to-text/package.json
new file mode 100644
index 00000000..55f2ea83
--- /dev/null
+++ b/samples/js/tutorial-voice-to-text/package.json
@@ -0,0 +1,9 @@
+{
+ "name": "tutorial-voice-to-text",
+ "version": "1.0.0",
+ "type": "module",
+ "main": "app.js",
+ "dependencies": {
+ "foundry-local-sdk": "*"
+ }
+}
diff --git a/samples/js/web-server-example/app.js b/samples/js/web-server-example/app.js
index 5e97edfc..b03bf9df 100644
--- a/samples/js/web-server-example/app.js
+++ b/samples/js/web-server-example/app.js
@@ -1,18 +1,24 @@
+//
+//
import { FoundryLocalManager } from 'foundry-local-sdk';
import { OpenAI } from 'openai';
+//
// Initialize the Foundry Local SDK
console.log('Initializing Foundry Local SDK...');
const endpointUrl = 'http://localhost:5764';
+//
const manager = FoundryLocalManager.create({
appName: 'foundry_local_samples',
logLevel: 'info',
webServiceUrls: endpointUrl
});
+//
console.log('✓ SDK initialized successfully');
+//
// Get the model object
const modelAlias = 'qwen2.5-0.5b'; // Using an available model from the list above
const model = await manager.catalog.getModel(modelAlias);
@@ -28,7 +34,9 @@ console.log('\n✓ Model downloaded');
console.log(`\nLoading model ${modelAlias}...`);
await model.load();
console.log('✓ Model loaded');
+//
+//
// Start the web service
console.log('\nStarting web service...');
manager.startWebService();
@@ -52,9 +60,11 @@ const response = await openai.chat.completions.create({
});
console.log(response.choices[0].message.content);
+//
// Tidy up
console.log('Unloading model and stopping web service...');
await model.unload();
manager.stopWebService();
console.log(`✓ Model unloaded and web service stopped`);
+//
diff --git a/samples/python/audio-transcription/Recording.mp3 b/samples/python/audio-transcription/Recording.mp3
new file mode 100644
index 0000000000000000000000000000000000000000..deb38418bf5fde82fe380add4a999d513baa9536
GIT binary patch
literal 329760
zcmd4330xC*w>Lgn2_#_HLRiFrf{_Y;fg&QZ83&3w34(wqi?)?Mv?OdIi#st0
zDzb`=8W!7XttMen2`G!UQf)IcE&h-)}+YON^auTKI?*A%zo!2*N1LIYgGGnHpDbw&ft|fCY#z$?S?P1vb+h9wcZe=@oA
zx2YqOkv+j%*4Ff8W7x-Eel(HN+0F^)2ey3o+Y{*I>Rs=ZSKj}0Zs{h2p5?$cmLwee
zG$tp@%=YPbCXJ&%;L=dQn@ByqxG
zg(B#F>ThGURTEhs8D{>)BJ~t;!G5pzCx;Uk^*v6uHyvb_84P}vk1&sapcx&fXgJGaJ%WHD?NO(7Lv-0ScfTwk;9o&y?=}6`>(M
zwyTqc6vcbM8SZBpc=oz^=b@Fs0QipkfApA!+{#7}9A7DdEFal@J1u#swT!bfk~a9V
z(`%KYKXsZ~!Q35U&%c=TAa(kqE9HZ|MR`}M;EM7RK3DDF+0Ux7UUz8Vt7yX^GMC-$
z2X1hR=m#nTxqMD`U$-XaU=caY*3{_$hV>UR>!p3&5-q2%J2|APf0!b>h~CBT53H|1
z?X9bHq}=xQs?rC4c4?Sk(b(~okzBrNi}qA-aNrhgkOlL=L)sBOHlA0yifV0gsCeo~
zjg{*n&L?Z!1*!+L${nb^;yx@`)bEmWE@IpD$rAN_Wu})jC?I=|XXVN%qsYjr=$Ufi
zSW)m%IoYB^dM&z3$>VqNc$5-pmsT^RVpW%vh{Ywcbj*%R)gi;>*f8poQgQ|5d1*&I
zJ-IxtaH@;gWzj_v2KiW72El?gQP)bso3r9M-
zdAz9(ttMT*+*&^K?)hqYwcIvcESFpV`7aMsDZ6y^2;CY=i=Us!5497tH`?2A;|yFo
zo~d<}VrZRCZVRQ_Ij#$)z%(T8f9VyJmCpkGZIYrjd-l2Y=$Qj#X
z#3Bs@2C5uI4x;&b@f>Q#e>SC>o;QzYG=qL1CtX$sbQhI(U+wQc;OX}^aUcn$@nqqw
z#P~7K#qw(60bbXnwZ4T%=Oj|HWR0h@&Zf@%pHkgs6g7VP8hYYz+I-I(~c9N7lEuhaE`|
zGrWSWu-lJ#^m*4Fb2lrxCb!0&s-U#I$Eq1ZMLa?JZIvgNRb5)yWY2Y%lTz(XS)^1R
ze3>X27GsGbJD$~mkGCbe;5w%K*qAb?@2pyeLQaGYXzr?CZPZx)u4eu+h
zAf?C}+8Zt`w2z&(x0`D3jI|eB@KS%Rxcfj(XLNNbVN$(2k%;anIg!%byeAmP-RtXB
zyFC44@wJf$@bK_>tURnssZu5A3r9V9)1I(vN|TMA=0;BDOozys8#
zfIgOX8xR6;lEeIbL^!@b`Tq|1?#$bV%P)Y8_|J)a^|D4gd#+u;9q0C|2A(@S3Vju%
zP#It?4!_+u+gIA(SDBeLFn#%E;H?1Vlx8BYcdJ1@j|=OaIwO{u;q_Yi%@zB3^9}l)
z>;@KPO5NgCBBK-Y%ACsGQGBhs9JTE46@mYtO;ovC#Aw~Mw%#8oIL7j8_c+EYh<3@7Z$RFW}&OD`;
zKA!iMUszm<8F#tgs>XS5@u#6x$8i>^zk62o!GwbY`WAN{*UVdm_wWoB)62K;A2*tr
zaZ{EtAJ3b|AAg);#+z?Dl+$69w`V@fOh=gLe;-7}LFBmAVZ-o=oa1A4c?o?vKa5=)
zzV=}2F#I3uEvhhLE$2CGbwmS|j{_fd>gHFU{9(Aa=&~vgn}%(}t;X_OIKD5JnHb~v
zZhC|7iv3&w?N;FL0qxLlaz|(7bcK=JjOV!Zedz7S_OTZ^*yDgln!cRA9R1enU(tY@
zIK+K9_&Ez^u%Eqedf(*DpUBe}%_^q@e&z-$T!R;$&
zRZi#$kvrNf%ZE)y44r6Qjxv>T%*`>)%VN&QG-!fib(G(^4W|;)PB(tsYLd%5-*)Tw
zsDE9wU4=K_+|CnosZ(qq(q2JTH*&cp7BkIYJ2;P;llf;+pOPO|Lf^C4*^oo
zdpD)%rHW%4%#o892#`}=#X``;-1+&qQugGTHG{I|SeLoT$V#Dr3~G15R4*aXSd5${
z4>2Q#=rosd(tEqSw76zv&48qe)9X3b_0>airYQj#N{bjWC6IOVK{R!^eCj|c58-N-
z(WM%S&R0oAg=ucC(n3)o8pPw+5eY-})%C|s36kPTH&Y7vENONq8ag;jRzZX5AR&(<
zXBNOdI*n31k1O=Cv(OsX*B>(>h{Z2Z(PujP8akbBDtX}hNyJxjmi1ZRPu5CCw6}OX
zZsIo9j4}~Lw}}?4KY`--{=WPxIurTe!yRw%MMWx=>Z~|XtL;wgY9Aexj9e%wX)Y-#
zDQy=gcKfgEf0BoNUDKG7IxyDJxmllDId;`Mta7YD1OG0oGF97j)XPeD%pF}tQz+fV
zf)*2T99wur9A{m^BXdo}!PW4eN8!sN-GbGzs^;|aQAKnq(+P7brlBSxE9K(q626S#
z7OP&zm(d9(+8J!IoT!Ly7UwJ7OK2j?eyyCChi>MdGSONW^JOb_(8lmGC}e$HwH=pL
zekQxT+^5xvaxftZm+k&uw+C(7s^9C}zBaV=Mh2}7^_*kVda#TkVvQp
zAOQTd*V8VSVoMl$$uJ~q2@>g9WD)>!EtH~9Cfd`_J{5g7DA8VMxgeGhlIC%x*&I2i
zT5aGKW@E>COFP>($0e%1df{8K($N1^^;rWB-OO3Qsap9bZ17zDk}xOgI)CcyyL!N*
zWDh-xTbihPrllJjWyp;yp!Jw$L8O2~J5c}Ws3wD<+Fij1I`AHjv5R);zOXiQ?fNw)Gvc`QvD%UhmNhq8b$%wXxy0lS
zH~vnX&cv>~p!$rc)i8q{UUYIwvvOj!%Ln*6Oq`QErf7A7%a$6lL2f+N5pjL0Bmc(Y
z{u^Af(vnP6JBkQW0!qQ;1io-e6{xgOQ)w$FXcTF_ihx+Ofqj)x9i}&gaCzb1_)PVXx6h6Zva5Oj%
zTgSD$tb`8wdmgn1_B_XD)#ALPs|Dx@$?iJIhf$L3I?-ag$i>x1Q)B_E#nyW#>gv*O
z%%M?J+Pv1q1#-&@V}0?IHb)Ggj-IP9?oIDIaa>>gNSo8TxVqIYH%}ZedbsNC099T3
zlim+fPns
z^C4@f0;|ZP`WL@D>-?ZNqBufU?P0`jVw{TJEA%Q6_8*~IX4
zad^l%a}S0))tF?lm}*ShOWRw#fly{)&ZIhh%&?#?dZ&m?BFoAw;kz<*PZ_ODR%Tu-
zEJBJ9f|STWU?NZnB*cg&rAdp0L3{GqLf>G`7GFuD?FDd{Ay`UF#jzKbVL9Jk>rY3cR+T>mXXGBMlBcYS-iVHZNfrI}mm*
zH+zGx?7QS^)%&>%^vM;ZfoxEq$pgEm+IwXWK*x##RBz3ytW;J?_hOp(&p}aDr?S#g
zrcIdIRs)a1rO%0pY~kK^eT%0*_%0t(T(7mFm`mL3>0iI0e)I2?g5T>-k8qvORN-?w+Po}O(*AL)hwsy!lc!rf
zb{gi&ue5q(x2|Yi@KfBYIFEvi!MJZ4+3@F#
ztSDy8FG3hJHi?8Fnjt>Ia3ax&fwUJmzWcdRMjT{k++X7_10hWcyVZO$1EG@H2u5_G
z5*Z8zEX`@`Rj+p;-Rd%-`t><5>}C-HUj$Q;;$Q|6;=~+R%Zd>(!X^|8&5;l$1`VMP
zEu2!!40bABMJsk@u$dU2zlT-?O`uO&sX4S$k^d@$&+MK3?+JY9k|tlFSUkE|=WxvDw}G*n(64g-s6S(d5Q5_sXEXFTY@)
ztc;XA|G?(GzwrIPkkJ3nhk&DV`7(ce>a*JWSvgH|Y{nP{n^(7FuwP;h(?aXaW7
zB+HigMt;Fh-4n2fU$T?681FG%{q^q@0!!FM7j}(#Iwjuds9lk`Yiw=dXv?jc^e_Da
zO+IVsrw}$6UYM-izqXj}DM%mUz5?^c`s(fUFx=eP4tA{7cIz*JbqI7S03$c9I-;%G
zqs5H!iAPFgTfrUfhK!$LKKUa)qiX#p7YZ_-)||NzcJ7YdXW-ES@th8lPM_{?d<8#4!|&-zwtV
z`G1NpP)oh9z?}ZMF<^RS=l0I+4WAA??!P&uE_FMx{nPEAqG{m%l`fj#lrGZmR5a9S
zS!6Cibs{HW4h>KaOKKzs6B;y~EtyY_d8bwmH~L0CvFEm+KDa(s=IwXbv1bOIb_i7F
z>~jzWDu)xUa$7)A+f8XYG!31hsr^snrnYCcMn0%MHU$d(D!46Rv*zssdo~Zlk9q4g
zKiS~fVdQ{h{X;(w{H;R`y1Z952My6_$F~eR(kvJ-dFW%ufNT4BW}xR$U&BP&gX-F8
zpG@Tg7(QSCxoViwT;=-J8mnquTH$FR@jSgU;Q=5XSUQ?Ky45g>b(OTNJv4H9C@5YO
z`d&b2ie23PxMIt7YV2J68#~*+a`U)B%JC~d{YB_pVdHGrb?zyqe`V|
zG?yJEd)7?@tLe$(93vVh8XY4N>g%oR!GWeA=V#9v^$)?yS+b5^Fbnk7XF5s?nuvlY
zJR>&J(Oin>`B+(i-(|dUPfBxz*}eR8+Aku0;=OCoSLC;+-%HNUO>`K5H0_=CKtH}ZS(9qT%Hz8-ZBOjl0#<66W~l$X%EwfdyNpxVW3}wXxLzTj*OlA}EXj
z;4xY_S5PaJI~J_I0$U}rh#s!ZKu}~*L4Oa=MC6Xewig(5MBpe{$Nk>+vrO@0E*;&{
z)m(Bo#gD%FN#~p_+-6Re-cf7ZIcYP=s$Ka6R#mc(WvgLPyI>AxS-MJ=(!BP_-Ko)n
z-!nb_!q-b$?>D`S%ZvE%%L8k(z2
znC9k^MkxkAc;me4dLT4#HpI>p5Cv?-WR*0{XK}DpBu#Up6qD*t@uKYja^s!D*5
z0<=-^REF*=w@I)|K%TfYue9Z7i~g#3KBHGEnnGYWoiO_j)A@
zKe&UD$UvzVFhQDKuWE0-cdDjTQz|QcwpX?H^NA3=)Ed&y?0ol
z&nZhRwW@fnrnJyi1`Vz-e5v*iEu+Pk**h4H=8lZ{wEeVwumAQHze6%28Hvpwnh%vd
zD|;q*tSK7T3<c}afwrui{81S=mJrchJ{I}oBVDivk)5+=kM(ijDxsa@R(dGhE8iw+0^n$z+OhFwoZe*BrB?
z97cMu3DnIJb_4Ls6ajfk4PvZ}Z4U%d2GUa9!fBR(S@2qZ`@ep1S9le|2)}2t4s`WG
zh+%|v5`_4)^%Go3SJbYMr8Ehf=!Uf?L3#gT|04dEfY5cgY6o==%DZG)-2RxWG0pZd
z7na2||0Tcwp0Pw^%p11#LVLbiMOsPezY!*IAykIhxDXE2i_rA&en$M=!1)|iFtO!y05QsM>~gm0*-uI6nm*Q(E6CuJlL+(S6rx(eIyvM9
zxQvGmx%<(>DM?-WW-!s#o}3y57fF72bStiVTe+K|)fHKq&7?I5;mI^KJcg4INd6->Vv~He9OvusP#Q^(SXi&)rFE{bPf{
zi~v?z{E0i?T)mJA?$H;b{}A6rRDRx(SB;HEMf(?aIs|_E1#g~z@~knB@!3H3G?*Ee
zd^C~HJ58(}lXXYFw
zb|rV%e-^fEct0?G$$Mn<4>MT`#?>KrV!uxDT)!tYSw{k2^RrFKK>o3x>t%g!4`Vsv
z;eESye9L~=E9(VM^pW~e`$S+4mQzAYAFXcTMv0g>mFJ@|(XyjU6^j+ZX6Y4iu=P=R
zz}#3=l)4e97U1;ZsV6Pvv0jXUeXENU6LCiOB#!Td-wHZAc4y-2_#hrX(o0LE&EmBI
zvFccLuzK@!6jMv)w+(n?acL-RgTp9UAUT-i=Emvpco`pq*4Hq=erw^sCQ=hTl2%
z#rhZYwMWK0-P5DpAr^Su3gyW{o4`c>Gh)a(I+{ZKl875=Te?bGh39E*<&)=7R{8aF
z<#TW{lm(u!W8}$x4IC^Bx1#V;DrD_)#XKi?
z{HZAUh$uXg!<|6|?zv)+_5^
ztGENFGJHhQ)>)oz*14nLKXGDTM{E<)XS=Q>W_d1!U9J%14X<@q)zVnCF3s`%)p2D=
z#!pEqQhdf>!Y3Z0Gk0_d3yD8d^NEL;=A`jmAJKN~vooS|4g`#PrtR(63-2qPf<7_4
z%ud06Z_W7b&f9SpM5jfh_+OQG>^{#nsX62O-nzf={rB|qdPFP>z2(d6?H$m(97u>5
z5SF^6r(Qpj>K=BTm+I=uJCaIWT-*QA_dg_9Va57XBqn#1K&gmI3?7jw22i!q)0*MN
z<3zi$m^>M_3h^d1ds#QLFYZ++WTBNPIuIGrJD^S=N_(X-?=7QQ!vJDJX}cbuuhmvw
zccV!XcSf)TyE@mf{GSx1MEFkxYf;;>uF{^1@*`Aj<{H@=P~%4jI+2Bzk|Y|c?8wXm
zZ$zqG6Pd|o(PjM(ms>!cW9SlU5*395;0o>%k|V`W@mueE#A<3^jMAu!+3y=2-Xg`~`m5HxCIfX4QUQyRK*%i+|0*
zYUlTdFj>VRli4Au22qkI){&TYgK@n~N(0C)VB=&?b)f@kJ?<6ziq5sIUwHu+{f4Ia
zrzgMZ+>rle-QUJ*Gbb;aaFRY74B2dSV|we-ih%q>Z$HmZ=w6n8(&p9!-+$msWJJy?
zV6bH@5me0HdYJ9byMHQoRH5BM?Xayrwr#P~q5GvZEef?{OXKrHeXcfiMUOmjmkG(H
zKBl6%rtEVol!nzJl4qQY6$%-pQmvletXnW)m=n!r&F)E+q#9%CaRL@ez*b$Xpc5$S
z-Ybc@qw*uljW}Vzyv6yQ{Ys
zAJ2t0&O4&hik
zw`4q2uaaF@g&+Zz>*Ie)i1TzyQ!Q$dl}_(k&=?L>76g{7{;xsd_8_HA{IT$XETy{;wDmfGSx8u{L7
z8I$>&8)BQ``OcDgOY-fhXBQORziKF|{s+GQB*4a4lw{}XgT=j)otEAMQA#gC(<5^7
z!p?#H=G+hLmpkwDXgfIQN?E9I78F`KB*h%IwBx%c2nr8Z7v>dO$_Y`P4v!8F)Ua3Y6#(HU~
zl@q#|N9MOzy2VvFX#4o5B6-OsR;oJfVjg9!6!M0E`bA*$z@5bXQ-6-X?gm%F&JWkU
zkzW`iT5kPLY0;#QCcpA40EJvI3qxY9OAEM_AlFl>D<};$){9rZ4(qx4JY2gxSDQ?$
zJG-ofmR`t?Zn45+zf9nyBbjFf*it-nOD9BVt07p6$NP#6d|AP)))g+!EiTe>z%6d%
zWWUzBg?|bLsffh5B^&V}AW0_xQvn|S_`AZcyLcwn+Wm&Lg(Kh{%MWy^V|!bKjZ5Bz
z_R&t+UbpF<9+@9fU?}+x55*+UOUTLe#A_fTzCzSo%$g@g`HF@cZ@6M7pa+QNf9}>R}{-@USQqkw7a7uX=h=
ze$SL22ljw?700*Ni)HMA?0okx{uwfrj%0!6y?XocD+sFMG4kQfKs8?O+6%P895^X1
zO~MBCMGP$U4Uc+mGF_6!53v6|Yc_x#n
z6YJWp8BRX{quCt-_1t0;7V*fqR;xXg;s0~d_{^1bhno);-{SEc&-p!SfP*4}6b8E~C(
zu(-;#o3^*dli;eWtgF~VUn{;jS>0xLsHV!4Jv5wn@|a?{?UCKFgOA!4{QN{5dHZo>
zCZxh8uiBKHycvii$e1daL+z*3V^|uTCUe_tUti_+@Y}cXMQQm>9ih+hy7*zQ^zm{?
zYct46Is!Py(kT`G@
z_Hbop-|d_JOhee<+Q{ctRgsx0pN!>hO!&1<_%vlCzr%*#m)g`M@=1SnJsu^n4KtThy)QPbVi!7V
zu4i#0lG$NmW*C|f*~qSRruE;wSzb1&*jH9LlXi1&nJO%EOK9dMbfHg9Y4QKDO8GXtmTJhe(wpicG1E_eteOf8x7>
z?q-$s1^W%Y5o(B#PY{V1sTp%ckYp(}AV7qO!nSNlfoU^59tM5kLT9+r>VxxbZLzY+
z)BTf`Rh8FI50sT%zdKM_dA)3KZ`oZ%`Q85OI&x}QD1HcjkXOe_uJg3B1plE6@x$e`
z3Sxy?y9THGo{?h|evsMS>aW&_WpUjFA&zG*Zjag|h2t5e#TZiaRU7mVBUHl*F=jnxnHAtM#-bxyU
zOSE<5q@5D%l{ObAe6q2RY^hLG=2#_sYf4?b@R-|3#;`y}?sHvOQ(q~f@Hxz$r`+lB+gRFOW#sigMe2$fNvClWA{8coo(RTu#47f!*<$mhY3)wGO&+XbU7WyQn1#u
zaMEBu*LV&!p0IK%F-b41+tpX+-V%lGLm~Cax|J)TZj!BT2lfEGo|d)6CI(Oc*LVCi
z|2*O6`FIKF=Y4PRoxzqu0u+pmt4yr4*TD02i&d3~P5E(=Ocf0A>+nN)MC(H9LWP@?
z*g>@qJFVZ+o0GaPYRmV1bQC)nT3VEz<|kd{O6HRJ!O3JYKYEqxVlH`_Ou>(xrcn68
zSP(!OajbSu_|xrv2s$F-$l
zFg}!8n>hH!q6-!4V|V^!eCv9jl}QcTV~b?jHIEb%*20q?|KPYxvg}CZj&Bu~E!j^n
zLLDL^tEGH|P6<@DVXbl#mWlNduA69Ke-FF5x4d%OBy^L6tdO}#qUwv7i@rAI
zCZQIn`2;5jEmRk(52;VWowtI?+N;D&u+FQ!@>Fa`ORNApb&0s+W_6waJArB`7s0@-
zR4`qnE9Y?e{lPDS&X4)vSlAnUQ38$veC9-pREr0Mh3$)D^3F5i)FF1ix5lq7OG1;j
z6|=}QC}0d_sn3cv31pu*Z;cp
z{;76X&gwD?NeBym$qM*~RLgh-J`#yos7b^R`qgEz8O^GZDU~B8WFdK&kwHPa
zs>jw2EtocIQg%ov-3m;Ttrr(+x{cV`6G_X^T|JSZZ!sKOk4zn>bhrH+*Z-bjuJ!2~
ze5Y-ZM{0-U28L+4oTybNW>olySnNcW+_p35Kr-EE)%ReZea`SHj}JQaHUhhTM=
zGEv0yuJ7t?7*TiKthQvuGZH<0UpU;Xwz%-Tf>Kg+>ilfjdqp!?qC_Lc@Kk8ClC5#9
ziZ#%Gm-1b`Ok5Fqbxy?NvS!dOmFJYYt9MVVT6MEwboqQUQ>s9GySG8x*@i-f^S5-<
z+bX3BPrX#>)<9NiqUc??S^C0mPsoMt_S}Upc>b}Fz4_ol?I))mN$S#utK6-@HaIzh
z{*|8o`VPaQzpuaVGM=dZF^=zV0TO}-@*r}m3?7F)GiV+P8NhPL9PaO~AWS&I*Hh|(>OHS>=&Se4VBmH{
zXu$ipv3G!}7?v@t>NJE%EATQZK$J^Bwu52h1vN8aVyp+jro6$I|4;J!Z#JbSENa!-
zNL7<)1gkHRt_rINXxJG>@!GO8tRnEyPFGic$j+Neze8DE3fGlOnWm7va$H#>x+apEl(=OK=MFrV0+U{iXWt7;bc8@H=wt?{
zMy9s!Z9!E)Fi?cCn%oiB8wr@#$Y@@=P#2d1v9kiW&4e`s6L324lv9?a-L5_>_ZCP#
z#HmNQB|ExAyoLZB!B{|nOhouhsn_B8GscjMSZpF*cnm8MnqPA#;YQ8o&m7ZdM2GF_VonXl`nR43;|OcfInbs(=*ysVg(u4RYo
z%YIkopl57)jX0-}1r9pIsZIvP#@P&d^&0xKl1
zorM@Nu=k$eUO)~zq_w*f4Z83|%xl~obJ{W{Y~-OpP-VWJQc
z??ir%1W@99gJ{xhb1AJ%NGePG405(H4ndttbatTL3E`MVE6G9*>SVUi%qjkCe1I`<
zyJwut`3QkVlL~tfyGzX(-?bH$(TN_%<2+JNw^c7ald}40-7WR1vSW~~^c{xdYgnFZ
zFfCjC0vBSMGTO?
zjIDtmC%ul4Ij%(KwpG%8wVkS%@`(bWWs{(xMLY7yLPIO2#KF}AaabI5=1d&ZoJ#=4
zg@UPBsSkI&36d)+AUQB
z5)O#KI8ZbGNOR;`a;H(z;D?t3YU>B&onA#azP~Q>vk_tg|EMqPzZQx;o?leOS-zz9
zh}hh=BDCJ=81v}c+cZPT2aV1WCtU*_`O)4?6&98pR8Mmx&=j{5easS_vIA&g36>1m
z__X={fsB>>-IJ;>{lkA8S=fokT9*s%PAVNyBSnBivML9AKe>Il#wcw0e8uDfcKpwm
zca)cw^1okc74Nf4aA~E%@XS0vV1=1km~NAF*BZ}+El0#i3yE<0I@v-~O~_bqY^ec$
zop6cS5^JfQDtA`7DS2w=24-0ZVo5_7M00)_)Gmq%#?lainmJC4>-(Gg*T-VhS2wR5
zzfN+N7Vr?Q#f?wbkIjwQJ6C;Ixg&YYYB5vfFZld!
zyFhSh(OC=F@ox4ot!Rg{DRZw+1m|93ZdVX`|Exut>|#}EpWojd`mp32D8z$atDs8n
zYIRj*roT_{qQ35(_NG-9ey_*h<+&t3hExsgfa|!g^5@#yIa#0~iZT4cE-g~|@!&`F`Gp*F!Y=~LOn(1i
zLkuOY$liG^6B@I>Wu_g3-J-pIOWH4@_wireKqo=yE$5+BQEC=Z3Qb8f9zi67SHP$U1xRwU5?s*?ME+e9WJ7&A#|2I
z;p)X$Bw5|^%{#x>HC(D{>kh~t5gcE^co+*hR9Xz`Q``eyBgP>znd9>^%Bw%|-JCJF
z`I}BrM%4x=&$P1a*D$P@H}>kteKcCC^Anuu2cJcF9u~mBz5WQ=&jVechT1R|wf}bi
z?W!oBz9W^HX-cTyE63~mUj31KM=YH%^jfCG`o^u9jE0R~G
zlz7|OP1{uon!V$IBdFVwp8VqZ%E+{|+c>`e63|cgL9#wP_r`v{z0VhF$jWaIu3ZBO
zM7VVnnt}5*K(gb7%yF~;v
z57a1h+5y?~n*I=6SQr<}wexlAP=8$wzpgS3*qVLSUYAlc#~>@XRi8RLXy+YPL9HC_
z^h+D81+zf|l`mW$(Y9G7p61m6sbm9C4Aa?^IcP8FhJa~1p12F~)@v-Iyi4;Tn>YKh
z1LT=uIT%D{ts!d5g%UT&^*uBF9+c!AJ9ZG5lNMlUO1vv${ewq&p8mlPyhgMg!w9i8
z<7v)Q->12a8QG0#%9|qszgKlv9B~`aFl`1kVkW?@#X_M=o7Thao4OA{oi}_2dA#7B
z;}lB}gINd6F?fG*WU@fK%S4cq1vZ8O?5m@?H8`+yP+B?yR&c`B+BUI#dF?6jrOn5m
zLwRWJsr1*Sg6NKOrISx#@#N!_lG4PWBH;W$G)j9j^{557RD7vG$1WFR27!}8NYFUfv62SMw8+>{1@iyn-6{)Yg
z(ygIWu87_RTOIP{v?CgE0$j
z$Jg!6xSPw)xusi;>W2h0=?OIw`xp1!`E
zEr7b2PQUkW>Ms@;edHi1t?GZdeBN)K*G05tDCfGb?$BzxFJ^CS{`v4ZL*`(0=kGsf
zR&+oT=&hH_bh-EUINit{|2{r9egYCib#V&+@z=8di)D-pTe$M{L5!$kFeAZE;=xFv
zGA=YmU9f5?3e_FRJRWgJAy)X>zBu&5v)*zx2=`32>c-a@)c75<^CJF
zyUun~x)F;Qwmm$X>RVI1&6)U*R)+CFmvq>4joHsKI8yxO?MCA&AqgOKTZpqEhIa}NGVw(Ui$
zA7!|f|C4|8Z1S5gAadxt?r9BJ?X`}OlrTEf-n{bQ;yKsZXx|U?vSpjT*F4*KNTJ)9
zG0SfK_`-qh{kd`_`+Lp#h8B!RsSSF5)cx&|w~znr`3}r_nCjHX&|!o!>Kp&>yx~2
z!2)l7qoaul#V!C15!W=!QLy5W}dRP*$WeBG;4f17)59=QMH>Tmw|^xWUS5E*9LU%q(u
zp*{Wk64Ufm1+PvupDHlyJzWjYvh<2DVEF=}M`gWrSiBhN_clCT{B)5N$2ZyK5hWa3
zv25rKzVBJaECn9Saw5p3FtU(Nu0F8E0_EITq%7<(o1&9v4(EIqbYOLTnJ+W1xj24x
zn8`3s`L1~tMmWte54kOd6NG5Wj#*aMo0WvEp^+M5ib-zQ1=bcXNN&d}-TL>s-7=iM
zcKT4|OJ9nu&~5o5B3#tlA+oaSdUoz{szd850pq>$YmsrbyRHe2GJFMxZPyvvMkGK=
zHTA_k-KFT2b(_UCT?0FJo%;ya?fwGlvK_1B-!F?U(432ac#7tsaQx-6{?If(v%lo`
zzY4~G`|;DP+@5PMQV!7fc|P-q)jPPI35`r%Wa4`qlG~2^#icKVgoWq%A3fii`LGTG
z;757!yS@=zWAQC&a%=7emlSGkepwc}W&JPVZQpdX{t|xY%UFY)GSwc@J9V)#VV^iM
zhFD_Xi{qQ(>NFF8r0&1_2H$Eq)KNoqPh&*R(;S20q4~>@-R$TCK2N0D5?TqZm{vlf
zSvfP=R%}ZsMFBTyT+scJ{qUjrc=9!pQBaYpY#e)rP4eHT2JB`(hn?hi$Qz4{NJh{?
z^PTwu**-_}1=@w`Yi1;eK<5DG0CoV|is2?KOLDeywqkf-Whj5KK_eqNAzJdwcaac+
z4N~kWbvEJCq|Ws0Mx>0$r>&gz1TFY7hQcNXZP%7%KFh5-u+vnUYbxvV3mY1|8w#EcORH(CZ|1GdFi;T`{oAqVT(ntXgp&9xBm|XsWfN!?yjvuHmPLWY!8iV
z{xJJ#K9Ai^HCh-`O#G0bX^IRlO5*e^ncl?dF@BbAZ2@5=%U^JOqZe*}9)M(6e)R@l
z+qmx(Ct7d+ev)gZcmO6JP{nd`^*J7m%apFSG;)q9v5fJxG{h_-84OyEKvpj(j-Dzx
zxavA9gtdz9G|w$>)pbYVw9(Ode&$G@c>tdpz$XcPNz5_^r-wo0uw`We+9@jrp^|MN
z3CkG5A!itEY4ts9vQPj8Qd$HH=0LfW5%dm-GWkRX*%`rt>PX{4@uM_$Q$&%~UQ|-a
z#(K>?UP?}JXCt0_Se)|d`AQ%$p?R1NSA2ennIvI^_ptwY{QWnyRT?_we_8__Z<^gvG|@V%R<(>6&I`56mwjoqD1>dG?WY$
zx@%ZK@4_nFErHj13=LzMU24hHs4HI3SC5v5Bsyz(HuOLB^Bs`i9aVqXSLW-qK3Z4l
zpk47=idJ?a(46I5=;o}+RwMwWRYU-jsGaDWHO;GG>`uIgfLqD}oA(la@Gy7RNd_xEsovLdi*?+J}V?$!KU#
z65B15_>de8J9;)bYq{oGv!TUFp&=q01~Ihkb2v3cOVO5SOHP|p)Lrz3mzP+I&q-aL
z*xU3%L&c+)UsNc*C~4YcLmZ}~S8XIO6HlejCO!J;f290Mi3|8lqH4jkl@*u9v#~y!
zhsB>|tK8E5*d%Fa{#%Nqxj5MhLZ&S9a&5krKC7iy-H(RwuLs0T-Mb_Wr5+xV3l!Np
z?mN%Q4){LeNJ7Rn%rXR{g;33j48qHhdVYP#Xafd%C!vQGUCkxY;O|3+8fOX
z;AQi6+1GQLf`ZC9S7%|lq!FqoII3P|yrgIs<*uI)9d`~05oc>Xf)B`sD-;Dz?@Te1
zirp>-er&@r%+PIkA3H64PPgID&Zk|Z1zQ!*mR!^SkHq(lYA(C1QBSE?&YB3O)apK`
z0vJM{Ql%!JE+FxRR6y5@A@X|!+v|Y5$h&l&n`W*c-CEn
z4Rr^VsJhc^43jpHR|oOpwZ(UZO=H3)69W+vNw?A0Oj;$zLir)Q0|)=K+^XO3(i+lz
z@#4fpp>O@R!TdGhwUWJy{`ROg@<)qf+gCUj6&mHs-s#+u4$#8v&rCydPnMHmR@?d8+!nKqFLh-Z%2kXrm=a3ePbR{WKJ+wIcS*X=0#*I
z!NhYcxNjbj_>IuJ52e=`(1zO
z=Yxe#LM-0-9ZgXdSTG@B4D;fUAagdmGL%-P60j@T&PC+xvT}3#B4By%vdd&ueQHgY
zv;Q+75ZV4k)MsUPo&8Dnrs`q00~o#<(jYch*>Sd$?5i|IyV>`%io+hf49TKZX&2_R
z{ix88lnrDjwEQI6c@ZPx8Fm1ZVp88n!Op_3p;n)Ep0tU><~T3ZIul_Ec3wH>+K|Ck
zt_?X&DWZByg#^ur
z7jm;_uRGIqWb@B=8Yhs^?8+j}W#xNQDtO=7sE#No9(TgX0=BCct!PR^(HGK2d@6L7
z#-HybB!;B35R_9{7``9jTvi>lKJf7tTgn`@p0nE)k>*Lfy1fY+-;ZEVX9*8TOV&
zYB0<)P){ly=`B3c)b_vdcBwEtlm3hhU0tBXT;P4dH)OL
z8~F_>l+`y0(-yG>WNJ~6to}fDIbG`(=v+kQ)Tj9{1yp?R^#fSiBBq_O5%qOx8!{BV
zmVl@Vp$;+)FKk-?V(CM+=pt&N8croegSf(Jc*z?Xf6vd*d=o9K<%?DOXtoaURJl`JD
zQ3^*06-;*sn7$$!Ae{Im)IU5?;bny>O~U`6{%CrB@F@MA%{&^lplk)Yvj5JMIx*!X
z3K9Rp=(=B{t;<|FpY%!ySP;DKZb~hcAVqpvxYC|SN)0OymDFH%g7CO0wGVc;1!Xek
z+!#XU8VaGz^GU)idSZV7UZWUkV4C!hm}Fu*qQYwu6-|&(W8Hv
z#oU{~HFd6Q-zy^_2@r;a01jc0DG|f0I4}r^5D)<|P?g0zNGqsSX>Ccu6d))liGWxc
z1cgQopslu6!zcz&aH!JVDr#}6tqQgki{!l*Zr%Hwea`PY@ApoJXy#$TleM1VzOMf@
zZ@U{QpHNrlWrFY0nc8m*S{k9{eq**hR{y49hsAMp)}Qj&qlU&qc5Zt=Kf;zt!FQ?v
z?o%j^INNZ?XDz7nC%SZFadn)j;qV%gt`-nZ%(}T$1BLzWcMCTCM{+Z&k3Kg9;mWXEN841~&XzI=E%zVFB
z>`jh!AH7^ez&~flS>PBK2#)oi7+N6e3uMwXy;ZNAFJa)VL~MJ$)Q3_ga#opkig(jn
zZR*??sPd8TD~>uefLdnY|?8u#6Z=s<6*-F-Yt%cJ#*clEy5ty)xdev!QNHhmE>
zM4deGDxi(9=qp^A$mPW@tGN6Sb=PzM@A5;2_Db~?y2^@x>4Z{UTwDJ*`
zQ#eU@0|K`HPA{I(v(f$F^!xD@hmi%Z^PwE`12Tg-!Ud7uiJx@_2HLm^U#D#gN`j(l%!_Yyk3E-m{hI;uceHpj-9{=3`
zIy0n$8Loy^vOp{6xs^W`TCZha9!Og1DE;}Hm4?UQUfycGzy|1m2K}JgBpnP$@OEWO
zm8-_lyR|08A`F@
zl`~~9(3b88Tg##h7R!!IcMYf_e68vRW
zQavO7vLJPj6XgW3aTqpqA1PZt)`@h&pq>Cg93#3RMVjwV^zW4TS>Yr;J<|O3d8S0T
zBi@idL7*V%NV~lmp3`M%gfn7Hc5CJdc=^l9CuiVNx~x9Q@@E6Pdz+2JUZUmiY0r=cwyWh9K7&4_KCOb=TBk*e}7-5cmcluPKXPR_f|
zbXwh0trnh9QMis%5>hX^Pwg8RwHj&V3yDk#-H5`&@9sqMOyyKf^E$_>a~l%wwa9}D
zKV8l~XOu8eFZZ=wWXiVLfR?Y8lg5)*nleL>eVtKPeD<~F$Z@t5qJrAoK&+}DpUsJV
z2Ff``G^a=Ees9^$bzYJXJPl6|0tsE)0x_B68=rFmgmyJ|Z8@tHM3CPl?C#NZrVR#9
zd{f9-UCc-zDu_8!Bap}8M7r@^YBEBmkWg(EB9}xPXs3kUaBQA}m3%tZzP~S?r)v-X
z&{OZ)7pq7R9zX>EUIK6_M9W2hC}oTobuQRxW;(a0+GcJK0I50`B(W$Ho%50yaN3k)
z6ulbV>#~|>6zz*j+6AW6PCml-ffIu9s2Dg!xmLOQTyd8payaOCL~M}t3p
zEURkSQ>EE{G(KB%F;aDA26T3Z1s^Zxr$Jx?{2!SrF;UbYJ4r%TFtFyYZ2$Nu+bA
zn~R$Tmps_>f#yoLiL~3=w}UKhG%0W;_d+izBx^bwfX1hnt01|k?sYU8VDLzc7C8yU
zP9TOxLaEatf@=X<_@y8pn^0#L%>|`c&ZrQxXqSP6TmwbcQfi7DR-(o9XCPNIj|2@j
zJb=m1QvlPt5-6ZTP_lvWJ0g+`SFo!fRKq?B1{^s>?JR<9*|N*kA-G=Ar2bNnslf&q
z7`&t{Tq0Dl+wl8S)x9n4B-4qj*wRL3Zu1p+5sRSJbeDjD?GDzyTCZ4L4szn}{8-Rm
zX~#v_#sYgI8}*(BZ~9{JuY>74{_Q6`zxNiDd9QN>U8V#Ar%<13z`x(258DHyB#_8u
zqszMUWZ0`N&wAd$178AyIlygpgC!gWNSJUAo{eof=8s?Ot%bX1qT%0LsAg*5yg8|<
zDciRLnN#1j@dE#CMP^p3&xg;4Ur@_6fFT4A58uELkk*|S5<&V#1Q9MjB9R1w+(;6L
zX3an`YbTdXp_oqv=R>CuK0D=wtKYqOm`${Og^M9@{M9%|dn|8h>cY5g5d~2oQ$l7K2f1;>p&Xf$_d{!wKK=%1kf=={MGbNUDM&^kfu$2#w|q%A`tEfk(Qbkw73G_a>D_|q4;KZwI208
z9vQ9xqZmgI&&rxwk@@RH>Y{^VQwJ7KIK{5E*?B&5ted>pHF&J#E1uDke6e2VO4+oL
zddnZmme_CCM?I;Ia9r~OLB^IV91g~8UbN6wl*7meNW($ABx)vQ}*
zk;i%E4v3y(F7(V_D|Tp4jNcwzpCHS9U3~0t9b-DM4xZ3E7l=DLuu%C(}P!;?ix{W@uq(SWzLb
zFf#=6Lw~<^0Q%GbuoDCZ_9elUxbfLr$zZfFME7Do0_dUxl2QUO*Fk1ruYab0pg-1K
zTXVrS0p^2AYb&nxtiJu3XCTf)_4M!iFX(f4FbJ4mZD>|eaS(Hg@%G!OZ%te!>cEr{c&Yx
z^xOsR=?;(F%wPAv+2M3$!|x|R!bS1dzeyZD{7Ctc$P2vquT^w
z8L}njk9Ykq)GW9gHXmRSYsNu2U%YAallg6AtNO%|*G38J&F{izU*GWYa)l}LBySNs
z)oAlg7&Dv$h5JcRxaZ1?C)E3(_QB~n$%TP{{Djw+rL^guMee^0`rSWvu1A-LZM6HE
ztCNP|H`l?|hek3uPbltPR44~Db3h8O#oliA%~&vwkQZ||fTq7<9{e)vt11?3o>r|c
z_uaQT#@3hY3qycKEC%x));i6q&F`PW|h%fG(!r9>ekd?
zAX^NU4GH$QCyNi0XMbG@e-5Ce)p#4xWD|Na)_+b!;n$|X9
zvU7J!Fs)fwOmu;!9G=kE!oD?H)&jz)GKZSVc(A(x!#neM(EY$tLZJKWX!w)>KzVy)
zWF%(*cOA29VWf9?nY;xYT-fn>X2}Yi8?yNm5J11>dtWac%mjHC$i4btcn}Di2j;K#
zo_G{gU-zx)AMx*#Wgqdc_sKHclxfN>uyK8jp+RZNOy!p8>!8+3aRb>Ed@zWxKo##?
zk8Lf+fCntp)#|c9^K#f!%u!c?g7*w)@E%7pvYVSdvhz~hj$}8&U%~=55wPczn0u|g
z4VdDXW_~obBDNYfAyc^-`-mo>IL5ShHhOHHZWE}|RB6gPfGo}nOuH$s;Wmw@l$uOq
z=*EA4e15k~xzKq$frv6$CMi2c71m}CR+&5R7xBu&mmCe7Mc2+V(WjbkIJFn#u{L*f
zh7Q?eNYC@eU!<-ndfgnQsfct>wg8d~ls_UXqXUwn1ESXbZoc+3cHe|rt98CF+t#st
znG*D0oH&eB(1lqxt3eI!k*n#0of|Z2(H@D3$=FHU7%5wP!@4ZOeF4bioZEo=4!?2W
z;Rg;a74QYzJ`~Mx}fVm`Y%!
z{bbcSOH0Gr_n}7w%?aAK=Uw}H1rCt+9J2Nd8nT$^JPBwt_^ZL6DmrrrFl!}4Ip`=l
zau6VEfXIAo){&An3SU+?p@+~TrKS2Ccq-7&Mq^I827&?~Q^~o(w|WY0Z0R&BS{yI0{cxfuPPc
zaFByhK_V$8UCnNHF-UZzp!S*_pg};`UxQwN-VgX8U`+=3zkm~qwQnmIWBdZ#x1{I3
zasFh@zC^w5gl=W_+B2A0(0y_!a0k
z>-p{kUjwc`yZEi_$&{``gOQhuE|>G0_4K84v+{@C{npbem_EQnqYA0gm;d>IVhghn
zm>PgWN{uLuv%A#Ti~s~p_-LF&7sUhfmQ0zCttj0AvhQCCPu@rC&n70`N%Zub^z?L1
zOz7C!vCYGi>jAeWI?rzKjrqFz?Ae$LQ^I=TNd-x74%C&DSZ&E3Do+Vb3Fo81gI8#8
zAH7G&boPzF+V}P1R+#_pT>8YClDHLhUPDRppy=z*K<`?xNwyMn>4sM1cnnxTY%@PTqqyHeG&ja{#(B!*LP3
zh)1b2Lkh$u1o?h;u@`Qb-Vp2zLX;>%9ea+yC>hLzGW&hS$27&qF4ma^{?q*T-Jbhj
zx}2m|3_7mTx&_+}ekcJw@f>S%zHgV6T6#QST$~*f*)^8bM-vY^kux6|`W!fLXMpJ2
ze?9Sf@|tFsM2DMDSNJMWGaB-nO;5X@HMvS$VWk)LaZpW0Q44>G?$J;5zhIwoV)L_w
ziEFbKZU2eo>03$-{oCmk@z-l_c!S)!*(;%WNs)fES=a^b076wW#jZyMv_?E~xpu`x
z`Y#gmclm_GN45z+
zf7iYUQw$H{&=v8xd{R72KCAqqjVaXELL5qp=ObP7xw_aD9Zi=IC#0%{1`2#MUd6@>
zartCKfk$;k3q~3Bl*YCVcWQb*G|2+M@_lVo5{pH)gc|F@W@PC3IJBjQmQIm0qg3!F5
z$3kr9yf*Lg66#kcsiF}+<eM2uLj|HIA3v5J{hQ;j
zna3C~3h&U$F;X=b$thadgaX=Fuw%|7a;Rhgr1N|BZ4&A$z{fjRzNZw|z4}N=ci|jF
zzGnb2-u~#sB~(`o<~e9q@g$z0omL+Lu&5A0!{&-?5!^<}*PnkR0{dWI(`PcQeU*#w
zBsXZ^zIW~WxT~`pPgD^BUQ1N!qHq$zv_lQWi{Fj3WW%)uY-}g#!fE>LD`N0nNrcz3
zV>PnZxTsM7@BH8RSEdeA8y)$L^Pku>%uYUq9g)H9(Ay9gId7hi#odz
zI*zwpIS)0cf<=rOxH|+ITZw&1;&q?n5}CDwXkVfGn(94OPqwGDAEu{oEY;zHwH5pX
zk$0dQn>o7QEbL9kU&)!jR=4N=mAtX6dglr4@3+4-P53RdOuqAzr~g_1`(N6N|L33N
z(?}z!>$iXtc*JK7FS+DKlgOcXXaySXP~-E3kR~rUck1wX(|)%L`$b(@@(%4bq)wKRz
zNn#*zqSHBciSVt_U4qX+;mPml^Xb-UVE$3#C7XNd;hJ7>d0h}K4n~F<~*%Bbl
zR`oE$LJaa98Q+v0yV=kHQ0CeKYQUPcN20Wmhku743@x0UUJ?*qFxXHsKv^4R^6+|k
znD=zAF*E}Fz=r|qJmGS*!)$iS`2>NG(Nn8(kUB^g$)!Su$`Xy3h1h8oM9c@*l%SOt
zKd^5~VA(;@m;i|q&1&o^Ll)
z>)O2im230IGnfDW9-sf`TlU|-V$xVlr(kYyb$+5TEWu0$^^|?$#nT_$;WeRts+{|P
zh?k3(d%Uq;{<{8i^yb|+xkt`ld((Ovc)EF_O*eKcDMI|)G#8}V4$n5B0J(eVvOG-2
zEdxpaZQ*0pl~Bb`k3@ln3>*-}!E5o6VDGH}
zt5|DOR2O4j3;5{;EppfBjN_1m0>+Q^HKRf7LdU7eIcoW$g{BiD%F
z9M#>kEa4i^(App>1l*VDAwlBsM8(lssn;G!3IE1AuVx^V*FCgM_kF&(g8Wt!$)iFJ
z0B~3Zt4r~@un#x~@Mz-+RSi`Q3cx)=wh)|@epdl1v+UI4cA%{QwhUAgaajB2*zeQj
zBbyh!_b)(c^0HlRm%WJ5hHIf}+XP2%hzO;0OZs!s^bm0p(a4`TI^Z9^L2d5x&0
z=Qr8I?K)L@5!p9CBmHg2GD`c1g10FN>HN6Y0E)_xMYNEe0r|VS*kqJJrE4K*3mpX3s5lB
z&Jd)KLXC#5v{?{_ikh#6C8-Mq0#sB;*^{00fJ({U54cGa%jU3nzbc