diff --git a/www/src/content/api-reference/csharp.md b/www/src/content/api-reference/csharp.md new file mode 100644 index 00000000..d4445d42 --- /dev/null +++ b/www/src/content/api-reference/csharp.md @@ -0,0 +1,681 @@ +--- +title: "C# / .NET API Reference" +description: Complete API reference for the Foundry Local C# SDK. Covers FoundryLocalManager, ICatalog, Model, OpenAIChatClient, OpenAIAudioClient, and configuration. +--- + +# C# / .NET API Reference + +Package: `Microsoft.AI.Foundry.Local` +Target Framework: .NET 8.0+ + +```bash +dotnet add package Microsoft.AI.Foundry.Local +``` + +For WinML hardware acceleration on Windows: + +```bash +dotnet add package Microsoft.AI.Foundry.Local.WinML +``` + +Namespace for all types: + +```csharp +using Microsoft.AI.Foundry.Local; +``` + +--- + +## FoundryLocalManager + +The singleton entry point for the Foundry Local SDK. Manages initialization, provides access to the model catalog, and controls the optional web service. Implements `IDisposable`. + +```csharp +public class FoundryLocalManager : IDisposable +``` + +### Static Properties + +| Property | Type | Description | +|----------|------|-------------| +| `IsInitialized` | `bool` | Whether the singleton has been created. | +| `Instance` | `FoundryLocalManager` | The singleton instance. Only valid after `CreateAsync` completes. | +| `Urls` | `string[]?` | Bound URLs after calling `StartWebServiceAsync`. Null if web service is not running. | + +### Static Methods + +#### `CreateAsync(configuration, logger, ct?)` + +Creates the singleton `FoundryLocalManager` instance. Must be called before accessing `Instance`. + +| Parameter | Type | Description | +|-----------|------|-------------| +| `configuration` | `Configuration` | Configuration settings. `AppName` is required. | +| `logger` | `ILogger` | Application logger. Use `NullLogger.Instance` to suppress SDK log output. | +| `ct` | `CancellationToken?` | Optional cancellation token. | + +**Returns:** `Task` + +**Throws:** `FoundryLocalException` if initialization fails. + +```csharp +var config = new Configuration { AppName = "MyApp", LogLevel = LogLevel.Information }; +await FoundryLocalManager.CreateAsync(config, logger); +var manager = FoundryLocalManager.Instance; +``` + +### Instance Methods + +#### `GetCatalogAsync(ct?)` + +Gets the model catalog instance. The catalog is populated on first use. + +| Parameter | Type | Description | +|-----------|------|-------------| +| `ct` | `CancellationToken?` | Optional cancellation token. | + +**Returns:** `Task` + +> **Note:** If using a WinML build, this call may trigger a one-off execution provider download. Call `EnsureEpsDownloadedAsync` first to separate the two operations. + +#### `StartWebServiceAsync(ct?)` + +Starts the optional OpenAI-compatible web service. After this call, `Urls` is populated with the bound addresses. The web service exposes: + +- `POST /v1/chat/completions` — Chat completions +- `GET /v1/models` — List downloaded models +- `GET /v1/models/{model_id}` — Get model details + +| Parameter | Type | Description | +|-----------|------|-------------| +| `ct` | `CancellationToken?` | Optional cancellation token. | + +**Returns:** `Task` + +#### `StopWebServiceAsync(ct?)` + +Stops the web service if it is running. + +| Parameter | Type | Description | +|-----------|------|-------------| +| `ct` | `CancellationToken?` | Optional cancellation token. | + +**Returns:** `Task` + +#### `EnsureEpsDownloadedAsync(ct?)` + +Downloads and registers execution providers. Only relevant when using the WinML package. Once downloaded, execution providers are cached and subsequent calls are fast. + +| Parameter | Type | Description | +|-----------|------|-------------| +| `ct` | `CancellationToken?` | Optional cancellation token. | + +**Returns:** `Task` + +#### `Dispose()` + +Releases all resources held by the manager. + +--- + +## Configuration + +Settings passed to `FoundryLocalManager.CreateAsync()`. + +```csharp +public class Configuration +``` + +| Property | Type | Required | Default | Description | +|----------|------|----------|---------|-------------| +| `AppName` | `string` | **Yes** | — | Your application name. Used to derive default paths. | +| `AppDataDir` | `string?` | No | `~/.{AppName}` | Application data directory. | +| `ModelCacheDir` | `string?` | No | `{AppDataDir}/cache/models` | Directory for downloaded model files. | +| `LogsDir` | `string?` | No | `{AppDataDir}/logs` | Directory for log files. | +| `LogLevel` | `LogLevel` | No | `LogLevel.Warning` | Logging level. | +| `Web` | `WebService?` | No | — | Web service configuration. | +| `AdditionalSettings` | `IDictionary?` | No | — | Additional key-value settings for the native core. | + +### WebService + +Optional configuration for the built-in web service. + +| Property | Type | Description | +|----------|------|-------------| +| `Urls` | `string?` | Semicolon-separated listener URLs (e.g. `"http://127.0.0.1:8080"`). | +| `ExternalUrl` | `Uri?` | URL of an external Foundry Local service to connect to. | + +```csharp +var config = new Configuration +{ + AppName = "MyApp", + LogLevel = LogLevel.Information, + Web = new WebService { Urls = "http://127.0.0.1:8080" } +}; +``` + +### LogLevel + +```csharp +public enum LogLevel +{ + Verbose, + Debug, + Information, + Warning, + Error, + Fatal +} +``` + +--- + +## ICatalog + +Interface for discovering and managing models. Obtained via `manager.GetCatalogAsync()`. + +```csharp +public interface ICatalog +``` + +### Properties + +| Property | Type | Description | +|----------|------|-------------| +| `Name` | `string` | The catalog name. | + +### Methods + +#### `ListModelsAsync(ct?)` + +Lists all available models in the catalog. + +| Parameter | Type | Description | +|-----------|------|-------------| +| `ct` | `CancellationToken?` | Optional cancellation token. | + +**Returns:** `Task>` + +```csharp +var catalog = await manager.GetCatalogAsync(); +var models = await catalog.ListModelsAsync(); +foreach (var model in models) +{ + Console.WriteLine($"{model.Alias} ({model.Variants.Count} variants)"); +} +``` + +#### `GetModelAsync(modelAlias, ct?)` + +Looks up a model by its alias (e.g. `"qwen2.5-0.5b"`). + +| Parameter | Type | Description | +|-----------|------|-------------| +| `modelAlias` | `string` | The model alias to look up. | +| `ct` | `CancellationToken?` | Optional cancellation token. | + +**Returns:** `Task` — The matching model, or `null` if not found. + +#### `GetModelVariantAsync(modelId, ct?)` + +Looks up a specific model variant by its unique ID. + +| Parameter | Type | Description | +|-----------|------|-------------| +| `modelId` | `string` | The unique model variant identifier. | +| `ct` | `CancellationToken?` | Optional cancellation token. | + +**Returns:** `Task` — The matching variant, or `null` if not found. + +#### `GetCachedModelsAsync(ct?)` + +Returns all model variants that have been downloaded to the local cache. + +| Parameter | Type | Description | +|-----------|------|-------------| +| `ct` | `CancellationToken?` | Optional cancellation token. | + +**Returns:** `Task>` + +#### `GetLoadedModelsAsync(ct?)` + +Returns all model variants currently loaded into memory. + +| Parameter | Type | Description | +|-----------|------|-------------| +| `ct` | `CancellationToken?` | Optional cancellation token. | + +**Returns:** `Task>` + +--- + +## Model + +Represents a logical AI model with one or more variants. Operations are performed on the currently selected variant. Implements `IModel`. + +```csharp +public class Model : IModel +``` + +### Properties + +| Property | Type | Description | +|----------|------|-------------| +| `Alias` | `string` | The model alias (e.g. `"qwen2.5-0.5b"`). | +| `Id` | `string` | The ID of the currently selected variant. | +| `Variants` | `List` | All available variants for this model. | +| `SelectedVariant` | `ModelVariant` | The currently selected variant. | + +### Methods + +#### `SelectVariant(variant)` + +Selects a specific variant for subsequent operations. + +| Parameter | Type | Description | +|-----------|------|-------------| +| `variant` | `ModelVariant` | The variant to select. Must be one of `Variants`. | + +**Returns:** `void` + +**Throws:** `FoundryLocalException` if the variant is not valid for this model. + +```csharp +// Select a specific variant (e.g. CPU variant) +var cpuVariant = model.Variants.First(v => v.Info.Runtime.DeviceType == DeviceType.CPU); +model.SelectVariant(cpuVariant); +``` + +#### `GetLatestVersion(variant)` + +Returns the latest version of the specified variant. + +| Parameter | Type | Description | +|-----------|------|-------------| +| `variant` | `ModelVariant` | The variant to check. | + +**Returns:** `ModelVariant` — The latest version. Same as input if already latest. + +**Throws:** `FoundryLocalException` if the variant is not valid for this model. + +#### `DownloadAsync(downloadProgress?, ct?)` + +Downloads the selected variant to the local cache. + +| Parameter | Type | Description | +|-----------|------|-------------| +| `downloadProgress` | `Action?` | Optional callback receiving progress (0–100). | +| `ct` | `CancellationToken?` | Optional cancellation token. | + +**Returns:** `Task` + +```csharp +await model.DownloadAsync( + progress => Console.Write($"\rDownloading: {progress:F1}%"), + cancellationToken); +``` + +#### `LoadAsync(ct?)` + +Loads the selected variant into memory for inference. + +| Parameter | Type | Description | +|-----------|------|-------------| +| `ct` | `CancellationToken?` | Optional cancellation token. | + +**Returns:** `Task` + +#### `UnloadAsync(ct?)` + +Unloads the selected variant from memory. + +| Parameter | Type | Description | +|-----------|------|-------------| +| `ct` | `CancellationToken?` | Optional cancellation token. | + +**Returns:** `Task` + +#### `IsCachedAsync(ct?)` + +Checks whether the selected variant is downloaded to the local cache. + +**Returns:** `Task` + +#### `IsLoadedAsync(ct?)` + +Checks whether the selected variant is loaded in memory. + +**Returns:** `Task` + +#### `GetPathAsync(ct?)` + +Gets the local file path of the selected variant. + +**Returns:** `Task` + +#### `RemoveFromCacheAsync(ct?)` + +Deletes the selected variant from the local cache. + +**Returns:** `Task` + +#### `GetChatClientAsync(ct?)` + +Creates an OpenAIChatClient for performing chat completions with this model. + +**Returns:** `Task` + +#### `GetAudioClientAsync(ct?)` + +Creates an OpenAIAudioClient for performing audio transcription with this model. + +**Returns:** `Task` + +--- + +## ModelVariant + +A specific variant of a model (e.g. a particular quantization or device target). Contains detailed metadata and implements the same lifecycle interface as `Model`. Implements `IModel`. + +```csharp +public class ModelVariant : IModel +``` + +### Properties + +| Property | Type | Description | +|----------|------|-------------| +| `Alias` | `string` | The model alias. | +| `Id` | `string` | Unique identifier of this variant. | +| `Version` | `int` | Version number. | +| `Info` | `ModelInfo` | Detailed metadata about this variant. | + +### Methods + +Same lifecycle methods as `Model`: `DownloadAsync()`, `LoadAsync()`, `UnloadAsync()`, `IsCachedAsync()`, `IsLoadedAsync()`, `GetPathAsync()`, `RemoveFromCacheAsync()`, `GetChatClientAsync()`, `GetAudioClientAsync()`. + +--- + +## ModelInfo + +Detailed metadata for a model variant. Accessed via `ModelVariant.Info`. + +| Property | Type | Description | +|----------|------|-------------| +| `Id` | `string` | Unique model variant identifier. | +| `Name` | `string` | Display name. | +| `Alias` | `string` | Model alias. | +| `Version` | `int` | Version number. | +| `Uri` | `string` | Download URI. | +| `FileSizeBytes` | `long` | File size in bytes. | +| `PromptTemplate` | `PromptTemplate` | Template for structuring prompts. | +| `Runtime` | `Runtime` | Runtime configuration (device type, execution provider). | +| `IsCached` | `bool` | Whether this variant is downloaded. | +| `ModelSettings` | `ModelSettings` | Model-specific parameters. | + +--- + +## OpenAIChatClient + +Client for performing chat completions. Uses OpenAI-compatible request/response types (from the Betalgo.Ranul.OpenAI SDK). + +```csharp +public class OpenAIChatClient +``` + +### Properties + +| Property | Type | Description | +|----------|------|-------------| +| `Settings` | `ChatSettings` | Configuration settings for chat completions. | + +### ChatSettings + +| Property | Type | Default | Description | +|----------|------|---------|-------------| +| `Temperature` | `float?` | — | Sampling temperature (0–2). | +| `MaxTokens` | `int?` | — | Maximum tokens to generate. | +| `TopP` | `float?` | — | Nucleus sampling threshold (0–1). | +| `FrequencyPenalty` | `float?` | — | Frequency penalty (-2 to 2). | +| `PresencePenalty` | `float?` | — | Presence penalty (-2 to 2). | + +### Methods + +#### `CompleteChatAsync(messages, ct?)` + +Executes a non-streaming chat completion. + +| Parameter | Type | Description | +|-----------|------|-------------| +| `messages` | `IEnumerable` | Chat messages. The system message is automatically added. | +| `ct` | `CancellationToken?` | Optional cancellation token. | + +**Returns:** `Task` + +```csharp +var chat = await model.GetChatClientAsync(); +chat.Settings.Temperature = 0.7f; +chat.Settings.MaxTokens = 256; + +var response = await chat.CompleteChatAsync(new[] +{ + ChatMessage.FromSystem("You are a helpful assistant."), + ChatMessage.FromUser("What is 2+2?") +}); +Console.WriteLine(response.Choices.First().Message.Content); +``` + +#### `CompleteChatStreamingAsync(messages, ct)` + +Executes a streaming chat completion. Returns an `IAsyncEnumerable` so you can process tokens as they arrive. + +| Parameter | Type | Description | +|-----------|------|-------------| +| `messages` | `IEnumerable` | Chat messages. | +| `ct` | `CancellationToken` | Cancellation token. | + +**Returns:** `IAsyncEnumerable` + +```csharp +await foreach (var chunk in chat.CompleteChatStreamingAsync( + new[] { ChatMessage.FromUser("Tell me a story") }, + cancellationToken)) +{ + var text = chunk.Choices?.FirstOrDefault()?.Delta?.Content; + if (text != null) Console.Write(text); +} +``` + +--- + +## OpenAIAudioClient + +Client for performing audio transcription. Uses OpenAI-compatible request/response types. + +```csharp +public class OpenAIAudioClient +``` + +### Properties + +| Property | Type | Description | +|----------|------|-------------| +| `Settings` | `AudioSettings` | Configuration settings for audio transcription. | + +### AudioSettings + +| Property | Type | Default | Description | +|----------|------|---------|-------------| +| `Language` | `string?` | — | Language of the audio (ISO 639-1 code). | +| `Temperature` | `float?` | — | Sampling temperature. | + +### Methods + +#### `TranscribeAudioAsync(audioFilePath, ct?)` + +Transcribes an audio file. + +| Parameter | Type | Description | +|-----------|------|-------------| +| `audioFilePath` | `string` | Path to the audio file. Supported format: mp3. | +| `ct` | `CancellationToken?` | Optional cancellation token. | + +**Returns:** `Task` + +```csharp +var audio = await model.GetAudioClientAsync(); +var result = await audio.TranscribeAudioAsync("recording.mp3"); +Console.WriteLine(result.Text); +``` + +#### `TranscribeAudioStreamingAsync(audioFilePath, ct)` + +Transcribes an audio file with streaming output. + +| Parameter | Type | Description | +|-----------|------|-------------| +| `audioFilePath` | `string` | Path to the audio file. Supported format: mp3. | +| `ct` | `CancellationToken` | Cancellation token. | + +**Returns:** `IAsyncEnumerable` + +```csharp +await foreach (var chunk in audio.TranscribeAudioStreamingAsync("recording.mp3", ct)) +{ + if (chunk.Text != null) Console.Write(chunk.Text); +} +``` + +--- + +## Enums and Types + +### DeviceType + +```csharp +public enum DeviceType +{ + Invalid = 0, + CPU = 1, + GPU = 2, + NPU = 3 +} +``` + +### Runtime + +| Property | Type | Description | +|----------|------|-------------| +| `DeviceType` | `DeviceType` | The target hardware device. | +| `ExecutionProvider` | `string` | The execution provider name (e.g. `"CUDAExecutionProvider"`). | + +### PromptTemplate + +| Property | Type | Description | +|----------|------|-------------| +| `System` | `string` | Template for system messages. | +| `User` | `string` | Template for user messages. | +| `Assistant` | `string` | Template for assistant messages. | +| `Prompt` | `string` | Overall prompt template. | + +### FoundryLocalException + +```csharp +public class FoundryLocalException : Exception +``` + +Base exception type for all errors thrown by the SDK. Includes logging context. + +--- + +## Complete Example + +```csharp +using Microsoft.AI.Foundry.Local; +using Microsoft.Extensions.Logging; + +// 1. Initialize +var logger = LoggerFactory.Create(b => b.AddConsole()).CreateLogger("MyApp"); +var config = new Configuration +{ + AppName = "MyApp", + LogLevel = LogLevel.Information +}; + +await FoundryLocalManager.CreateAsync(config, logger); +var manager = FoundryLocalManager.Instance; + +// Optional: download execution providers (WinML only) +await manager.EnsureEpsDownloadedAsync(); + +// 2. Discover and select a model +var catalog = await manager.GetCatalogAsync(); +var model = await catalog.GetModelAsync("qwen2.5-0.5b"); + +// 3. Download with progress +await model.DownloadAsync(progress => + Console.Write($"\rDownloading: {progress:F1}%")); +Console.WriteLine("\nDownload complete."); + +// 4. Load into memory +await model.LoadAsync(); + +// 5. Chat completion +var chat = await model.GetChatClientAsync(); +chat.Settings.Temperature = 0.7f; +chat.Settings.MaxTokens = 256; + +// Non-streaming +var response = await chat.CompleteChatAsync(new[] +{ + ChatMessage.FromUser("What is Foundry Local?") +}); +Console.WriteLine(response.Choices.First().Message.Content); + +// Streaming +await foreach (var chunk in chat.CompleteChatStreamingAsync( + new[] { ChatMessage.FromUser("Tell me a joke") }, + CancellationToken.None)) +{ + var text = chunk.Choices?.FirstOrDefault()?.Delta?.Content; + if (text != null) Console.Write(text); +} + +// 6. Cleanup +await model.UnloadAsync(); +manager.Dispose(); +``` + +## Web Service Example + +Start an OpenAI-compatible REST endpoint alongside the SDK: + +```csharp +var config = new Configuration +{ + AppName = "MyApp", + Web = new WebService { Urls = "http://127.0.0.1:8080" } +}; + +await FoundryLocalManager.CreateAsync(config, logger); +var manager = FoundryLocalManager.Instance; + +var catalog = await manager.GetCatalogAsync(); +var model = await catalog.GetModelAsync("qwen2.5-0.5b"); +await model.DownloadAsync(); +await model.LoadAsync(); + +// Start the web service +await manager.StartWebServiceAsync(); +Console.WriteLine($"Web service running at: {string.Join(", ", manager.Urls!)}"); + +// The service exposes OpenAI-compatible endpoints: +// POST /v1/chat/completions +// GET /v1/models +// GET /v1/models/{model_id} + +// Stop when done +await manager.StopWebServiceAsync(); +await model.UnloadAsync(); +manager.Dispose(); +``` diff --git a/www/src/content/api-reference/index.md b/www/src/content/api-reference/index.md new file mode 100644 index 00000000..297c3b42 --- /dev/null +++ b/www/src/content/api-reference/index.md @@ -0,0 +1,104 @@ +--- +title: API Reference +description: Complete API reference for the Foundry Local SDKs. Build AI-powered applications with JavaScript/TypeScript or C#/.NET. +--- + +# API Reference + +Foundry Local provides native SDKs for building AI-powered applications that run entirely on-device. Both SDKs share the same architecture and concepts, so code patterns transfer across languages. + +## Available SDKs + +| SDK | Package | Install | +|-----|---------|---------| +| **JavaScript / TypeScript** | `foundry-local-sdk` | `npm install foundry-local-sdk` | +| **C# / .NET** | `Microsoft.AI.Foundry.Local` | `dotnet add package Microsoft.AI.Foundry.Local` | + +## Core Concepts + +Every Foundry Local application follows the same workflow regardless of language: + +1. **Initialize** — Create a `FoundryLocalManager` with your app configuration +2. **Discover** — Use the `Catalog` to browse available models +3. **Download** — Pull a model to your local cache (with progress tracking) +4. **Load** — Load the model into memory for inference +5. **Infer** — Create a `ChatClient` or `AudioClient` and run completions or transcriptions +6. **Cleanup** — Unload models when done to free resources + +## Architecture Overview + +Both SDKs expose the same class hierarchy: + +- **FoundryLocalManager** — Singleton entry point. Initializes the native core, provides access to the catalog and optional web service. +- **Catalog** — Discovers and lists models from the model registry. Returns `Model` objects grouped by alias. +- **Model** — A logical model (e.g. "qwen2.5-0.5b") that may have multiple variants (quantizations, device targets). Delegates lifecycle operations to its selected variant. +- **ModelVariant** — A specific model binary (e.g. "qwen2.5-0.5b-cpu-int4"). Handles download, load, unload, and cache operations. +- **ChatClient** — Performs chat completions (synchronous and streaming) against a loaded model. Follows the OpenAI Chat Completions API structure. +- **AudioClient** — Performs audio transcription (synchronous and streaming) against a loaded model. Follows the OpenAI Audio API structure. +- **Configuration** — Specifies app name, directories, log level, and optional web service settings. + +## Quick Comparison + +### Initialization + +**JavaScript:** + +```typescript +import { FoundryLocalManager } from 'foundry-local-sdk'; + +const manager = FoundryLocalManager.create({ + appName: 'MyApp', + logLevel: 'info' +}); +``` + +**C#:** + +```csharp +using Microsoft.AI.Foundry.Local; + +var config = new Configuration { AppName = "MyApp", LogLevel = LogLevel.Information }; +await FoundryLocalManager.CreateAsync(config, logger); +var manager = FoundryLocalManager.Instance; +``` + +### Chat Completion + +**JavaScript:** + +```typescript +const model = await manager.catalog.getModel('qwen2.5-0.5b'); +await model.download(); +await model.load(); + +const chat = model.createChatClient(); +const response = await chat.completeChat([ + { role: 'user', content: 'Hello!' } +]); +``` + +**C#:** + +```csharp +var catalog = await manager.GetCatalogAsync(); +var model = await catalog.GetModelAsync("qwen2.5-0.5b"); +await model.DownloadAsync(); +await model.LoadAsync(); + +var chat = await model.GetChatClientAsync(); +var response = await chat.CompleteChatAsync(new[] { + ChatMessage.FromUser("Hello!") +}); +``` + +## Detailed References + +- [JavaScript / TypeScript API Reference](/docs/api-reference/javascript) — Full class and method documentation for the JS SDK +- [C# / .NET API Reference](/docs/api-reference/csharp) — Full class and method documentation for the C# SDK + +## Source Code + +- [JavaScript SDK on GitHub](https://github.com/microsoft/Foundry-Local/tree/main/sdk_v2/js) +- [C# SDK on GitHub](https://github.com/microsoft/Foundry-Local/tree/main/sdk_v2/cs) +- [JavaScript Samples](https://github.com/microsoft/Foundry-Local/tree/main/samples/js) +- [C# Samples](https://github.com/microsoft/Foundry-Local/tree/main/samples/cs) diff --git a/www/src/content/api-reference/javascript.md b/www/src/content/api-reference/javascript.md new file mode 100644 index 00000000..c487fb7f --- /dev/null +++ b/www/src/content/api-reference/javascript.md @@ -0,0 +1,545 @@ +--- +title: JavaScript / TypeScript API Reference +description: Complete API reference for the Foundry Local JavaScript SDK. Covers FoundryLocalManager, Catalog, Model, ChatClient, AudioClient, and configuration. +--- + +# JavaScript / TypeScript API Reference + +Package: `foundry-local-sdk` + +```bash +npm install foundry-local-sdk +``` + +All classes are exported from the package root: + +```typescript +import { + FoundryLocalManager, + Catalog, + Model, + ModelVariant, + ChatClient, + AudioClient +} from 'foundry-local-sdk'; +``` + +--- + +## FoundryLocalManager + +The singleton entry point for the Foundry Local SDK. Manages initialization of the native core and provides access to the Catalog and web service. + +### Static Methods + +#### `FoundryLocalManager.create(config)` + +Creates and returns the singleton `FoundryLocalManager` instance. + +| Parameter | Type | Description | +|-----------|------|-------------| +| `config` | `FoundryLocalConfig` | Configuration object. `appName` is required. | + +**Returns:** `FoundryLocalManager` + +```typescript +const manager = FoundryLocalManager.create({ + appName: 'MyApp', + logLevel: 'info' +}); +``` + +### Properties + +| Property | Type | Description | +|----------|------|-------------| +| `catalog` | `Catalog` | The Catalog instance for discovering and managing models. | +| `urls` | `string[]` | URLs where the web service is listening. Empty if web service is not running. | + +### Methods + +#### `startWebService()` + +Starts the local OpenAI-compatible web service. After calling this, use the `urls` property to get the bound addresses. If no listener address is configured, defaults to `127.0.0.1:0` (random ephemeral port). + +**Returns:** `void` + +**Throws:** `Error` if starting the service fails. + +#### `stopWebService()` + +Stops the local web service. + +**Returns:** `void` + +**Throws:** `Error` if stopping the service fails. + +--- + +## FoundryLocalConfig + +Configuration object passed to `FoundryLocalManager.create()`. + +| Property | Type | Required | Default | Description | +|----------|------|----------|---------|-------------| +| `appName` | `string` | **Yes** | — | Your application name. Used to derive default paths. | +| `logLevel` | `string` | No | `'warning'` | Log level: `'trace'`, `'debug'`, `'info'`, `'warn'`, `'error'`, or `'fatal'`. | +| `appDataDir` | `string` | No | `~/.{appName}` | Application data directory. | +| `modelCacheDir` | `string` | No | `{appDataDir}/cache/models` | Directory for downloaded model files. | +| `logsDir` | `string` | No | `{appDataDir}/logs` | Directory for log files. | +| `webServiceUrls` | `string` | No | — | Semicolon-separated listener URLs for the web service (e.g. `"http://127.0.0.1:8080"`). | +| `serviceEndpoint` | `string` | No | — | URL of an external Foundry Local service to connect to instead of using the embedded core. | +| `libraryPath` | `string` | No | — | Path to the native library. Normally auto-detected. | +| `additionalSettings` | `Record` | No | — | Additional key-value settings passed to the native core. | + +--- + +## Catalog + +Discovers and lists AI models available in the system. Accessed via `manager.catalog`. + +### Properties + +| Property | Type | Description | +|----------|------|-------------| +| `name` | `string` | The name of the catalog. | + +### Methods + +#### `getModels()` + +Lists all available models in the catalog. + +**Returns:** `Promise` + +```typescript +const models = await manager.catalog.getModels(); +for (const model of models) { + console.log(model.alias, model.variants.length, 'variants'); +} +``` + +#### `getModel(alias)` + +Retrieves a model by its alias (e.g. `"qwen2.5-0.5b"`). + +| Parameter | Type | Description | +|-----------|------|-------------| +| `alias` | `string` | The alias of the model to retrieve. | + +**Returns:** `Promise` + +**Throws:** `Error` if alias is empty or model is not found. + +#### `getModelVariant(modelId)` + +Retrieves a specific model variant by its unique ID. + +| Parameter | Type | Description | +|-----------|------|-------------| +| `modelId` | `string` | The unique identifier of the model variant. | + +**Returns:** `Promise` + +**Throws:** `Error` if modelId is empty or variant is not found. + +#### `getCachedModels()` + +Returns all model variants that have been downloaded to the local cache. + +**Returns:** `Promise` + +#### `getLoadedModels()` + +Returns all model variants currently loaded into memory. + +**Returns:** `Promise` + +--- + +## Model + +Represents a logical AI model (e.g. "qwen2.5-0.5b") that may have multiple variants. Operations are performed on the currently selected variant. Implements `IModel`. + +### Properties + +| Property | Type | Description | +|----------|------|-------------| +| `alias` | `string` | The model alias (e.g. `"qwen2.5-0.5b"`). | +| `id` | `string` | The ID of the currently selected variant. | +| `isCached` | `boolean` | Whether the selected variant is downloaded to the local cache. | +| `path` | `string` | Local file path of the selected variant. | +| `variants` | `ModelVariant[]` | All available variants for this model. | + +### Methods + +#### `selectVariant(modelId)` + +Selects a specific variant by its ID. + +| Parameter | Type | Description | +|-----------|------|-------------| +| `modelId` | `string` | The ID of the variant to select. | + +**Returns:** `void` + +**Throws:** `Error` if no variant with the given ID exists. + +#### `download(progressCallback?)` + +Downloads the selected variant to the local cache. + +| Parameter | Type | Description | +|-----------|------|-------------| +| `progressCallback` | `(progress: number) => void` | Optional callback receiving download progress (0–100). | + +**Returns:** `Promise` + +```typescript +await model.download((progress) => { + console.log(`Download: ${progress.toFixed(1)}%`); +}); +``` + +#### `load()` + +Loads the selected variant into memory for inference. + +**Returns:** `Promise` + +#### `unload()` + +Unloads the selected variant from memory. + +**Returns:** `Promise` + +#### `isLoaded()` + +Checks whether the selected variant is currently loaded in memory. + +**Returns:** `Promise` + +#### `removeFromCache()` + +Deletes the selected variant from the local cache. + +**Returns:** `void` + +#### `createChatClient()` + +Creates a ChatClient for performing chat completions with this model. + +**Returns:** `ChatClient` + +#### `createAudioClient()` + +Creates an AudioClient for performing audio transcription with this model. + +**Returns:** `AudioClient` + +#### `addVariant(variant)` + +Adds a variant to this model. Automatically selects it if it is cached and the current one is not. + +| Parameter | Type | Description | +|-----------|------|-------------| +| `variant` | `ModelVariant` | The variant to add. Must share the same alias. | + +**Returns:** `void` + +**Throws:** `Error` if the variant's alias does not match. + +--- + +## ModelVariant + +A specific variant of a model (e.g. a particular quantization or device target). Contains detailed model metadata and implements the same lifecycle interface as `Model`. Implements `IModel`. + +### Properties + +| Property | Type | Description | +|----------|------|-------------| +| `alias` | `string` | The model alias. | +| `id` | `string` | The unique identifier of this variant. | +| `isCached` | `boolean` | Whether this variant is downloaded locally. | +| `path` | `string` | Local file path of this variant. | +| `modelInfo` | `ModelInfo` | Detailed metadata about this variant. | + +### Methods + +Same lifecycle methods as `Model`: `download()`, `load()`, `unload()`, `isLoaded()`, `removeFromCache()`, `createChatClient()`, `createAudioClient()`. + +--- + +## ModelInfo + +Detailed metadata for a model variant. Returned by `ModelVariant.modelInfo`. + +| Property | Type | Description | +|----------|------|-------------| +| `id` | `string` | Unique model variant identifier. | +| `name` | `string` | Display name. | +| `alias` | `string` | Model alias. | +| `version` | `number` | Version number. | +| `uri` | `string` | Download URI. | +| `fileSizeBytes` | `number` | File size in bytes. | +| `promptTemplate` | `PromptTemplate` | Template for structuring prompts. | +| `runtime` | `Runtime` | Runtime configuration (device type, execution provider). | +| `isCached` | `boolean` | Whether this variant is downloaded. | +| `modelSettings` | `ModelSettings` | Model-specific parameters. | + +--- + +## ChatClient + +Client for performing chat completions with a loaded model. Follows the OpenAI Chat Completions API structure. + +### Properties + +| Property | Type | Description | +|----------|------|-------------| +| `settings` | `ChatClientSettings` | Configuration settings for chat completions. | + +### Methods + +#### `completeChat(messages)` + +Performs a non-streaming chat completion. + +| Parameter | Type | Description | +|-----------|------|-------------| +| `messages` | `any[]` | Array of message objects with `role` and `content` fields. | + +**Returns:** `Promise` — The chat completion response. + +**Throws:** `Error` if messages are invalid or completion fails. + +```typescript +const response = await chat.completeChat([ + { role: 'system', content: 'You are a helpful assistant.' }, + { role: 'user', content: 'What is 2+2?' } +]); +console.log(response.choices[0].message.content); +``` + +#### `completeStreamingChat(messages, callback)` + +Performs a streaming chat completion. The callback receives each response chunk as it arrives. + +| Parameter | Type | Description | +|-----------|------|-------------| +| `messages` | `any[]` | Array of message objects. | +| `callback` | `(chunk: any) => void` | Callback invoked for each streaming chunk. | + +**Returns:** `Promise` — Resolves when the stream completes. + +**Throws:** `Error` if messages or callback are invalid, or streaming fails. + +```typescript +await chat.completeStreamingChat( + [{ role: 'user', content: 'Tell me a story' }], + (chunk) => { + const delta = chunk.choices?.[0]?.delta?.content; + if (delta) process.stdout.write(delta); + } +); +``` + +--- + +## ChatClientSettings + +Configuration for chat completion requests. Assign to `chatClient.settings` before calling completion methods. + +| Property | Type | Default | Description | +|----------|------|---------|-------------| +| `temperature` | `number?` | — | Sampling temperature (0–2). Higher values produce more random output. | +| `maxTokens` | `number?` | — | Maximum number of tokens to generate. | +| `topP` | `number?` | — | Nucleus sampling threshold (0–1). | +| `topK` | `number?` | — | Top-K sampling parameter. | +| `frequencyPenalty` | `number?` | — | Penalizes tokens based on frequency in the output so far (-2 to 2). | +| `presencePenalty` | `number?` | — | Penalizes tokens based on presence in the output so far (-2 to 2). | +| `n` | `number?` | — | Number of completions to generate. | +| `randomSeed` | `number?` | — | Random seed for reproducible outputs. | + +```typescript +const chat = model.createChatClient(); +chat.settings.temperature = 0.7; +chat.settings.maxTokens = 512; +``` + +--- + +## AudioClient + +Client for performing audio transcription with a loaded model. Follows the OpenAI Audio API structure. + +### Properties + +| Property | Type | Description | +|----------|------|-------------| +| `settings` | `AudioClientSettings` | Configuration settings for audio operations. | + +### Methods + +#### `transcribe(audioFilePath)` + +Transcribes an audio file. + +| Parameter | Type | Description | +|-----------|------|-------------| +| `audioFilePath` | `string` | Path to the audio file (supported format: mp3). | + +**Returns:** `Promise` — The transcription result. + +**Throws:** `Error` if the file path is invalid or transcription fails. + +```typescript +const result = await audio.transcribe('./recording.mp3'); +console.log(result.text); +``` + +#### `transcribeStreaming(audioFilePath, callback)` + +Transcribes an audio file with streaming output. + +| Parameter | Type | Description | +|-----------|------|-------------| +| `audioFilePath` | `string` | Path to the audio file. | +| `callback` | `(chunk: any) => void` | Callback invoked for each streaming chunk. | + +**Returns:** `Promise` — Resolves when the stream completes. + +**Throws:** `Error` if the file path or callback is invalid, or streaming fails. + +```typescript +await audio.transcribeStreaming('./recording.mp3', (chunk) => { + if (chunk.text) process.stdout.write(chunk.text); +}); +``` + +--- + +## AudioClientSettings + +Configuration for audio transcription requests. Assign to `audioClient.settings` before calling transcription methods. + +| Property | Type | Default | Description | +|----------|------|---------|-------------| +| `language` | `string?` | — | Language of the audio (ISO 639-1 code, e.g. `"en"`). | +| `temperature` | `number?` | — | Sampling temperature for transcription. | + +--- + +## Enums and Types + +### DeviceType + +```typescript +enum DeviceType { + Invalid = 0, + CPU = 1, + GPU = 2, + NPU = 3 +} +``` + +### Runtime + +| Property | Type | Description | +|----------|------|-------------| +| `deviceType` | `DeviceType` | The target hardware device. | +| `executionProvider` | `string` | The execution provider name (e.g. `"CUDAExecutionProvider"`). | + +### PromptTemplate + +| Property | Type | Description | +|----------|------|-------------| +| `system` | `string` | Template for system messages. | +| `user` | `string` | Template for user messages. | +| `assistant` | `string` | Template for assistant messages. | +| `prompt` | `string` | Overall prompt template. | + +--- + +## Complete Example + +```typescript +import { FoundryLocalManager } from 'foundry-local-sdk'; + +async function main() { + // 1. Initialize + const manager = FoundryLocalManager.create({ + appName: 'MyApp', + logLevel: 'info' + }); + + // 2. Discover and select a model + const model = await manager.catalog.getModel('qwen2.5-0.5b'); + console.log(`Model: ${model.alias}, Variants: ${model.variants.length}`); + + // 3. Download with progress + await model.download((progress) => { + process.stdout.write(`\rDownloading: ${progress.toFixed(1)}%`); + }); + console.log('\nDownload complete.'); + + // 4. Load into memory + await model.load(); + + // 5. Chat completion + const chat = model.createChatClient(); + chat.settings.temperature = 0.7; + chat.settings.maxTokens = 256; + + // Non-streaming + const response = await chat.completeChat([ + { role: 'user', content: 'What is Foundry Local?' } + ]); + console.log(response.choices[0].message.content); + + // Streaming + await chat.completeStreamingChat( + [{ role: 'user', content: 'Tell me a joke' }], + (chunk) => { + const text = chunk.choices?.[0]?.delta?.content; + if (text) process.stdout.write(text); + } + ); + + // 6. Cleanup + await model.unload(); +} + +main().catch(console.error); +``` + +## Web Service Example + +Start an OpenAI-compatible REST endpoint alongside the SDK: + +```typescript +import { FoundryLocalManager } from 'foundry-local-sdk'; + +const manager = FoundryLocalManager.create({ + appName: 'MyApp', + webServiceUrls: 'http://127.0.0.1:8080' +}); + +const model = await manager.catalog.getModel('qwen2.5-0.5b'); +await model.download(); +await model.load(); + +// Start the web service +manager.startWebService(); +console.log('Web service running at:', manager.urls); + +// The service exposes OpenAI-compatible endpoints: +// POST /v1/chat/completions +// GET /v1/models +// GET /v1/models/{model_id} + +// Stop when done +manager.stopWebService(); +await model.unload(); +``` diff --git a/www/src/lib/components/document/doc-renderer.svelte b/www/src/lib/components/document/doc-renderer.svelte index fba20a64..4c0c5ab0 100644 --- a/www/src/lib/components/document/doc-renderer.svelte +++ b/www/src/lib/components/document/doc-renderer.svelte @@ -27,7 +27,9 @@ 'css', 'svelte', 'shell', - 'tsx' + 'tsx', + 'csharp', + 'python' ] }); }); diff --git a/www/src/lib/config.ts b/www/src/lib/config.ts index 4489d3a0..312f742a 100644 --- a/www/src/lib/config.ts +++ b/www/src/lib/config.ts @@ -44,6 +44,11 @@ export let navItems: NavItem[] = [ title: 'Docs', href: 'https://learn.microsoft.com/en-us/azure/ai-foundry/foundry-local/get-started', icon: BookOpen + }, + { + title: 'API Reference', + href: '/docs/api-reference', + icon: Code } ];