Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
38 changes: 38 additions & 0 deletions docs/features/model-top-p-settings/plan.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,38 @@
# Implementation Plan

## Approach

Implement `topP` as an optional generation setting, mirroring the existing session generation settings pipeline while avoiding a forced default.

## Affected Interfaces

- `src/shared/types/agent-interface.d.ts`: add optional `topP` to `SessionGenerationSettings`.
- `src/shared/contracts/common.ts`: add optional `topP` to route schemas.
- `src/shared/utils/generationSettingsValidation.ts`: validate `topP` as a finite number in `[0.1, 1]`.
- `src/main/presenter/sqlitePresenter/tables/deepchatSessions.ts`: add `top_p` storage and migration.
- `src/main/presenter/agentRuntimePresenter/index.ts`: sanitize, persist, and pass `topP` through runtime model config.
- `src/main/presenter/llmProviderPresenter/aiSdk/runtime.ts`: include `topP` in AI SDK `generateText` and `streamText` calls and request traces when defined.
- `src/renderer/src/stores/ui/draft.ts`: carry draft `topP` overrides for new sessions.
- `src/renderer/src/components/chat/ChatStatusBar.vue`: show and persist the compact `topP` control.
- i18n `chat.json` and `settings.json` files: add `topP` label, hover description, and validation.
- `topP` number inputs use min `0.1`, max `1`, and step `0.1` so values align with common AI SDK/provider constraints.

## Data Flow

1. User edits `topP` in ChatStatusBar.
2. Draft sessions store it in Pinia; active sessions call `sessions.updateGenerationSettings`.
3. Main process validates and stores `topP` as part of session generation settings.
4. Agent runtime adds `topP` to runtime `ModelConfig`.
5. AI SDK runtime includes `topP` only when defined.

## Compatibility

- Existing databases migrate by adding nullable `top_p`.
- Existing sessions return no `topP` unless previously set.
- Requests without `topP` preserve current behavior.

## Test Strategy

- Run formatting and i18n generation required by repository guidelines.
- Run lint to catch type/schema/template issues.
- Prefer focused type/lint validation over provider integration tests because this is a pass-through parameter.
36 changes: 36 additions & 0 deletions docs/features/model-top-p-settings/spec.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,36 @@
# Model Top P Settings

## User Need

Users need to adjust `top_p` for chat models because many upstream model APIs support nucleus sampling and expose it as a generation parameter.

## Goal

Add an optional per-session `topP` generation setting for text chat requests and pass it through to AI SDK text generation when the user explicitly sets it.

## Acceptance Criteria

- Users can set `topP` from the chat model advanced settings panel for regular text chat models.
- `topP` accepts values greater than or equal to 0.1 and less than or equal to 1.
- The Top P UI uses the plain label "Top P" and moves explanatory copy into a hover help icon.
- Existing conversations and new sessions continue to work when no `topP` is set.
- `topP` persists with DeepChat session generation settings and survives app restart.
- Text `generateText` and streaming requests pass `topP` to AI SDK only when it is defined.
- Existing Voice.ai TTS `topP` configuration remains independent.

## Constraints

- Follow current typed route/contracts and presenter boundaries.
- Use `topP` internally and let SDK/provider layers map provider payload details.
- Do not default-send `topP: 1`; omit the parameter when unset to preserve provider defaults.
- Use i18n keys for all user-facing strings.
- SQLite schema migration must be backward compatible.

## Non-Goals

- Provider-specific `top_p` compatibility matrices.
- Building a full Provider DB `top_p` capability matrix.

## Open Questions

- None.
10 changes: 10 additions & 0 deletions docs/features/model-top-p-settings/tasks.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,10 @@
# Tasks

- [x] Create SDD artifacts for optional model `topP` setting.
- [x] Add shared types, contracts, and validation support for `topP`.
- [x] Add SQLite session persistence and migration for `topP`.
- [x] Propagate `topP` through runtime model config and AI SDK calls.
- [x] Add renderer draft state and advanced settings UI for `topP`.
- [x] Add i18n strings for `topP` controls and validation.
- [x] Refine Top P UI to use a help tooltip and `[0.1, 1]` number input range.
- [x] Run `pnpm run format`, `pnpm run i18n`, and `pnpm run lint`.
22 changes: 22 additions & 0 deletions docs/issues/chat-top-p-tooltip/plan.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,22 @@
# Plan

## Approach

The current nested `Tooltip` is fragile inside the model settings `Popover` and can render beneath or outside the visible stacking context. Replace it with a controlled lightweight hover/focus panel anchored next to the Top P label. This keeps tooltip-style interaction while avoiding portal stacking conflicts.

## Affected Interfaces

- `src/renderer/src/components/chat/ChatStatusBar.vue`

## Data Flow

No data flow changes. The same `chat.advancedSettings.topPDescription` i18n string is displayed.

## Compatibility

No persisted configuration changes. The chat settings Top P clamp behavior remains unchanged.
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️ Potential issue | 🟡 Minor | ⚡ Quick win

Plan text is stale relative to this PR’s Top P behavior.

Line 17 says clamp behavior is unchanged, but this PR also tightens Top P min/clamp behavior in ChatStatusBar.vue (e.g., min set to 0.1 and commit-time clamping). Please update the plan so reviewers/testers aren’t working from incorrect assumptions.

🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@docs/issues/chat-top-p-tooltip/plan.md` at line 17, Update the plan text to
reflect that Top P clamp behavior was changed in ChatStatusBar.vue: state that
the minimum Top P is now 0.1 (previously lower), that commit-time clamping is
applied (values are clamped when committing/ saving), and any persistence/tests
should expect the tightened min/clamp behavior rather than "unchanged";
reference ChatStatusBar.vue and the Top P min/clamp change so reviewers/testers
use the correct assumptions.


## Test Strategy

- Static verification of template/script changes.
- Run project formatting/lint commands if package tooling is available.
28 changes: 28 additions & 0 deletions docs/issues/chat-top-p-tooltip/spec.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,28 @@
# Chat Top P Tooltip

## User Need

The chat settings panel Top P help icon must show explanatory text reliably. The settings dialog should keep inline description text, while the chat status bar should keep a hover tooltip interaction.

## Goal

Fix the Top P help content in `ChatStatusBar.vue` so it is visible above the settings popover and has a readable width.

## Acceptance Criteria

- Hovering the Top P help icon in the chat settings panel shows the Top P description.
- The tooltip is not hidden behind the model settings popover.
- The tooltip content has a readable constrained width for the long description.
- The settings dialog Top P description remains inline text.
- The existing Top P range clamp behavior remains aligned between chat and settings pages.

## Constraints

- Keep changes focused to the chat settings Top P help behavior.
- Preserve existing i18n keys and shadcn/reka UI styling patterns where practical.
- Do not introduce new dependencies.

## Non-goals

- Redesign the full chat settings panel.
- Change Top P semantics or persisted data shape.
5 changes: 5 additions & 0 deletions docs/issues/chat-top-p-tooltip/tasks.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,5 @@
# Tasks

- [x] Inspect current Top P tooltip implementation and popover stacking.
- [x] Add a layer-safe tooltip-style hover/focus panel for Top P in chat settings.
- [ ] Verify formatting and report any unavailable validation tooling.
31 changes: 29 additions & 2 deletions src/main/presenter/agentRuntimePresenter/index.ts
Original file line number Diff line number Diff line change
Expand Up @@ -154,6 +154,11 @@ type PackageJsonManifest = {
scripts?: Record<string, unknown>
}

function normalizeTopP(value: unknown): number | undefined {
const numeric = parseFiniteNumericValue(value)
return numeric !== undefined && numeric >= 0.1 && numeric <= 1 ? numeric : undefined
}

function readPackageJsonManifest(workdir: string): PackageJsonManifest | null {
try {
const packageJsonPath = path.join(workdir, 'package.json')
Expand Down Expand Up @@ -202,6 +207,7 @@ type PersistedSessionGenerationRow = {
permission_mode: PermissionMode
system_prompt: string | null
temperature: number | null
top_p: number | null
context_length: number | null
max_tokens: number | null
timeout_ms: number | null
Expand Down Expand Up @@ -1237,7 +1243,6 @@ export class AgentRuntimePresenter implements IAgentImplementation {
if (!state && !dbSession) {
throw new Error(`Session ${sessionId} not found`)
}

const providerId = state?.providerId ?? dbSession?.provider_id
const modelId = state?.modelId ?? dbSession?.model_id
if (!providerId || !modelId) {
Expand Down Expand Up @@ -1987,6 +1992,7 @@ export class AgentRuntimePresenter implements IAgentImplementation {
const modelConfig: ModelConfig = {
...baseModelConfig,
temperature: generationSettings.temperature,
topP: generationSettings.topP,
contextLength: generationSettings.contextLength,
maxTokens: capAgentRequestMaxTokens(generationSettings.maxTokens, contextBudgetLength),
timeout: generationSettings.timeout,
Expand Down Expand Up @@ -3328,6 +3334,9 @@ export class AgentRuntimePresenter implements IAgentImplementation {
if (sessionRow.temperature !== null) {
patch.temperature = sessionRow.temperature
}
if (sessionRow.top_p !== null) {
patch.topP = sessionRow.top_p
}
if (sessionRow.context_length !== null) {
patch.contextLength = sessionRow.context_length
}
Expand Down Expand Up @@ -3375,6 +3384,9 @@ export class AgentRuntimePresenter implements IAgentImplementation {
if (Object.prototype.hasOwnProperty.call(requestedPatch, 'temperature')) {
patch.temperature = sanitized.temperature
}
if (Object.prototype.hasOwnProperty.call(requestedPatch, 'topP')) {
patch.topP = sanitized.topP
}
if (Object.prototype.hasOwnProperty.call(requestedPatch, 'contextLength')) {
patch.contextLength = sanitized.contextLength
}
Expand Down Expand Up @@ -3402,6 +3414,9 @@ export class AgentRuntimePresenter implements IAgentImplementation {
if (Object.prototype.hasOwnProperty.call(requestedPatch, 'imageGeneration')) {
patch.imageGeneration = sanitized.imageGeneration
}
if (Object.prototype.hasOwnProperty.call(requestedPatch, 'videoGeneration')) {
patch.videoGeneration = sanitized.videoGeneration
}

return patch
}
Expand All @@ -3412,6 +3427,7 @@ export class AgentRuntimePresenter implements IAgentImplementation {
return {
systemPrompt: settings.systemPrompt,
temperature: settings.temperature,
topP: settings.topP,
contextLength: settings.contextLength,
maxTokens: settings.maxTokens,
timeout: settings.timeout,
Expand All @@ -3420,7 +3436,8 @@ export class AgentRuntimePresenter implements IAgentImplementation {
reasoningVisibility: settings.reasoningVisibility,
verbosity: settings.verbosity,
forceInterleavedThinkingCompat: settings.forceInterleavedThinkingCompat,
imageGeneration: settings.imageGeneration
imageGeneration: settings.imageGeneration,
videoGeneration: settings.videoGeneration
}
}

Expand Down Expand Up @@ -3466,6 +3483,7 @@ export class AgentRuntimePresenter implements IAgentImplementation {
fixedTemperatureKimi?.temperature ??
parseFiniteNumericValue(modelConfig.temperature) ??
0.7,
topP: normalizeTopP(modelConfig.topP),
contextLength: contextLengthDefault,
timeout:
timeoutDefault >= MODEL_TIMEOUT_MIN_MS && timeoutDefault <= MODEL_TIMEOUT_MAX_MS
Expand Down Expand Up @@ -3607,6 +3625,15 @@ export class AgentRuntimePresenter implements IAgentImplementation {
}
}

if (Object.prototype.hasOwnProperty.call(patch, 'topP')) {
const normalizedTopP = normalizeTopP(patch.topP)
if (normalizedTopP !== undefined) {
next.topP = normalizedTopP
} else {
delete next.topP
}
}
Comment thread
zhangmo8 marked this conversation as resolved.

if (Object.prototype.hasOwnProperty.call(patch, 'timeout')) {
const error = validateGenerationNumericField('timeout', patch.timeout)
const numeric = toValidNonNegativeInteger(parseFiniteNumericValue(patch.timeout))
Expand Down
3 changes: 3 additions & 0 deletions src/main/presenter/configPresenter/modelConfig.ts
Original file line number Diff line number Diff line change
Expand Up @@ -184,6 +184,7 @@ export class ModelConfigHelper {
contextLength: resolveModelContextLength(model.limit?.context),
timeout: DEFAULT_MODEL_TIMEOUT,
temperature: 0.6,
topP: undefined,
vision: isImageInputSupported(model),
speechRecognition: false,
functionCall: resolveModelFunctionCall(model.tool_call),
Expand Down Expand Up @@ -488,6 +489,7 @@ export class ModelConfigHelper {
...DEFAULT_MODEL_CAPABILITY_FALLBACKS,
timeout: DEFAULT_MODEL_TIMEOUT,
temperature: 0.6,
topP: undefined,
type: ModelType.Chat,
apiEndpoint: ApiEndpointType.Chat,
endpointType: undefined,
Expand All @@ -512,6 +514,7 @@ export class ModelConfigHelper {
contextLength: storedConfig.contextLength ?? finalConfig.contextLength,
timeout: storedConfig.timeout ?? finalConfig.timeout,
temperature: storedConfig.temperature ?? finalConfig.temperature,
topP: storedConfig.topP ?? finalConfig.topP,
vision: storedConfig.vision ?? finalConfig.vision,
speechRecognition: storedConfig.speechRecognition ?? finalConfig.speechRecognition,
functionCall: storedConfig.functionCall ?? finalConfig.functionCall,
Expand Down
6 changes: 5 additions & 1 deletion src/main/presenter/llmProviderPresenter/aiSdk/runtime.ts
Original file line number Diff line number Diff line change
Expand Up @@ -1178,7 +1178,8 @@ export async function runAiSdkGenerateText(
maxOutputTokens: maxTokens,
...(shouldSendTemperature && resolvedTemperature !== undefined
? { temperature: resolvedTemperature }
: {})
: {}),
...(normalizedModelConfig.topP !== undefined ? { topP: normalizedModelConfig.topP } : {})
}
Comment thread
zhangmo8 marked this conversation as resolved.

await context.emitRequestTrace?.(normalizedModelConfig, {
Expand All @@ -1195,6 +1196,7 @@ export async function runAiSdkGenerateText(
...(shouldSendTemperature && resolvedTemperature !== undefined
? { temperature: resolvedTemperature }
: {}),
...(normalizedModelConfig.topP !== undefined ? { topP: normalizedModelConfig.topP } : {}),
maxOutputTokens: maxTokens
})

Expand Down Expand Up @@ -1380,6 +1382,7 @@ export async function* runAiSdkCoreStream(
...(shouldSendTemperature && resolvedTemperature !== undefined
? { temperature: resolvedTemperature }
: {}),
...(normalizedModelConfig.topP !== undefined ? { topP: normalizedModelConfig.topP } : {}),
tools: tools.map((tool) => tool.function.name)
}

Expand All @@ -1398,6 +1401,7 @@ export async function* runAiSdkCoreStream(
...(shouldSendTemperature && resolvedTemperature !== undefined
? { temperature: resolvedTemperature }
: {}),
...(normalizedModelConfig.topP !== undefined ? { topP: normalizedModelConfig.topP } : {}),
maxOutputTokens: maxTokens
})

Expand Down
5 changes: 4 additions & 1 deletion src/main/presenter/sqlitePresenter/schemaCatalog.ts
Original file line number Diff line number Diff line change
Expand Up @@ -117,6 +117,7 @@ const CATALOG_DEFINITIONS: CatalogDefinition[] = [
repairableColumns: {
system_prompt: 'ALTER TABLE deepchat_sessions ADD COLUMN system_prompt TEXT;',
temperature: 'ALTER TABLE deepchat_sessions ADD COLUMN temperature REAL;',
top_p: 'ALTER TABLE deepchat_sessions ADD COLUMN top_p REAL;',
context_length: 'ALTER TABLE deepchat_sessions ADD COLUMN context_length INTEGER;',
max_tokens: 'ALTER TABLE deepchat_sessions ADD COLUMN max_tokens INTEGER;',
thinking_budget: 'ALTER TABLE deepchat_sessions ADD COLUMN thinking_budget INTEGER;',
Expand All @@ -131,7 +132,9 @@ const CATALOG_DEFINITIONS: CatalogDefinition[] = [
'ALTER TABLE deepchat_sessions ADD COLUMN force_interleaved_thinking_compat INTEGER;',
reasoning_visibility: 'ALTER TABLE deepchat_sessions ADD COLUMN reasoning_visibility TEXT;',
image_generation_options_json:
'ALTER TABLE deepchat_sessions ADD COLUMN image_generation_options_json TEXT;'
'ALTER TABLE deepchat_sessions ADD COLUMN image_generation_options_json TEXT;',
video_generation_options_json:
'ALTER TABLE deepchat_sessions ADD COLUMN video_generation_options_json TEXT;'
},
typeCheckedColumns: [
'summary_cursor_order_seq',
Expand Down
Loading