Currently, the Not Diamond SDK requires calling client.chat.completions.create() to generate a session_id before submitting feedback via metric.feedback(). This tightly couples feedback submission to completion generation through the SDK or API.
In many real-world use cases, users may already have completions generated externally (e.g., from self-hosted models, other APIs, or offline inference) and want to:
Submit feedback on those completions
Use Not Diamond’s routing and feedback learning without invoking .create() or re-generating completions
Avoid wasting compute/time by duplicating requests just to get session IDs
Current limitation:
Feedback submission requires a valid session_id, which currently can only be obtained by calling .create(). There is no public API or SDK method to:
Manually register a session (session ID + model/provider info)
Submit feedback linked to that manually registered session
Proposed solution:
Add support for:
A new SDK/API method to manually create/register sessions with arbitrary metadata, allowing users to associate existing completions with sessions.
Allow submitting feedback referencing these manually registered sessions without calling .create().
Possibly support a “dry-run” or “session-only” mode in .create() that skips actual generation but returns a valid session ID.
Benefits:
Enables users to integrate Not Diamond feedback with completions generated outside the SDK/API.
Avoids duplicate generation calls and wasted compute.
Makes Not Diamond routing usable in more diverse real-world scenarios (self-hosted, batch-generated, multi-cloud, etc.).
Example use case:
I have a custom LLM inference pipeline outside Not Diamond. I want to use ND’s routing feedback system without re-generating completions. Currently, I have to call .create() anyway to get a session ID, which causes wasted calls and costs.
Additional context:
Code inspection shows that metric.feedback() requires a session_id obtained only from .create(). There is no public API to register a session independently.
Thank you!
Currently, the Not Diamond SDK requires calling client.chat.completions.create() to generate a session_id before submitting feedback via metric.feedback(). This tightly couples feedback submission to completion generation through the SDK or API.
In many real-world use cases, users may already have completions generated externally (e.g., from self-hosted models, other APIs, or offline inference) and want to:
Submit feedback on those completions
Use Not Diamond’s routing and feedback learning without invoking .create() or re-generating completions
Avoid wasting compute/time by duplicating requests just to get session IDs
Current limitation:
Feedback submission requires a valid session_id, which currently can only be obtained by calling .create(). There is no public API or SDK method to:
Manually register a session (session ID + model/provider info)
Submit feedback linked to that manually registered session
Proposed solution:
Add support for:
A new SDK/API method to manually create/register sessions with arbitrary metadata, allowing users to associate existing completions with sessions.
Allow submitting feedback referencing these manually registered sessions without calling .create().
Possibly support a “dry-run” or “session-only” mode in .create() that skips actual generation but returns a valid session ID.
Benefits:
Enables users to integrate Not Diamond feedback with completions generated outside the SDK/API.
Avoids duplicate generation calls and wasted compute.
Makes Not Diamond routing usable in more diverse real-world scenarios (self-hosted, batch-generated, multi-cloud, etc.).
Example use case:
I have a custom LLM inference pipeline outside Not Diamond. I want to use ND’s routing feedback system without re-generating completions. Currently, I have to call .create() anyway to get a session ID, which causes wasted calls and costs.
Additional context:
Code inspection shows that metric.feedback() requires a session_id obtained only from .create(). There is no public API to register a session independently.
Thank you!