Skip to content

feat: allow multipart tool responses to include a metadata field. #4865

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Draft
pavelgj wants to merge 2 commits into
base: main
Choose a base branch
from
+35 −1
Draft

feat: allow multipart tool responses to include a metadata field. #4865

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
32f7605
feat: allow multipart tool responses to include a metadata field.
pavelgj Mar 1, 2026
85f7098
feat: Add metadata field to Message class.
pavelgj Mar 1, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions genkit-tools/common/src/types/parts.ts
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -196,6 +196,7 @@ export type Part = z.infer<typeof PartSchema>;
export const MultipartToolResponseSchema = z.object({
output: z.unknown().optional(),
content: z.array(PartSchema).optional(),
metadata: z.record(z.unknown()).optional(),
});

export type MultipartToolResponse = z.infer<typeof MultipartToolResponseSchema>;
4 changes: 4 additions & 0 deletions genkit-tools/genkit-schema.json
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -906,6 +906,10 @@
"items": {
"$ref": "#/$defs/Part"
}
},
"metadata": {
"type": "object",
"additionalProperties": {}
}
},
"additionalProperties": false
Expand Down
3 changes: 2 additions & 1 deletion go/ai/gen.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -326,7 +326,8 @@ type ModelResponseChunk struct {
// MultipartToolResponse represents a tool response with both structured output and content parts.
type MultipartToolResponse struct {
// Content holds additional message parts providing context or details.
Content []*Part `json:"content,omitempty"`
Content []*Part `json:"content,omitempty"`
Metadata map[string]any `json:"metadata,omitempty"`
// Output contains the structured output data from the tool.
Output any `json:"output,omitempty"`
Comment on lines +329 to 332
Copy link
Contributor

@gemini-code-assist gemini-code-assist bot Mar 1, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

medium

The new Metadata field is inserted between Content and the comment for Output, which makes the code harder to read. It also lacks a comment explaining its purpose, unlike other fields in the struct.

For better readability and consistency, I suggest moving the Metadata field to the end of the struct and adding a descriptive comment.

I see that this file is auto-generated. If a manual edit is not possible, this might indicate an issue in the jsonschemagen tool that needs to be addressed, as it seems to be misplacing comments when sorting struct fields alphabetically.

	Content  []*Part        "json:\"content,omitempty\""
	// Output contains the structured output data from the tool.
	Output   any            "json:\"output,omitempty\""
	// Metadata contains arbitrary key-value data associated with this response.
	Metadata map[string]any "json:\"metadata,omitempty\""

}
Expand Down
1 change: 1 addition & 0 deletions js/ai/src/generate/resolve-tool-requests.ts
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -132,6 +132,7 @@ export async function resolveToolRequest(
output: multipartResponse.output,
content: multipartResponse.content,
} as ToolResponse,
metadata: multipartResponse.metadata,
});

return { response };
Expand Down
1 change: 1 addition & 0 deletions js/ai/src/parts.ts
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -187,6 +187,7 @@ export type Part = z.infer<typeof PartSchema>;
export const MultipartToolResponseSchema = z.object({
output: z.unknown().optional(),
content: z.array(PartSchema).optional(),
metadata: z.record(z.unknown()).optional(),
});

export type MultipartToolResponse = z.infer<typeof MultipartToolResponseSchema>;
1 change: 1 addition & 0 deletions js/ai/src/tool.ts
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -301,6 +301,7 @@ export type MultipartToolFn<I extends z.ZodTypeAny, O extends z.ZodTypeAny> = (
) => Promise<{
output?: z.infer<O>;
content?: Part[];
metadata?: Record<string, any>;
}>;

export function defineTool<I extends z.ZodTypeAny, O extends z.ZodTypeAny>(
Expand Down
2 changes: 2 additions & 0 deletions js/ai/tests/generate/generate_test.ts
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -632,6 +632,7 @@ describe('generate', () => {
return {
output: 'main output',
content: [{ text: 'part 1' }],
metadata: { custom: 'data' },
};
}
);
Expand Down Expand Up @@ -699,6 +700,7 @@ describe('generate', () => {
},
],
},
metadata: { custom: 'data' },
},
],
},
Expand Down
22 changes: 22 additions & 0 deletions js/ai/tests/tool_test.ts
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -135,6 +135,28 @@ describe('defineInterrupt', () => {
});
});

it('should define a multipart tool returning metadata', async () => {
const t = defineTool(
registry,
{ name: 'test_meta', description: 'test', multipart: true },
async () => {
return {
output: 'main output',
content: [{ text: 'part 1' }],
metadata: { customField: 123 },
};
}
);
assert.equal(t.__action.metadata.type, 'tool.v2');
assert.equal(t.__action.actionType, 'tool.v2');
const result = await t({});
assert.deepStrictEqual(result, {
output: 'main output',
content: [{ text: 'part 1' }],
metadata: { customField: 123 },
});
});

it('should handle fallback output', async () => {
const t = defineTool(
registry,
Expand Down
1 change: 1 addition & 0 deletions py/packages/genkit/src/genkit/core/typing.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -936,6 +936,7 @@ class MultipartToolResponse(BaseModel):
model_config: ClassVar[ConfigDict] = ConfigDict(alias_generator=to_camel, extra='forbid', populate_by_name=True)
output: Any | None = None
content: list[Part] | None = None
metadata: dict[str, Any] | None = None


class RerankerRequest(BaseModel):
Expand Down
Loading