Update custom tool display documentation

[shea-parkes]

I'm trying to do a custom tool display as documented here:
https://shiny.posit.co/py/docs/genai-tools.html#custom-display

Here is a small example:

import os

from chatlas import ChatAzureOpenAI
from chatlas import ContentToolResult
from shiny import App
from shiny import ui


class RedBoxResult(ContentToolResult):
    """A silly example"""

    def tagify(self):
        print("DEBUG: Custom tagify method was called!")

        return ui.div(
            ui.h3("My Custom Result", style="color: white;"),
            ui.p(f"Tool Value: {self.value["the_thing_to_be_red"]}", style="color: white;"),
            style="background-color: red; padding: 20px; border-radius: 10px; margin: 10px 0;",
        )


def test_tool(message: str):
    """
    A test tool that takes a message and returns it in a red box.
    """
    print(f"DEBUG: Tool called with message: {message}")

    return RedBoxResult(value={"the_thing_to_be_red": message})


app_ui = ui.page_fillable(ui.chat_ui("chat"))


def server(input, output, session):
    chat = ui.Chat(id="chat")

    chat_client = ChatAzureOpenAI(
        endpoint=os.environ["AZUREOPENAI_ENDPOINT"],
        deployment_id=os.environ["AZUREOPENAI_DEPLOYMENT_NAME"],
        api_version=os.environ["AZUREOPENAI_API_VERSION"],
        api_key=os.environ["AZUREOPENAI_API_KEY"],
        system_prompt="Be a helpful assistant",
    )
    chat_client.register_tool(test_tool)

    @chat.on_user_submit
    async def _(user_input: str):
        stream = await chat_client.stream_async(user_input, content="all")
        await chat.append_message_stream(stream)


app = App(app_ui, server)

Here's a screenshot of using the above app:

The logs show the RedBoxResult.tagify method was never called.

Is this related perhaps to the note # Currently, OpenAI only allows for text content in tool results in _provider_openai_completions.py? If so, I do understand, but it would be good to document that back at https://shiny.posit.co/py/docs/genai-tools.html#custom-display

[cpsievert]

Hey, thanks for reporting! We recently changed/improved the overall approach to custom tool displays -- those docs need to get updated.

In the meantime, this directory has the full range of possibilities:

https://github.com/posit-dev/shinychat/tree/main/pkg-py/tests/playwright/tools/weather

If you'd like to share more about what you're trying to achieve/customize, I can help point you to the relevant app.