Clean up internal LLM API

[eb8680]

This refactoring PR reorganizes the internals of handlers.llm around four new Operations, each of which (call_assistant, call_user, call_system, call_tool) is responsible for constructing all Messages for one role ("assistant", "user", "system", "tool") in the completions API.

It also avoids the anti-pattern of handling Template.__apply__, except as a temporary expedient in RetryLLMHandler, and includes some general code cleanup of completions.py.

I'm putting it up as a draft for discussion, since I haven't finished getting tests to pass and there are still some outstanding design questions, particularly around the desired semantics of failure and repetition.

[eb8680]

I know I suggested this API, but to address #495 I'm realizing we probably want to modify the interfaces of these new operations so that they return both a Message and correctly decoded data:

type ToolCallId = ...


class DecodedToolCall[T](typing.NamedTuple):
  tool: Tool[..., T]
  bound_args: inspect.BoundArguments
  id: ToolCallId
  final: bool


@Operation.define
def call_assistant[T](
  messages: Sequence[Message],
  encoding: Encodable[T, pydantic.BaseModel | str],
  **kwargs
) -> tuple[Message, T | None, Sequence[DecodedToolCall]]:
  ...
  message: Message = ...
  if message.tool_calls:
    decoded_tool_calls: Sequence[DecodedToolCall] = ...
    return message, None, decoded_tool_calls
  else:
    result: T = encoding.decode(encoding.deserialize(message.content))
    return message, result, ()


@Operation.define
def call_tool[T](
  tc: DecodedToolCall[T],
) -> tuple[Message, T]:
  result: T = tc.tool(*tc.bound_args.args, **tc.bound_args.kwargs)
  encoding = Encodable.define(type(result))
  msg: Message = Message(role="tool", content=encoding.serialize(encoding.encode(result)), tool_call_id=tc.id)
  return msg, result

[kiranandcode]

@eb8680 , ready for review!

[eb8680]

This removes RetryLLMHandler, is that still intended?

[kiranandcode]

Leaving RetryLLMHandler for another PR

[eb8680]

OK, in that case it's good to merge. You'll have to approve and merge it yourself since I'm the original author of the PR.

[kiranandcode]

Gotcha, will do once tests pass! Rebasing the synthesis on top of this, and working on that in parallel