Caching the pipeline outputs

[jdluzen]

I'd like to get your thoughts on how to cache the various outputs from the different stages of the pipeline, primarily the output from the text encoder. That step seems to be a substantial amount of time for a generation. But from a users view, I can imagine a scenario where they'd want to regenerate with the same prompt.
I haven't given the implementation part much thought, but probably default to some kind of interface and in memory storage.

[saddam213]

For the text_encoders we should be able to cache the PromptResult from CreatePromptAsync directly on the pipeline

If Prompt or NegativePrompt changes then re-run the models, else return the last generated PromptResult

In OnnxStack there was a separate GenerateBatch call that cached the prompt when doing batches but for TensorStack I think this should be done by default, I don't see a reason why we should ever need to regenerate a prompt, but we could also add a flag to ignore caching in the GenerateOptions

Caching other parts of the pipeline could be handy, but probably best to wait for a use case before making he pipelines too complicated

[saddam213]

Just had an injection into my spine so apparently will be pain free for a day or two so I might look at this tomorrow, unless you would like to?

[jdluzen]

Ack, do what you need to do man. I'll just poke at the branch for a bit. Specifically caching to disk/DB.