How to reset a Llama object back to initial state

[jacob-lee]

I want to sample the output of the same query multiple times independently. My (limited) understanding is that the history of these queries change subsequent outputs. So, I guess I could do:

outputs = []
for _ in range(30):
    llm = Llama(model_path="model.gguf")
    outputs.append(llm('my query goes here'))

but re-instantiating the model seems a bit heavy, and I was wondering if there were a way to simply reset the model. I see that there is a reset method of the llm object, but as far as i can tell it simply sets n_tokens attribute to 0. Is that really enough to set the model back to its initial state?

[Egorkin-enabled]

Hello through centuries!

Just move out Llama instance & use create_completion instead. This way story is not collecting.

import llama_cpp

# Enable unlimited caching.
cache = llama_cpp.LlamaCache()
model.set_cache(cache)

# Your promts.
prompts = [ "What capital of Paris?", "Who are you", "..." ]

# Our outputs.
outputs = []

for p in prompts:
    outputs.append(model.create_completion(prompt=p)['choices'][0]['text'])

Hope it helps even through times!