LLMBasedFaithfulness does not work when using LLMs other than OpenAI

[kelvinchanwh]

In continuous_eval/metrics/generation/text/llm_based.py, line 39, the full model parameters are not being passed to the LLMBasedContextCoverage class. This means that the code defaults to calling OpenAI API even if another LLM is passed as self.model

Changes requied

AS-IS:
context_coverage = LLMBasedContextCoverage(use_few_shot=self.use_few_shot)

TO-BE:

context_coverage = LLMBasedContextCoverage(model=self._llm, use_few_shot=self.use_few_shot)

[yisz]

Great catch, thanks @kelvinchanwh!

@ellipsis-dev can you fix this and submit a PR.

[kelvinchanwh]

The code change should be to the following instead since model is stored in self._llm

line 39:
AS-IS: context_coverage = LLMBasedContextCoverage(use_few_shot=self.use_few_shot)
TO-BE: context_coverage = LLMBasedContextCoverage(model=self._llm, use_few_shot=self.use_few_shot)