mismatch when inference

[ziyanxzy]

when i run the example for this repo:

it will have the issue:

[jpablomch]

Thanks, @ziyanxzy, for your interest in Shears! Please check if your environment matches the setup described here. If you intend only to perform inference, apply the PEFT inference patch to PEFT.

git clone https://github.com/huggingface/peft.git
cd peft && git checkout v0.5.0 && git apply --ignore-space-change --ignore-whitespace peft-modifications-for-shears-inference-usage.patch&& pip install -e . && cd ..

Ref: https://huggingface.co/IntelLabs/shears-mpt-7b-50-gsm8k-heuristic-adapter#how-to-use.