LLM deployment on RK3566?

[Gooddz1]

Hello, does the rkllama project support deploying llm (e.g., 0.5B parameters) on the RK3566 chip? If supported, what is the expected performance and is it considered feasible? Any guidance would be helpful. Thanks!

[danielferr85]

No. RKLLAMA use the native rkllm library and the library doesnt support it.

When the rkllm library support it, I will make the required changes to include it to the RKLLAMA. I have an OPI 3b that i also want to use with RKLLAMA

[Gooddz1]

Can the quantization method of this project be used in rknn-toolkit2's model quantization conversion? Is it only applicable to rknn-llm or RKLLAMA?