Skip to content

ReelzOrg/llm_service

Repository files navigation

Reelz LLM Service

This service is responsible for handling the LLM requests and responses.

Setup

  1. Install dependencies:
uv sync

Make sure to have the LLM models installed in your environment which includes - gemma3:1b, qwen3:4b, qwen2.5-coder:0.5b

  1. Run the service:
python app/llm_service.py

About

This is a grpc server which talks to the node.js client and receives the user prompts and streams responses from llms

Topics

Resources

Stars

Watchers

Forks

Releases

No releases published

Packages

 
 
 

Contributors