Installation instructions

Setting up HarpIA Model Gateway

HarpIA Model Gateway can run using a local Python installation or a Docker container.

From a local Python installation

1. Install Python ≥ 3.11 with pip.

2. Create a Python environment and activate it:

   python3 -m venv .venv
   source ./.venv/bin/activate

3. Install Poetry.

4. Install the dependencies:

   poetry install --no-root

5. Create a configuration file. Create a copy of the config_TEMPLATE.py file and edit it to choose the models that will be provided. Follow the instructions in the file.

6. Test an answer provider by sending a single message on a terminal, as in this example (replace ./config/config1.py with the path to your configuration file, and replace ECHO with the name of an answer provider as defined in the configuration file):

   python run_gateway.py --config=./config/config1.py cli --provider="ECHO"

7. Start the server (replace 42774 with the desired port):

   python run_gateway.py --config=./config/config1.py server --host=0.0.0.0 --port=42774

   Temporarily, only during the installation until everything is working, --debug may be added to the end of the command to enable debugging messages during the setup.

8. While the server is running, send a test request (replace change the port and the name of the answer provider accordingly):

   curl -X POST http://localhost:42774/send -H "Content-Type: application/json" -d '{"query": "this is a test", "answer_provider": "ECHO"}'
   

   It should print a JSON response. In the example above, one of the lines will be "text": "THIS IS A TEST" (i.e. the query converted to uppercase by the answer provider ECHO).

From the Docker container

1. Install Docker.

2. Build the Docker image:

   docker build -t harpia-model-gateway:1.0 -f containers/prod/Dockerfile .

3. Create a configuration file. Create a copy of the config_TEMPLATE.py file and edit it to choose the models that will be provided. Follow the instructions in the file.
   

   IMPORTANT: If you use Ollama or any other model that requires accessing localhost addresses:

   • Define a virtual name for the host's localhost that will be accessible from the container (e.g. ollamaserver);
   • Use that address in your configuration file (e.g. "ollama_address": "http://ollamaserver:11434/api");
   • In the remaining steps, add --add-host=ollamaserver:host-gateway (replacing ollamaserver with the chosen name). The commands will look like docker run --add-host=ollamaserver:host-gateway --rm [...].
   • Make sure that the communication between the containers is not blocked by firewalls or system settings.
   • Standalone Ollama blocks external connections by default, preventing the communication with HarpIA Model Gateway's container. Either configure Ollama server to allow external connections by setting OLLAMA_HOST to 0.0.0.0, or run Ollama from its Docker image.

4. Test an answer provider by sending a single message on a terminal, as in this example (replace ./config/config1.py with the path to your configuration file, and replace ECHO with the name of an answer provider as defined in the configuration file):

   docker run --rm -it --name harpia-gateway -v './config/config1.py':/cfg.py harpia-model-gateway:1.0 --config=/cfg.py cli --provider="ECHO"

5. Start the server (replace ./config/config1.py with the path to your configuration file, optionally replace all instances of 42774 with the desired port):

   docker run --rm -d --name harpia-gateway -v './config/config1.py':/cfg.py -p 42774:42774 harpia-model-gateway:1.0 --config=/cfg.py server --host=0.0.0.0 --port=42774

   Temporarily, only during the installation until everything is working, --debug may be added to the end of the command to enable debugging messages during the setup, and -d can be removed to keep the container in the foreground (which allows killing it with Ctrl+C).

   If the configuration file reads environment variables (such as API keys), they can be defined after docker run:

   docker run -e FIRSTVAR=firstvalue -e SECONDVAR=secondvalue --rm [...]

6. While the server is running, send a test request (replace change the port and the name of the answer provider accordingly):

   curl -X POST http://localhost:42774/send -H "Content-Type: application/json" -d '{"query": "this is a test", "answer_provider": "ECHO"}'
   

   It should print a JSON response. In the example above, one of the lines will be "text": "THIS IS A TEST" (i.e. the query converted to uppercase by the answer provider ECHO).