cld2labs/ubuntu22.04-deployment-scripts#49
cld2labs/ubuntu22.04-deployment-scripts#49HarikaDev296 wants to merge 21 commits intoopea-project:mainfrom
Conversation
amberjain1
left a comment
amberjain1
left a comment
There was a problem hiding this comment.
Reviewed the general structure.
Will wait for validation to complete before reviewing the iac folder.
third_party/Dell/ubuntu-22.04/EI/single-node/user-guide-apisix.md
Outdated
Show resolved
Hide resolved
There was a problem hiding this comment.
Can we only capture specific instructions that differ as compared to the main readme guides. For generic steps shall we start pointing to common setup and user-guides so that we do not duplicate documentation?
There was a problem hiding this comment.
This documentation is done as per Dell's guidance
There was a problem hiding this comment.
Shall we create a separate common document (or point to the one that is already present in docs folder) and only capture delta's here?
There was a problem hiding this comment.
I also agree. There are a lot of duplicate instructions. I think the changes are minimal such that we can have just one README for setting up both apisix and genai gateway. The testing inference portions seems more elaborate, so those can be split into two.
There was a problem hiding this comment.
This documentation is done as per Dell's guidance
|
@bagd0003 please add your review comments here |
third_party/Dell/ubuntu-22.04/EI/single-node/user-guide-genai.md
Outdated
Show resolved
Hide resolved
third_party/Dell/ubuntu-22.04/EI/single-node/user-guide-genai.md
Outdated
Show resolved
Hide resolved
third_party/Dell/ubuntu-22.04/EI/single-node/user-guide-genai.md
Outdated
Show resolved
Hide resolved
Co-authored-by: alexsin368 <109180236+alexsin368@users.noreply.github.com>
third_party/Dell/ubuntu-22.04/EI/single-node/user-guide-genai.md
Outdated
Show resolved
Hide resolved
third_party/Dell/ubuntu-22.04/EI/single-node/user-guide-genai.md
Outdated
Show resolved
Hide resolved
third_party/Dell/ubuntu-22.04/EI/single-node/user-guide-genai.md
Outdated
Show resolved
Hide resolved
| ``` | ||
| **Mount ISO** | ||
| ```bash | ||
| ./mount-iso.sh |
There was a problem hiding this comment.
I had to change all .sh scripts to be executable before I could run them.
chmod 755 ./*.sh
Either add this step or change the permissions of the .sh files before commiting them.
There was a problem hiding this comment.
Updated readme to run chmod +x mount.sh before executing the script
There was a problem hiding this comment.
@bagd0003 Heerak mentioned you had concerns in the way that the ISO is being downloaded from the public ubuntu url, followed by installation via terraform. Please recommend if there is a better way Dell prefers before we upstream this code
There was a problem hiding this comment.
is this related to ISO mount, already updated the script so that it can pick http and https?
| -p Linux123! \ | ||
| -t hf_xxxxxxxxxxxxx \ | ||
| -g gaudi3 \ | ||
| -m "1" |
There was a problem hiding this comment.
Does the user have to input a model ID or can the user select from the list of models in the menu? Better to make it flexible for the user to choose the desired model to deploy.
Updated user guide for Intel AI for Enterprise Inference to clarify setup instructions and improve readability.
|
|
||
| ```bash | ||
| cd ~ | ||
| git clone https://github.com/opea-project/Enterprise-Inference.git |
There was a problem hiding this comment.
All these deployment scripts are inside the Enterprise-Inference repo already. Can we avoid re-downloading it?
Updated README.md to include new instructions for mounting ISO and clarify mandatory Terraform variables.
Added instructions for changing file permissions and running the script.
third_party/Dell/ubuntu-22.04/EI/single-node/troubleshooting.md
Outdated
Show resolved
Hide resolved
third_party/Dell/ubuntu-22.04/EI/single-node/user-guide-apisix.md
Outdated
Show resolved
Hide resolved
third_party/Dell/ubuntu-22.04/EI/single-node/user-guide-apisix.md
Outdated
Show resolved
Hide resolved
third_party/Dell/ubuntu-22.04/EI/single-node/user-guide-genai.md
Outdated
Show resolved
Hide resolved
|
|
||
| ```bash | ||
| wget -O deploy-enterprise-inference.sh \ | ||
| https://raw.githubusercontent.com/cld2labs/Enterprise-Inference/refs/heads/cld2labs/ubuntu22.04-deployment-scripts/third_party/Dell/ubuntu-22.04/iac/deploy-enterprise-inference.sh |
There was a problem hiding this comment.
Isnt cloning a better option? so its less hardcoded for better maintenance?
| -m "1" \ | ||
| -r | ||
| ``` | ||
|
|
There was a problem hiding this comment.
The following info should be included here:
- the model ID numbers corresponding to the model names in the EI pre-validated models list
- how to put in a model from HF (for Gaudi, will also need to specify TP size)
- how to install another model or uninstall an existing model without tearing down the kubernetes deployment
| MODELS="Enter Model number" | ||
| DEPLOYMENT_MODE="keycloak" | ||
| DEPLOY_OBSERVABILITY="off" | ||
| KEYCLOAK_CLIENT_ID="api" |
There was a problem hiding this comment.
change the default values for keycloak to match these values: https://github.com/opea-project/Enterprise-Inference/blob/main/core/inventory/inference-config.cfg
keycloak_client_id=my-client-id
keycloak_admin_user=your-keycloak-admin-user
keycloak_admin_password=changeme
please update these values anywhere else in the script or other files
4 participants
Adding ubuntu deployment scripts under third-party dell folder.