Skip to content

Commit a0244e3

Browse files
Bloodis94localai-botmudlerrichiejp
authored
feat(install): added complete process for installing nvidia drivers on fedora without pulling X11 (#5246)
* Update installation script for improved compatibility and clarity - Renamed VERSION to LOCALAI_VERSION to avoid conflicts with system variables. - Enhanced NVIDIA and CUDA repository installation for DNF5 compatibility. - Adjusted default Fedora version handling for CUDA installation. - Updated Docker image tag handling to use LOCALAI_VERSION consistently. - Improved logging messages for repository and LocalAI binary downloads. - Added a temporary bypass for nvidia-smi installation on Fedora Cloud Edition. * Enhance log functions with ANSI color formatting - Added ANSI escape codes for improved log styling: light blue for info, orange for warnings, and red for errors. - Updated all log functions (`info`, `warn`, `fatal`) to include bold and colored output. Signed-off-by: Alessandro Pirastru <[email protected]> * feat: Enhance log functions with ANSI color formatting - Added ANSI escape codes for improved log styling: light blue for info, orange for warnings, and red for errors. - Updated all log functions (`info`, `warn`, `fatal`) to include bold and colored output. Signed-off-by: Alessandro Pirastru <[email protected]> * chore: ⬆️ Update ggml-org/llama.cpp to `ecda2ec4b347031a9b8a89ee2efc664ce63f599c` (#5238) :arrow_up: Update ggml-org/llama.cpp Signed-off-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com> Co-authored-by: mudler <[email protected]> * fix(stablediffusion-ggml): Build with DSD CUDA, HIP and Metal flags (#5236) Signed-off-by: Richard Palethorpe <[email protected]> * feat(install): enhance script with choice functions and logs - Added custom `choice_info`, `choice_warn`, and `choice_fatal` functions for interactive input logging. - Adjusted Docker volume creation message for better clarity. - Included NVIDIA driver check log for improved feedback to users. - Added consistent logging before starting LocalAI Docker containers across configurations. Signed-off-by: Alessandro Pirastru <[email protected]> * feat(install): add Fedora NVIDIA driver installation option - Introduced a new function to install NVIDIA kernel drivers on Fedora using akmod packages. - Added user prompt to choose between installing drivers automatically or exiting for manual setup. - Integrated the new function into the existing Fedora-specific CUDA toolkit installation workflow. Signed-off-by: Alessandro Pirastru <[email protected]> * fix(install): correct repository ID for DNF5 configuration - Update repository ID from 'nome-repo' to 'nvidia-cuda' for DNF5. - Ensures the correct repository name matches expected configuration. - Fix prevents potential misconfiguration during installation process. Signed-off-by: Alessandro Pirastru <[email protected]> * feat(install): enhance NVIDIA driver handling on Fedora - fixed `install_cuda_driver_yum` function call in `install_fedora_nvidia_kernel_drivers` - Added `cuda-toolkit` for Fedora installations, as recommended by RPM Fusion. - Adjusted driver repository commands for compatibility with DNF5. - Improved URL and version handling for package manager installations. Signed-off-by: Alessandro Pirastru <[email protected]> * Refactor NVIDIA driver installation process in install.sh - Removed redundant empty lines for cleaner formatting. - Standardized URL formatting by removing unnecessary quotes around URLs. - Reverted logic by removing Fedora-specific exclusions for cuda-toolkit and using `cuda-drivers` universally. - Refined repository addition for `dnf` by explicitly setting `id` and `name` parameters for clarity and accuracy. - Fixed minor formatting inconsistencies in parameter passing. Signed-off-by: Alessandro Pirastru <[email protected]> * feat: Update NVIDIA module installation warning in install script - Clarified that Akmod installation may inhibit the reboot command. - Added a cautionary note to the warning to inform users of potential risks. Signed-off-by: Alessandro Pirastru <[email protected]> * Update NVIDIA driver installation warning message - Clarify prerequisites by noting the need for rpmfusion free/nonfree repos. - Improve formatting of the warning box for better readability. - Inform users that the script will install missing repos if necessary. Signed-off-by: Alessandro Pirastru <[email protected]> --------- Signed-off-by: Alessandro Pirastru <[email protected]> Signed-off-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com> Signed-off-by: Richard Palethorpe <[email protected]> Co-authored-by: LocalAI [bot] <[email protected]> Co-authored-by: mudler <[email protected]> Co-authored-by: Richard Palethorpe <[email protected]>
1 parent d663962 commit a0244e3

File tree

1 file changed

+95
-6
lines changed

1 file changed

+95
-6
lines changed

‎docs/static/install.sh

100644100755
Lines changed: 95 additions & 6 deletions
Original file line numberDiff line numberDiff line change
@@ -31,19 +31,44 @@ set -o noglob
3131
#set -x
3232

3333
# --- helper functions for logs ---
34+
# ANSI escape codes
35+
LIGHT_BLUE='\033[38;5;117m'
36+
ORANGE='\033[38;5;214m'
37+
RED='\033[38;5;196m'
38+
BOLD='\033[1m'
39+
RESET='\033[0m'
40+
3441
info()
3542
{
36-
echo ' ' "$@"
43+
echo -e "${BOLD}${LIGHT_BLUE}" '[INFO] ' "$@" "${RESET}"
3744
}
3845

3946
warn()
4047
{
41-
echo '[WARN] ' "$@" >&2
48+
echo -e "${BOLD}${ORANGE}" '[WARN] ' "$@" "${RESET}" >&2
4249
}
4350

4451
fatal()
4552
{
46-
echo '[ERROR] ' "$@" >&2
53+
echo -e "${BOLD}${RED}" '[ERROR] ' "$@" "${RESET}" >&2
54+
exit 1
55+
}
56+
57+
# --- custom choice functions ---
58+
# like the logging functions, but with the -n flag to prevent the new line and keep the cursor in line for choices inputs like y/n
59+
choice_info()
60+
{
61+
echo -e -n "${BOLD}${LIGHT_BLUE}" '[INFO] ' "$@" "${RESET}"
62+
}
63+
64+
choice_warn()
65+
{
66+
echo -e -n "${BOLD}${ORANGE}" '[WARN] ' "$@" "${RESET}" >&2
67+
}
68+
69+
choice_fatal()
70+
{
71+
echo -e -n "${BOLD}${RED}" '[ERROR] ' "$@" "${RESET}" >&2
4772
exit 1
4873
}
4974

@@ -342,7 +367,7 @@ install_cuda_driver_yum() {
342367
DNF_VERSION=$($PACKAGE_MANAGER --version | grep -oE '[0-9]+\.[0-9]+\.[0-9]+' | head -n1 | cut -d. -f1)
343368
if [ "$DNF_VERSION" -ge 5 ]; then
344369
# DNF5: Use 'addrepo' to add the repository
345-
$SUDO $PACKAGE_MANAGER config-manager addrepo --id=nome-repo --set=name="nvidia-cuda" --set=baseurl="https://developer.download.nvidia.com/compute/cuda/repos/$1$2/$(uname -m)/cuda-$1$2.repo"
370+
$SUDO $PACKAGE_MANAGER config-manager addrepo --id=nvidia-cuda --set=name="nvidia-cuda" --set=baseurl="https://developer.download.nvidia.com/compute/cuda/repos/$1$2/$(uname -m)/cuda-$1$2.repo"
346371
else
347372
# DNF4: Use '--add-repo' to add the repository
348373
$SUDO $PACKAGE_MANAGER config-manager --add-repo https://developer.download.nvidia.com/compute/cuda/repos/$1$2/$(uname -m)/cuda-$1$2.repo
@@ -367,6 +392,64 @@ install_cuda_driver_yum() {
367392
$SUDO $PACKAGE_MANAGER -y install cuda-drivers
368393
}
369394

395+
install_fedora_nvidia_kernel_drivers(){
396+
397+
#We want to give the user the choice to install the akmod kernel drivers or not, since it could break some setups
398+
warn "+------------------------------------------------------------------------------------------------+"
399+
warn "| WARNING: |"
400+
warn "| Looks like the NVIDIA Kernel modules are not installed. |"
401+
warn "| |"
402+
warn "| This script can try to install them using akmod-nvidia. |"
403+
warn "| - The script need the rpmfusion free and nonfree repos and will install them if not available. |"
404+
warn "| - The akmod installation can sometimes inhibit the reboot command. |"
405+
warn "| |"
406+
warn "| Otherwise you can exit the install script and install them yourself. |"
407+
warn "| NOTE: you will need to reboot after the installation. |"
408+
warn "+------------------------------------------------------------------------------------------------+"
409+
410+
while true; do
411+
choice_warn "Do you wish for the script to try and install them? (akmod/exit) ";
412+
read Answer
413+
414+
if [ "$Answer" = "akmod" ]; then
415+
416+
DNF_VERSION=$($PACKAGE_MANAGER --version | grep -oE '[0-9]+\.[0-9]+\.[0-9]+' | head -n1 | cut -d. -f1)
417+
418+
OS_NAME=$ID
419+
OS_VERSION=$VERSION_ID
420+
FREE_URL="https://mirrors.rpmfusion.org/free/fedora/rpmfusion-free-release-${OS_VERSION}.noarch.rpm"
421+
NONFREE_URL="https://mirrors.rpmfusion.org/nonfree/fedora/rpmfusion-nonfree-release-${OS_VERSION}.noarch.rpm"
422+
423+
curl -LO "$FREE_URL"
424+
curl -LO "$NONFREE_URL"
425+
426+
if [ "$DNF_VERSION" -ge 5 ]; then
427+
# DNF5:
428+
$SUDO $PACKAGE_MANAGER install -y "rpmfusion-nonfree-release-$(rpm -E %fedora).noarch.rpm" "rpmfusion-nonfree-release-$(rpm -E %fedora).noarch.rpm"
429+
$SUDO $PACKAGE_MANAGER install -y akmod-nvidia
430+
else
431+
# DNF4:
432+
$SUDO $PACKAGE_MANAGER install -y "rpmfusion-nonfree-release-$(rpm -E %fedora).noarch.rpm" "rpmfusion-nonfree-release-$(rpm -E %fedora).noarch.rpm"
433+
$SUDO $PACKAGE_MANAGER install -y akmod-nvidia
434+
fi
435+
436+
$SUDO rm "rpmfusion-free-release-$(rpm -E %fedora).noarch.rpm"
437+
$SUDO rm "rpmfusion-nonfree-release-$(rpm -E %fedora).noarch.rpm"
438+
439+
install_cuda_driver_yum $OS_NAME '41'
440+
441+
info "Nvidia driver installation complete, please reboot now and run the Install script again to complete the setup."
442+
exit
443+
444+
elif [ "$Answer" = "exit" ]; then
445+
446+
aborted
447+
else
448+
warn "Invalid choice. Please enter 'akmod' or 'exit'."
449+
fi
450+
done
451+
}
452+
370453
# ref: https://docs.nvidia.com/cuda/cuda-installation-guide-linux/index.html#ubuntu
371454
# ref: https://docs.nvidia.com/cuda/cuda-installation-guide-linux/index.html#debian
372455
install_cuda_driver_apt() {
@@ -485,7 +568,7 @@ install_docker() {
485568
$SUDO systemctl start docker
486569
fi
487570

488-
info "Starting LocalAI Docker container..."
571+
info "Creating LocalAI Docker volume..."
489572
# Create volume if doesn't exist already
490573
if ! $SUDO docker volume inspect local-ai-data > /dev/null 2>&1; then
491574
$SUDO docker volume create local-ai-data
@@ -526,16 +609,18 @@ install_docker() {
526609
IMAGE_TAG=${LOCALAI_VERSION}-aio-gpu-nvidia-cuda-12
527610
fi
528611

612+
info "Checking Nvidia Kernel Drivers presence..."
529613
if ! available nvidia-smi; then
530-
#TODO Temporary Bypass for Fedora Headless (Cloud Edition), need to find a way to install nvidia-smi without pulling x11
531614
OS_NAME=$ID
532615
OS_VERSION=$VERSION_ID
533616

534617
case $OS_NAME in
535618
debian|ubuntu) $SUDO apt-get -y install nvidia-cuda-toolkit;;
619+
fedora) install_fedora_nvidia_kernel_drivers;;
536620
esac
537621
fi
538622

623+
info "Starting LocalAI Docker container..."
539624
$SUDO docker run -v local-ai-data:/build/models \
540625
--gpus all \
541626
--restart=always \
@@ -554,6 +639,7 @@ install_docker() {
554639
IMAGE_TAG=${LOCALAI_VERSION}-aio-gpu-hipblas
555640
fi
556641

642+
info "Starting LocalAI Docker container..."
557643
$SUDO docker run -v local-ai-data:/build/models \
558644
--device /dev/dri \
559645
--device /dev/kfd \
@@ -573,6 +659,7 @@ install_docker() {
573659
IMAGE_TAG=${LOCALAI_VERSION}-aio-gpu-intel-f32
574660
fi
575661

662+
info "Starting LocalAI Docker container..."
576663
$SUDO docker run -v local-ai-data:/build/models \
577664
--device /dev/dri \
578665
--restart=always \
@@ -590,6 +677,8 @@ install_docker() {
590677
if [ "$USE_AIO" = true ]; then
591678
IMAGE_TAG=${LOCALAI_VERSION}-aio-cpu
592679
fi
680+
681+
info "Starting LocalAI Docker container..."
593682
$SUDO docker run -v local-ai-data:/models \
594683
--restart=always \
595684
-e MODELS_PATH=/models \

0 commit comments

Comments
 (0)