Here to show my experience about playing with ROCm with runable code, step-by-step tutorial to help you reproduce what I have did. If you have iGPU or dGPU of AMD, you may try Machine Learning with them.
NOTICE : For more easier tracking my update, I use 🆕 and 🔥 to flag the new hot topics.
- Deploy Deepseek-R1 in one GPU -AMD Instinct™ MI300X 🔥
- Deploy Llama 3.2 Vision quickly on AMD ROCm with Ollama
- Deploy vLLM service with Kubernetes over AMD ROCm GPU
- Deploy LLM with Radeon iGPU 780M
- Examples of using vLLM with ROCm 🆕 🔥
- Help scripts to fast use vLLM with ROCm
- Example: using docker compose to run multiple containers of vllm serve. e.g. gpu=0,1 for container-1 and gpu=6,7 for container-2.
- vLLM
- Neural Magic vLLM, nm-vllm
- AIBrix
- KubeAI : AI Inferencing Operator
- vLLM Production Stack
- RAG_LLM_QnA_Assistant, Step-by-step tutorial repo project to setup RAG Apps with ROCm
- Ask4ROCm_Chatbot, An chatbot app drive by RAG solution.
- LLM_Voice_Assistant , Use STT/TTS model from Picovoice.
- Easy-Wav2Lip-ROCm, Easy run Wav2Lip with ROCm over AMD GPU. Way2Lip is a project of Generalized Lip Sync Models
- Run EchoMimic with ROCm EchoMimic: Lifelike Audio-Driven Portrait Animations through Editable Landmark Conditioning
- Run LLama-3.2-vision with ROCm Ollama+Llama-3.2-vision+ROCm
- Deploy vLLM service with Kubernetes over AMD ROCm GPU , Turoial with sample codes.
- Play Qwen2.5–Omni with AMD GPU 🆕 🔥
These projects may not offical announce to support ROCm GPU. But they work fine base on my verification.
- Tutorial: vLLM deploy
- Summary: Awesome-Agent-Framework - https://github.com/AgentSpaceAI/Awesome-Agent-Framework - https://github.com/kyrolabs/awesome-agents
@misc{ Playing with ROCm,
author = {He Ye (Alex)},
title = {Playing with ROCm: share my experience and practice},
howpublished = {\url{https://alexhegit.github.io/}},
year = {2024--}
}