@@ -88,20 +88,18 @@ We will backport bug fixes to AutoGPTQ on a case-by-case basis.
88
88
89
89
## Model Support ( 🚀 GPTQModel only )
90
90
[ Ready to deply quantized models] ( https://hf.co/ModelCloud )
91
-
92
- | Model | | | | | | | | |
93
- | ---------------- | --- | --------------------- | --- | --------- | --- | ---------- | --- | --- |
94
- | Baichuan | ✅ | EXAONE 3.0 | 🚀 | Llama 1/2/3 | ✅ | Phi/Phi-3 | 🚀 | |
95
- | Bloom | ✅ | Falon | ✅ | Llama 3.2 Vision | 🚀 | Qwen | ✅ | |
96
- | ChatGLM | 🚀 | Gemma 2 | 🚀 | LongLLaMA | ✅ | Qwen2MoE | 🚀 | |
97
- | CodeGen | ✅ | GPTBigCod | ✅ | MiniCPM3 | 🚀 | RefinedWeb | ✅ | |
98
- | Cohere | ✅ | GPTNeoX | ✅ | Mistral | ✅ | StableLM | ✅ | |
99
- | DBRX Converted | 🚀 | GPT-2 | ✅ | Mixtral | ✅ | StarCoder2 | ✅ | |
100
- | Deci | ✅ | GPT-J | ✅ | MOSS | ✅ | XVERSE | ✅ | |
101
- | DeepSeek-V2 | 🚀 | GRIN-MoE | 🚀 | MPT | ✅ | Yi | ✅ | |
102
- | DeepSeek-V2-Lite | 🚀 | InternLM 1/2.5 | 🚀 | OPT | ✅ | | | |
103
-
104
91
92
+ | Model | | | | | | | | | |
93
+ | ---------------- | --- | ------- | --- | ---------------- | --- | ---------- | --- | --- | --- |
94
+ | Baichuan | ✅ | EXAONE 3.0 | 🚀 | InternLM 1/2.5 | 🚀 | OPT | ✅ | | |
95
+ | Bloom | ✅ | Falon | ✅ | Llama 1/2/3 | ✅ | Phi/Phi-3 | 🚀 | | |
96
+ | ChatGLM | 🚀 | Gemma 2 | 🚀 | Llama 3.2 Vision | 🚀 | Qwen | ✅ | | |
97
+ | CodeGen | ✅ | GPTBigCod | ✅ | LongLLaMA | ✅ | Qwen2MoE | 🚀 | | |
98
+ | Cohere | ✅ | GPTNeoX | ✅ | MiniCPM3 | 🚀 | RefinedWeb | ✅ | | |
99
+ | DBRX Converted | 🚀 | GPT-2 | ✅ | Mistral | ✅ | StableLM | ✅ | | |
100
+ | Deci | ✅ | GPT-J | ✅ | Mixtral | ✅ | StarCoder2 | ✅ | | |
101
+ | DeepSeek-V2 | 🚀 | Granite | 🚀 | MOSS | ✅ | XVERSE | ✅ | | |
102
+ | DeepSeek-V2-Lite | 🚀 | GRIN-MoE | 🚀 | MPT | ✅ | Yi | ✅ | | |
105
103
## Compatiblity
106
104
107
105
We aim for 100% compatibility with models quanted by AutoGPTQ <= 0.7.1 and will consider syncing future compatibilty on a case-by-case basis.
0 commit comments