Skip to content

@ModelCloud ModelCloud.ai

Change the repository type filter

All

    Repositories list

    12 repositories

    • GPTQModel

      Public
      Production ready LLM model compression/quantization toolkit with hw accelerated inference support for both cpu/gpu via HF, vLLM, and SGLang.
      transformersquantizationoptimumpeftvllmgptqsglang
      Python
      •
      Apache License 2.0
      •87•589•31•12•Updated May 29, 2025May 29, 2025

    • lm-evaluation-harness

      Public
      A framework for few-shot evaluation of language models.
      Python
      •
      MIT License
      •2.4k•0•0•0•Updated Apr 17, 2025Apr 17, 2025

    • LogBar

      Public
      A unified Logger and ProgressBar util with zero dependencies.
      Python
      •
      Apache License 2.0
      •0•4•1•0•Updated Apr 1, 2025Apr 1, 2025

    • vllm

      Public
      A high-throughput and memory-efficient inference and serving engine for LLMs
      Python
      •
      Apache License 2.0
      •7.7k•0•0•0•Updated Mar 27, 2025Mar 27, 2025

    • rockthem

      Public
      Cuda
      •
      Apache License 2.0
      •0•0•0•0•Updated Mar 13, 2025Mar 13, 2025

    • Tokenicer

      Public
      A (nicer) tokenizer you want to use for model inference and training: with all known peventable gotchas normalized or auto-fixed.
      trainingtokenizerinferencetoken
      Python
      •
      Apache License 2.0
      •2•8•0•1•Updated Mar 12, 2025Mar 12, 2025

    • platinum-benchmarks

      Public
      Python
      •
      Creative Commons Attribution 4.0 International
      •1•0•0•0•Updated Mar 6, 2025Mar 6, 2025

    • peft

      Public
      🤗 PEFT: State-of-the-art Parameter-Efficient Fine-Tuning.
      Python
      •
      Apache License 2.0
      •1.9k•0•0•0•Updated Mar 4, 2025Mar 4, 2025

    • sglang

      Public
      SGLang is a fast serving framework for large language models and vision language models.
      Python
      •
      Apache License 2.0
      •1.9k•0•0•0•Updated Mar 4, 2025Mar 4, 2025

    • Device-SMI

      Public
      Self-contained Python lib with zero-dependencies that give you a unified device properties for gpu, cpu, and npu. No more calling separate tools such as nvidia-smi or /proc/cpuinfo and parsing it yourself.
      devicecpugpusminpuxpu
      Python
      •
      Apache License 2.0
      •1•11•2•1•Updated Mar 1, 2025Mar 1, 2025

    • transformers

      Public
      🤗 Transformers: State-of-the-art Machine Learning for Pytorch, TensorFlow, and JAX.
      Python
      •
      Apache License 2.0
      •29k•0•0•0•Updated Feb 12, 2025Feb 12, 2025

    • optimum

      Public
      🚀 Accelerate inference and training of 🤗 Transformers, Diffusers, TIMM and Sentence Transformers with easy to use hardware optimization tools
      Python
      •
      Apache License 2.0
      •541•1•0•0•Updated Feb 7, 2025Feb 7, 2025