“Package‐style” Dependency Management for Models (npm/Pip‑style)

[JoeGaffney]

Hey, this may have been discussed before or diffusers might not be the correct place to discuss. (let me know) I am coming from the point of view of purely being a consumer of models and noticing the large amount of potential copies and re-downloads.

🚀 Concept

Many Hugging Face model variants—whether fine‑tunes, adapters, or language‑specific forks—share large portions of their weights with a common base checkpoint. Similarly, tasks like diffusion often reuse the same T5 text encoder across multiple pipelines. Yet today, every variant or pipeline bundles the full encoder + decoder (or diffusion U‑Net), leading to redundant uploads and downloads.

A lightweight “package manager” layer would let you:

• Declare one or more bases (e.g.
  • Core transformer: [email protected]
  • Diffusion U‑Net: [email protected])
• Upload only the components you’ve changed (e.g. a new decoder head or adapter weights)
• Download smartly, pulling shared pieces from bases you already have in cache

---

🎯 Why It Matters

• Bandwidth & Storage: CI pipelines, Docker builds, and on‑prem deployments fetch only the modified components—be they large (text encoder) or small (classification head)—instead of full checkpoints every time.
• Cache Harmony: Local caches (~/.cache/huggingface) track components by content hash, preventing duplicate copies of popular encoders or U‑Nets.
• Provenance & Security: Variants reference signed base packages; you audit and trust the core model separately from custom deltas.
• Reproducibility: A lockfile (e.g. hf-lock.json) pins exact versions of all referenced packages and deltas, similar to package-lock.json or Pipfile.lock.

---

🤝 Analogy to npm / pip

• npm: projects list dependencies in package.json and fetch only missing modules.
• pip: requirements.txt or Pipfile pins exact versions; wheels avoid rebuilding the same code.
• Our ask: let model repos declare dependencies on other model “packages,” so from_pretrained("my-diffusion-pipeline") behaves like npm install—reusing existing parts and pulling only what’s new.

[yiyixuxu]

so we are going to have a ModularLoader that I think should support what you're asking
#11462

[JoeGaffney]

Hey, this looks promising! There’s a lot to unpack, and it’s still a bit unclear to me whether it actually saves on downloading shared weights or mainly helps manage what’s already been downloaded locally.

With this, is it possible for people to create models and model cards that explicitly reference other published models, so downloads can be shared and duplicated weights avoided? Potentially even a model card that doesn’t contain any weights itself but just points to others in an ensemble.

That’s actually what I’m most interested in right now.