Skip to content

[Examples] Renamed some example groups for better extensibility #2641

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 2 commits into from
May 15, 2025
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
3 changes: 2 additions & 1 deletion docs/assets/stylesheets/extra.css
Original file line number Diff line number Diff line change
Expand Up @@ -790,6 +790,7 @@ body {
display: flex;
}

/* Comment to siwtch to sections in sidebar */
.md-sidebar__inner > .md-nav--primary > .md-nav__list:not(.md-post__meta) > .md-nav__item > .md-nav > .md-nav__list > .md-nav__item:not(.md-nav__item--section) {
display: none;
}
Expand Down Expand Up @@ -861,7 +862,7 @@ body {
}

.md-sidebar--primary .md-nav__link, .md-sidebar--post .md-nav__link {
padding: 5px 15px 5px;
padding: 5px 15px 4px;
margin-top: 0;
}

Expand Down
2 changes: 1 addition & 1 deletion docs/blog/posts/amd-on-runpod.md
Original file line number Diff line number Diff line change
Expand Up @@ -42,7 +42,7 @@ you can now specify an AMD GPU under `resources`. Below are a few examples.
Here's an example of a [service](../../docs/concepts/services.md) that deploys
Llama 3.1 70B in FP16 using [TGI :material-arrow-top-right-thin:{ .external }](https://huggingface.co/docs/text-generation-inference/en/installation_amd){:target="_blank"}.

<div editor-title="examples/deployment/tgi/amd/service.dstack.yml">
<div editor-title="examples/inference/tgi/amd/service.dstack.yml">

```yaml
type: service
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -110,7 +110,7 @@ efficient manner.

### NVIDIA

NVIDIA remains the top accelerator supported by `dstack`. Recently, we introduced a [NIM example](../../examples/deployment/nim/index.md)
NVIDIA remains the top accelerator supported by `dstack`. Recently, we introduced a [NIM example](../../examples/inference/nim/index.md)
for model deployment, and we continue to enhance support for the rest of NVIDIA's ecosystem.

### AMD
Expand Down
2 changes: 1 addition & 1 deletion docs/blog/posts/dstack-metrics.md
Original file line number Diff line number Diff line change
Expand Up @@ -7,7 +7,7 @@ image: https://dstack.ai/static-assets/static-assets/images/dstack-stats-v2.png
categories:
- AMD
- NVIDIA
- Monitoring
- Metrics
---

# Monitoring essential GPU metrics via CLI
Expand Down
4 changes: 2 additions & 2 deletions docs/blog/posts/metrics-ui.md
Original file line number Diff line number Diff line change
Expand Up @@ -5,7 +5,7 @@ description: "TBA"
slug: metrics-ui
image: https://dstack.ai/static-assets/static-assets/images/dstack-metrics-ui-v3-min.png
categories:
- Monitoring
- Metrics
- AMD
- NVIDIA
---
Expand Down Expand Up @@ -55,6 +55,6 @@ For persistent storage and long-term access to metrics, we still recommend setti
metrics from `dstack`.

!!! info "What's next?"
1. See [Monitoring](../../docs/guides/monitoring.md)
1. See [Metrics](../../docs/guides/metrics.md)
2. Check [dev environments](../../docs/concepts/dev-environments.md), [tasks](../../docs/concepts/tasks.md), [services](../../docs/concepts/services.md), and [fleets](../../docs/concepts/fleets.md)
3. Join [Discord :material-arrow-top-right-thin:{ .external }](https://discord.gg/u8SmfwPpMd){:target="_blank"}
2 changes: 1 addition & 1 deletion docs/blog/posts/mpi.md
Original file line number Diff line number Diff line change
Expand Up @@ -101,5 +101,5 @@ as well as use MPI for other tasks.

!!! info "What's next?"
1. Learn more about [dev environments](../../docs/concepts/dev-environments.md), [tasks](../../docs/concepts/tasks.md), [services](../../docs/concepts/services.md), and [fleets](../../docs/concepts/fleets.md)
2. Check the [NCCL tests](../../examples/distributed-training/nccl-tests/index.md) example
2. Check the [NCCL tests](../../examples/clusters/nccl-tests/index.md) example
2. Join [Discord :material-arrow-top-right-thin:{ .external }](https://discord.gg/u8SmfwPpMd){:target="_blank"}
6 changes: 3 additions & 3 deletions docs/blog/posts/prometheus.md
Original file line number Diff line number Diff line change
Expand Up @@ -5,7 +5,7 @@ description: "TBA"
slug: prometheus
image: https://dstack.ai/static-assets/static-assets/images/dstack-prometheus-v3.png
categories:
- Monitoring
- Metrics
- NVIDIA
---

Expand Down Expand Up @@ -46,7 +46,7 @@ Overall, `dstack` collects three groups of metrics:
| **Runs** | Run metrics include run counters for each user in each project. |
| **Jobs** | A run consists of one or more jobs, each mapped to a container. Job metrics offer insights into execution time, cost, GPU model, NVIDIA DCGM telemetry, and more. |

For a full list of available metrics and labels, check out [Monitoring](../../docs/guides/monitoring.md).
For a full list of available metrics and labels, check out [Metrics](../../docs/guides/metrics.md).

??? info "NVIDIA"
NVIDIA DCGM metrics are automatically collected for `aws`, `azure`, `gcp`, and `oci` backends,
Expand All @@ -60,7 +60,7 @@ For a full list of available metrics and labels, check out [Monitoring](../../do
only accessible through the UI and the [`dstack metrics`](dstack-metrics.md) CLI.

!!! info "What's next?"
1. See [Metrics](../../docs/guides/monitoring.md)
1. See [Metrics](../../docs/guides/metrics.md)
1. Check [dev environments](../../docs/concepts/dev-environments.md),
[tasks](../../docs/concepts/tasks.md), [services](../../docs/concepts/services.md),
and [fleets](../../docs/concepts/fleets.md)
Expand Down
4 changes: 2 additions & 2 deletions docs/blog/posts/tpu-on-gcp.md
Original file line number Diff line number Diff line change
Expand Up @@ -50,7 +50,7 @@ and [vLLM :material-arrow-top-right-thin:{ .external }](https://github.com/vllm-

=== "Optimum TPU"

<div editor-title="examples/deployment/optimum-tpu/service.dstack.yml">
<div editor-title="examples/inference/optimum-tpu/service.dstack.yml">

```yaml
type: service
Expand Down Expand Up @@ -83,7 +83,7 @@ and [vLLM :material-arrow-top-right-thin:{ .external }](https://github.com/vllm-
the official Docker image can be used instead of `dstackai/optimum-tpu:llama31`.

=== "vLLM"
<div editor-title="examples/deployment/vllm/service-tpu.dstack.yml">
<div editor-title="examples/inference/vllm/service-tpu.dstack.yml">

```yaml
type: service
Expand Down
1 change: 0 additions & 1 deletion docs/changelog/index.md

This file was deleted.

2 changes: 1 addition & 1 deletion docs/docs/concepts/gateways.md
Original file line number Diff line number Diff line change
Expand Up @@ -71,7 +71,7 @@ To delete a gateway, pass the gateway configuration to [`dstack delete`](../refe
<div class="termy">

```shell
$ dstack delete -f examples/deployment/gateway.dstack.yml
$ dstack delete -f examples/inference/gateway.dstack.yml
```

</div>
Expand Down
6 changes: 3 additions & 3 deletions docs/docs/concepts/services.md
Original file line number Diff line number Diff line change
Expand Up @@ -558,6 +558,6 @@ If one replica of a multi-replica service fails with retry enabled,
1. Read about [dev environments](dev-environments.md), [tasks](tasks.md), and [repos](repos.md)
2. Learn how to manage [fleets](fleets.md)
3. See how to set up [gateways](gateways.md)
4. Check the [TGI :material-arrow-top-right-thin:{ .external }](../../examples/deployment/tgi/index.md){:target="_blank"},
[vLLM :material-arrow-top-right-thin:{ .external }](../../examples/deployment/vllm/index.md){:target="_blank"}, and
[NIM :material-arrow-top-right-thin:{ .external }](../../examples/deployment/nim/index.md){:target="_blank"} examples
4. Check the [TGI :material-arrow-top-right-thin:{ .external }](../../examples/inference/tgi/index.md){:target="_blank"},
[vLLM :material-arrow-top-right-thin:{ .external }](../../examples/inference/vllm/index.md){:target="_blank"}, and
[NIM :material-arrow-top-right-thin:{ .external }](../../examples/inference/nim/index.md){:target="_blank"} examples
File renamed without changes.
22 changes: 11 additions & 11 deletions docs/examples.md
Original file line number Diff line number Diff line change
Expand Up @@ -38,10 +38,10 @@ hide:
</a>
</div>

## Distributed training
## Clusters

<div class="tx-landing__highlights_grid">
<a href="/examples/distributed-training/nccl-tests"
<a href="/examples/clusters/nccl-tests"
class="feature-cell sky">
<h3>
NCCL tests
Expand All @@ -51,7 +51,7 @@ hide:
Run multi-node NCCL tests with MPI
</p>
</a>
<a href="/examples/distributed-training/rccl-tests"
<a href="/examples/clusters/rccl-tests"
class="feature-cell sky">
<h3>
RCCL tests
Expand All @@ -61,7 +61,7 @@ hide:
Run multi-node RCCL tests with MPI
</p>
</a>
<a href="/examples/distributed-training/a3mega-clusters"
<a href="/examples/clusters/a3mega"
class="feature-cell sky">
<h3>
A3 Mega
Expand All @@ -71,7 +71,7 @@ hide:
Set up GCP A3 Mega clusters with optimized networking
</p>
</a>
<a href="/examples/distributed-training/a3high-clusters"
<a href="/examples/clusters/a3high"
class="feature-cell sky">
<h3>
A3 High
Expand All @@ -83,10 +83,10 @@ hide:
</a>
</div>

## Deployment
## Inference

<div class="tx-landing__highlights_grid">
<a href="/examples/deployment/sglang"
<a href="/examples/inference/sglang"
class="feature-cell">
<h3>
SGLang
Expand All @@ -95,7 +95,7 @@ hide:
Deploy DeepSeek distilled models with SGLang
</p>
</a>
<a href="/examples/deployment/vllm"
<a href="/examples/inference/vllm"
class="feature-cell">
<h3>
vLLM
Expand All @@ -104,7 +104,7 @@ hide:
Deploy Llama 3.1 with vLLM
</p>
</a>
<a href="/examples/deployment/tgi"
<a href="/examples/inference/tgi"
class="feature-cell">
<h3>
TGI
Expand All @@ -113,7 +113,7 @@ hide:
Deploy Llama 4 with TGI
</p>
</a>
<a href="/examples/deployment/nim"
<a href="/examples/inference/nim"
class="feature-cell">
<h3>
NIM
Expand All @@ -122,7 +122,7 @@ hide:
Deploy a DeepSeek distilled model with NIM
</p>
</a>
<a href="/examples/deployment/trtllm"
<a href="/examples/inference/trtllm"
class="feature-cell">
<h3>
TensorRT-LLM
Expand Down
4 changes: 2 additions & 2 deletions docs/overrides/main.html
Original file line number Diff line number Diff line change
Expand Up @@ -117,9 +117,9 @@

<div class="tx-footer__section">
<div class="tx-footer__section-title">Examples</div>
<a href="/examples#distributed-training" class="tx-footer__section-link">Distributed training</a>
<a href="/examples#fine-tuning" class="tx-footer__section-link">Fine-tuning</a>
<a href="/examples#deployment" class="tx-footer__section-link">Deployment</a>
<a href="/examples#clusters" class="tx-footer__section-link">Clusters</a>
<a href="/examples#inference" class="tx-footer__section-link">Inference</a>
<a href="/examples#accelerators" class="tx-footer__section-link">Accelerators</a>
<a href="/examples#llms" class="tx-footer__section-link">LLMs</a>
<a href="/examples#misc" class="tx-footer__section-link">Misc</a>
Expand Down
12 changes: 6 additions & 6 deletions examples/accelerators/amd/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -11,7 +11,7 @@ Llama 3.1 70B in FP16 using [TGI :material-arrow-top-right-thin:{ .external }](h

=== "TGI"

<div editor-title="examples/deployment/tgi/amd/.dstack.yml">
<div editor-title="examples/inference/tgi/amd/.dstack.yml">

```yaml
type: service
Expand Down Expand Up @@ -44,7 +44,7 @@ Llama 3.1 70B in FP16 using [TGI :material-arrow-top-right-thin:{ .external }](h

=== "vLLM"

<div editor-title="examples/deployment/vllm/amd/.dstack.yml">
<div editor-title="examples/inference/vllm/amd/.dstack.yml">

```yaml
type: service
Expand Down Expand Up @@ -98,7 +98,7 @@ Llama 3.1 70B in FP16 using [TGI :material-arrow-top-right-thin:{ .external }](h

> To speed up the `vLLM-ROCm` installation, we use a pre-built binary from S3.
> You can find the task to build and upload the binary in
> [`examples/deployment/vllm/amd/` :material-arrow-top-right-thin:{ .external }](https://github.com/dstackai/dstack/blob/master/examples/deployment/vllm/amd/){:target="_blank"}.
> [`examples/inference/vllm/amd/` :material-arrow-top-right-thin:{ .external }](https://github.com/dstackai/dstack/blob/master/examples/inference/vllm/amd/){:target="_blank"}.

!!! info "Docker image"
If you want to use AMD, specifying `image` is currently required. This must be an image that includes
Expand Down Expand Up @@ -228,16 +228,16 @@ $ WANDB_API_KEY=...
$ WANDB_PROJECT=...
$ WANDB_NAME=axolotl-amd-llama31-train
$ HUB_MODEL_ID=...
$ dstack apply -f examples/deployment/vllm/amd/.dstack.yml
$ dstack apply -f examples/inference/vllm/amd/.dstack.yml
```

</div>

## Source code

The source-code of this example can be found in
[`examples/deployment/tgi/amd` :material-arrow-top-right-thin:{ .external }](https://github.com/dstackai/dstack/blob/master/examples/deployment/tgi/amd){:target="_blank"},
[`examples/deployment/vllm/amd` :material-arrow-top-right-thin:{ .external }](https://github.com/dstackai/dstack/blob/master/examples/deployment/vllm/amd){:target="_blank"},
[`examples/inference/tgi/amd` :material-arrow-top-right-thin:{ .external }](https://github.com/dstackai/dstack/blob/master/examples/inference/tgi/amd){:target="_blank"},
[`examples/inference/vllm/amd` :material-arrow-top-right-thin:{ .external }](https://github.com/dstackai/dstack/blob/master/examples/inference/vllm/amd){:target="_blank"},
[`examples/fine-tuning/axolotl/amd` :material-arrow-top-right-thin:{ .external }](https://github.com/dstackai/dstack/blob/master/examples/fine-tuning/axolotl/amd){:target="_blank"} and
[`examples/fine-tuning/trl/amd` :material-arrow-top-right-thin:{ .external }](https://github.com/dstackai/dstack/blob/master/examples/fine-tuning/trl/amd){:target="_blank"}

Expand Down
6 changes: 3 additions & 3 deletions examples/accelerators/intel/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -12,7 +12,7 @@ using [TGI on Gaudi :material-arrow-top-right-thin:{ .external }](https://github
and [vLLM :material-arrow-top-right-thin:{ .external }](https://github.com/HabanaAI/vllm-fork){:target="_blank"}.

=== "TGI"
<div editor-title="examples/deployment/tgi/intel/.dstack.yml">
<div editor-title="examples/inference/tgi/intel/.dstack.yml">

```yaml
type: service
Expand Down Expand Up @@ -60,7 +60,7 @@ and [vLLM :material-arrow-top-right-thin:{ .external }](https://github.com/Haban

=== "vLLM"

<div editor-title="examples/deployment/vllm/intel/.dstack.yml">
<div editor-title="examples/inference/vllm/intel/.dstack.yml">

```yaml
type: service
Expand Down Expand Up @@ -162,7 +162,7 @@ Once the configuration is ready, run `dstack apply -f <configuration file>`.
<div class="termy">

```shell
$ dstack apply -f examples/deployment/vllm/.dstack.yml
$ dstack apply -f examples/inference/vllm/.dstack.yml

# BACKEND REGION RESOURCES SPOT PRICE
1 ssh remote 152xCPU,1007GB,8xGaudi2:96GB yes $0 idle
Expand Down
8 changes: 4 additions & 4 deletions examples/accelerators/tpu/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -24,7 +24,7 @@ and [vLLM :material-arrow-top-right-thin:{ .external }](https://github.com/vllm-

=== "Optimum TPU"

<div editor-title="examples/deployment/tgi/tpu/.dstack.yml">
<div editor-title="examples/inference/tgi/tpu/.dstack.yml">

```yaml
type: service
Expand Down Expand Up @@ -56,7 +56,7 @@ and [vLLM :material-arrow-top-right-thin:{ .external }](https://github.com/vllm-
the official Docker image can be used.

=== "vLLM"
<div editor-title="examples/deployment/vllm/tpu/.dstack.yml">
<div editor-title="examples/inference/vllm/tpu/.dstack.yml">

```yaml
type: service
Expand Down Expand Up @@ -179,8 +179,8 @@ Note, `v5litepod` is optimized for fine-tuning transformer-based models. Each co
## Source code

The source-code of this example can be found in
[`examples/deployment/tgi/tpu` :material-arrow-top-right-thin:{ .external }](https://github.com/dstackai/dstack/blob/master/examples/deployment/tgi/tpu){:target="_blank"},
[`examples/deployment/vllm/tpu` :material-arrow-top-right-thin:{ .external }](https://github.com/dstackai/dstack/blob/master/examples/deployment/vllm/tpu){:target="_blank"},
[`examples/inference/tgi/tpu` :material-arrow-top-right-thin:{ .external }](https://github.com/dstackai/dstack/blob/master/examples/inference/tgi/tpu){:target="_blank"},
[`examples/inference/vllm/tpu` :material-arrow-top-right-thin:{ .external }](https://github.com/dstackai/dstack/blob/master/examples/inference/vllm/tpu){:target="_blank"},
and [`examples/fine-tuning/optimum-tpu` :material-arrow-top-right-thin:{ .external }](https://github.com/dstackai/dstack/blob/master/examples/fine-tuning/trl){:target="_blank"}.

## What's next?
Expand Down
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
# GCP A3 High clusters
# GCP A3 High

This example shows how to set up a GCP A3 High cluster with [GPUDirect-TCPX](https://cloud.google.com/compute/docs/gpus/gpudirect)
optimized NCCL communication and run [NCCL Tests](https://github.com/NVIDIA/nccl-tests) on it using `dstack`.
Expand Down
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
# GCP A3 Mega clusters
# GCP A3 Mega

This example shows how to set up a GCP A3 Mega cluster with [GPUDirect-TCPXO](https://cloud.google.com/kubernetes-engine/docs/how-to/gpu-bandwidth-gpudirect-tcpx-autopilot)
optimized NCCL communication and run [NCCL Tests](https://github.com/NVIDIA/nccl-tests) on it using `dstack`.
Expand Down
13 changes: 0 additions & 13 deletions examples/deployment/infinity/.dstack.yml

This file was deleted.

11 changes: 0 additions & 11 deletions examples/deployment/infinity/README.md

This file was deleted.

14 changes: 0 additions & 14 deletions examples/deployment/lorax/.dstack.yml

This file was deleted.

Loading
Loading