Skip to content

Commit bb0f188

Browse files
committed
Document Sync by Tina
1 parent c1038b8 commit bb0f188

File tree

7 files changed

+17
-19
lines changed

7 files changed

+17
-19
lines changed

docs/stable/dev/_category_.json

Lines changed: 0 additions & 7 deletions
This file was deleted.

docs/stable/dev/custom_backends.md

Whitespace-only changes.

docs/stable/intro.md

Lines changed: 17 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -4,7 +4,24 @@ sidebar_position: 1
44

55
# Serverless LLM
66

7+
<!-- logo (../images/serverlessllm.jpg) -->
8+
![ServerlessLLM](../images/serverlessllm.jpg)
9+
10+
ServerlessLLM is a fast, affordable, and easy-to-use library designed for multi-LLM serving, also known as [Serverless Inference](https://docs.aws.amazon.com/sagemaker/latest/dg/serverless-endpoints.html), [Inference Endpoint](https://huggingface.co/inference-endpoints/dedicated), or [Model Endpoints](https://learn.microsoft.com/en-us/azure/machine-learning/concept-endpoints?view=azureml-api-2). This library is ideal for environments with limited GPU resources, as it allows efficient dynamic loading of models onto GPUs. By supporting high levels of GPU multiplexing, it maximizes GPU utilization without the need to dedicate GPUs to individual models.
11+
712
## Documentation
813

914
### Getting Started
1015

16+
- [Install ServerlessLLM](./getting_started/installation.md)
17+
- [Deploy a ServerlessLLM cluster on your local machine](./getting_started/quickstart.md)
18+
- [Deploy ServerlessLLM using Docker (Recommended)](./getting_started/docker_quickstart.md)
19+
- [Deploy ServerlessLLM on a multi-machine cluster](./getting_started/multi_machine_setup.md)
20+
21+
### ServerlessLLM Store
22+
23+
- [Use ServerlessLLM Store in your own code](./store/quickstart.md)
24+
25+
### ServerlessLLM CLI
26+
27+
- [ServerlessLLM CLI Documentation](./cli/cli_api.md)

docs/stable/serve/_category_.json

Lines changed: 0 additions & 8 deletions
This file was deleted.

docs/stable/serve/docker_deployment.md

Lines changed: 0 additions & 2 deletions
This file was deleted.

docs/stable/serve/openai_compatible_server.md

Lines changed: 0 additions & 2 deletions
This file was deleted.

0 commit comments

Comments
 (0)