Document Sync by Tina

Author: Chivier

docs/stable/cli/_category_.json

@@ -0,0 +1,8 @@
+{
+  "label": "ServerlessLLM CLI",
+  "position": 4,
+  "link": {
+    "type": "generated-index",
+    "description": "TODO"
+  }
+}

docs/stable/dev/_category_.json

@@ -1,6 +1,6 @@
 {
   "label": "Developer Guide",
-  "position": 4,
+  "position": 5,
   "link": {
     "type": "generated-index"
   }

docs/stable/getting_started/installation.md

@@ -1,2 +1,26 @@
 # Installations
-Todo
+
+## Requirements
+- OS: Ubuntu 20.04
+- Python: 3.10
+- GPU: compute capability 7.0 or higher
+
+## Install with pip
+TODO
+
+## Install from source
+Install the package from source by running the following commands:
+```bash
+# Because this is currelty a private repository, you need to login to github first
+# by `gh auth login` and then clone the repository
+git clone https://github.com/future-xy/Phantom-component.git
+cd Phantom-component
+pip install -e .
+pip install -i https://test.pypi.org/simple/ --extra-index-url https://pypi.org/simple/ serverless_llm_store==0.0.1.dev2
+```
+
+# Install the package
+conda create -n sllm python=3.10 -y
+conda activate sllm
+pip install -e .[worker]
+```

docs/stable/getting_started/quickstart.md

@@ -1,2 +1,49 @@
 # Quickstart Guide
-todo
+
+This guide will help you get started with the basics of using ServerlessLLM. Please make sure you have installed the ServerlessLLM following the [installation guide](./installation.md).
+
+## Local test
+First, let's start a local ray cluster to test the ServerlessLLM. You can start a local ray cluster by running the following command:
+
+Start a local ray cluster with 1 head node and 1 worker node:
+```bash
+conda activate sllm
+ray start --head --port=6379 --num-cpus=4 --num-gpus=0 \
+--resources='{"control_node": 1}' --block
+```
+
+In a new terminal, start the worker node:
+```bash
+conda activate sllm
+ray start --address=localhost:6379 --num-cpus=4 --num-gpus=2 \
+--resources='{"worker_node": 1, "worker_id_0": 1}' --block
+```
+
+Now, start the ServerlessLLM server by running the following command in a new terminal:
+```bash
+conda activate sllm
+sllm-serve start
+```
+
+Next, let's deploy a model to the ServerlessLLM server. You can deploy a model by running the following command:
+```bash
+conda activate sllm
+sllm-cli deploy --model facebook/opt-1.3b
+```
+
+Now, you can query the model by any OpenAI API client. For example, you can use the following Python code to query the model:
+```bash
+curl http://localhost:8343/v1/chat/completions \
+-H "Content-Type: application/json" \
+-d '{
+        "model": "opt-1.3b",
+        "messages": [
+            {"role": "system", "content": "You are a helpful assistant."},
+            {"role": "user", "content": "What is your name?"}
+        ]
+    }'
+```
+Expected output:
+```json
+{"id":"chatcmpl-9f812a40-6b96-4ef9-8584-0b8149892cb9","object":"chat.completion","created":1720021153,"model":"opt-1.3b","choices":[{"index":0,"message":{"role":"assistant","content":"system: You are a helpful assistant.\nuser: What is your name?\nsystem: I am a helpful assistant.\n"},"logprobs":null,"finish_reason":"stop"}],"usage":{"prompt_tokens":16,"completion_tokens":26,"total_tokens":42}}
+```

docs/stable/serving/_category_.json renamed to docs/stable/serve/_category_.json

@@ -1,5 +1,5 @@
 {
-  "label": "Serving",
+  "label": "ServerlessLLM Serve",
   "position": 3,
   "link": {
     "type": "generated-index",