Search results

Feature Request: Add Prometheus Metrics Endpoint to trtllm-serve

feature requestNew feature or request. This includes new model, dtype, functionality supportNew feature or request. This includes new model, dtype, functionality support

OpenAI APItrtllm-serve's OpenAI-compatible API: endpoint behavior, req/resp formats, feature parity.trtllm-serve's OpenAI-compatible API: endpoint behavior, req/resp formats, feature parity.

triagedIssue has been triaged by maintainersIssue has been triaged by maintainers

Status: Open.

#4926 In NVIDIA/TensorRT-LLM;

· Nekofish-L opened on Jun 5, 2025

Coalesce text diffs in streaming requests.

Community want to contributePRs initiated from CommunityPRs initiated from Community

OpenAI APItrtllm-serve's OpenAI-compatible API: endpoint behavior, req/resp formats, feature parity.trtllm-serve's OpenAI-compatible API: endpoint behavior, req/resp formats, feature parity.

triagedIssue has been triaged by maintainersIssue has been triaged by maintainers

Status: Open (in progress).

NVIDIA/TensorRT-LLMnumber 4923

#4923 In NVIDIA/TensorRT-LLM;

· pathorn opened on Jun 5, 2025

[fix] Make add_special_tokens to false for completions API

Community want to contributePRs initiated from CommunityPRs initiated from Community

OpenAI APItrtllm-serve's OpenAI-compatible API: endpoint behavior, req/resp formats, feature parity.trtllm-serve's OpenAI-compatible API: endpoint behavior, req/resp formats, feature parity.

triagedIssue has been triaged by maintainersIssue has been triaged by maintainers

Status: Open (in progress).

NVIDIA/TensorRT-LLMnumber 4583

#4583 In NVIDIA/TensorRT-LLM;

· Pernekhan opened on May 22, 2025

Are multimodal models supported by trtllm-serve?

OpenAI APItrtllm-serve's OpenAI-compatible API: endpoint behavior, req/resp formats, feature parity.trtllm-serve's OpenAI-compatible API: endpoint behavior, req/resp formats, feature parity.

triagedIssue has been triaged by maintainersIssue has been triaged by maintainers

Status: Open.

#2714 In NVIDIA/TensorRT-LLM;

· xiaoyuzju opened on Jan 23, 2025

trtllm-serve without any output Qwne2.5-7b

bugSomething isn't workingSomething isn't working

OpenAI APItrtllm-serve's OpenAI-compatible API: endpoint behavior, req/resp formats, feature parity.trtllm-serve's OpenAI-compatible API: endpoint behavior, req/resp formats, feature parity.

Status: Open.

#2667 In NVIDIA/TensorRT-LLM;

· Justin-12138 opened on Jan 8, 2025