Search results

Broken openai api

bugSomething isn't workingSomething isn't working

Status: Open.

#5393 In NVIDIA/TensorRT-LLM;

· Alcanderian opened on Jun 21, 2025

Server stuck with high load

bugSomething isn't workingSomething isn't working

Status: Open.

#5391 In NVIDIA/TensorRT-LLM;

· k-l-lambda opened on Jun 21, 2025

RuntimeError: CUDA error: CUBLAS_STATUS_EXECUTION_FAILED when calling <code>cublasGemmEx( handle, opa, opb, m, n, k, &amp;falpha, a, CUDA_R_16BF, lda, b, CUDA_R_16BF, ldb, &amp;fbeta, c, CUDA_R_16BF, ldc, compute_type, CUBLAS_GEMM_DEFAULT_TENSOR_OP)</code>

bugSomething isn't workingSomething isn't working

Status: Open.

#5390 In NVIDIA/TensorRT-LLM;

· moguizhizi opened on Jun 21, 2025

[TensorRT-LLM][ERROR] Assertion failed: Error occurred when running GEMM!

bugSomething isn't workingSomething isn't working

Status: Open.

#5389 In NVIDIA/TensorRT-LLM;

· momaek opened on Jun 21, 2025

invalid request_id in <code>MTPSpecMetadata</code>

bugSomething isn't workingSomething isn't working

Status: Open.

#5386 In NVIDIA/TensorRT-LLM;

· k-l-lambda opened on Jun 20, 2025

Mixtral-8x7B-Instruct awq-w4a8 output shows duplicated Chinese text

bugSomething isn't workingSomething isn't working

Status: Open.

#5379 In NVIDIA/TensorRT-LLM;

· wanzhenchn opened on Jun 20, 2025

Poor performance after FP8 Quantization for Llama 3.1 on PyTorch backend

bugSomething isn't workingSomething isn't working

Status: Open.

#5370 In NVIDIA/TensorRT-LLM;

· geaned opened on Jun 19, 2025

GPTManagerBenchmark Hangs with INT8 KV Cache - GPU Utilization Stays at 0%

bugSomething isn't workingSomething isn't working

need more infoFurther info is required from the requester for devs to helpFurther info is required from the requester for devs to help

Status: Open.

#5304 In NVIDIA/TensorRT-LLM;

· kurosakiharachan opened on Jun 18, 2025

Qwen3-235B-A22B shape error with EP=8

bugSomething isn't workingSomething isn't working

triagedIssue has been triaged by maintainersIssue has been triaged by maintainers

Status: Open.

#5292 In NVIDIA/TensorRT-LLM;

· rzhao01 opened on Jun 17, 2025

Incompatibility between dp_attention and enable_chunked_prefill causes service hang on exceeding max_num_tokens

bugSomething isn't workingSomething isn't working

triagedIssue has been triaged by maintainersIssue has been triaged by maintainers

Status: Open.

#5267 In NVIDIA/TensorRT-LLM;

· Nekofish-L opened on Jun 17, 2025

[Torchflow] Can't run torchflow on llamaforcausallm with fp32. rmsnorm(at::Tensor&amp;, at::Tensor&amp;, at::Tensor&amp;, double, bool)::&lt;lambda()&gt; failed to dispatch data type Float

bugSomething isn't workingSomething isn't working

need more infoFurther info is required from the requester for devs to helpFurther info is required from the requester for devs to help

Status: Open.

#5260 In NVIDIA/TensorRT-LLM;

· michaelfeil opened on Jun 17, 2025

Investigate TRTLLM runtime repetitive issue

AutoDeploy

bugSomething isn't workingSomething isn't working

Status: Open.

#5254 In NVIDIA/TensorRT-LLM;

· Fridah-nv opened on Jun 16, 2025