Skip to content

Commit a4310f5

Browse files
feat: use non-streaming tool calls for streaming tool calls
Signed-off-by: Nathan Weinberg <[email protected]>
1 parent 55c018a commit a4310f5

File tree

1 file changed

+1
-0
lines changed

1 file changed

+1
-0
lines changed

src/ramalama_stack/openai_compat.py

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -129,6 +129,7 @@ async def convert_chat_completion_request(
129129
payload.update(
130130
tool_choice=request.tool_config.tool_choice.value
131131
) # we cannot include tool_choice w/o tools, server will complain
132+
payload.update(stream=False) # llama.cpp does not support streaming tool calls
132133

133134
if request.logprobs:
134135
payload.update(logprobs=True)

0 commit comments

Comments
 (0)