Open
Description
Is your feature request related to a problem? Please describe.
Several use cases require access to the token log probabilities from the completion response. This includes techniques such as FLARE Agents, "SelfRAG" in LlamaIndex, Guidance, as well as, the ability to develop logic to evaluate the confidence level of the generated answer.
Describe the solution you'd like
Basically the same as the OpenAI API.
Describe alternatives you've considered
Additional context