Skip to content

epic: Remove model.yml file #2151

Open
Open
@qnixsynapse

Description

@qnixsynapse

Goal

Currently, with GGUF model files we ship a model.yml file which contains extra parameters not generally available in GGUF files such as inference parameters and so on. Given that cortex also/will use vllm as a backend, consider removing model.yml file and use gguf-editor to set extra parameters in the GGUF itself.

Metadata

Metadata

Assignees

Labels

type: epicA major feature or initiative

Type

No type

Projects

Status

Investigating

Status

Eng Planning

Relationships

None yet

Development

No branches or pull requests

Issue actions