Skip to content

bug: Dropping GGUF metadata when creating model.yaml for Model Downloaded via URL #3558

Closed
@Van-QA

Description

@Van-QA

Describe the bug
When downloading the HF GGUF model via URL import from the model hub, the model is using default settings instead of the correct settings. The fixed context length is set to 2048, the prompt template is incorrect, and there is an issue with the stop word. However, importing GGUF directly seems to resolve these issues.

Steps to reproduce

  1. Download the HF GGUF model from the model hub using URL import.
  2. Check the settings for context length, prompt template, and stop word.
  3. Compare the settings with the correct settings for the GGUF model.

Expected behavior
The HF GGUF model downloaded via URL import should have the correct settings for context length, prompt template, and stop word, matching the settings when GGUF is imported directly.

Additional context
The issue seems to be specific to downloading the HF GGUF model via URL import from the model hub. Importing GGUF directly does not exhibit the same issue with default settings.

Comparing model.json, the GGUF direct import in on the right
image

Metadata

Metadata

Assignees

Labels

Type

No type

Projects

Status

Completed

Relationships

None yet

Development

No branches or pull requests

Issue actions