Cortex.cpp: llama.cpp Engine

[dan-menlo]

Overview

• We submodule llama.cpp
• How we handle different llama.cpp versions (e.g. AVX, CUDA etc)
• How we differ from "stock" llama.cpp
• What we implement as part of llamacpp-engine

Related

• fix: #3549, #3552 - Inference on CPU is slower on Jan 0.5.3 jan#3602
  • Documents which llama.cpp versions we bundle into Cortex/Jan
  • Switching logic for CPU/GPU inference