Update on the development branch

[kaiyux]

Hi,

The TensorRT-LLM team is pleased to announce that we are pushing an update to the development branch (and the Triton backend) this January 23rd, 2024.

This update includes:

• Model Support
  • Mamba support (see examples/mamba/README.md)
    • The support is limited to beam width = 1 and single-node single-GPU
  • Nougat support (see examples/multimodal/README.md#nougat)
  • Qwen-VL support (see examples/qwenvl/README.md)
• Features
  • Support batch size > 1 for Medusa (The support is limited to the Python runtime)
  • Support Bart OOTB functionality
  • Support FP16 fMHA on NVIDIA V100 GPU
• API
  • Remove optional parameter maxNumSequences for GPT manager
• Bug fixes
  • Minor bug fixes
• Performance
  • Integrate XQA kernels for GPT-J (beamWidth=4)
• Documentation
  • Add documentation for LoRA (see docs/source/lora.md)
  • Update AWQ INT4 weight only quantization documentation for GPT-J

Thanks,
The TensorRT-LLM Engineering Team