Large Language Models
24 views
Model Merging
Quick Definition
Combining fine-tuned LLMs without additional training
Full Definition
Combining multiple fine-tuned LLMs into a single model without additional training.
Examples
model soup, weight merging, multi-task models
Related Terms
fine-tuning
ensemble
More Large Language Models Terms
GPT
OpenAI's family of autoregressive large language models
Mistral
Efficient open-source LLMs with strong performance
Grounding LLM
Connecting LLM outputs to verifiable external sources
Temperature
Parameter controlling randomness in text generation
Positional Encoding
Incorporating token position information into transformers
Sparse Attention
Attention attending to only a subset of tokens