Large Language Models
372 views
Model Merging
Quick Definition
Combining fine-tuned LLMs without additional training
Full Definition
Combining multiple fine-tuned LLMs into a single model without additional training.
Examples
model soup, weight merging, multi-task models
Related Terms
fine-tuning
ensemble
More Large Language Models Terms
System Prompt
Special prompt defining AI assistant behavior and constraints
Next Token Prediction
Core LLM objective predicting the next token from context
Quantization LLM
Reducing LLM weight precision for efficient inference
Mixture of Experts
Architecture routing tokens to specialized expert sub-networks
GPT
OpenAI's family of autoregressive large language models
Masked Language Model
Training objective predicting randomly masked input tokens