Large Language Models
330 views
Alignment Tax
Quick Definition
Performance cost of aligning AI with human preferences
Full Definition
The performance cost incurred when aligning AI models to follow human preferences and safety guidelines.
Examples
capability reduction after RLHF, safety vs helpfulness tradeoff
Related Terms
rlhf
ai-alignment
fine-tuning
More Large Language Models Terms
Attention Head
Individual attention mechanism learning specific patterns
Jailbreaking
Circumventing AI safety restrictions through prompt techniques
Linear Attention
Attention with linear complexity for long sequences
Top-K Sampling
Generation strategy considering top k most probable tokens
Inference LLM
Using a trained LLM to generate outputs from inputs
Nucleus Sampling
Generation using cumulative probability threshold for token selection