Artificial Intelligence
154 views
Model Distillation
Quick Definition
Creating smaller models that replicate larger model behavior
Full Definition
Creating smaller models that replicate the behavior of larger teacher models for efficient deployment.
Examples
TinyBERT, mobile models, edge deployment
Related Terms
pruning
quantization
model-compression
More Artificial Intelligence Terms
Federated Learning
Distributed ML training without sharing raw data
Speech Synthesis
Converting written text into natural-sounding spoken audio
Quantization
Reducing model precision from 32-bit to lower bit representations
Reward Modeling
Training models to predict human preferences for RLHF
Adversarial Attack
Crafting inputs that fool AI models into wrong predictions
Multimodal AI
AI that processes and integrates multiple data types