Large Language Models
165 views
Streaming
Quick Definition
Sending LLM tokens to clients as they are generated
Full Definition
Sending LLM tokens to clients as they are generated for real-time responsiveness.
Examples
chat interfaces, real-time responses, SSE
Related Terms
inference-optimization
user-experience
More Large Language Models Terms
Flash Attention
Memory-efficient attention using GPU SRAM block computation
Function Calling
Structured mechanism for LLMs to generate API function calls
Adapter Layer
Small trainable modules in frozen transformer layers
Semantic Search
Search technique understanding meaning rather than keywords
Masked Language Model
Training objective predicting randomly masked input tokens
Linear Attention
Attention with linear complexity for long sequences