Large Language Models
31 views
Streaming
Quick Definition
Sending LLM tokens to clients as they are generated
Full Definition
Sending LLM tokens to clients as they are generated for real-time responsiveness.
Examples
chat interfaces, real-time responses, SSE
Related Terms
inference-optimization
user-experience
More Large Language Models Terms
BPE
Subword tokenization algorithm merging frequent character pairs
Linear Attention
Attention with linear complexity for long sequences
GPT
OpenAI's family of autoregressive large language models
Guardrails LLM
Frameworks for monitoring and controlling LLM I/O
Prompt Template
Reusable structured format with dynamic input placeholders
Tree-of-Thought
Framework exploring multiple reasoning paths for optimal solutions