Your chatbot loses context after 10 turns. How do you maintain long conversation context?
LLM & AILevel: IntermediateTroubleshooting Scenarios
Your chatbot loses context after 10 turns. How do you maintain long conversation context?
Hints
- Identify the stage: pretraining, adaptation, generation or evaluation.
- Name the inputs, constraints and a case where the approach fails.
Answer
Three main strategies, often combined:
- Sliding window — keep last N turns in context. Simple; loses old information.
- Summarization memory — periodically compress old turns into a summary. Lossy but compact.
- Vector memory (RAG-style) — embed and store all turns; retrieve relevant ones on demand. Precise recall, adds latency.
Hybrid approach: keep recent turns verbatim (short-term), summary of earlier session (medium-term), vector store of important facts mentioned (long-term). Choose strategy based on what users actually need to remember.
Source and licence: JustInCache & contributors — Awesome AI Interviews · Apache-2.0