How do you implement conversation memory for a chatbot?
LLM & AILevel: IntermediateMemory Management
How do you implement conversation memory for a chatbot?
Hints
- Identify the stage: pretraining, adaptation, generation or evaluation.
- Name the inputs, constraints and a case where the approach fails.
Answer
Option 1 — Buffer memory: keep all turns in context. Simple; runs out of context window quickly.
Option 2 — Sliding window: keep last N turns. Loses early context abruptly.
Option 3 — Summarization memory:
if len(history) > threshold:
summary = llm.summarize(history[:-recent_k])
context = [summary_message] + history[-recent_k:]
Option 4 — Vector memory: embed each turn, retrieve by relevance to current query. Precise recall of specific facts; adds latency.
Option 5 — Hybrid (recommended for production):
- Last K turns verbatim (short-term, recency)
- Summary of earlier session (medium-term, general context)
- Vector store of important facts (long-term, specific recall)
Source and licence: JustInCache & contributors — Awesome AI Interviews · Apache-2.0