Skip to content
Quantoros

How do you implement conversation memory for a chatbot?

LLM & AILevel: IntermediateMemory Management

How do you implement conversation memory for a chatbot?

Hints
  • Identify the stage: pretraining, adaptation, generation or evaluation.
  • Name the inputs, constraints and a case where the approach fails.

Answer

Option 1 — Buffer memory: keep all turns in context. Simple; runs out of context window quickly.

Option 2 — Sliding window: keep last N turns. Loses early context abruptly.

Option 3 — Summarization memory:

if len(history) > threshold:
    summary = llm.summarize(history[:-recent_k])
    context = [summary_message] + history[-recent_k:]

Option 4 — Vector memory: embed each turn, retrieve by relevance to current query. Precise recall of specific facts; adds latency.

Option 5 — Hybrid (recommended for production):

  • Last K turns verbatim (short-term, recency)
  • Summary of earlier session (medium-term, general context)
  • Vector store of important facts (long-term, specific recall)

Source and licence: JustInCache & contributors — Awesome AI Interviews · Apache-2.0

Related questions