How do you handle multi-turn conversations with LLMs?
LLM & AILevel: IntermediateProduction Prompt Design
How do you handle multi-turn conversations with LLMs?
Hints
- Identify the stage: pretraining, adaptation, generation or evaluation.
- Name the inputs, constraints and a case where the approach fails.
Answer
Multi-turn conversations require managing a conversation history that grows over time:
messages = [
{"role": "system", "content": "You are a helpful assistant."},
{"role": "user", "content": "What is RAG?"},
{"role": "assistant", "content": "RAG is..."},
{"role": "user", "content": "How does it compare to fine-tuning?"}, # new turn
]
Context management:
- Truncation: drop oldest turns when approaching context limit (risky: loses context)
- Summarization: replace oldest N turns with a compressed summary
- Sliding window: keep last K turns + initial system context
Topic switching: model tends to anchor to early context. When user changes topics, explicitly refresh context: "New topic: [X]. Previous conversation: [summary]."
Source and licence: JustInCache & contributors — Awesome AI Interviews · Apache-2.0