Guides
LLM engineer roadmap: how to become an AI engineer
LLM engineering is about making language models useful and reliable inside products. Prompting is the easy part; the job is retrieval, evaluation, fine-tuning, serving and knowing when the model is wrong. Here is what to learn and in what order.
What an LLM engineer does
An LLM or AI engineer designs systems around language models: chooses the model, connects it to data through retrieval, constrains outputs, adds tools, measures quality and cost, and ships it safely. The best engineers treat the model as an unreliable component and build evaluation and guardrails around it.
Prerequisites
- Solid Python, APIs, async code, testing and basic web services.
- ML basics: train/validation/test, metrics, overfitting.
- Deep learning basics in PyTorch; ideally you have trained a small transformer yourself.
Understand the transformer
You do not need to pretrain large models, but you should know tokenisation, embeddings, attention, the decoder-only architecture, context windows, sampling (temperature, top-p) and why models hallucinate. Implementing a small GPT once makes every later decision clearer.
Retrieval-augmented generation (RAG)
- Chunking strategies and metadata.
- Embeddings, vector indexes and hybrid search (dense plus keyword).
- Reranking with cross-encoders.
- Citations: answers that point to verifiable sources.
- Evaluating retrieval (recall@k) separately from answer quality.
Structured outputs, tools and agents
- JSON schemas and validated structured output.
- Function calling and tool design with clear contracts.
- Agent loops with step limits, timeouts and safe, permissioned tools.
- Prompt injection and data exfiltration risks, and how to contain them.
Fine-tuning
Fine-tune when prompting and retrieval are not enough: for a consistent format, a domain style or a smaller, cheaper model. Learn supervised fine-tuning (SFT), parameter-efficient methods such as LoRA and QLoRA, dataset curation, and how to prove the adapter improved quality without breaking other behaviour.
Evaluation and regression testing
- Golden datasets built from real user questions.
- Automatic metrics where possible, LLM-as-judge with calibration against human labels.
- Regression tests that run before every prompt or model change.
- Tracking cost and latency alongside quality.
Serving and operations
- Quantisation (for example GGUF or 4-bit) and its effect on quality.
- Inference servers such as vLLM, batching and the KV cache.
- Observability: traces, failures, user feedback and reproducible releases.
Portfolio projects
- A RAG assistant over a real document set, with retrieval and answer evaluations.
- A LoRA fine-tune with a before/after evaluation report.
- A tool-using agent with explicit limits and a test suite of failure cases.
- A locally served model with a latency and cost benchmark.
Working with a mentor
Quantoros has an LLM & AI Engineering track of 8 sprints — from the transformer to controllable generation, SFT and LoRA, structured output and function calling, RAG with verifiable sources, evaluation before release, local serving with GGUF and vLLM, safe agents and observability. It builds on the Deep Learning stage, where you write a small GPT yourself. Mentoring is 1:1 with code review and mock interviews.
Resources
- Neural Networks: Zero to Hero — Andrej Karpathy ↗Build a GPT from scratch to understand the internals.
- Hugging Face LLM Course ↗Transformers, fine-tuning and the open-model ecosystem.
- Stanford CS336: Language Modeling from Scratch ↗Deep dive into building and training language models.
- vLLM documentation ↗High-throughput LLM serving.
- OWASP Top 10 for LLM Applications ↗Security risks such as prompt injection.
Questions
Do I need to be an ML engineer first?
Not necessarily, but ML basics and some deep learning make you much stronger, especially for evaluation and fine-tuning.
Is prompt engineering a career?
On its own it is narrow. Employers hire engineers who can build retrieval, evaluation, tools and serving around models.
Should I use closed APIs or open models?
Learn both. Open models teach you serving, quantisation and fine-tuning; APIs are common in products.
What is the most important skill?
Evaluation: being able to measure whether a change made the system better.