Guides
GPU programming for machine learning: a roadmap
Model speed and cost are decided on the GPU. ML performance engineers understand where time goes, write kernels when libraries are not enough and integrate them into PyTorch safely. This roadmap goes from the first kernel to a tested custom operator.
Why GPU skills matter for ML engineers
Training and inference costs dominate many ML budgets. Engineers who can profile a model, find the bottleneck and fix it — by fusing operations, changing memory layout or writing a kernel — are rare and valuable in ML infrastructure, inference and research engineering teams.
The mental model: how a GPU works
- Streaming multiprocessors run thousands of threads grouped into warps of 32 that execute together.
- Memory hierarchy: registers and shared memory are fast and small; global memory is large and slow.
- Most ML kernels are memory-bound: moving data costs more than arithmetic.
- The roofline model tells you whether a kernel is limited by bandwidth or by compute.
First steps in CUDA
- Grids, blocks and threads; computing a global index.
- A vector addition kernel and checking it against a reference.
- Coalesced memory access and why stride patterns matter.
- Measuring time correctly: warm-up, synchronisation, repeated runs.
Core kernels every ML performance engineer should write
- Fused softmax: one pass over memory instead of several, with a numerically stable maximum.
- Occupancy and throughput tuning for that kernel.
- Tiled matrix multiplication using shared memory.
- Block ordering and autotuning of tile sizes.
- Layer normalisation forward pass with a parallel reduction.
Triton: the productive path
Triton lets you write GPU kernels in Python with block-level primitives, while the compiler handles many low-level details. It is widely used for custom fused operations in PyTorch models and is the fastest way to get from an idea to a fast kernel. Learn CUDA concepts for the mental model and Triton for daily work.
Integrating with PyTorch
- Write a custom operator in C++/CUDA and register it with PyTorch.
- Add autograd support with a backward function.
- Test correctness with gradcheck and operator checks, including edge cases and dtypes.
- Benchmark against the PyTorch baseline and document when your operator wins.
Profiling and inference optimisation
- PyTorch profiler, Nsight Systems for timelines and Nsight Compute for kernels.
- Mixed precision and its numerical traps.
- Batching, KV caching and quantisation for LLM inference.
- The idea behind FlashAttention: tiling attention to avoid materialising the full matrix.
Portfolio projects
- A fused softmax or layer norm kernel with a benchmark report.
- A tiled matmul with autotuning and a roofline analysis.
- A custom PyTorch operator with autograd and tests.
- A profiling case study: a model made faster with measurements before and after.
Working with a mentor
The Quantoros GPU & ML Performance track has 8 sprints: the first vector-add kernel, a fused softmax, softmax occupancy and throughput, tiled matmul, block ordering and autotuning, layer norm forward, a PyTorch operator in C++/CUDA, and testing with autograd. Free GPUs in Colab and Kaggle are enough. Mentoring is 1:1 with review of kernels, benchmarks and profiles.
Resources
- CUDA C++ Programming Guide — NVIDIA ↗The reference for the CUDA programming model.
- GPU MODE ↗Community lectures and resources on GPU programming for ML.
- Triton documentation and tutorials ↗Fused softmax, matmul and layer norm tutorials.
- How to Optimize a CUDA Matmul Kernel — Simon Boehm ↗Step-by-step matmul optimisation.
- PyTorch custom C++ and CUDA operators ↗Official tutorial for custom operators.
Questions
Do I need my own GPU to learn CUDA?
No. Free GPUs in Google Colab and Kaggle notebooks are enough for learning and for most portfolio projects.
Should I learn CUDA or Triton first?
Learn the CUDA concepts first for the mental model, then use Triton for productive kernel work.
How much C++ do I need?
Basic C++ is enough to start; you will need more for custom PyTorch operators.
Is this useful outside ML?
Yes. The same skills apply to scientific computing and to quant development where fast numerical code matters.