Article
System Design Interview: Monitoring Truck Shipments with Cameras. Part 1
A system design walkthrough from a Computer Vision lead interview: verifying construction-site truck shipments with cameras, tracking and scales. Part one of a series.
This is the first part of a five-part series about a single job interview.
I recently interviewed with a startup that had just raised a pre-seed round. I was being considered for the role of Computer Vision and ML team lead. First came an HR screen, then a 2.5-hour conversation with the founder, and then a system design round with a technical auditor.
The product is built for construction sites: cameras, sensors and drones help monitor shipments and the movement of materials. I'll walk through what happened in that last interview and how I structured my answer.
Let's start with the trucks
What does the business actually need: to estimate how full the truck is, or to verify the order?
A half-empty truck may be carrying exactly what was ordered. So first we pin down the units of measurement, the acceptable error and what the dispatcher is supposed to do with the result.
Say the goal is to catch a wrong shipment before the truck leaves. Then we need the order, the measurements and a link to the specific trip. Fill percentage on its own doesn't solve the problem.
How I'd build the CV part
I'd start with the measurement zone: a camera mounted overhead or at an angle has to see what's inside the truck bed. If all it sees is the side panel, a bigger model won't give you the missing viewpoint.
For vehicle detection I'd go with YOLO11s, and for segmenting the truck bed I'd try YOLO11s-seg. Both models get fine-tuned on data from the site.
For tracking I'd start with ByteTrack: it associates detections by motion and box overlap, and also uses low-confidence detections to keep a track going. If trucks frequently get swapped after occlusions, I'd compare it against BoT-SORT with a ReID model validated on trucks.
The segmented bed is cropped from the original frame and fed into a ResNet-18 trained to distinguish "empty, low, medium, full". We account for the ordering of the classes: confusing neighboring states is less critical than mistaking an empty bed for a full one.
The result is aggregated over several usable frames once loading is complete. A tarp, heavy occlusion or a bad image should yield "insufficient data", not a random class.
Camera or scales
The camera sees a full bed, the scales show an underload. Which one do you trust?
These are different measurements. Visual fullness is not a substitute for mass. For mass you need scales; for physical volume you could consider LiDAR combined with the known geometry of the truck bed.
Every arrival gets a session_id. The order, the measurements and the timestamps are attached to it. A track_id isn't enough for this: the track can break while the trip is still in progress.
A late scale reading is matched by the time of measurement, not the time the message was received. Otherwise the previous truck's data ends up attached to the next one.
How it all fits together
On site there's a local GPU node: video ingestion through DeepStream, models running in TensorRT, and a separate tracking stage. The fill classifier only runs on suitable observations inside the measurement zone.
The backend tracks the trip's states: arrived, loading, measured, verified, departed. When there's a discrepancy, it stores the event and sends the dispatcher the order, the measurements and the frames. A retried send must not create a second issue in the records.
On test trips we evaluate detection, tracking and classification separately. But the main metrics are the share of correctly verified trips, missed discrepancies, manual checks, and alerts raised before departure. A good mAP doesn't mean the dispatcher got the right decision in time.