Computer Vision Systems • Edge Deployment (MobileNet, EfficientNet-Lite)Hard⏱️ ~2 min
Edge Deployment Failure Modes: Quantization Drift, Thermal Throttling, and NMS Explosions
Quantization drift occurs when post training quantization causes large accuracy drops for out of distribution inputs. Models trained on well lit, high quality images can lose 5 to 10 mAP points when quantized to 8 bit integers and deployed on devices with low light noise, motion blur, or heavy JPEG compression. If validation mAP drops more than 2 to 3 points after quantization, switch to quantization aware training with a calibration set that covers real device camera characteristics, including lens distortions and sensor noise.
Thermal throttling is the silent killer of edge performance. Running inference at 100 percent utilization heats the system on chip (SOC). After tens of seconds, the device reduces clock speeds to prevent damage. Latency that started at 12ms rises to 25ms or higher, and frame rate collapses. Battery drain accelerates. Mitigate this with duty cycling: run inference only when the viewfinder is visible, reduce resolution when temperature sensors report thermal stress, and cap workload to leave thermal headroom. Apple and Google both implement adaptive frame rates in their camera pipelines to prevent thermal runaway.
Non maximum suppression can become the bottleneck in crowded scenes. NMS scales with the square of candidate boxes. A detection head tuned for sparse scenes might output 100 candidates per frame, taking 2ms to process. In a crowded scene with 500 candidates, NMS time jumps to 30ms or more, blowing the entire frame budget. Solutions include class agnostic NMS that processes all classes together, top K filtering per pyramid level before NMS, and tuning confidence thresholds to reduce candidates. Memory allocator stalls are another common trap: frequent tensor allocation in tight loops causes unpredictable pauses, with p99 latency spiking from 20ms to 70ms due to fragmentation.
💡 Key Takeaways
•Quantization drift from post training integer quantization can drop mAP by 5 to 10 points on low light or compressed images; use quantization aware training with representative calibration data
•Thermal throttling after tens of seconds at full utilization doubles latency from 12ms to 25ms and causes frame rate collapse; implement adaptive duty cycling and resolution reduction
•Non maximum suppression scales quadratically with candidate boxes; crowded scenes can push NMS from 2ms to 30ms, requiring top K filtering and class agnostic processing
•Memory allocator stalls from frequent tensor allocation cause p99 latency spikes from 20ms to 70ms; preallocate fixed size buffers and use memory pools to eliminate runtime allocation
•Sensor pipeline mismatches where training uses center crops but deployment uses wide aspect ratios can degrade accuracy by 10 to 15 percent; maintain consistent preprocessing with letterbox padding
📌 Examples
Mobile app experiences 8 mAP point drop after quantization due to low light images from device cameras; switching to quantization aware training with night photos recovers 6 points
IoT camera running continuous detection heats SOC to thermal limit in 45 seconds, causing latency to jump from 15ms to 32ms and frame rate to drop from 30 fps to 18 fps
Detection model outputs 500 candidates in crowded mall scene, NMS takes 35ms and misses frame deadline; adding per level top 50 filtering reduces NMS to 4ms