Computer Vision on Edge Devices: Models, Frameworks, and Trade-offs
Computer Vision on Edge Devices
Running vision models on edge hardware — IP cameras, Raspberry Pi clusters, industrial gateways — removes cloud round-trips and keeps sensitive imagery local.
Model selection
Start with a model that fits your hardware budget:
| Model family | Use case | Typical size |
|---|---|---|
| MobileNetV3 | Classification | 5–15 MB |
| YOLOv8-nano | Object detection | 6–12 MB |
| EfficientNet-Lite | High-accuracy classification | 20–50 MB |
Prune and quantize before deployment. INT8 quantization often cuts inference time in half with minimal accuracy loss on vision tasks.
Runtime frameworks
- TensorFlow Lite — best for mobile and ARM edge devices
- ONNX Runtime — cross-platform, good GPU support
- OpenVINO — optimized for Intel CPUs and VPUs
- TensorRT — NVIDIA Jetson and GPU deployments
Pick one runtime and standardize. Supporting four runtimes in production is painful.
Pipeline design
A typical edge vision pipeline:
- Capture frame (camera SDK)
- Preprocess (resize, normalize)
- Inference
- Post-process (NMS for detection, threshold filtering)
- Act locally (alert, store event, trigger actuator)
- Send summary to cloud (not raw frames)
Sending only detections and metadata to the cloud reduces bandwidth by orders of magnitude.
Hardware considerations
Thermal throttling on fanless devices is real. Profile sustained inference under load, not just burst benchmarks. Add a frame-skip strategy when the device overheats.
Power budgets matter for battery-powered cameras — duty-cycling inference to 1–2 FPS can extend battery life from hours to days.
