Downloads · 30 days
0
sriramk-edge-ai/edge-ai-yolov8n-demo
edge-ai-yolov8n-demo is a object detection model from sriramk-edge-ai. Use it when you need objects located in an image. It is set up for tensorrt. The card lists the license as apache-2.0.
This repository documents my hands-on exploration of edge-AI inference performance across platforms and numerical precisions. Each phase benchmarks YOLOv8n under different hardware, precision modes, and input resoluti…
Downloads · 30 days
0
Access
Public
Updated Nov 2, 2025
Repo size
65.5 MB
Likes
0
Public
Click a slice to open those files.
.mp438.2 MB · 58%
From the Hugging Face model README
This repository documents my hands-on exploration of edge-AI inference performance across platforms and numerical precisions.
Each phase benchmarks YOLOv8n under different hardware, precision modes, and input resolutions to understand how each factor impacts real-time performance.
All tests used a single publicly available annotated video:
A Busy Intersection Road in London – by Mikhail Nilov (Pexels)
The first phase establishes an FP32 CPU baseline using a 640×640 video input.
It reflects an unoptimized, CPU-only inference path to provide a realistic reference for later GPU acceleration.
test.mp4, from Pexels – London Intersection)| Resolution | FPS | Precision | Accelerator | Framework |
|---|---|---|---|---|
| 320×320 | 27.38 | FP32 | CPU | Ultralytics YOLOv8n |
| 480×480 | 25.53 | FP32 | CPU | Ultralytics YOLOv8n |
| 640×640 | 22.15 | FP32 | CPU | Ultralytics YOLOv8n |
🎬 Demo: demo_clip.mp4
Phase 2 compares FP32 vs FP16 inference on the Jetson Orin Nano across three input sizes (640, 480, 320).
The goal is to quantify both precision impact and resolution scaling using TensorRT optimization.
test.mp4, from Pexels – London Intersection)| Resolution | FPS (FP32) | FPS (FP16) | Accelerator | Framework |
|---|---|---|---|---|
| 320×320 | 195.684 | 298.535 | GPU (TensorRT) | YOLOv8n |
| 480×480 | 128.831 | 205.119 | GPU (TensorRT) | YOLOv8n |
| 640×640 | 75.7129 | 132.516 | GPU (TensorRT) | YOLOv8n |
🎬 Demo: jetson_demo.mp4
To illustrate the performance scaling between FP32 and FP16, the following chart compares YOLOv8n throughput (FPS) across three resolutions on the Jetson Orin Nano.

🧮 Observations
| Precision | Description | Typical Use |
|---|---|---|
| FP32 | Full-precision; highest accuracy, slower throughput | CPU / training |
| FP16 | Half-precision; faster compute, small accuracy trade-off | GPU / TensorRT |
| INT8 (Planned) | Quantized; fastest, needs calibration | NPU / Hailo / edge |
Key insight: both precision and input resolution are powerful levers for optimizing edge AI latency and power draw.
🔭 Phase 3 (Planned) — Raspberry Pi 5 + Hailo 8 NPU
The next step is to benchmark the same YOLOv8n pipeline on a Raspberry Pi 5 + Hailo 8 accelerator to compare:
- NPU vs GPU vs CPU performance
- Power efficiency / throughput trade-offs
- Layer-wise operator mapping (Conv / BN / ReLU etc.)
While resolving CUDA/PyTorch compatibility for JetPack 6.x, I ran live YOLOv8n inference directly on the CPU to keep progress moving forward. Below are the end-to-end frame stage timings captured during live webcam inference.
| Resolution | Preprocessing (ms) | Inference (ms) | Postprocessing (ms) | Approx FPS |
|---|---|---|---|---|
| 640×640 | 2.0 ms | 435.6 ms | 2.3 ms | ~2.2 FPS |
| 480×480 | 4.2 ms | 289.8 ms | 1.8 ms | ~3.4 FPS |
| 320×320 | 1.6 ms | 148.5 ms | 1.4 ms | ~6.4 FPS |
Demo video: Live_webcam
Live video Frame (CPU): Frame_Capture
Key Takeaways
Next Step (Phase 4)
Run the same live webcam pipeline using the TensorRT FP16 engine inside the NVIDIA l4t-ml container to demonstrate GPU acceleration and quantify the performance gain.
# Export model
yolo export model=yolov8n.pt format=onnx simplify=True
# Build engines
# FP32 engine
/usr/src/tensorrt/bin/trtexec \
--onnx=yolov8n.onnx \
--saveEngine=yolov8n_fp32.engine \
--minShapes=images:1x3x320x320 \
--optShapes=images:1x3x480x480 \
--maxShapes=images:1x3x640x640 \
--avgRuns=50
# FP16 engine
/usr/src/tensorrt/bin/trtexec \
--onnx=yolov8n.onnx \
--saveEngine=yolov8n_fp32.engine \
--minShapes=images:1x3x320x320 \
--optShapes=images:1x3x480x480 \
--maxShapes=images:1x3x640x640 \
--fp16 \
--avgRuns=50
# Run inference for each resolution
# FP32 benchmarks
/usr/src/tensorrt/bin/trtexec --loadEngine=yolov8n_fp32.engine --shapes=images:1x3x320x320 --avgRuns=50
/usr/src/tensorrt/bin/trtexec --loadEngine=yolov8n_fp32.engine --shapes=images:1x3x480x480 --avgRuns=50
/usr/src/tensorrt/bin/trtexec --loadEngine=yolov8n_fp32.engine --shapes=images:1x3x640x640 --avgRuns=50
# FP16 benchmarks
/usr/src/tensorrt/bin/trtexec --loadEngine=yolov8n_fp16.engine --shapes=images:1x3x320x320 --avgRuns=50
/usr/src/tensorrt/bin/trtexec --loadEngine=yolov8n_fp16.engine --shapes=images:1x3x480x480 --avgRuns=50
/usr/src/tensorrt/bin/trtexec --loadEngine=yolov8n_fp16.engine --shapes=images:1x3x640x640 --avgRuns=50
# Phase 3
pip install ultralytics opencv-python
#640x640
yolo detect model=yolov8n.pt source=0 imgsz=640 device=cpu save=True
#480x480
yolo detect model=yolov8n.pt source=0 imgsz=480 device=cpu save=True
#320x320
yolo detect model=yolov8n.pt source=0 imgsz=320 device=cpu save=True