|
Title:
|
POWER CONSUMPTION BENCHMARK FOR EMBEDDED AI INFERENCE |
|
Author(s):
|
Zijie Ning, Maarten Vandersteegen, Kristof Van Beeck, Toon Goedemé
and Patrick Vandewalle |
|
ISBN:
|
978-989-8704-62 |
|
Editors:
|
Paula Miranda and Pedro Isaías |
|
Year:
|
2024 |
|
Edition:
|
Single |
|
Keywords:
|
AI Inference, Energy Efficiency, Embedded Systems, FPGA, ASIC |
|
Type:
|
Short |
|
First Page:
|
355 |
|
Last Page:
|
360 |
|
Language:
|
English |
|
Cover:
|
|
|
Full Contents:
|
click to dowload 
|
|
Paper Abstract:
|
This study provides an in-depth analysis of power consumption for embedded computer vision platforms, focusing on the
energy efficiency of AI inference tasks. As AI models grow in complexity and usage, the power consumption during
inference, rather than training, becomes a critical factor due to its prolonged nature. Given the multitude of different
embedded hardware architectures, it is far from trivial to make an optimal hardware choice for an embedded application at
hand. We evaluate the energy performance of MobileNetV2 and ResNet-50 on four embedded platforms: KV260, ROCK
3A, Coral Mini, and Jetson Nano, which represent the hardware architectures FPGA, ASIC, and GPU. Our measurements
are conducted using direct current input to ensure accuracy and platform independence. The results indicate that FPGA and
ASIC platforms demonstrate significantly better energy efficiency than GPU-based systems. The KV260 platform (FPGA,
Int8) consumes about five times less energy than the Jetson Nano (GPU, FP16) hardware, and Int8 quantization is at least
1.6 times more energy efficient than FP16. Moreover, surprisingly, our experiments indicate that the stand-by power of
most embedded platforms is of the same order or larger than the power consumed by running the AI model. This study
aims to guide researchers and engineers in selecting greener embedded systems for AI applications, promoting low-carbon
and environmentally responsible practices in scientific computing. |
|
|
|
|
|
|
