Optimizing Deep Neural Network Inference on Edge TPU Devices through Integer Quantization Protocols

International Journal of Computer Science and Technology | Vol. 4, No. 8, August 2023 | pp. 1–8

DOI: 10.46882/2023/IJCST/000223

Article Type: Original Research Paper

Title: Optimizing Deep Neural Network Inference on Edge TPU Devices through Integer Quantization Protocols

Names of Authors: Amina Al-Mansoor¹, Dieter Schmidt²

Authors’ Affiliations: ¹Department of Computer Engineering, King Fahd University of Petroleum and Minerals, Dhahran, Saudi Arabia; ²Institute for Computer Architecture, Technical University of Munich, Munich, Germany

Abstract: Running complex object detection networks on localized edge hardware remains limited by the high memory requirements and power consumption profiles of floating-point processing layers. This research presents an integer quantization protocol designed to compress deep learning networks for edge-layer Tensor Processing Unit (TPU) deployments. Our methodology implements an optimized post-training quantization algorithm that maps 32-bit floating-point parameters (FP32) into 8-bit integer formats (INT8). The system uses a dynamic scaling factor calculation across intermediate network layers to minimize quantization noise accumulation. We evaluated this compression architecture using MobileNet-V3 and YOLO-V5 models running on embedded edge TPU modules. The experimental measurements indicate a 72.3% reduction in the total memory space required by the neural model. The total frame inference rate increased by 2.8x, achieving a processing latency of 14 ms per image frame. Crucially, the overall classification accuracy degraded by only 0.42% compared to the original uncompressed model framework.

Keywords: Edge TPU, Model Compression, Integer Quantization, Deep Neural Networks, Computer Vision, Resource Optimization

Manuscript Timeline: Received: February 22, 2023; Revised: April 15, 2023; Accepted: May 25, 2023; Published: August 14, 2023

Citation: Al-Mansoor, A., & Schmidt, D. (2023). Optimizing Deep Neural Network Inference on Edge TPU Devices through Integer Quantization Protocols. International Journal of Computer Science and Technology, 4(8), 1–8. DOI: 10.46882/2023/IJCST/000223

Subscribe to read the full article: https://internationalscholarsjournals.org/subscribe-to-read