IRUS Total

Model pruning enables efficient federated learning on edge devices

File Description SizeFormat 
Model-Pruning-FL-2022.pdfAccepted version6.84 MBAdobe PDFView/Open
Title: Model pruning enables efficient federated learning on edge devices
Authors: Jiang, Y
Wang, S
Valls, V
Bong Jun, K
Wei-Han, L
Leung, K
Tassiulas, L
Item Type: Journal Article
Abstract: Federated learning (FL) allows model training from local data collected by edge/mobile devices while preserving data privacy, which has wide applicability to image and vision applications. A challenge is that client devices in FL usually have much more limited computation and communication resources compared to servers in a data center. To overcome this challenge, we propose PruneFL--a novel FL approach with adaptive and distributed parameter pruning, which adapts the model size during FL to reduce both communication and computation overhead and minimize the overall training time, while maintaining a similar accuracy as the original model. PruneFL includes initial pruning at a selected client and further pruning as part of the FL process. The model size is adapted during this process, which includes maximizing the approximate empirical risk reduction divided by the time of one FL round. Our experiments with various datasets on edge devices (e.g., Raspberry Pi) show that: 1) we significantly reduce the training time compared to conventional FL and various other pruning-based methods and 2) the pruned model with automatically determined size converges to an accuracy that is very similar to the original model, and it is also a lottery ticket of the original model.
Issue Date: 25-Apr-2022
Date of Acceptance: 1-Apr-2022
URI: http://hdl.handle.net/10044/1/96730
DOI: 10.1109/TNNLS.2022.3166101
ISSN: 1045-9227
Publisher: Institute of Electrical and Electronics Engineers
Start Page: 1
End Page: 13
Journal / Book Title: IEEE Transactions on Neural Networks and Learning Systems
Copyright Statement: © 2022 IEEE. Personal use of this material is permitted. Permission from IEEE must be obtained for all other uses, in any current or future media, including reprinting/republishing this material for advertising or promotional purposes, creating new collective works, for resale or redistribution to servers or lists, or reuse of any copyrighted component of this work in other works. See http://www.ieee.org/publications_standards/publications/rights/index.html for more information.
Sponsor/Funder: IBM United Kingdom Ltd
Funder's Grant Number: PO 4603 458 249
Keywords: Science & Technology
Computer Science, Artificial Intelligence
Computer Science, Hardware & Architecture
Computer Science, Theory & Methods
Engineering, Electrical & Electronic
Computer Science
Computational modeling
Data models
Adaptation models
Collaborative work
Distributed databases
Efficient training
federated learning (FL)
model pruning
Artificial Intelligence & Image Processing
Publication Status: Published online
Appears in Collections:Computing
Electrical and Electronic Engineering