ASYNCHRONOUS CENTRALIZED STREAMING DATA PROCESSING IN DISTRIBUTED REAL-TIME SYSTEMS
DOI:
https://doi.org/10.17721/3041-2323.2025.122-136Keywords:
asynchronous federated learning, distributed real-time systems, centralized coordination, neural network weight aggregation, FedAvg, FedAsync, robust aggregation, FedAdam, FedYogi, update ageAbstract
This paper presents a structured and pragmatic analysis of several aggregation methods for asynchronous centralized federated learning: classical federated averaging (FedAvg), asynchronous gradient/parametric advancement (FedAsync-style methods), robust aggregation based on coordinate-weighted medians, and adaptive optimizers such as FedAdam and FedYogi. The study focuses on their suitability under conditions of heterogeneous client time, update staleness, non-IID (independently and identically distributed) data partitions, and the limited real-time created in distributed real-time systems. The sensitivity analysis of the methods to delayed and out-of-date updates is performed, and the communication and computational costs at the central node are also refined. A semi-hypothetical, realistic experimental study is conducted using non-IID datasets with failure modes including node shutdown and system time drift. Comparative findings from the study showed that FedAvg degrades sharply with high age and skewed participation; reliable aggregation can unexpectedly amplify the impact of age-old but structurally consistent ages; and adaptive methods exhibit a nontrivial tension between rapid convergence and instability when delay patterns change over time.
References
P. Blanchard, E. M. El Mhamdi, R. Guerraoui, and J. Stainer, “Machine learning with adversaries: Byzantine tolerant gradient descent,” in Proc. NIPS, 2017.
G. C. Buttazzo, Hard Real-Time Computing Systems: Predictable Scheduling Algorithms and Applications, 3rd ed. Springer, 2017.
T. Chen, G. Giannakis, T. Sun, and W. Yin, “LAG: Lazily aggregated gradient for communication-efficient distributed learning,” IEEE Trans. Signal Process., vol. 68, pp. 4365–4379, 2020.
P. Kairouz et al., “Advances and open problems in federated learning,” Foundations and Trends in Machine Learning, vol. 14, no. 1–2, pp. 1–210, 2021.
H. Kopetz, Real-Time Systems: Design Principles for Distributed Embedded Applications, 2nd ed. Springer, 2011.
H. B. McMahan, E. Moore, D. Ramage, S. Hampson, and B. A. y Arcas, “Communication-efficient learning of deep networks from decentralized data,” in Proc. AISTATS, 2017 (arXiv:1602.05629, 2016).
S. Reddi et al., “Adaptive federated optimization,” in Proc. ICLR, 2021.
P. Tabuada, “Event-triggered real-time scheduling of stabilizing control tasks,” IEEE Trans. Autom. Control, vol. 52, no. 9, pp. 1680–1685, 2007.
C. Xie, S. Koyejo, and I. Gupta, “Asynchronous federated optimization,” arXiv:1903.03934, 2019.
D. Yin, Y. Chen, R. Kannan, and P. Bartlett, “Byzantine-robust distributed learning: Towards optimal statistical rates,” in Proc. ICML, 2018.
Downloads
Published
Issue
Section
License
Copyright (c) 2025 Applied information systems and technologies in the digital society
This work is licensed under a Creative Commons Attribution 4.0 International License.
