The next PGR seminar is taking place this Friday 4th April at 2PM in JC 1.33a
Below are the Titles and Abstracts for Zihan and Berné’s talks – Please do come along if you are able.
Zihan Zhang
Title: FedOptima: Optimizing Resource Utilization in Federated Learning
Abstract: Federated learning (FL) systems facilitate distributed machine learning across a server and multiple devices. However, FL systems have low resource utilization limiting their practical use in the real world. This inefficiency primarily arises from two types of idle time: (i) task dependency between the server and devices, and (ii) stragglers among heterogeneous devices. We propose FedOptima, a resource-optimized FL system designed to simultaneously minimize both types of idle time; existing systems do not eliminate or reduce both at the same time. FedOptima offloads the training of certain layers of a neural network from a device to server using three innovations. First, devices operate independently of each other using asynchronous aggregation to eliminate straggler effects, and independently of the server by utilizing auxiliary networks to minimize idle time caused by task dependency. Second, the server performs centralized training using a task scheduler that ensures balanced contributions from all devices, improving model accuracy. Third, an efficient memory management mechanism on the server increases scalability of the number of participating devices. Four state-of-the-art offloading-based and asynchronous FL methods are chosen as baselines. Experimental results show that compared to the best results of the baselines on convolutional neural networks and transformers on multiple lab-based testbeds, FedOptima (i) achieves higher or comparable accuracy, (ii) accelerates training by 1.9x to 21.8x, (iii) reduces server and device idle time by up to 93.9% and 81.8%, respectively, and (iv) increases throughput by 1.1x to 2.0x.
Berné Nortier
Title: Shortest paths and optimal transport in higher-order systems
Abstract: One of the defining features of complex networks is the connectivity properties that we observe emerging from local interactions. Nevertheless, not all networks describe interactions which are merely pairwise. Recently, different frameworks for modelling non-dyadic, higher-order, interactions have been proposed, garnering much attention. Of these, hypergraphs have emerged as a versatile and powerful tool to model such higher-order networks. However, the connectivity properties of real-world hypergraphs remain largely understudied. A first, data-driven, work introduces a measure to characterise higher-order connectivity and quantify the relevance of non-dyadic ties for efficient shortest paths in a diverse set of empirical networks with and without temporal information. The analysis presents a nuanced picture.
A second work (in progress) considers higher-order simplicial networks within the context of optimal transport, where shortest paths do not always lead to optimal resource allocation. We extend the existing framework to the higher-order setting to explore to what degree this additional degree of freedom influences the flux of resources in a system of interest.