| Funding Scheme : |
General Research Fund |
| Project Number : |
17203122 |
| Project Title(English) : |
Decentralized, Private and Communication-Efficient Clustered Federated Learning among Self-Motivated Nodes in Byzantine Networks |
| Project Title(Chinese) : |
拜占庭網絡中自我驅動節點之間的去中心化、私有、高通訊效率的集成聯邦學習 |
| Principal Investigator(English) : |
Dr Chan, Hubert Tsz Hong |
| Principal Investigator(Chinese) : |
|
| Department : |
School of Computing and Data Science |
| Institution : |
The University of Hong Kong |
| Co - Investigator(s) : |
|
| Panel : |
Engineering |
| Subject Area : |
Computing Science & Information Technology |
| Exercise Year : |
2022 / 23 |
| Fund Approved : |
1,086,185 |
| Project Status : |
On-going
|
| Completion Date : |
14-6-2026 |
Abstract as per original application
(English/Chinese): |
This project will investigate clustered federated learning (FL) among self-motivated nodes whose datasets may be sampled from different distributions. FL is introduced for the setting when the empirical data is partitioned among many nodes that participate in a distributed protocol to generate a common model vector, which would hopefully provide good prediction or classification accuracy on another dataset resembling the combined distribution over all nodes. Because of data privacy or communication cost, typically each node keeps its empirical dataset to itself, on which local computation is performed to produce model parameters that are communicated between the nodes.
Since the nodes may sample their datasets from very different distributions, a common model vector may not necessarily be suitable for an individual node. In this project, we consider the scenario in which each node is a self-motivated agent, whose goal is to learn a model vector that is suitable for itself. A 2021 work by Donahue and Kleinberg applied the concept of hedonic game to FL and studied how nodes are motivated to form clusters (aka coalitions) such that information is only exchanged between nodes within a cluster. One limitation of their approach is that the decisions on forming coalitions are based only on whether the reduced variance due to more sampled data from different nodes can offset the increased variance due to difference in data distributions. On the contrary, we will also consider stability notions of coalition structures that depend on information derived from the nodes’ empirical datasets, because intuitively nodes with datasets drawn from similar distributions would benefit one another by performing FL together.
Because our protocols will achieve privacy and communication efficiency, the model parameters exchanged between nodes will inherently contain random noise. In addition to investigating how these random noises influence the convergence of model vectors in the clusters, we will also consider how this will affect the stability of the resulting coalition structures. Furthermore, to apply the protocols in practical settings, we will develop reward mechanisms to incentivize nodes to cooperate. We will also employ security techniques to defend against Byzantine nodes that might attempt to disrupt the protocols. This project will contribute theoretically to the game and model convergence analysis of clustered FL and impact the practical design of secure, private and efficient FL protocols.
本課題將會研究自我驅動的節點之間的集成聯邦學習，當中這些節點的數據集可能自不同的分佈中採樣。當一個分布式協議中的經驗性數據被劃分存入協議中不同節點以生成一個公共模型向量時，引入聯邦學習將有助於爲另一個在所有節點的聯合分佈上與其相似的數據集提供一個良好的預測或良好的分類準度。出於數據隱私或通訊成本考量，通常每個節點會自我保留其經驗性數據集，在本地對其執行運算後，生成用於與其他節點進行通訊的相關模型參數。
由於不同的節點可能自非常不同的分佈中採樣，一個公共模型向量未必適用於單個節點。在本課題中，我們將會考慮這樣一個情景：每個節點都是一個自我驅動的代理人，而其目標是通過學習得到一個適用於自身的模型向量。Donahue 與 Kleinberg 於 2021 年的一項工作將享樂博弈的概念應用於聯邦學習，並研究不同節點是如何被驅動形成集羣（亦稱聯盟），從而使得信息只會在同一個集羣的節點之間進行交換。該研究的一個局限在於，其關乎是否形成聯盟的決定僅基於一個因素：自不同節點中採樣更多數據，從而導致的方差減少是否能夠抵消數據分佈差異導致的方差增加。與之相反，直觀上看，對保有採樣自相似分佈數據集的節點，讓它們共同執行聯邦學習將可使其互相受益，爲此，我們還將考慮聯盟結構的穩定性概念，而該結構依賴從節點的經驗性數據集中得到的信息。
由於我們的協議會實現隱私和高通訊效率，節點之間交換的模型參數將會固有地帶有隨機噪聲。除了探究這些隨機噪聲會如何影響集羣模型向量的收斂性外，我們亦會考慮其將如何影響得到的聯盟結構的穩定性。此外，爲了能夠在實際環境中應用該協議，我們將研發獎勵機制，以激勵節點之間進行合作。我們亦會應用不同的安全技術，以抵御可能會嘗試破壞協議的拜占庭節點。本課題將爲集成聯邦學習的博弈和模型收斂分析提供理論貢獻，並影響安全、私有和高效的聯邦學習協議的實際設計。
|
| Research Outcome |
Layman's Summary of
Completion Report: |
Not yet submitted
|
