SCIENTIFIC NEWS AND
INNOVATION FROM ÉTS
Blockchain Technologies for Decentralized Machine Learning - By : Zahra Batool, Kaiwen Zhang, Matthew Toews,

Blockchain Technologies for Decentralized Machine Learning


Zahra Batool
Zahra Batool Author profile
Zahra Batool is a Ph.D. student in the FUSÉE lab at the ÉTS Department of Software and IT Engineering. Her research interests include artificial intelligence, decentralized machine learning, Blockchain and mobile edge computing.

Kaiwen Zhang
Kaiwen Zhang Author profile
Kaiwen Zhang is a professor in the Department of Software and IT Engineering at ÉTS.

Matthew Toews
Matthew Toews Author profile
Matthew Toews is a professor in the Systems Engineering Department at ÉTS. His research work is related to geometry, probability theory and information theory, sound, photography, video and medical image data.

Purchased from Istockphoto. Copyright.

SUMMARY

Federated Learning (FL) operates in an iterative manner by selecting a subset of clients in each round, asking them to train using their local data. The clients then share their model with the server, which aggregates all client models into a single global model. Traditional FL approaches typically select random sets of clients for each round, which can negatively impact the quality of the trained model. Moreover, clients are often reluctant to participate in collaborative learning due to a lack of secure monetization mechanism. This article presents FL-MAB (FL-Multi-Auction using Blockchain), a client selection mechanism for FL operation in a smart contract, which rewards clients for their participation in using cryptocurrencies.

Introduction

Data generated by the Internet of Things (IoT) are often not a single entity but rather distributed among many parties. However, due to new data privacy laws and transmission overhead, it is impractical to gather all the data from IoT devices to the central cloud in conducting centralized Machine Learning model training.

Federated Learning (FL) emerged from Machine Learning and overcomes the privacy issue by eliminating the need for centralized storage and enabling the collaborative training of a shared model with distributed data. In traditional FL settings, client selection is random for training in each FL round. These randomly selected clients in FL might have heterogeneous resources, i.e. battery level and internet connectivity. Therefore, any client within a network may go offline during the model training process, which will affect the global model accuracy. Moreover, there are also some malicious clients who intentionally want to jeopardize the FL ecosystem by intimidating the global model through attacks.

The server can be biased by being involved in nepotism—endorsing its peers by selecting and rewarding them. Moreover, resource-rich clients are demotivated to participate in collaborative training due to a lack of secure monetization mechanism. Therefore, it is imperative to develop a secure, privacy-preserving FL framework that motivates resource-rich clients to participate, giving them rewards based on their contribution.

Blockchain Integration with FL

A blockchain system is a distributed tamper-proof database which provides auditability and accountability. Therefore, a blockchain-backed federated learning network promotes the federation of more devices by providing a secure FL System. The blockchain enables interaction between network entities—clients and server—and compels them to be fair by providing auditability, traceability, non-repudiation, and secure monetization as shown in Figure 1.

Figure 1: Blockchain integration with FL

A Proposed Federated Learning Protocol

The proposed client selection protocol FL-MAB consists of three phases—auction, federated learning, and reward phase. First, client selection will be performed by blockchain-based smart contracts through auctioning. Then, the clients will submit their bid with resources, i.e. total samples of local data, bandwidth, and internet connectivity. Finally, high-scoring clients will be nominated as winners by the blockchain.

In the FL phase, winners will start training their local model by using their private data. The local model hash will be saved on-chain to avoid tampering. Finally, the server will perform an aggregation to calculate the global model. The contribution of every client in the global model will be calculated in the rewarding phase. Finally, the clients will be incentivized based on their performance using cryptocurrency.

Validating the Proposed Approach

We compared our solution with Random FL, a traditional FL approach where clients are randomly selected by the server to participate in the FL rounds. Figure 2 shows that FL-MAB increases the convergence speed and accuracy of the model for the MNIST-O handwritten character recognition task after the 20th round of training compared to RandomFL.

Figure 3 also shows an increase in the model performance of the CIFAR-10 image classification task. In comparison with RandomFL, the accuracy of FL-MAB remains markedly higher over 20th  FL round of training.

Performance of the FL-MAB system

Figure 2 : MNIST—Accuracy comparison

Performance of the FL-MAB system

Figure 3: CIFAR-10 Accuracy comparison

We implemented three different methods to calculate the marginal contribution of clients, i.e., Shapley value, contribution index, and influence function. Figure 4 illustrates that our heuristic method, the Shapley-based contribution index, is approximately 34% faster as it calculates the marginal contribution of clients by approximating the coalition models.

Performance of the FL-MAB system

Figure 4: Comparison of reward distribution methods

Conclusion

We proposed a solution to promote a Federated Learning process while providing both client selection and fair monetization using smart contracts. Our experiments demonstrate that our solution reduced the training rounds by eliminating straggler clients and improved the accuracy of the model. Furthermore, our reward heuristic provides fair monetization while being significantly faster to execute than the baseline.

Additional Information

For more information on this research, please read the following paper: Batool, Z., Zhang, K., & Toews, M. (2022, April). FL-MAB: client selection and monetization for blockchain-based federated learning. In Proceedings of the 37th ACM/SIGAPP Symposium on Applied Computing (pp. 299-307).

Zahra Batool

Author's profile

Zahra Batool is a Ph.D. student in the FUSÉE lab at the ÉTS Department of Software and IT Engineering. Her research interests include artificial intelligence, decentralized machine learning, Blockchain and mobile edge computing.

Program : Information Technology Engineering 

Research laboratories : FUSÉE- Fast Unified Scalable Event processing Event messaging 

Author profile

Kaiwen Zhang

Author's profile

Kaiwen Zhang is a professor in the Department of Software and IT Engineering at ÉTS.

Program : Software Engineering  Information Technology Engineering 

Research laboratories : FUSÉE- Fast Unified Scalable Event processing Event messaging 

Author profile

Matthew Toews

Author's profile

Matthew Toews is a professor in the Systems Engineering Department at ÉTS. His research work is related to geometry, probability theory and information theory, sound, photography, video and medical image data.

Program : Automated Manufacturing Engineering 

Research laboratories : LIVIA – Imaging, Vision and Artificial Intelligence Laboratory 

Author profile


Research laboratories :

FUSÉE- Fast Unified Scalable Event processing Event messaging 

Field(s) of expertise :

Blockchains 

Get the latest scientific news from ÉTS