Contribution Evaluation

In the landscape of DeAI, federated learning stands as a pivotal approach, leveraging data contributions from various providers to enhance model performance. The efficacy of federated learning hinges on the quality of contributed data. It is a key component in incentive mechanisms how to evaluate contribution of DeAI participants.

However, the decentralized nature of federated learning introduces vulnerabilities, notably the risk of malicious actors attempting to exploit the system for undeserved rewards. These attackers may engage in various fraudulent activities, such as submitting fake, redundant, or low-quality data to inflate their rewards.

To mitigate such risks and ensure the integrity of the federated learning process, researchers have proposed diverse methods to evaluate the quality of contribution. Data Shapley is an equitable data valuation metric that quantifies the the contribution of individual data points to a learning task. Metrics such as training loss reduction and accuracy enhancement serve as pivotal benchmarks in evaluating the efficacy of participants' contributions within incentive mechanisms. These mechanisms not only incentivize data providers to offer high-quality data but also safeguard against fraudulent behavior.