|
1 |
Introduction
AI is moving towards edge devices with the availability of massively distributed data sources and the increase in computing power for handheld and wireless devices such as smartphones or self-driving cars. This has generated growing interest to develop large-scale distributed machine learning paradigms. However, traditional learning approaches cannot be readily applied for a large-scale distributed learning system. In this regard, we propose a novel design philosophy called democratized learning (Dem-AI) whose goal is to build large-scale distributed learning systems that rely on the self-organization of distributed learning agents that are well-connected, but limited in learning capabilities. Correspondingly, inspired from the societal groups of humans, the specialized groups of learning agents in the proposed Dem-AI system are self-organized in a hierarchical structure to collectively perform learning tasks more efficiently. As such, the Dem-AI learning system can evolve and regulate itself based on the underlying duality of two processes that we call specialized and generalized processes.
Inspired by the hierarchical social structures in existing organizations, the Dem-AI system aims to provide unique features of democracy in a distributed learning system as follows:
- Learning agents are self-organized into appropriate hierarchical groups based on their learning characteristics. This voluntary process mediates contributions from all members in the collaborative learning for solving multiple complex tasks and building corresponding hierarchical generalized knowledge.
- The shared hierarchical generalized learning knowledge supports these learning agents to improve their learning performance by reducing individual biases. As a result, the new learning agents can speed up their learning process with the existing group knowledge, and further contribute to expanding the generalization capability of whole groups.
Research Challenges
- Edge-Assisted Distributed and Democratized Learning
- Specialized Learning and Generalization mechanism in Democratized Learning
- Hierarchical self-organization mechanism in Democratized Learning
- Multi-task Democratized Learning
References
- Minh N. H. Nguyen, Shashi Raj Pandey, Kyi Thar, Nguyen H. Tran, Mingzhe Chen, Walid Saad, Choong Seon Hong, “Distributed and Democratized Learning: Philosophy and Research Challenges,” Submitted to IEEE Computational Intelligence Magazine (Major Revision).
- Minh N. H. Nguyen, Shashi Raj Pandey, Tri Nguyen Dang, Eui-Nam Huh, Choong Seon Hong, Nguyen H. Tran, and Walid Saad. "Self-organizing Democratized Learning: Towards Large-scale Distributed Learning Systems." arXiv:2007.03278 (2020).
|
