| Project Detail |
Developing faster classic learning algorithms
The advancement of machine learning technology has brought unparalleled benefits to various sectors, facilitating improved and more accessible automation. Presently, learning algorithms are categorised into deep learning, which excels in environments abundant with data, and classic algorithms, which are better suited for data-scarce environments. In this context, the ERC-funded TUCLA project seeks to delve into classic algorithms and enhance their speed and efficacy. The project’s main focus is on studying fundamental algorithms like bagging and boosting to establish a learning theoretical framework for designing innovative boosting and bagging algorithms.
Machine learning has evolved from being a relatively isolated discipline to have a disruptive influence on all areas of science, industry and society. Learning algorithms are typically classified into either deep learning or classic learning, where deep learning excels when data and computing resources are abundant, whereas classic algorithms shine when data is scarce. In the TUCLA project, we expand our theoretical understanding of classic machine learning, with a particular emphasis on two of the most important such algorithms, namely Bagging and Boosting. As a result of this study, we shall provide faster learning algorithms that require less training data to make accurate predictions. The project accomplishes this by pursuing several objectives:
1. We will establish a novel learning theoretic framework for proving generalization bounds for learning algorithms. Using the framework, we will design new Boosting algorithms and prove that they make accurate predictions using less training data than what was previously possible. Moreover, we complement these algorithms by generalization lower bounds, proving that no other algorithm can make better use of data.
2. We will design parallel versions of Boosting algorithms, thereby allowing them to be used in combination with more computationally expensive base learning algorithms. We conjecture that success in this direction may lead to Boosting playing a more central role also in deep learning.
3. We will explore applications of the classic Bagging heuristic. Until recently, Bagging was not known to have significant theoretical benefits. However, recent pioneering work by the PI shows that Bagging is an optimal learning algorithm in an important learning setup. Using these recent insights, we will explore theoretical applications of Bagging in other important settings. |
