German Federal Ministry of Education and Research is funding two AI junior research groups at Paderborn University
Artificial intelligence (AI) is one of the key technologies of the future. Whether language assistants, smart homes or pandemic forecasting, AI helps us in many areas and is an inextricable part of our everyday lives. Research in this area is more important than ever in order to fully understand rapid technical developments, further develop technology in a targeted fashion, and put it to practical use. Since September, two new AI junior research groups at Paderborn University have been examining how to optimise and more effectively deploy machine learning. The young researchers’ aim is firstly to use the combination of machine learning and expert knowledge to improve the quality of dynamic system models, and secondly to make training deep neural networks using ‘multi-objective optimisation’ models a more robust, efficient and interactive process. The Federal Ministry of Education and Research (BMBF) is funding both groups for three years with a total of approximately 1.8 million euros.
The two teams are planning to work closely together in order to pool their AI skills as effectively as possible. Paderborn offers the ideal infrastructure for this: thanks to the Paderborn Center for Parallel Computing (PC2), researchers have access to state-of-the-art HPC (High Performance Computing) hardware for their functional and demonstration tests. At the end of the project, the junior research groups’ methods, software tools and data will be made available free of charge as open-access material to a wide community of users.
Combining the benefits of expert knowledge and data for modelling
From temperature estimates in electric motors through to forecasting COVID-19 spread dynamics or the unemployment rate, there are numerous dynamic systems in engineering, economics, social sciences, physics, biology, chemistry and medicine that can be described using mathematical models. However, modelling these complex, dynamic systems is often a challenge. “In the past there has been a clear trend away from expert models towards black-box models developed using machine learning. Both have their own particular pros and cons”, explains Dr.-Ing. Oliver Wallscheid of Paderborn’s Department of Electrical Engineering and Information Technology and head of one of the new AI junior research groups. The aim of the researchers working with Wallscheid is to combine the benefits of expert-driven and data-driven modelling in order to significantly improve model accuracy in terms of precision, robustness and complexity for numerous applications. As part of the ‘Automated modelling and model validation of dynamic systems using machine learning and prior expert knowledge’ project (ML-Expert), they are examining how fundamentally different modelling paradigms – developed by experts or by AI – could be combined in a hybrid form of modelling.
Wallscheid explains the issues experienced in the past: “Whilst expert-based approaches reflect system behaviour in a robust, interpretable way, these often take a lot of time and staff resources and also include systematic deviations based on simplifications. Black-box models (i.e. data-based models), on the other hand, can sometimes be generated quickly without any significant prior knowledge. But although this produces precise and scalable models, they are often difficult to interpret and are not robust as regards outliers.”
Efficient, resource-oriented data generation, modelling and model validation should enable quicker development and application cycles in the future. “This offers significant added value given the increasing skills shortage, in particular for cost-relevant industrial applications such as the automotive, energy or automation sectors. However, our work should not be limited to these sectors, but rather take an interdisciplinary approach”, Wallscheid emphasises.
Ideal compromise solutions for machine learning techniques
Machine learning techniques also form part of numerous other applications. For example, deep neural networks (DNNs for short) enable intelligent image recognition or language processing. The rise in available capacity enables the construction of increasingly large, deep and complex DNNs. This progress also brings with some additional challenges: ideally, the construction and training of DNNs should achieve various, sometimes contradictory objectives as effectively as possible. “Unlike many technological and social sectors where taking multiple criteria into account is a matter of course, cancer treatment for example, thus far the huge potential of a multi-objective approach in machine learning has remained largely untapped”, explains Jun.-Prof. Dr. Sebastian Peitz of the Department of Computer Science. The head of the new AI junior research group is seeking to change this with the project entitled ‘Multicriteria machine learning – efficiency, robustness, interactivity and system knowledge’ (MultiML). The aim is to develop multi-objective optimisation methods in order to make DNN training more robust, efficient and interactive and thus significantly improve it. Furthermore, the additional incorporation of system knowledge should enable the construction of extremely efficient methods tailored to specific problems.
Peitz cites a few examples: “In virtually all areas of technology, business and society, there is the dilemma of similarly important yet competing criteria: electric vehicles need to drive quickly and have a large range, a product to be manufactured needs to offer high quality and low production costs, and political decision-making requires examination of both economic and environmental aspects.” The ongoing challenge: identifying and selecting ideal compromise solutions, known as ‘Pareto optima’. ‘Machine learning also involves numerous criteria that all need to be met as effectively as possible, such as robustness in the face of incomplete input data, generalisation beyond training data, or optimum compliance with physical laws’, the computer scientist continues.
Developing multi-objective optimisation methods for machine learning should enable algorithms that determine the set of ideal compromises in a highly efficient way. “Among other things, an awareness of all compromise solutions enables users to make very well-informed and conscious trade-offs and adapt learning techniques to the situation by reprioritising individual objectives”, Peitz explains.
