Hybrid Reinforcement Learning with Baseline Controllers

28.03.2025, Abschlussarbeiten, Bachelor- und Masterarbeiten

Background

Reinforcement learning (RL) has shown great promise in industrial automation, but its real-world applicability is often limited by key challenges. The exploration phase can lead to unsafe and erratic behavior, making direct deployment impractical. Additionally, accurate simulation models for complex systems are often unavailable, and even when simulations are feasible, the simulation-to-reality gap remains a significant obstacle. These challenges frequently drive practitioners toward conventional control methods, which are not always the optimal solution.

This project aims to develop hybrid methods that integrate reinforcement learning with conventional control approaches to improve the control of dynamic systems in industrial settings.

Possible Research Directions

The specific focus of the project can be tailored based on the student's interests and expertise. Potential directions include:

• RL Algorithm Deployment & Evaluation
  • Implement and evaluate existing RL-based control strategies in industrial simulations.
  • Explore novel algorithmic improvements to enhance safety and robustness.
• Simulation Development
  • Design and build physics-based simulations of industrial automation systems.
  • Work with Python or Unity to create realistic environments.

What We Offer

• The opportunity to contribute to cutting-edge research at the intersection of academia and industry, in collaboration with Siemens AG.
• A collaborative research environment with expertise in AI, industrial automation, and optimization.
• Flexibility in defining the project scope based on your interests.

Preferred Qualifications

• Strong programming skills in Python, ideally with experience in reinforcement learning or machine learning libraries (e.g., TensorFlow, PyTorch, Stable Baselines3).
• Basic knowledge of control theory and familiarity with RL concepts.
• (Optional) Experience with uncertainty quantification techniques.
• (Optional) Familiarity with Unity for simulation development.

How to Apply

Send an email to olivia.garland@tum.de with the following:

• CV
• A brief introduction outlining your background and motivation, as well as which area of the project you are interested in
• Academic Transcripts

References

1. Tian, Haozhe, et al. "Reinforcement Learning with Adaptive Regularization for Safe Control of Critical Systems." arXiv preprint arXiv:2404.15199 (2024).
2. Zhang, Haichao, We Xu, and Haonan Yu. "Policy expansion for bridging offline-to-online reinforcement learning." arXiv preprint arXiv:2302.00935 (2023).
3. Körwer, Niklas, et al. "AI-Based Motion Control with System Dynamics Flexibility." 2024 Energy Conversion Congress & Expo Europe (ECCE Europe). IEEE, 2024.
4. Gassert, Philipp, and Matthias Althoff. "Stepping Out of the Shadows: Reinforcement Learning in Shadow Mode." arXiv preprint arXiv:2410.23419 (2024).

Kontakt: olivia.garland@tum.de