Research Projects on Mid-Training of Large Language Models

18.07.2025, Abschlussarbeiten, Bachelor- und Masterarbeiten

Are you curious about the training paradigm of Large Language Models (LLMs)? Are you excited by recent breakthroughs in mid-training techniques that go beyond standard pretraining-finetuning paradigms? This project aims to enhance the reasoning ability of LLMs by exploring the mid-training phase of LLMs, which is an emerging research direction that bridges the gap between general pretraining and task-specific finetuning. Are you ready to explore this new research direction that has great potential? We are interested in supervising talented students and publishing high-quality publications with practical applications. Our lab is led by Prof. Volker Tresp from LMU, who is one of PIs of a European Laboratory for Learning and Intelligent Systems (ELLIS) fellowship program and the first distinguished research scientist at Siemens Corporate Technology, and PI of Munich Center of Machine Learning (MCML). We have already published several high-quality publications at top international conferences (Neurips, ICLR, ACL, EMNLP, NAACL, AAAI, UAI, CIKM, etc.). During this thesis project, you can collaborate with many talented PhDs. You will have the opportunity to develop advanced research skills while exploring the promising research direction of mid-training for LLMs.

Topic Description:
Concrete topics include Mid-Training Paradigms, Large Language Models, LLM Reasoning etc.

Requirements:
● Study (Bachelor/Master) in computer science, electrical engineering, physics, or mathematics;
● Good understanding of machine learning and deep learning;
● Good understanding of transformer and LLMs as a plus;
● Good programming skills in Python and at least one deep learning framework;
● Strong motivation and independent working

Additional Information:
Start: from now on
Length of the work: 6-9 months

Contact information:
If you are interested, please contact mengyue.wang@tum.de

We are looking forward to hearing from you!

Kontakt: mengyue.wang@tum.de