MA/BA: Developing AI Agents with Context-Awareness for Augmented Reality

LLM-based agent being augmented into the room

13.02.2026, Diplomarbeiten, Bachelor- und Masterarbeiten

The vision behind the EnvSLAM project, is expanding Augmented Reality with spatial context. For example, smart characters (so-called LLM-based agents) can understand their environment and react towards it.

A second iteration of EnvSLAM focused on its integration into Unity. This means, we can run C++ code inside of the game engine and access OpenCV and other libraries. The result is a framework that combines the spatial understanding of ORB-SLAM3 with the classification through a segmentation neural network or an object detector, in this case Yolo11n. This framework then can be deployed on mobile devices, such as smartphones.

In the next steps, we want to expand this framework with a data structure to allow agents to access the spatial context and built scenarios for user studies.

You need to be a student at the School of Computation, Information and Technology (CIT), e.g., Computer Science, Games Engineering, Robotics, or others. Having experience with Unity or another game engine is a plus. Additionally, you need to be not afraid to work with C++ applications. There is time to catch up, by having some buffer for preparations and learning before registering the thesis.

If you are interested, feel free to contact us via mail: christian.eichhorn@tum.de and jonas.weigand@tum.de

Kontakt: Christian Eichhorn: christian.eichhorn@tum.de, Jonas Weigand: jonas.weigand@tum.de

More Information

2

The segmentation neural network in use., (Type: image/png, Size: 153.6 kB) Save attachment