EngageCam: Personalizing Engagement Detection in Online Learning with Generative AI – IDP/Master’s thesis

11.04.2025, Abschlussarbeiten, Bachelor- und Masterarbeiten

Description:
Are you passionate about advancing AI in online learning? Join the EngageCam project to explore cutting-edge techniques for personalized engagement detection.

In this project, you'll:

• Artificially augment datasets by generating synthetic images/videos using Generative AI models, such as:
  • StarGAN-v2
  • FOMM (website), Avatarify (GitHub)
  • Diffusion Motion Transfer, Awesome Video Diffusion
  • Viggle
• Evaluate model performance through advanced methods, such as similarity measures and engagement/emotion detection using trained ML models.

Dive into personalization by:

• Leveraging federated learning to improve engagement detection accuracy for unseen users, starting with just one image per unseen user.
• Identifying and utilizing the 3 most similar participants from the dataset (e.g., via k-nearest neighbors, triplet loss) to fine-tune the model for tailored engagement analysis.

Outcome:
Develop a robust, personalized engagement detection system that adapts to unique user needs in online learning environments.

Start Date: immediately

Datasets:
EngageNet, DAiSEE (video datasets of users participating in online learning – 10s long videos)

Skills Required:

• Advanced knowledge of Machine Learning: Experience with training and evaluating ML models.
• Proficiency in Python: Expertise in using ML libraries such as PyTorch or scikit-learn.

Good to have: Experience with Generative AI: Knowledge of techniques for synthetic data generation, such as GANs or diffusion models.

How to apply: Please write an e-mail to us with a short introduction, CV, and transcripts.

Kontakt: anna.bodonhelyi@tum.de, mengdi.wang@tum.de