Face Anonymization and Human Pose Estimation in the Operating Room

07.08.2024, Abschlussarbeiten, Bachelor- und Masterarbeiten

Overview

This project aims to anonymize people in an operating room, but keep face and emotions visible on the video stream. Moreover, the people in the operating room should be tracked full body to enable avatar like overlays or full body deepfakes in future work.

Background & Motivation

Surgery videos are essential for analyzing surgeries, teaching and performance measures. However, for privacy reasons the persons in the operating room should be anonymized. Moreover, movements of the people in the operating room are essential for phase detection, scene graphs etc.. Deep fake approaches enable face anonymization, however, they usually capture forward looking people instead of crowded operating room situations. Considering different angles for the capturing process of the face as well as multiple faces and occluded parts of the face. Additionally, full body tracking allows complete anonymization by overlaying avatars.

Student’s Task

The student should apply and extend state-of-the-art face anonymization and full body-tracking for OR recordings/live feed. The individual person shall keep the same anonymized face over the video by keeping face expressions present. In case of a bachelor thesis/guided research project we can focus on either the face anonymization or full body tracking. https://github.com/wngTn/disguisor

Technical Prerequisites

The student should have some experience in data processing and deep learning. Knowledge in 3D computer vision is beneficial. Additional knowledge in terms of different sensor types etc are beneficial as well but not mandatory.

Please send your transcript of records, CV and motivation to: Hannah Schieber (hannah.schieber@tum.de) with CC to hex-thesis.ortho@mh.tum.de

Literature
[1] Bastian, L., Wang, T. D., Czempiel, T., Busam, B., & Navab, N. (2023). DisguisOR: holistic face anonymization for the operating room. International Journal of Computer Assisted Radiology and Surgery, 18(7), 1209-1215.
[2] Hintz, J., Rühe, J., & Siegert, I. AnonEmoFace: Emotion Preserving Facial Anonymization.
[3] Rosberg, F., Aksoy, E. E., Englund, C., & Alonso-Fernandez, F. (2023). FIVA: Facial Image and Video Anonymization and Anonymization Defense. In Proceedings of the IEEE/CVF International Conference on Computer Vision (pp. 362-371).
[4] Barattin, S., Tzelepis, C., Patras, I., & Sebe, N. (2023). Attribute-preserving face dataset anonymization via latent code optimization. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (pp. 8001-8010).
https://github.com/bastianlb/segmentOR https://github.com/wngTn/disguisor Kadkhodamohammadi, A., Gangi, A., de Mathelin, M., & Padoy, N. (2017, March). A multi-view RGB-D approach for human pose estimation in operating rooms. In 2017 IEEE winter conference on applications of computer vision (WACV) (pp. 363-372). IEEE.
Srivastav, V., Issenhuth, T., Kadkhodamohammadi, A., de Mathelin, M., Gangi, A., & Padoy, N. (2018). MVOR: A multi-view RGB-D operating room dataset for 2D and 3D human pose estimation. arXiv preprint arXiv:1808.08180.

Kontakt: hannah.schieber@tum.de, hex-thesis.ortho@mh.tum.de