AI guided automatic CT Data Transfer Function Parametrization for Path Tracing Visualization

19.03.2025, Abschlussarbeiten, Bachelor- und Masterarbeiten

In volumetric visualization, transfer functions are usually manually parameterized to map CT scan densities to specific colors, enabling differentiation of anatomical structures in ray marching or path tracing. This project explores automating this mapping to account for variations in captured data and streamline the process.

The Human-Centered Computing and Extended Reality Lab of the Professorship for Machine Intelligence in Orthopedics seeks applicants for Bachelor/Master Thesis for the Summer Semester 2025.

Project Description

CT scans are routinely used in medical procedures, with grayscale slices serving as the standard representation. However, 3D visualizations, particularly those generated via path tracing, offer more realistic and intuitive depictions of anatomical structures. Since CT data contains only density values, an appropriate mapping from density to color is required. Fine-tuning these transfer functions for specific use cases is often time-consuming and challenging to automate. This project investigates how deep learning-based segmentation can be leveraged to drive automatic transfer function generation, reducing manual effort while improving adaptability.

Key research areas include:

• Applying deep learning for CT scan segmentation
• Exploring methods to approximate densities from segmented regions and designing user interfaces (visual or parametric) for adjustments
• Investigating techniques for automated transfer function creation and optimization
• Path tracer scripting for visualization integration

Recommended background (or motivation in learning):

• Basic knowledge of computer graphics
• Interest in visualization and path tracing
• Experience with deep learning model training and application
• Interest in working with medical data
• Some experience with C++ and Python

Please send your transcript of records, CV and motivation to: Constantin Kleinbeck (constantin.kleinbeck@tum.de) with CC to hex-thesis.ortho@mh.tum.de

Literatur

[1] D. Engel, L. Sick, and T. Ropinski, “Leveraging Self-Supervised Vision Transformers for Segmentation-based Transfer Function Design,” IEEE Trans. Visual. Comput. Graphics, pp. 1–12, 2024, doi: 10.1109/TVCG.2024.3401755.
[2] P. Ljung, J. Krüger, E. Groller, M. Hadwiger, C. D. Hansen, and A. Ynnerman, “State of the Art in Transfer Functions for Direct Volume Rendering,” Computer Graphics Forum, vol. 35, no. 3, pp. 669–691, 2016, doi: 10.1111/cgf.12934.
[3] J. Wasserthal et al., “TotalSegmentator: Robust Segmentation of 104 Anatomic Structures in CT Images,” Radiology: Artificial Intelligence, vol. 5, no. 5, p. e230024, Sep. 2023, doi: 10.1148/ryai.230024.
[4] M. Berger, J. Li, and J. A. Levine, “A Generative Model for Volume Rendering,” IEEE Trans. Visual. Comput. Graphics, vol. 25, no. 4, pp. 1636–1650, Apr. 2019, doi: 10.1109/TVCG.2018.2816059.
[5] J. Winslow, Y. Zhang, and E. Samei, “A method for characterizing and matching CT image quality across CT scanners from different manufacturers,” Med. Phys., vol. 44, no. 11, pp. 5705–5717, Nov. 2017, doi: 10.1002/mp.12554.

Kontakt: hex-thesis.ortho@mh.tum.de, constantin.kleinbeck@tum.de