Master's Thesis: Infrastructure Support for Black-Box End-to-End Autonomous Driving – Strategies, Scenarios and Benchmarking

05.10.2026, Abschlussarbeiten, Bachelor- und Masterarbeiten

The Chair of Robotics, Artificial Intelligence and Real-Time Systems offers a thesis on supporting vehicle end-to-end (E2E) driving systems with roadside infrastructure perception. The vehicle's driving function is treated as a closed black box. The thesis identifies the situations in which infrastructure knowledge improves driving safety and the integration strategies through which it can do so without access to the model's internals.

Motivation

E2E driving models map sensor data directly to trajectories and are becoming the dominant paradigm in industry. In production vehicles, these systems are typically closed: their internal representations can be neither accessed nor modified. Existing cooperative-perception approaches for E2E driving mostly fuse infrastructure data inside the model and rely on joint training, which is not possible for closed systems.

Urban intersections remain a weak spot for vehicle-only perception. Typical cases are pedestrians occluded by trucks, hidden cross traffic, and sensors degraded by weather. Roadside units observe exactly these situations and can share their perception via standardized V2X messages. A closed E2E system can benefit from this knowledge through two routes:

• Output side: infrastructure acts as a runtime checker that evaluates the vehicle's planned trajectory and constrains it when necessary. This checker architecture is also attractive from a functional safety perspective.
• Input side: infrastructure knowledge is conveyed through interfaces the driving model already supports.

Both routes come with different trade-offs between safety benefit, false interventions, comfort and robustness to communication imperfections. A systematic comparison of these trade-offs across scenarios has not yet been carried out.

Goal

The thesis develops and benchmarks infrastructure support strategies for black-box E2E driving in closed-loop CARLA simulation. The result is a systematic map of the scenarios, infrastructure configurations and communication conditions in which each strategy improves safety. It also yields design recommendations for infrastructure support of closed driving systems.

Approach

Pretrained E2E models, for example from Bench2DriveZoo, are used strictly as black boxes. Only their inputs and outputs are accessed, and no driving model is trained.

Infrastructure perception is provided at two levels. The first is an idealized level based on perfect knowledge of the infrastructure's field of view. The second is a realistic level using standardized V2X messages, with ETSI CAM/CPM as the reference format, including latency, packet loss, localization error and limited coverage. All strategies are evaluated against the stand-alone vehicle system in a scenario suite of safety-critical situations.

Your tasks

• Review the literature on E2E driving, cooperative V2X perception and runtime safety monitoring
• Set up closed-loop evaluation of pretrained E2E agents in CARLA
• Design a scenario suite of safety-critical situations, such as occluded vulnerable road users, unprotected turns and hidden cross traffic, with varying infrastructure coverage and environmental conditions
• Integrate infrastructure perception at idealized and realistic V2X levels, including a configurable communication model
• Implement an infrastructure-based trajectory checker with graded intervention (warn, constrain, veto)
• Implement at least one input-side support strategy
• Evaluate safety benefit, false-intervention rate and robustness
• Derive design recommendations

Related work

Closed-loop evaluation and pretrained E2E models:

• Bench2Drive: https://github.com/Thinklab-SJTU/Bench2Drive
• Bench2DriveZoo: https://github.com/Thinklab-SJTU/Bench2DriveZoo
• Bench2Drive-Robust: https://github.com/Thinklab-SJTU/Bench2Drive-Robust
• CARLA Garage: https://github.com/autonomousvision/carla_garage

Cooperative E2E driving:

• UniV2X: https://arxiv.org/abs/2404.00717
• V2X-VLM: https://arxiv.org/abs/2408.09251
• OmniV2X: https://arxiv.org/abs/2606.21165

Runtime safety:

• Argus: https://arxiv.org/abs/2511.09032
• Safety monitor using forward reachable sets: https://arxiv.org/abs/2507.22389
• Real-time safeguarding of motion planners (TUM): https://arxiv.org/abs/2507.07444
• Responsibility-Sensitive Safety (RSS): https://arxiv.org/abs/1708.06374

V2X communication:

• ETSI ITS ROS2 Messages: https://github.com/ika-rwth-aachen/etsi_its_messages

Your profile

• Master's student in Computer Science, Robotics, Electrical Engineering or a related field
  
• Strong interest in autonomous driving, safety and cooperative perception
• Experience with CARLA, ROS2, motion planning or V2X communication is a plus

What you will gain

• Hands-on experience with state-of-the-art E2E driving models and closed-loop benchmarking
• Expertise in runtime safety monitoring for learned driving systems
• Practical knowledge of standardized V2X-based cooperative perception
• Insight into an industry-relevant topic at the interface of AI, safety and intelligent infrastructure
• The opportunity to contribute to a scientific publication

How to apply

Please send your CV and transcript of records to erik-leo.hass@tum.de.

Kontakt: erik-leo.hass@tum.de