Foundation Models Modulo Formal Methods and Uncertainty Quantification
Abstract -- The rapid advancement of multimodal foundation models in language and vision has opened new possibilities for autonomous systems. These models offer broad perceptual and reasoning capabilities, but integrating them into the design and deployment of autonomous agents remains a challenge. Their outputs often lack guarantees, may misalign with domain-specific tasks, and introduce uncertainty that can compromise safety and reliability. In this talk, I explore how foundation models can be adapted and refined to meet the stringent requirements of autonomy by combining them with tools from formal methods and uncertainty quantification. I will discuss a method for automatically fine-tuning pre-trained language models for control tasks using formal specification-guided synthesis of verifiable automaton-based controllers, without relying on human feedback. I will also present a framework for disentangling and mitigating uncertainty in multimodal planning pipelines by separating perceptual and decision uncertainty, applying conformal prediction, and introducing a formal-methods-driven quantification technique. These approaches, validated in simulated and real-world robotic scenarios, show that principled adaptation of foundation models—modulo formal verification and calibrated uncertainty—can significantly improve both task performance and safety guarantees.
Bio -- Ufuk Topcu is a Professor in the Department of Aerospace Engineering and Engineering Mechanics at The University of Texas at Austin. He is a core faculty member of the Oden Institute for Computational Engineering and Sciences, Texas Robotics, and Machine Learning Laboratory, and he directs the Autonomous Systems Group. His research lies at the intersection of formal methods, reinforcement learning, and control theory, with a focus on the theoretical and
algorithmic foundations for the design and verification of autonomous systems. He is a recipient of the IEEE Control System Society Antonio Ruberti Young Researcher Prize.