Designing a Neural Control Barrier Function for Physical Systems Using Counterfactual Reasoning

Abstract

Safety certifications are crucial for complex robotic systems operating in diverse environments alongside humans. Current approaches often tightly couple safety mechanisms with specific tasks or environments, creating challenges for certifying systems deployed in real-world settings like warehouses or homes. This paper proposes a pipeline to design an Neural Control Barrier Function (NCBF) that serves as a rigid safety filter, decoupling safety guarantees from task execution to ensure liveness, scalability, and robustness. We simultaneously train an optimized data-driven, actor-critic safety policy to steer the robot away from the unsafe set of states. This thesis relies on existing semantic counterfactual reasoning approaches involving vision-language models (VLMs) to "imagine" unsafe scenarios from real safe trajectories, addressing the data sparsity problem in training learned CBFs. It expands upon these by incorporating world models that can rollout trajectories in latent space and attempts to test the pipeline on two datasets taken on different robots, a manipulator and a navigator. The navigation pipeline was implemented fully, whereas the manipulation pipeline was shown to be possible but not fully implemented due to compute constraints. In testing the developed Control Barrier Function on the navigation dataset, it was found that although the safe and unsafe boundary could be learned to a high accuracy, the optimizer function trained as a safety policy often reached a trade-off and frequently misfired on safe episodes. The results suggested this pipeline may be useful but requires further development on the preprocessing and training for the final Control Barrier Function and optimizer.