Using ontologies for adaptive planning and robust execution in robotic manipulation

Robotic manipulation tasks face many challenges when dealing with real scenarios that require the perception and understanding of the environment, planning with procedures that can adapt to the actual situation, and monitoring and recovery capabilities to ensure successful execution despite uncertainties and errors. The use of knowledge in terms of ontologies can address these needs by providing reasoning capabilities that allow the robot to be aware of the situation and the task and to dynamically create execution structures that adapt its behaviors in a flexible and robust manner. In this line, this paper discusses the work done at the Institute of Industrial and Control Engineering (IOC-UPC) that uses ontologies combined with Large Language Models to plan and automatically generate Behavior Trees with monitoring and recovery capabilities, which allow for the control of robot execution for successful performance in real scenarios.