LLM Enabled Social Robots – Transforming Aged Care Through AI

This research work provides personalized a model for elderly care by integrating Large Language Models (LLMs) to the Pepper, a social humanoid robot. The facial and emotional recognition is achieved using the robot’s onboard sensors, including cameras and microphones.Using these, pepper captures real-time facial expressions from elderly individuals, enabling empathetic interactions. The visual data are processed on external hardware, such as the NVIDIA Jetson Nano, where the advanced tasks like fine-tuned facial emotion analysis are performed. A proof of concept system demonstrated Pepper’s ability to infer emotional states and deliver dynamic, tailored responses through efficient processing that combines lightweight on-device tasks with external AI resources. Two curated datasets, FER-2013 (containing facial images labeled with emotions) and FFHQ (featuring high-resolution facial images), enhance the model’s accuracy by enabling it to combine fine-grained visual details, such as facial features and expressions, with a deeper understanding of emotional context and meaning. The results confirm that AI-powered Pepper can effectively deliver empathetic care, marking a significant advancement in human-robot interaction and healthcare technologies, particularly for elderly care.