Continual Learning based on Memory Update Model for Multimodal User Modeling

Recently, various artificial intelligence applications that utilize advanced machine learning models have been developed to address real-world problems. However, as we are well aware, such applications require large-scale training data, complex network architectures, and significant time for the training process. This paper proposes a model that supports online real-time learning using an accumulative memory structure as a way to overcome these limitations of existing machine learning models. The proposed model extracts latent vectors in feature space from stimuli received from the external environment and converts them into sparse distribution representations through scalar encoding and spatial pooling. Sparse distribution representations are known to mimic the structures used by the neural systems of living organisms for memory. Whenever new stimuli are received, the accumulated results of sparse distribution representations derived from each element of the latent vector are used to form memory through hierarchical clustering, reflecting morphological similarities. During the continuous updating process of this memory, the similarity recalculation, incorporating weights derived from the cumulative distributions of each cluster, enhances the performance of cluster formation based on the morphological characteristics of the stimuli. Experimental results using the MNIST dataset confirm that the model can achieve appropriate clustering even for inputs that are morphologically difficult to distinguish. The proposed continuous learning model, through memory updates, is also applicable to multimodal user modeling.