BERT-DRE: BERT with Deep Recursive Encoder for Natural Language Sentence\n Matching

This paper presents a deep neural architecture, for Natural Language Sentence\nMatching (NLSM) by adding a deep recursive encoder to BERT so called BERT with\nDeep Recursive Encoder (BERT-DRE). Our analysis of model behavior shows that\nBERT still does not capture the full complexity of text, so a deep recursive\nencoder is applied on top of BERT. Three Bi-LSTM layers with residual\nconnection are used to design a recursive encoder and an attention module is\nused on top of this encoder. To obtain the final vector, a pooling layer\nconsisting of average and maximum pooling is used. We experiment our model on\nfour benchmarks, SNLI, FarsTail, MultiNLI, SciTail, and a novel Persian\nreligious questions dataset. This paper focuses on improving the BERT results\nin the NLSM task. In this regard, comparisons between BERT-DRE and BERT are\nconducted, and it is shown that in all cases, BERT-DRE outperforms BERT. The\nBERT algorithm on the religious dataset achieved an accuracy of 89.70%, and\nBERT-DRE architectures improved to 90.29% using the same dataset.\n