A hybrid approach to Azerbaijani lemmatization: integrating rule-based methods with ML PoS tagging

Lemmatization of a language is the process of finding the lemma of a given word. Although this concept has been straightforward to apply for analytic languages such as English, it required an innovative approach specifically designed for languages that fall under the category of agglutinative languages due to their complex morphology. This paper addresses the above-mentioned issue by presenting a hybrid approach combining Rule-Based and Machine Learning methodologies. The approach mentioned solves the problem with 96% accuracy and has already produced significant results in projects like sentiment analysis, by increasing its accuracy by 8.9%.