Fine Tuning LLMs for Low Resource Languages

Large Language Models (LLMs) hold immense potential, but their data hunger can limit its performance in processing languages with limited resources. This research study explores the techniques for fine-tuning LLMs specifically for low-resource settings. This study highlights the need for language-specific approaches that satisfy the unique characteristics of each language. Further, this study analyzes various fine-tuning techniques, including parameter-efficient methods (like Low Rank Adaption (LoRA) and Quantized Low Rank Adaption (QLoRA)), instruction tuning, and Representation Fine-Tuning (ReFT). By promoting these strategies, this work aims to bridge the language gap and empower low-resource languages within the LLM landscape.