Tibetan Large Model Based on Efficient Parameter Fine Tuning
A large model refers to a deep learning model with many parameters,which has powerful representation learning and generation capabilities and has had a profound impact on fields such as natural language processing.With the continuous advancement of technology,large models have made breakthroughs in performance and application scope,becoming a research hotspot in the field of artificial intelligence.However,the development of large models also faces some challenges,such as high model training costs,parameter redundancy,and limitations in cross language applications.Specifically,in the field of Tibetan,which has unique language characteristics,research on large models is still in its early stages and lacks corresponding models and resource support.In response to the above issues,this article proposes an efficient parameter fine-tuning method based on LORA and constructs the Tibetan-Llama2 and Tibetan-Alpaca models based on the Llama2 model architecture.After incremental pre-training with large-scale data and instruction fine-tuning,they can understand and generate long Tibetan texts,demonstrate their multitasking learning ability,and have broad application prospects in multiple fields.
natural language processingtibetan language modelefficient parameter fine-tuningincremental pre-traininginstruction fine-tuning
NETL
NSTL
万方数据
