YUQORI N-GRAM MODELLARINI O’ZBEK TILI MATNLARIGA QOʻLLASH
Keywords:
Til modellari, n-gram til modeli, o'rtacha logarifmik ehtimollik, modelni baholash, Laplas silliqlash, mashinali o’qitishAbstract
Tabiiy tilni qayta ishlashda kontekstdagi keyingi qaysi so’zni bashorat qilish vazifasi tilni modellashtirish deb ataladi. N-gram tahlili tilni qayta ishlashning muhim usuli boʻlib, u til tuzilishini tushunishga va gapdagi fragmentdan keyin qanday so`z kelishini bashorat qilishga yordam beradi. N-gramlarni tahlil qilish matn yaratish, imloni tuzatish va hissiyotlarni tahlil qilish kabi murakkab vazifalarda foydalidir. NLP modellari til qoliplarini yaxshiroq tushunishi va qaysi so'zlar birgalikda paydo bo'lishini o'rganish orqali samaraliroq bashorat qilishlari mumkin. N-gram modellari mashina tarjimasi, chatbotlar va qidiruv tizimlari kabi turli xil NLP ilovalarini yaxshilashga yordam beradi.
References
Booch, G., Jacobson, I., & Rumbaugh, J. (1996). The unified modeling language. Unix Review, 14(13), 5.
Tan, M., Zhou, W., Zheng, L., & Wang, S. (2012). A scalable distributed syntactic, semantic, and lexical language model. Computational Linguistics, 38(3), 631-671.
Brown, P. F., Della Pietra, V. J., Desouza, P. V., Lai, J. C., & Mercer, R. L. (1992). Class-based n-gram models of natural language. Computational linguistics, 18(4), 467-480.
Hockey, B. A., & Rayner, M. (2005, July). Comparison of grammar-based and statistical language models trained on the same data. In Proceedings of the AAAI Workshop on Spoken Language Understanding (pp. 9-10).
Mikolov, T. (2012). Statistical language models based on neural networks.
Kim, Y., Jernite, Y., Sontag, D., & Rush, A. (2016, March). Character-aware neural language models. In Proceedings of the AAAI conference on artificial intelligence (Vol. 30, No. 1).
B.Elov, Sh.Sirojiddinov, Sh.Hamroyeva, E.Adalı, Z.Xusainova (2023, September). Pos Taging of Uzbek Text Using Hidden Markov Model. In 2023 8th International Conference on Computer Science and Engineering (UBMK) (pp. 63-68). IEEE.
B.Elov, Sh.Khamroeva, R.Alayev, Z.Khusainova, U.Yodgorov (2023). Methods of processing the uzbek language corpus texts. International Journal of Open Information Technologies, 11(12), 143-151.
Liu, Y., & Zhang, M. (2018). Neural network methods for natural language processing.
Chang, Y., Wang, X., Wang, J., Wu, Y., Yang, L., Zhu, K., ... & Xie, X. (2024). A survey on evaluation of large language models. ACM Transactions on Intelligent Systems and Technology, 15(3), 1-45.
Chen, M., Tworek, J., Jun, H., Yuan, Q., Pinto, H. P. D. O., Kaplan, J., ... & Zaremba, W. (2021). Evaluating large language models trained on code. arXiv preprint arXiv:2107.03374.
Chen, S. F., Beeferman, D., & Rosenfeld, R. (1998). Evaluation metrics for language models.
Chang, Y., Wang, X., Wang, J., Wu, Y., Yang, L., Zhu, K., ... & Xie, X. (2024). A survey on evaluation of large language models. ACM Transactions on Intelligent Systems and Technology, 15(3), 1-45.
B.Elov (2022). N-gram til modellari vositasida o ‘zbek tilida matn generatsiya qilish. Computer linguistics: problems, solutions, prospects, 1(1).
B. Elov, A. Abdullayev, A., N.Xudoyberganov. (2024). O‘zbek tili korpusi matnlari asosida til modellarini yaratish. «Contemporary technologies of computational linguistics», 2(22.04), 344-353
Downloads
Published
How to Cite
Issue
Section
License
Copyright (c) 2024 Botir Elov, Ruhillo Alayev, Abdulla Abdullayev, Narzillo Aloyev
This work is licensed under a Creative Commons Attribution 4.0 International License.
