TABIIY TIL QOLIPLARINI N-GRAM METODI VOSITASIDA ANIQLASH
Keywords:
n-gram metodi, Til modellari, til korpusi, til qoliplari, unigram, bigram, trigram, mashinali o’qitishAbstract
N-gram metodi – matn ma'lumotlaridagi qolip va munosabatlarni aniqlash uchun tabiiy tilni qayta ishlash (Natural Language Processing, NLP)da qo'llaniladigan matnni tahlil qilish usuli. Ushbu metod matnni n-gram deb ataladigan kichikroq birliklarga bo'lish va matn ma'lumotlari haqida tushunchaga ega bo'lish uchun ushbu n-grammlarning chastotasi va ularning korpusda tarqalishini tahlil qilishni o'z ichiga oladi. N-gramlar so'zlar, belgilar yoki boshqa har qanday mazmunli matn birliklaridan iborat bo'lishi mumkin. Til korpusidahi N-gramlar tahlili muhim ahamiyatga ega bo`lib, u matn ma'lumotlarini tahlil qilish va ma'lumotlar ichidagi qolip va munosabatlarni aniqlashning sodda, ammo samarali usulini taklif qiladi. N-gram metodi tilni modellashtirish, matnni tasniflash va his-tuyg`ularni tahlil qilish kabi turli xil NLP ilovalarini ishlab chiqish uchun foydali bo'lishi mumkin. N-gram tahlili tilni modellashtirishda matn ma'lumotlaridagi qolip va munosabatlarni aniqlash hamda tabiiy tilni qayta ishlash vazifalari uchun bashoratli modellarni yaratish uchun qo`llaniladi. Shungdek, N-gram tahlili matnni tasniflashda matnning asosiy xususiyatlarini aniqlash va matnni oldindan belgilangan toifalarga ajratish uchun ishlatiladi.
References
Pauls, A., & Klein, D. (2011, June). Faster and smaller n-gram language models. In Proceedings of the 49th annual meeting of the Association for Computational Linguistics: Human Language Technologies (pp. 258-267).
Takahashi, S., & Morimoto, T. (2012, November). N-gram language model based on multi-word expressions in web documents for speech recognition and closed-captioning. In 2012 International Conference on Asian Language Processing (pp. 225-228). IEEE.
Chakraborty, R., Deka, M., & Sarma, S. K. (2024). Syntactic Category based Assamese Question Pattern Extraction using N-grams. Procedia Computer Science, 235, 214-230.
B. Elov, A. Abdullayev, A., N.Xudoyberganov. (2024). O‘zbek tili korpusi matnlari asosida til modellarini yaratish. Contemporary technologies of computational linguistics, 2(22.04), 344-353.
Cavnar, W. B., & Trenkle, J. M. (1994, April). N-gram-based text categorization. In Proceedings of SDAIR-94, 3rd annual symposium on document analysis and information retrieval (Vol. 161175, p. 14).
Náther, P. (2005). N-gram based Text Categorization. Comenius University, Bratislava, Slovakia.
Ahmed, B., Cha, S. H., & Tappert, C. (2004). Language identification from text using n-gram based cumulative frequency addition. Proceedings of Student/Faculty Research Day, CSIS, Pace University, 12(1).
Zens, R., & Ney, H. (2006, June). N-gram posterior probabilities for statistical machine translation. In Proceedings on the Workshop on Statistical Machine Translation (pp. 72-77).
Majumder, P., Mitra, M., & Chaudhuri, B. B. (2002, November). N-gram: a language independent approach to IR and NLP. In International conference on universal knowledge and language (Vol. 2).
Dey, A., Jenamani, M., & Thakkar, J. J. (2018). Senti-N-Gram: An n-gram lexicon for sentiment analysis. Expert Systems with Applications, 103, 92-105.
Downloads
Published
How to Cite
Issue
Section
License
Copyright (c) 2024 Mastura Primova
This work is licensed under a Creative Commons Attribution 4.0 International License.