O‘ZBEK TILI MATNLARINI NAIVE BAYES USULI ASOSIDA SENTIMENT TAHLIL QILISH

Botir Elov; Abdulla Abdullayev; Nizomaddin Xudayberganov

Авторы

Botir Elov Alisher Navoiy nomidagi Toshkent davlat oʻzbek tili va adabiyoti universiteti
Abdulla Abdullayev Urganch innovatsion university
Nizomaddin Xudayberganov Alisher Navoiy nomidagi Toshkent davlat o‘zbek tili va adabiyoti universiteti

Ключевые слова:

Sentiment tahlil, Naive Bayes, matn tasnifi, chastota vektorlari, tokenlash, mashinali o‘qitish, TF-IDF vektorizatsiyasi, morfologik xususiyatlar, Matnni oldin ishlash (preprocessing), NLP (Tabiiy tilni qayta ishlash)

Аннотация

Ushbu maqolada o‘zbek tilidagi matnlarni sentiment tahlil qilishda Naive Bayes (NB) usulining samaradorligi va cheklovlari tadqiq qilindi. Tadqiqotning asosiy maqsadi matnlarning hissiy ohangini (ijobiy, salbiy yoki neytral) aniqlash uchun Naive Bayes modelini qo‘llash va uning samaradorligini baholashdan iborat. O‘zbek tili milliy korpusidan olingan matnlar to‘plami matnlar tozalash, tokenizatsiya va chastota vektorlariga aylantirish bosqichlaridan o‘tkazilib, model uchun tayyorlandi. TF-IDF vektorizatsiyasi asosida qurilgan model 4,000 ta ijtimoiy tarmoq sharhlaridan iborat dataset yordamida o‘qitilib, 75.47% aniqlik (accuracy) natijasiga erishdi. Modelning aniqligi va F1-score ko‘rsatkichlari asosida baholangan natijalar oddiy va qisqa matnlarda yuqori samaradorlikni ko‘rsatdi. O‘zbek tiliga xos morfologik murakkabliklar (masalan, so‘z qo‘shimchalari, izohlovchi shakllar) modelning baʼzi murakkab iboralarni noto‘g‘ri talqin qilishiga sabab bo‘lishi aniqlandi. Qiyosiy tahlil shuni ko‘rsatdiki, NB Logistik Regressiyaga nisbatan 7% pastroq, lekin Decision Treesga nisbatan 15% tezroq ishlaydi. Matnni boshlang`ich ishlash (nomuhim so‘zlarni olib tashlash, kichik harflarga o‘tkazish) bilan modelning ishonchliligi 5% ga oshirildi. Biroq, murakkab sintaksis va kontekstga bog‘liq sentimentlarni tahlil qilishda modelning cheklovlari aniqlandi. Tadqiqot o‘zbek tili uchun sentiment tahlilining rivojlanishiga hissa qo‘shadi va usulning boshqa tillardagi modellar bilan taqqoslash imkonini beradi. Tadqiqot shuningdek, NBning mustaqillik gipotezasi tufayli so‘zlar o‘rtasidagi bog‘liqlikni eʼtiborsiz qoldirishi kabi cheklovlarini taʼkidlaydi. Kelajakda n-gram modellari va kontekstni hisobga oluvchi yondashuvlar bilan ushbu cheklovlarni bartaraf etish mumkinligi ko‘rsatilgan. Maqola yakunida NB usulining mijozlar sharhlarini tahlil qilish, ijtimoiy media monitoringi va taʼlim sohasidagi qisqa matnlarni baholash kabi amaliy dasturlarda qo‘llanilishi tavsiya etiladi.

Библиографические ссылки

Lewis, D. D. (1995, September). A sequential algorithm for training text classifiers: Corrigendum and additional data. In Acm Sigir Forum (Vol. 29, No. 2, pp. 13-19). New York, NY, USA: ACM.

McCallum, A., & Nigam, K. (1998, July). A comparison of event models for naive bayes text classification. In AAAI-98 workshop on learning for text categorization (Vol. 752, No. 1, pp. 41-48).

Nigam, K., McCallum, A. K., Thrun, S., & Mitchell, T. (2000). Text classification from labeled and unlabeled documents using EM. Machine learning, 39, 103-134.

Rennie, J. D., Shih, L., Teevan, J., & Karger, D. R. (2003). Tackling the poor assumptions of naive bayes text classifiers. In Proceedings of the 20th international conference on machine learning (ICML-03) (pp. 616-623).

Domingos, P., & Pazzani, M. (1997). On the optimality of the simple Bayesian classifier under zero-one loss. Machine learning, 29, 103-130.

Pang, B., Lee, L., & Vaithyanathan, S. (2002). Thumbs up? Sentiment classification using machine learning techniques. arXiv preprint cs/0205070.

Witten, I. H., Frank, E., Hall, M. A., Pal, C. J., & Data, M. (2005, June). Practical machine learning tools and techniques. In Data mining (Vol. 2, No. 4, pp. 403-413). Amsterdam, The Netherlands: Elsevier.

Frank, E., & Bouckaert, R. R. (2006). Naive bayes for text classification with unbalanced classes. In Knowledge Discovery in Databases: PKDD 2006: 10th European Conference on Principles and Practice of Knowledge Discovery in Databases Berlin, Germany, September 18-22, 2006 Proceedings 10 (pp. 503-510). Springer Berlin Heidelberg.

Elov, B. B., Khamroeva, S. M., Alayev, R. H., Khusainova, Z. Y., & Yodgorov, U. S. (2023). Methods of processing the uzbek language corpus texts. International Journal of Open Information Technologies, 11(12), 143-151.

Boltayevich, E. B., Turapovna, I. S., & Ibragimovna, T. G. (2024, November). Tagging Units in the Text and the Bayes Algorithm. In 2024 IEEE 3rd International Conference on Problems of Informatics, Electronics and Radio Engineering (PIERE) (pp. 1840-1843). IEEE.