Bildiri, tam metinConference paper 2026
When the Measure Fails Before the Model Does: Generative AI as a Source of Tourist Information in Kyrgyz, Turkish and English
ÖzetAbstract
This study set out to compare the quality of the answers that generative artificial intelligence models give about the tourism resources of Kyrgyzstan in Kyrgyz, Turkish and English. Fifteen standardised tourist questions were put to ChatGPT, Gemini and DeepSeek in the three languages on 26 July 2026, and the resulting 135 answers were examined through content analysis. Three results held firm. Answer length followed the same order in every model, from Kyrgyz at the short end to English at the long end. Several answers carried serious factual errors, the most striking of which described a lake in Türkiye in place of Lake Issyk-Kul and placed the city of Osh inside Uzbekistan. When the same questions were repeated on 14 September 2026 under identical settings, those errors had gone. The fourth result was not the one that had been planned. The attempt to rate the answers on five criteria did not produce workable agreement between coders. The figure reached 0.93 for two coders in English but fell to 0.49 once a third expert joined, and it stood at 0.02 in Turkish and 0.03 in Kyrgyz. It was determined that the coders were in fact close to one another, since their scores fell within one point of each other in ninety to one hundred per cent of cases; the difficulty was that almost every answer received a four or a five, which leaves a reliability coefficient nothing to work with. From this it can be understood that a five-point rubric does not discriminate between answers in these languages, and that counting and fact-checking carry the evidential weight that rating was expected to carry.
- coder reliability
- generative artificial intelligence
- Kyrgyz language
- Kyrgyzstan
- less-resourced languages
- tourist information
AtıfCite
Vural, M., Kazakbaeva, Z., Alieva, N., Yilmaz, R., & Toktomatova, A. (2026). When the Measure Fails Before the Model Does: Generative AI as a Source of Tourist Information in Kyrgyz, Turkish and English. In 8. Uluslararası Türk Dünyası Turizm Kongresi. Ankara Hacı Bayram Veli Üniversitesi, Turizm Fakültesi.