GUI Agentlar: Benchmark Ko'rsatgichlar Orqasidagi Haqiqat

Sinab Ko'ring

Hozir sinab ko'rishingiz mumkin bo'lgan bir tajriba bor. Eng zamonaviy GUI agentni olib, unga tanish bir saytni ko'rsating. Brauzer zoomini 70% ga tushiring.

Hech narsa o'zgarmadi. Dizayn bir xil. Tugmalar o'z joyida. Faqat matn kichikroq.

Model esa muvaffaqiyatsiz bo'ladi.

Bu maxsus holat emas. Bu benchmarklar o'lchaydigan narsa bilan real dunyoda kerak bo'lgan narsa o'rtasidagi chuqur tafovutni ko'rsatadi.

Benchmark Aldamchiligi

Raqamlarni to'g'ri tushunish kerak. Hozirgi GUI modellar ScreenSpot-v2 kabi benchmarklarda 90% dan yuqori aniqlik ko'rsatmoqda. Ko'pchilik buni "bu muammo hal qilindi" deb o'qiydi.

Lekin bu raqamlar nimalarni qamrab olmaydi?

ScreenSpot-v2 va shunga o'xshash benchmarklar muzlatilgan screenshotlar ustida sinov o'tkazadi. Bir xil sahifa, bir xil ko'rinish, har doim. Real saytlar esa bunday ishlaydi - foydalanuvchilar zoom qiladi, dizaynerlar yangilaydi, ranglar o'zgaradi.

Model o'zgaruvchanlikka o'rganmagan - u maxsus screenshotlarni eslab qolgan. Baland benchmark ko'rsatgichlari aslida xotira sig'imini o'lchayapti, haqiqiy vizual tushunishni emas.

GUI-Perturbed tadqiqotchilari aynan shu muammoni o'lchashga harakat qildi. Ular vizual sahifalarni turli yo'llar bilan o'zgartirib, aniqlik qancha tushishini o'lchashdi. Natijalar ishlab chiqarish tizimlarida ishlaydigan har bir kishi uchun tashvishli.

Uch Tomonlama Moslash Muammosi

GUI grounding nima talab qiladi? Model screenshot va "submit tugmasini bosing" kabi buyruqni ko'rganda, bir vaqtning o'zida uch xil moslashish kerak:

Vizual moslash - piksel namunalarini interfeys elementlariga moslashtirish. Tugmaning shakli, rangi va o'lchami model uchun tanilishi kerak.

Funksional moslash - element aslida nima qilishini tushunish. Kiritish maydoni va yorliq turlicha ko'rinadi, bosiladigan tugma esa statik ikonkadan farq qiladi.

Geometrik moslash - fazoviy munosabatlarni hal qilish. "Qidiruv panelidan yuqoridagi tugma" yoki "yorliqdan o'ngdagi maydon" - bu narsalarning qayerda joylashganini tushunish kerak, faqat qanday ko'rinishini emas.

Eng muammoli qismi: ko'pchilik benchmarklari barchasini birga o'lchaydi. Model 85% olganda, uchchala qismni ham yaxshi bajargani yoki faqat vizual qismni bilib, geometriyani taxmin qilganini bilib bo'lmaydi.

Modellar Qayerda Buziladi

1. Fazoviy Ko'rsatmalar Juda Zaif

Bu eng katta muammo. "Submit tugmasini bosing" dan "aloqa shakli ustidagi tugmasini bosing" ga o'tganda, aniqlik 27 dan 56 punktgacha tushadi. 27 punkt - bu tashvishli. 56 punkt esa ishlab chiqarishda foydalanishga yaroqsiz.

Model aniq nomlangan tugmani topa oladi. Undan fazoda qayerda ekanini so'rashingiz bilan, natija keskin yomonlashadi.

Bu xususiyat muhim, chunki odamlar tabiiy tilda ko'rsatmalar beradi. "Pastga siljiting va shaklni bosing" - bu odatiy usul. Ammo joriy modellar buni qila olmaydi.

2. Vizual O'zgarishlar Ham Ta'sir Qiladi

Zoom tajribasi noyob holat emas. Brauzer zoomini 70% ga o'zgartirish barcha modellarda 2-6 punkt aniqlikni tushiradi. Bu halokatli emas, lekin o'ylab ko'ring: model bir o'lchamda elementlarni tanishga o'rgangan, o'lcham o'zgarganda tanish buziladi.

Real foydalanuvchilar zoom qiladi. Turli monitorlar turli DPI sozlamalariga ega. Bu kundalik holatlar.

Yana bir muhim xulosa: modellar insonlar kabi o'lchamga bog'liq bo'lmagan tasvirlar yaratmayapti - ular o'qigan vaqtdagi o'lchamlardagi ko'rinishni eslayapti.

3. Mantiqiy Qadamlarni Afzalliklari va Kamchiliklari Bor

Harakat qilishdan oldin o'ylash qo'shish qiyin vazifalarda yordam beradi, lekin oson vazifalarda aslida ishlashni yomonlashtiradi. Model qachon o'ylash va qachon to'g'ridan-to'g'ri harakat qilishni bilishi kerak.

Bu amaliy muammo yaratadi. Hamma joyda mantiqiy qadamlarni yoqib bo'lmaydi - yoki qachon o'ylashni hal qiluvchi router kerak, yoki ikkala rejimda ham yaxshi ishlaydigan model kerak.

Post-Training Haqiqatan Ham Nima Beradi?

Eng achinarli natija: ko'proq GUI ga yo'naltirilgan qo'shimcha training muammolarning hech birini hal qilmaydi.

Uchta model bir xil baza checkpointni ulashadi, lekin turli miqdorda GUI maxsus fine-tuning o'tkazgan. Qo'shimcha training benchmark ko'rsatgichlarini oshirdi. Ammo vizual扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰letika扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰letika扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰letika扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰letika扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰letika扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰letika扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰letika扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰letika扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰letika扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰letika扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰letika扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰letika扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰letika扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰letika扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰letika扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰letika扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰letika扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰letika扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰letika扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰letika扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰letika扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰letika扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰letika扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰letika扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰letika扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰letika扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰letika扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰letika扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰letika扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰letika扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰letika扰动扰动扰动扰动扰动扰动扰动扰动扰letika扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰letika扰动扰动扰动扰动扰letika扰动扰动扰动扰动扰动扰动扰动扰动扰letika扰动扰动扰动扰动扰动扰letika扰动扰动扰动扰动扰letika扰动扰动扰动扰动扰letika扰动扰动扰动扰letika扰动扰动扰letika扰动扰letika扰letika

Buning ma'nosi shundaki, benchmarklardagi yaxshilanishlar qisman aldamchi bo'lishi mumkin. Modellar testga ko'proq moslashyapti, asosiy vazifani emas. Ular benchmarkni aniqroq "eslayapti", lekin umumiy qobiliyatlarini rivojlantirmayapti.

Model tanlashda bu juda muhim farq. "ScreenSpot-v2 da 92% oladi" degani shundan iboratki, model screenshotlardagi GUI elementlarini taniydi. Bu uning real web saytlarning o'zgaruvchanligiga dosh bera oladimi - bu boshqa savol.

Builderlar Uchun Qanday Qilib

Agar siz computer-use agentlar ustida qurayotgan bo'lsangiz, bir nechta narsa esda tuting:

Ishlab chiqarish muhiti benchmark muhitidan qiyinroq bo'ladi. Agar siz faqat belgilangan sahifalar ustida sinov o'tkazayotgan bo'lsangiz, real foydalanishdagi ishlashni o'lchamayapsiz. Perturbatsion testingni o'z ichiga olgan baholash tizimini yarating - turli zoom darajalarida, CSS o'zgarishlari bilan, yangi dizayndagi sahifalarda sinab ko'ring.

Fazoviy ko'rsatmalar bilan ishlashga alohida e'tibor bering. Agar ilovangiz tabiiy tildagi ko'rsatmalarni qo'llasa (masalan, "sarlavhadan pastdagi variantni tanlang"), joriy umumiy maqsadli modellar qiynaladi. Bu ko'rsatma formatlarini cheklash, aniq koordinata bashorat qilishni qo'shish yoki fazoviy mantiq uchun maxsus modellardan foydalanish kerakligini anglatadi.

Dizayn o'zgarishlarini kuzatib boring. Maqsadli saytlar o'z maketlarini o'zgartirganda, agentingizning aniqlik darajasi to'satdan tushishi mumkin - model yomonlashgani uchun emas, u hali ko'rmagan vizual konfiguratsiyaga duch kelgani uchun. Element joylashuv strategiyasini keshlash va siljishni kuzatishni ko'rib chiqing.

Kelajak Haqida

Bu tadqiqot computer-use agentlar foydasiz degani emas. Bu shundan dalolatki, sohadagi eng muhim narsalarni o'lchash usullari yaxshilanishi kerak - benchmark ko'rsatgichlar emas, balki mustahkamlik.

Yaxshi tomoni shundaki, muammolar endi ko'rinadi va o'lchanadi. GUI-Perturbed usullabos bir aniq qadamlari bo'yicha modellarni sinash imkonini beradi. Agar siz bunday tizimlarni yaratayotgan yoki sotib olayotgan bo'lsangiz, statik benchmark ko'rsatgichlari emas, balki扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰letika扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰letika扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰letika扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰letika扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰letika扰动扰动扰动扰动扰动扰动扰动扰动扰letika扰动扰动扰动扰动扰动扰letika扰动扰动扰动扰动扰letika扰动扰动扰动扰letika扰动扰letika扰letika扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰动扰letika扰动扰动扰动扰动扰动扰动扰动扰动扰letika扰动扰动扰动扰动扰动扰letika扰动扰动扰动扰动扰letika扰动扰动扰letika扰动扰动扰动扰动扰动扰letika扰动扰动扰letika扰动扰动扰letika扰动扰letika扰动扰动扰letika扰动扰letika扰动扰letika

Vizual, funksional va geometrik tushunish birgalikda ishlashi kerak - bu uch tomonlama muammo haqiqiy va hal qilinadigan. Uni hal qilish keyingi avlod ishonchli AI agentlarini yaratadi - foydalanuvchilaringiz yashaydigan tartibsiz, o'zgaruvchan dunyoda haqiqatan ishlaydigan agentlarni.

Hozircha, 90%+ benchmark ko'rsatgichlarini boshlang'ich nuqta deb qarang, yakuniy maqsad emas. Foydalanuvchilaringiz sizga minnatdor bo'ladi - AI yordamchilari zoom qilingan brauzerni muammosiz ishlatganda.