Zanen kama-da-gaskiya a cikin Ƙarfafa Gaskiya na Wayar Hannu (AR) yana da ƙuntatawa ta asali saboda rashin ingantaccen bayanin haske na kowane bangare a kowane wurin zane cikin sauri. Na'urorin wayar hannu na yanzu ba za su iya ɗaukar cikakken hoto na 360° daga wurin da ake son sanya abu na zahiri ba. Yin amfani da bayanan haske daga wurin kallon mai amfani yana haifar da zane mara inganci, wanda ba ya canzawa a sarari, wanda ke karya shiga cikin labarin.
Tsarin Xihe yana gabatar da sabon mafita ta hanyar amfani da ci gaban hangar nesa 3D na wayar hannu—kamar na'urar LiDAR da na'urori masu zurfin gani da aka haɗa—don kimanta hasken muhalli. Shi ne tsarin taimakon gefe wanda aka tsara don samar da ingantaccen kiyasin haske mai canzawa a sarari cikin sauri (har zuwa kusan 20ms), yana ba da damar samun ƙwarewar AR mai inganci akan na'urorin masu amfani.
Tsarin ginin Xihe ya dogara ne akan samfurin abokin ciniki-gefe-ukaɗa, yana inganta kowane ɓangare don ƙayyadaddun ƙuntatawa na AR na wayar hannu: ƙarancin lissafi akan na'ura, jinkirin hanyar sadarwa, da buƙatar kama-da-gaskiya na fahimta.
Tsarin aiki ya ƙunshi: 1) Na'urar wayar hannu tana ɗaukar gungu matsakaici (point cloud) na muhalli ta amfani da na'urar zurfin gani (misali, LiDAR). 2) Sabon algorithm na zaɓi yana matsawa wannan bayanan. 3) Ana aika bayanan da aka sarrafa zuwa ukaɗar gefe da ke ɗauke da samfurin koyon zurfi don kiyasin haske. 4) Ana mayar da ƙayyadaddun haske da aka kiyasta (misali, ƙididdiga na Spherical Harmonics) zuwa na'urar don zana abubuwa na zahiri.
Wani sabon abu mai mahimmanci shine dabarar zaɓi mai inganci wacce aka samo daga bincike na zahiri na bayanan cikin gida na 3D. Maimakon sarrafa cikakken gungu matsakaici mai yawa, Xihe yana zaɓar ɓangaren maki da suka fi dacewa don kiyasin haske (misali, maki akan saman da ke da takamaiman ma'auni ko kaddarorin albedo). Wannan yana rage yawan bayanan da ake ɗauka sosai ba tare da asarar inganci mai mahimmanci ba.
Don rage jinkiri, ana yin farkon sarrafa gungu matsakaici (tacewa, daidaitawa, zaɓi) akan GPU na na'urar wayar hannu. Wannan hanyar da aka keɓance tana tabbatar da cewa babban aikin sarrafawa kafin aikawa ba zai zama cikas ba.
Samfurin koyon zurfi mai rikitarwa don fahimtar haske daga tsarin 3D yana gudana akan ukaɗar gefe. Xihe yana amfani da tsarin ɓoyayyen bayanai na musamman don ƙara matsawa bayanan gungu matsakaici da aka zaɓa kafin aikawa, yana rage jinkirin hanyar sadarwa da amfani da bandeji.
Xihe ya haɗa da dabarar kunna mai hankali. Ba ya yin sabon kiyasin haske don kowane firam. A maimakon haka, yana kimanta lokacin da yanayin haske ko matsayin mai amfani/maimaitawa ya canza sosai har ya cancanci sabuntawa. Bugu da ƙari, yana ba da hanyoyin tabbatar da daidaiton lokaci tsakanin ƙiyasin, yana hana ƙyalli ko canje-canje masu tsangwama a cikin yanayin AR da aka zana.
Ana wakiltar haske sau da yawa ta amfani da Spherical Harmonics (SH). Matsalar kiyasin ta asali za a iya tsara ta azaman nemo ƙididdiga na SH $\mathbf{l}$ waɗanda suka fi bayyana hasken da aka lura $B(\mathbf{n})$ a maki na saman tare da ma'auni $\mathbf{n}$, idan aka ba da albedo $\rho$:
$B(\mathbf{n}) = \rho \int_{\Omega} L(\omega) (\mathbf{n} \cdot \omega)^+ d\omega \approx \rho \sum_{i} l_i Y_i(\mathbf{n})$
Inda $L(\omega)$ shine hasken da ke faruwa, $Y_i$ ayyukan tushe ne na SH, kuma $(\cdot)^+$ shine abin da aka matse. Cibiyar sadarwar jijiya ta Xihe tana koyon taswira $f_\theta$ daga gungu matsakaici da aka zaɓa $P$ zuwa waɗannan ƙididdiga: $\mathbf{l} = f_\theta(P)$.
Dabarar zaɓi tana nufin zaɓar maki $p_i \in P$ waɗanda suka ƙaru da ƙimar bayanai don warware wannan matsalar juyawa, sau da yawa suna mai da hankali kan maki tare da alamun da ba na Lambertian ba ko takamaiman alaƙar lissafi.
Yanayi: Sanya bututun tukwane na zahiri akan teburin katako a cikin ɗakin zama tare da taga a gefe ɗaya da fitila a ɗayan.
Takardar tana kimanta Xihe ta amfani da aikace-aikacen AR na wayar hannu na tunani. Ma'auni suna mai da hankali kan ingancin kiyasin da jinkiri har zuwa ƙarshe.
20.67 ms
Matsakaici kowace kiyasin
9.4%
Mafi kyau fiye da ma'aunin cibiyar sadarwar jijiya na zamani
~100x
Ragewa daga gungu matsakaici na danye
An auna inganci ta hanyar kwatanta hotunan da aka zana na abubuwa na zahiri a ƙarƙashin hasken da aka kiyasta na Xihe da gaskiyar zane ta amfani da taswirorin muhalli da aka sani. Xihe ya fi ma'aunin cibiyar sadarwar jijiya na zamani da 9.4% dangane da ma'aunin kamancen hoto na yau da kullun (mai yiyuwa PSNR ko SSIM). An samo wannan ribar ne saboda sanin tsarin gini na 3D da gungu matsakaici ya bayar, sabanin hanyoyin da suka dogara kawai akan hotunan kamara na 2D.
Hanyar har zuwa ƙarshe tana samun matsakaicin jinkiri na 20.67 milliseconds kowace kiyasin haske, cikin kasafin kuɗi don AR cikin sauri (yawanci 16ms don 60 FPS). Wannan yana samuwa ta hanyar ingantaccen sarrafawa akan na'ura da ingantacciyar hanyar sadarwa. Hanyar kunna da dacewa tana ƙara rage yawan aikin lissafi na kowane firam.
Hangen Nesa na Asali: Xihe ba wani ƙarin ci gaba ne kawai a cikin zanen jijiya ba; yaudara ne na matakin tsarin wanda a ƙarshe ya haɗa tazarar tsakanin ka'idar zane mai zurfi da gaskiyar tauraron kayan aikin wayar hannu. Hangen nesa na asali shine cewa sabon yaduwar na'urori masu auna nesa 3D na wayar hannu (LiDAR) ba don auna ɗakuna kawai ba ne—shi ne maɓalli da ya ɓace don warware matsalar "haske daga ko'ina" wacce ta addabi AR na wayar hannu tsawon shekaru goma. Yayin da ayyuka kamar NeRF: Wakiltar Yanayi azaman Filayen Haske na Jijiya don Haɗin Kallo (Mildenhall et al., 2020) suka burge da cikakken sake gina yanayi, ba su da iyaka ga amfani da wayar hannu cikin sauri. Xihe yana guje wa wannan tarko da hankali ta hanyar rashin ƙoƙarin sake gina komai; a maimakon haka, yana amfani da bayanan 3D azaman bayanin lissafi mai yawa, don ƙuntata matsalar kiyasin haske, wanda ya fi dacewa.
Tsarin Hankali: Hankalin takardar yana da gamsarwa: 1) Kama-da-gaskiya yana buƙatar haske mai canzawa a sarari. 2) Wayoyin hannu ba za su iya ɗauka kai tsaye ba. 3) Amma yanzu suna iya ɗaukar lissafi na 3D cikin arha. 4) Lissafi yana nuna ƙuntatawa na haske (misali, kusurwa mai duhu da kusa da taga). 5) Don haka, yi amfani da cibiyar sadarwar jijiya don koyon taswirar "lissafi → haske". 6) Don yin sauri cikin sauri, inganta kowane mataki sosai: zaɓi bayanan 3D, tura babban fahimta zuwa gefe, kuma kada a yi kiyasin sai dai idan ya cancanta. Wannan kwararar daga ma'anar matsalar zuwa tsarin aiki yana da tsabta musamman.
Ƙarfi & Kurakurai: Babban ƙarfinsa shine aikin aiki. Kunna da dacewa da daidaiton lokaci alamun injiniya ne don samfurin gaskiya, ba kawai nunin bincike ba. Algorithm ɗin zaɓi yana da wayo, 'ya'yan itace masu sauƙi waɗanda ke haifar da riba mai yawa. Duk da haka, tsarin yana da kurakurai na asali. Ya dogara gaba ɗaya akan ingancin na'urar auna zurfin; aikin a cikin muhalli mara rubutu ko mai haske yana da shakku. Samfurin taimakon gefe yana gabatar da dogaro da hanyar sadarwa, yana haifar da bambancin jinkiri da damuwa game da sirri—tunani aikace-aikacen ƙira na cikin gida na AR yana watsa taswirorin 3D na gidanku zuwa ukaɗa. Bugu da ƙari, kamar yadda aka lura a cikin binciken Microsoft HoloLens, kiyasin haske wani ɓangare ne kawai na wasan guntun haɗin kai; kiyasin kayan duniya na gaskiya yana da mahimmanci daidai don haɗuwa mara tsage, matsalar da Xihe ya kauce.
Hangennesa Masu Aiki: Ga masu bincike, abin da za a ɗauka shine a ƙara ƙarfi kan hanyoyin haɗin gwiwa na lissafi-jijiya. Koyon tsantsa yana da nauyi sosai; lissafi tsantsa yana da sauƙi. Makomar tana cikin tsare-tsare kamar Xihe waɗanda ke amfani da ɗaya don jagorantar ɗayan. Ga masu haɓakawa, wannan takarda ta zama tsari: idan kuna gina aikace-aikacen AR na wayar hannu mai mahimmanci, dole ne yanzu ku ɗauki bayanan na'urar auna nesa 3D azaman shigarwa na farko. Fara ƙirar samfuri tare da APIs na zurfin ARKit/ARCore nan da nan. Ga masu yin guntu, buƙatar ƙarin ƙarfi, injunan jijiya akan na'ura da ingantattun na'urori masu auna zurfin zai ƙara ƙarfi kawai—inganta don wannan hanyar. Xihe ya nuna cewa hanyar zuwa AR mai kama-da-gaskiya na masu amfani ba kawai game da ingantattun algorithms ba ne, amma game da haɗin gwiwar ƙira na algorithms, kayan aiki, da tsarin gini tare.