Saurin Kiyaye Haske na Cikin Gida Mai Bambanta a Wuri daga Hoton RGB Guda

1. Gabatarwa

Kiyaye hasken wuri daga hoto guda matsala ce ta asali amma ba ta da tabbas a hangen nesa na kwamfuta, mai mahimmanci ga aikace-aikace kamar gaskiyar da aka ƙara (AR) da zane-zane dangane da hoto. Hanyoyin gargajiya sun dogara da abubuwa da aka sani (na'urorin gwada haske) ko ƙarin bayanai (zurfi, ra'ayoyi da yawa), suna iyakance amfani. Hanyoyin koyon zamani, kamar na Gardner da sauransu [8], suna hasashen haske na duniya amma sun kasa ɗaukar yanayin mai bambanta a wuri na hasken cikin gida, inda kusanci da tushen haske da toshewa ke haifar da bambance-bambance na gida masu mahimmanci. Tsarin AR na kasuwanci (misali, ARKit) suna ba da ƙididdiga na asali na haske amma ba su da inganci don sake haskaka abubuwa cikin gaskiya.

Wannan takarda ta gabatar da hanyar ainun lokaci don kiyaye hasken cikin gida mai bambanta a wuri daga hoton RGB guda. Idan aka ba da hoto da wurin pixel na 2D, Cibiyar Sadarwar Jijiya ta Convolutional (CNN) tana hasashen wakilcin Harmonics na Siffar Zobe (SH) na mataki na 5 na hasken a wannan takamaiman wuri a ƙasa da 20ms, yana ba da damar shigar da abu na zahiri a ko'ina a cikin wurin.

2. Hanyoyi

Babban ra'ayi shine horar da CNN don koma baya ga coefficients na Harmonics na Siffar Zobe dangane da wurin hoto na 2D.

2.1 Tsarin Cibiyar Sadarwa

Cibiyar sadarwa tana ɗaukar shigarwa guda biyu: hoton RGB da aka shigar da da'irar 2D $(u, v)$ wanda aka daidaita zuwa $[-1, 1]$. Hoton yana wucewa ta cikin mai ɓoyayyen sifa (misali, dangane da ResNet). Ana sarrafa da'irar 2D ta cikin sassan da aka cika don samar da ɓoyayyen wuri. Sifofin hoto da ɓoyayyen wuri ana haɗa su, yawanci ta hanyar haɗawa ko hanyoyin kulawa, kafin mai ɓoyayyar bayani ta ƙayyade ƙarshen coefficients na SH don tashoshi na RGB. Wannan ƙira tana ƙayyadad da hasashen haske akan wurin sarari.

2.2 Wakilcin Harmonics na Siffar Zobe

Ana wakilta haske a wani batu ta amfani da Harmonics na Siffar Zobe na mataki na 5. SH yana ba da ƙayyadaddun wakilci, dangane da mitar, na aiki akan siffar zobe. Irradiance $E$ a wurin saman tare da al'ada $\mathbf{n}$ ana kiyasta shi kamar haka:

$E(\mathbf{n}) \approx \sum_{l=0}^{L} \sum_{m=-l}^{l} c_{l}^{m} Y_{l}^{m}(\mathbf{n})$

inda $L=5$, $Y_{l}^{m}$ su ne ayyukan tushe na SH, kuma $c_{l}^{m}$ su ne coefficients da cibiyar sadarwa ta hasashe (coefficients 9 kowane tashar launi, jimlar 27 don RGB). Wannan ƙaramin fitarwa shine makullin zuwa ƙididdiga na ainun lokaci.

3. Gwaje-gwaje & Sakamako

3.1 Kimantawa ta Ƙididdiga

An kimanta hanyar akan tarin bayanai na roba da na gaske. Ma'auni sun haɗa da Kuskuren Angular tsakanin taswirorin muhalli da aka hasashe da na gaskiya da RMSE akan abubuwan da aka zana. Hanyar da aka gabatar mai bambanta a wuri ta ci gaba da fi da hanyar ƙididdiga na haske na duniya na Gardner da sauransu [8], musamman ga wurare da suka yi nisa da tsakiyar hoton inda haske ya bambanta.

3.2 Nazarin Mai Amfani

An gudanar da nazarin fahimta na mai amfani inda mahalarta suka kwatanta abubuwa na zahiri da aka sake haskaka ta amfani da haske daga hanyoyi daban-daban. Sakamakon ya nuna fifiko mai ƙarfi (kusan 75%) ga zane-zanen da aka samar ta amfani da hasken da aka gabatar mai bambanta a wuri fiye da waɗanda aka yi amfani da ƙididdiga na duniya daga [8], yana tabbatar da mahimmancin fahimta na tasirin haske na gida.

3.3 Aikin Ainun Lokaci

Cibiyar sadarwa ta cimma lokutan ƙididdiga na ƙasa da millisecond 20 akan GPU na matakin laptoɗp (Nvidia GTX 970M). Wannan aikin yana ba da damar aikace-aikacen AR na ainun lokaci inda za a iya sabunta haske nan da nan yayin da abu na zahiri ko kyamara ke motsawa.

4. Bincike na Fasaha & Fahimta ta Asali

Fahimta ta Asali: Babban nasarar takardar ba wani ƙirar ƙididdiga na haske kawai ba ce; juyawa ce mai dabarun daga tsarin haske na mai da hankali kan wuri zuwa na mai da hankali kan batu. Yayin da aikin fasaha na baya kamar na Gardner da sauransu (wanda sau da yawa ake kwatanta shi da ƙa'idodin fassarar hoto-zuwa-hoto irin na CycleGAN don matsalolin da ba su da tabbas) suka ɗauki hoton gaba ɗaya don fitar da haske ɗaya na duniya, wannan aikin ya gane cewa ga AR, hasken da kawai ke da mahimmanci shine hasken a takamaiman wurin shigarwa. Wannan sauyi ne mai zurfi wanda ya dace da buƙatun zane-zane na ainun lokaci, inda masu inuwa ke ƙididdige haske kowane guntu, ba kowane wuri ba.

Kwararar Ma'ana: Ma'anar tana da sauƙi mai kyau: 1) Amince da bambancin sarari a matsayin matsala ta farko a cikin saitunan cikin gida (wanda aka goyi bayan ƙa'idodin radiometry na asali daga manyan tushe kamar Lissafin Zane-zane na Kajiya). 2) Zaɓi wakilci (SH) wanda duka yana bayyana ra'ayi don hasken cikin gida mai ƙarancin mitar kuma ya dace da masu zane-zane na ainun lokaci (misali, ta hanyar PRT ko kimanta SH kai tsaye a cikin masu inuwa). 3) Ƙirƙiri cibiyar sadarwa wanda a fili yana ɗaukar wuri a matsayin shigarwa, yana tilasta masa koyon taswirar daga mahallin hoton gida zuwa sigogin SH na gida. Bayanan horo, wanda wataƙila aka samar daga wuraren roba ko wuraren 3D da aka ɗauka tare da sanannen haske, yana koya wa cibiyar sadarwa don danganta alamun gani (inuwowi, zubar da launi, fitattun haske) da yanayin haske na gida.

Ƙarfi & Kurakurai: Babban ƙarfi shine amfaninsa. Lokacin aiki na <20ms da fitarwar SH sun sa ya zama maganin "shigar da" ga injunan AR na yanzu, bambanci mai ban mamaki da hanyoyin da ke fitar da cikakkun taswirorin muhalli na HDR. Yanayinsa mara lissafi wata dabara ce mai wayo, ta yin amfani da CNN a matsayin wakili don binciken haske mai sarƙaƙiya. Duk da haka, kurakurai suna da mahimmanci. Na farko, a zahiri tsaka-tsaki ne na haske daga bayanan horo. Ba zai iya hasashen haske a cikin yankunan da ba a gani gaba ɗaya ba (misali, a cikin kabad ɗin da aka rufe). Na biyu, SH na mataki na 5, yayin da yake da sauri, ya kasa ɗaukar cikakkun bayanai na haske mai mitar girma kamar inuwowi masu kaifi daga ƙananan tushen haske—iyaka da aka sani na kiyasin SH. Na uku, aikin sa yana da alaƙa da bambancin saitin horonsa; yana iya kasawa a cikin yanayi masu sabon salo.

Fahimta Mai Aiki: Ga masu bincike, hanyar gaba a bayyane take: 1) Ƙirar Haɗin gwiwa: Haɗa ƙaƙƙarfan SH da aka hasashe tare da filin radiance na jijiya mai sauƙi (NeRF) ko ƙaramin saitin hasken batu na zahiri da aka koya don dawo da tasirin mitar girma. 2) Ƙididdiga mara tabbas: Ya kamata cibiyar sadarwa ta fitar da ma'aunin amincewa don hasashenta, mai mahimmanci ga aikace-aikacen AR masu mahimmanci na aminci. 3) Wurare Masu Ƙarfi: Hanyar yanzu ta tsaya. Gaba gaba shine ƙididdiga na haske mai daidaitawa na ɗan lokaci don wurare masu ƙarfi da tushen haske masu motsi, wataƙila ta hanyar haɗa kwararar gani ko cibiyoyin sadarwa masu maimaitawa. Ga masu aiki, wannan hanyar ta shirya don haɗin kai na matukin jirgi cikin ƙa'idodin AR na wayar hannu don haɓaka gaskiyar gaske sosai fiye da abubuwan da SDK na yanzu ke bayarwa.

5. Misalin Tsarin Bincike

Yanayi: Kimanta ƙarfin hanyar a cikin yanayin kusurwa.
Shigarwa: Hoton ɗaki inda wani kusurwa yana da inuwa mai zurfi, nesa da kowane taga ko tushen haske. Za a sanya wani abu na zahiri a cikin wannan kusurwa mai duhu.
Aiwatar da Tsarin:

1. Tambayar Mahalli: Cibiyar sadarwa tana karɓar hoton da da'irar (u,v) na kusurwar da ke da inuwa.
2. Binciken Sifa: Mai ɓoyayyen bayani yana cire sifofin da ke nuna ƙarancin haske, rashin hanyoyin haske kai tsaye, da yuwuwar jefa launi daga bangon da ke kusa (hasken mahalli).
3. Hasashe: Sifofin da aka haɗa suna jagorantar mai ɓoyayyar bayani don hasashen coefficients na SH da ke wakiltar yanayin haske mai ƙarancin ƙarfi, watsawa, da yuwuwar son zuciya na launi.
4. Tabbatarwa: Abu na zahiri da aka zana ya kamata ya bayyana a fili yana da haske kaɗan, tare da inuwowi masu laushi da launuka masu laushi, sun dace da mahallin gani na kusurwa. Rashin nasara zai kasance idan abu ya bayyana yana da haske kamar wanda yake tsakiyar ɗaki, yana nuna cibiyar sadarwa ta yi watsi da yanayin sarari.

6. Aikace-aikace na Gaba & Jagorori

• AR/VR Mai Ci gaba: Bayan shigar da abu, don kasancewar avatar na gaskiya inda mutumin zahiri dole ne a haskaka shi daidai da mahallin gida da suka bayyana suna ciki.
• Daukar Hoto na Lissafi: Gudanar da kayan aikin gyara hoto masu sane da sarari (misali, "sake haskaka wannan mutum" daban da "sake haskaka wannan abu").
• Robobi & Tsare-tsare masu cin gashin kansu: Samar da robobi da saurin fahimta, mara lissafi, na hasken wuri don inganta fahimtar kayan da tsarawa.
• Zane-zane na Jijiya: Yin aiki azaman saurin fifikon haske don ayyukan zane-zane na baya-baya ko don fara ƙirar ƙira masu sarƙaƙi amma masu sauri kamar NeRF.
• Bincike na Gaba: Ƙara zuwa wuraren waje, ƙirar canje-canjen haske masu ƙarfi, da haɗawa da lissafi a fakaice (misali, daga mai ƙididdige zurfi na monocular) don ƙarin ingantaccen tunani na ganuwa.

7. Nassoshi

1. Kajiya, J. T. (1986). Lissafin zane-zane. ACM SIGGRAPH Kwamfuta Graphics.
2. Gardner, M., da sauransu. (2017). Koyon Hasashen Haske na Cikin Gida daga Hoton Guda. ACM TOG.
3. Zhu, J., da sauransu. (2017). Fassarar Hoto-zuwa-Hoto mara Haɗin gwiwa ta amfani da Cibiyoyin Adawa masu Daidaitaccen Zagaye (CycleGAN). ICCV.
4. Ramamoorthi, R., & Hanrahan, P. (2001). Wakilci mai inganci don taswirorin muhalli na irradiance. ACM SIGGRAPH.
5. Apple Inc. (2017, 2018). ARKit Takardun Bayani da Zaman WWDC.
6. Mildenhall, B., da sauransu. (2020). NeRF: Wakiltar Wurare azaman Filayen Radiance na Jijiya don Haɗin Duba. ECCV.
7. Garon, M., Sunkavalli, K., Hadap, S., Carr, N., & Lalonde, J. (2019). Saurin Kiyaye Haske na Cikin Gida Mai Bambanta a Wuri. arXiv:1906.03799.