Ƙididdigar Haske na Cikin Gida Mai Gyara Daga Hoton Guda

1. Gabatarwa

Haɗa abubuwan zamani cikin hotunan duniya na gaskiya da gaske yana da mahimmanci ga aikace-aikace tun daga tasirin musamman zuwa gaskiyar da aka ƙara (AR). Hanyoyin gargajiya kamar haske bisa hoto (IBL) suna buƙatar binciken haske na zahiri, suna iyakance isa ga waɗanda ba ƙwararru ba. Wannan takarda tana magance buƙatar ƙididdigar haske ta atomatik daga hoton guda, tare da mai da hankali kan ƙirƙirar wakilcin da ba kawai daidai ba ne har ma yana da fassara kuma mai gyara ta masu amfani. Kalubalen ainihin yana cikin daidaita gaskiya da sarrafa mai amfani.

2. Ayyukan Da Suka Gabata

Hanyoyin da suka gabata suna nuna alamar ƙara rikitarwa a cikin wakilci:

• Taswirorin Muhalli [11,24,17]: Suna ɗaukar cikakken hasken siffar zobe amma suna haɗa hanyoyin haske da muhalli, suna sa gyara zaɓi ya zama mai wahala.
• Wakilcin Ƙididdiga/Mai Yawa (Lighthouse [25], Li et al. [19], Wang et al. [27]): Suna amfani da ƙididdiga masu yawa ko grid na Gaussians masu siffar zobe don ingantaccen gaskiya, haske mai canzawa a sarari. Duk da haka, suna da ƙididdiga masu yawa kuma ba su da gyara mai sauƙi.
• Wakilcin Ƙididdiga [10]: Suna ƙirƙira hasken mutum ɗaya tare da ƙididdiga masu sauƙi (matsayi, ƙarfi) amma sun kasa ɗaukar cikakkun bayanai masu yawa da ake buƙata don nunin haske mai kama da gaskiya.

Marubutan sun gano gibi: babu wata hanyar da ta wanzu da ta cika duk ma'auni uku don wakilcin mai gyara: rabuwar sassa, sarrafa sauƙi, da sakamako mai kama da gaskiya.

3. Hanyar Da Aka Gabatar

Tsarin da aka gabatar yana ƙididdigar haske daga hoton RGB guda na wurin cikin gida.

3.1. Wakilcin Haske

Sabon abu shine wakilcin haɗaɗɗe:

• Tushen Haske na Ƙididdiga: Haske na 3D mai sauƙi (misali, haske mai shugabanci ko yanki) wanda aka ayyana ta ƙididdiga masu sauƙi kamar matsayi na 3D $(x, y, z)$, alkibla $(\theta, \phi)$, da ƙarfi $I$. Wannan yana ba da damar sauƙin sarrafa mai amfani (misali, motsa haske da linzamin kwamfuta) kuma yana samar da inuwa mai ƙarfi, bayyananne.
• Taswirar Siffa Mai Siffa Ba ta Ƙididdiga ba: Siffar muhalli ta HDR mai haɗawa wacce ke ɗaukar cikakkun bayanai na haske da rikice-rikicen nunawa daga tagogi, saman mai sheki, da sauransu, waɗanda ƙirar ƙididdiga ba za ta iya wakilta ba.
• Tsarin Tsarin Wurin 3D Mai Karkace: Ƙididdigar lissafi (bangon, bene, rufi) don sanya haske daidai da jefa inuwa a sararin 3D.

Ma'auni na zane don wurin saman zai iya zama kusan: $L_o(\omega_o) = L_{o, parametric} + L_{o, texture}$, inda aka tara gudummawar.

3.2. Tsarin Ƙididdiga

An horar da ƙirar koyo mai zurfi don hasashen waɗannan sassa tare daga hoton shigarwa. Yiwuwar hanyar sadarwa tana da rassa daban-daban ko kawuna don hasashen ƙididdigan haske na ƙididdiga, samar da siffar muhalli, da ƙididdigar tsarin ɗaki, yana amfani da bayanan wuraren cikin gida tare da sanannen haske.

4. Gwaje-gwaje & Sakamako

4.1. Ƙididdiga na Ƙididdiga

An kimanta hanyar akan ma'auni na yau da kullun don ƙididdigar haske da shigar abu na zamani:

• Daidaiton Haske: Ma'auni kamar Kuskuren Matsakaicin Matsakaici (MSE) ko Kuskuren Kuskure akan taswirorin muhalli da aka hasashen idan aka kwatanta da gaskiyar ƙasa.
• Ingancin Sake Haske: Ma'auni kamar PSNR, SSIM, ko LPIPS tsakanin zane-zane na abubuwan zamani da aka saka ta amfani da hasken da aka ƙididdige da zane-zane ta amfani da hasken gaskiya.

Takardar ta yi iƙirarin cewa hanyar tana samar da sakamako masu gasa idan aka kwatanta da hanyoyin zamani marasa gyara, yana nuna ƙaramin sadaukarwa a cikin daidaito don babban riba a cikin amfani.

4.2. Ƙididdiga na Halaye

Hoto na 1 a cikin PDF yana tsakiya: Yana nuna hoton shigarwa, sassan hasken da aka ƙididdige, zanen abubuwan zamani da aka saka (armadillo na zinariya da siffar zobe), da zanen ƙarshe bayan mai amfani ya gyara matsayin haske a hanyar hulɗa. Sakamakon ya nuna:

• Inuwa & Nunawa Masu Kama da Gaskiya: Haske na ƙididdiga yana ƙirƙirar inuwa mai ƙarfi mai yuwuwa, yayin da siffar ke ba da haske mai gamsarwa akan abubuwan zinariya.
• Gyara Mai Tasiri Hujjar gani cewa motsa tushen haske yana canza alkiblar inuwa da ƙarfi ta hanyar da ta dace da zahiri, yana ba da damar sarrafa fasaha.

5. Bincike na Fasaha & Fahimta

6. Tsarin Bincike & Misali

Tsari: Tsarin Rabuwa-Kima don AI Mai Gyara

Don bincika takardun "AI mai gyara" iri ɗaya, kimanta tare da axes uku da aka samo daga wannan aikin:

1. Axis na Rabuwa: Ta yaya tsafta ƙirar ta raba abubuwa daban-daban na bambancin (misali, matsayin haske da launin haske da siffar muhalli)? Za a iya gyara su da kansu?
2. Axis na Ƙarfin Sarrafawa: Menene naúrar sarrafa mai amfani? Shin babban matakin zamewa ("haske"), ƙididdiga na matsakaici (haɗin gwiwar haske XYZ), ko sarrafa ƙananan lambobin ɓoye?
3. Axis na Kiyaye Aminci: Lokacin da aka gyara wani sashi, shin fitarwa ya kasance mai yuwuwar zahiri da kuma gaskiya? Shin gyara wani sashi yana haifar da kayan fasaha a wani?

Misalin Aikace-aikace: Kimanta ƙirar "Gyara Hoton Hoto" na hasashe.

• Rabuwa: Shin ya raba haske mai mahimmanci, cika haske, da hasken baya? (Yayi kyau). Ko kuma daidaita haske mai mahimmanci shima yana canza launin fata? (Mummuna).
• Ƙarfin Sarrafawa: Shin mai amfani zai iya ja da tushen haske na 3D na zamani a kusa da fuskar batun? (Yayi kyau, kamar wannan takarda). Ko sarrafawa yana iyakance ga "tsararrun studio" da aka saita? (Ƙasa da gyara).
• Kiyaye Aminci: Lokacin motsa haske mai mahimmanci, shin inuwar da ke ƙarƙashin hanci da haɓa sun sabunta daidai ba tare da haifar da kaifi ko hayaniya mara kyau ba? (Gwaji mai mahimmanci).

Ta hanyar amfani da wannan tsari, mutum zai iya kimanta balagagge da amfanin aiki na kowane tsarin da ke da'awar gyara da sauri.

7. Aikace-aikace na Gaba & Hanyoyi

• AR na Masu Amfani & Kafofin Watsa Labarun Zamantakewa: Ƙididdigar haske na lokaci-lokaci akan na'urorin hannu don tacewa na Instagram mafi gaskiya ko ruwan tabarau na Snapchat waɗanda ke hulɗa daidai da hasken ɗaki.
• Zanen Cikin Gida & Gidaje na Gaskiya: Tsarin zamani inda ba kawai aka saka kayan daki ba har ma an sake haskaka su don dacewa da lokuta daban-daban na rana ko tare da sabbin kayan aikin haske na zamani waɗanda ke jefa inuwa mai gaskiya.
• Fim & Gabatarwar Wasan Kwaikwayo: Saurin toshe saitin haske don wuraren zamani bisa ga hoton wurin duniya na gaskiya da ake nufi.
• Hanyoyin Bincike na Gaba:
  1. Ƙididdigar Haske Da Yawa: Ƙaddamar da wakilcin don sarrafa hanyoyin haske na ƙididdiga da yawa ta atomatik.
  2. Musanya Musanya na Jijiyoyi: Yin amfani da harshe na halitta ko zane-zane masu karkace ("ja inuwa anan") don jagorantar gyare-gyare, yana sa kayan aikin ya fi samun dama.
  3. Fahimtar Wuri Mai Ƙarfi: Ƙididdigar haske a cikin jerin bidiyo, la'akari da hanyoyin haske masu motsi (misali, mutum yana tafiya ta taga).
  4. Haɗawa tare da Ƙirar Waɗanda Aka Watsa: Yin amfani da ƙididdigan haske da aka ƙididdige, masu gyara azaman sharadi don ƙirar ƙirar hoto don ƙirƙirar bambance-bambancen wuri a ƙarƙashin sabon haske.

8. Nassoshi

1. Weber, H., Garon, M., & Lalonde, J. F. Editable Indoor Lighting Estimation. A cikin Proceedings of ... (Takardar yanzu).
2. Debevec, P. (1998). Rendering synthetic objects into real scenes: Bridging traditional and image-based graphics with global illumination and high dynamic range photography. Proceedings of SIGGRAPH.
3. Lombardi, S., et al. (2019). Lighthouse: Predicting Lighting Volumes for Spatially-Coherent Illumination. CVPR.
4. Li, Z., et al. (2020). Learning to Reconstruct Shape and Spatially-Varying Reflectance from a Single Image. SIGGRAPH Asia.
5. Wang, Q., et al. (2021). IBRNet: Learning Multi-View Image-Based Rendering. CVPR.
6. Hold-Geoffroy, Y., et al. (2019). Deep Outdoor Illumination Estimation. CVPR.
7. Zhu, J.Y., et al. (2017). Unpaired Image-to-Image Translation using Cycle-Consistent Adversarial Networks. ICCV.
8. Mildenhall, B., et al. (2020). NeRF: Representing Scenes as Neural Radiance Fields for View Synthesis. ECCV.