NeedleLight: Tsinke-Tsinke na Tsinke don Kimanta Hasken da Asarar Sufuri na Spherical

1. Gabatarwa & Bayyani

Kimanta hasken daga hoto guda matsala ce mai mahimmanci amma mara daidaito a cikin hangen nesa na kwamfuta da zane-zane, wadda ke da muhimmanci ga aikace-aikace kamar sake haskaka mai ƙarfi (HDR) a cikin ƙarin/gaskiyar gaskiya. Babban kalubale shine fahimtar cikakkiyar yanayi na haske na spherical, HDR daga ƙaramin filin gani, shigarwar mai ƙarancin ƙarfi (LDR). Hanyoyin gargajiya suna ƙirƙirar haske ko dai a cikin yankin mitoci (misali, Spherical Harmonics) ko yankin sarari (misali, taswirar muhalli, spherical Gaussians), kowannensu yana da iyakoki masu mahimmanci. Hanyoyin yankin mitoci ba su da ƙayyadaddun wuri, suna ɓata hanyoyin haske da raunana inuwa. Hanyoyin yankin sarari sau da yawa suna fama da gama gari ko rikitarwar horo kuma ƙila ba su sarrafa bayanan mitoci a sarari ba, wanda ke haifar da rashin daidaiton sake haskawa.

Wannan takarda ta gabatar da NeedleLight, sabon tsari wanda ke haɗa wannan tazara ta hanyar amfani da needlets—wani nau'in wavelet na spherical—a matsayin haɗin gwiwar tushen mitoci-sarari don wakilcin haske. Manyan ƙirƙira sun haɗa da dabarar rarefi don ƙididdiga na needlet da sabon Asarar Sufuri na Spherical (STL) dangane da ka'idar sufuri mafi kyau don jagorantar koma bayan sigogi tare da sanin sarari.

2. Hanyoyi & Tsarin Fasaha

Hanyar NeedleLight tana kimanta ƙididdiga na needlet daga hoton shigar, waɗanda ake amfani da su don sake gina taswirar hasken.

2.1 Tushen Needlet don Hasken

Needlets sune wavelet na spherical na ƙarni na biyu waɗanda ke ba da firam ɗaure a kan siffar spherical, suna ba da ingantattun kaddarorin ƙayyadaddun wuri a cikin mitoci (kamar SH) da sarari (ba kamar SH ba). Aikin haske $L(\omega)$ akan siffar spherical naúrar $S^2$ ana iya rarraba shi kamar haka:

$$L(\omega) = \sum_{j=0}^{\infty} \sum_{k=1}^{N_j} \beta_{j,k} \psi_{j,k}(\omega)$$

inda $\psi_{j,k}$ su ne ayyukan needlet a matakin ƙuduri $j$ da fihirisar wuri $k$, kuma $\beta_{j,k}$ su ne ƙididdiga masu dacewa. Wannan yana ba da damar wakilcin haske mai rikitarwa mai ƙarancin ƙuduri da yawa.

2.2 Tsinke-Tsinke na Needlet ta hanyar Matsakaicin Maƙasudi

Ƙididdiga na needlet na danye na iya zama masu yawa. Takardar ta gabatar da aikin matsakaicin maƙasudi $T_{\lambda}(\cdot)$ da ake amfani da shi yayin horo don haɓaka rarefi:

$$\hat{\beta}_{j,k} = T_{\lambda}(\beta_{j,k})$$

Wannan aikin yana sanya ƙididdiga ƙasa da matsakaicin matsakaici $\lambda$, wanda ake koya ko samo shi dangane da rarraba makamashi. Rarefin yana mai da hankalin samfurin akan mahimman abubuwan haske (misali, manyan hanyoyin haske), yana inganta daidaiton kimantawa da ƙarfi.

2.3 Asarar Sufuri na Spherical (STL)

Don koma bayan ƙididdiga na needlet masu ƙayyadaddun wuri yadda ya kamata, asarar L2 mara hankali bai isa ba. Masu rubutun sun ba da shawarar Asarar Sufuri na Spherical (STL), wanda ya dogara akan ka'idar Sufuri Mafi Kyau (OT). Don tsinkayar taswirar haske da gaskiya $\hat{L}$ da $L$, waɗanda ake ɗauka a matsayin rarraba akan $S^2$, STL yana lissafta nisan da aka gyara na Wasserstein:

$$\mathcal{L}_{STL}(\hat{L}, L) = \inf_{\pi \in \Pi(\hat{L}, L)} \int_{S^2 \times S^2} c(\omega, \omega') d\pi(\omega, \omega') + \lambda_{reg} R(\pi)$$

inda $c(\omega, \omega')$ farashin geodesic ne akan siffar spherical, $\Pi$ shine saitin shirye-shiryen sufuri, kuma $R$ mai daidaitawa ne. STL a zahiri yana la'akari da tsarin sarari na haske, yana haifar da mafi kyawun kiyaye inuwa mai kaifi da iyakokin hanyoyin haske.

3. Sakamakon Gwaji & Kimantawa

An kimanta NeedleLight akan daidaitattun bayanai kamar Laval Indoor HDR da ma'auni na roba.

3.1 Ma'aunin Ƙididdiga

Takardar ta ba da shawarar ma'aunin taswirar haske kai tsaye (misali, kuskuren kusurwa akan siffar spherical) don guje wa ɓarkewar kimantawa dangane da zane. NeedleLight ya ci gaba da fiye da hanyoyin zamani (misali, Garon et al. [15], Gardner et al. [13]) a cikin ma'auni da yawa, yana nuna raguwa mai mahimmanci a cikin kuskure (an ruwaito kamar ~15-20% inganci a cikin kuskuren kusurwa).

Mahimman Abubuwan Ayyuka

Mafi Girman Daidaito: Ƙananan kuskuren kusurwa idan aka kwatanta da hanyoyin tushen SH da SG.
Ingantaccen Gama Gari: Ayyuka masu ƙarfi a cikin fage daban-daban na cikin gida da waje.
Ingantaccen Wakilci: Tsinke-tsinke na needlet suna buƙatar ƙananan sigogi masu aiki fiye da wakilci mai yawa.

3.2 Bincike na Halaye & Kwatancen Gani

Hoto na 1 a cikin takardar yana ba da kwatancen gani mai ƙarfi. Hanyoyi kamar Garon et al. [15] (tushen SH) suna samar da haske mai santsi sosai tare da raunana inuwa. Gardner et al. [13] (tushen SG) na iya dawo da wasu kaifi amma na iya haifar da kayan aiki ko rasa cikakkun bayanai na mitoci masu yawa. Akasin haka, sakamakon NeedleLight ya yi daidai da Gaskiyar Gaskiya, yana ɗaukar shugabanci, ƙarfi, da girman sarari na hanyoyin haske daidai, yana haifar da inuwa mai ƙarfi da haske mai haske akan abubuwa na roba da aka saka.

Bayanin Chati/Hoto: Grid 2x2 yana nuna sakamakon sake haskawa. Subfigure (a) yana nuna sakamako mai ɓacin rai, mara inuwa daga hanyar yankin mitoci. Subfigure (b) yana nuna sakamako tare da wasu ƙayyadaddun wuri amma yuwuwar kayan aiki daga hanyar yankin sarari. Subfigure (c) (Namu) yana nuna sake haskawa mai kaifi, daidai tare da inuwa da aka ayyana da kyau. Subfigure (d) yana nuna Gaskiyar Gaskiya don kwatantawa.

4. Bincike na Cibiyar & Fassarar ƙwararru

Fahimtar Cibiya: NeedleLight ba kawai ingantacciyar ci gaba ba ce; canjin tsari ne wanda ya yi nasarar haɗa yankunan mitoci da sarari don kimanta haske. Babban nasara shine gane cewa haske a zahiri siginar mitoci da yawa, mai ƙayyadaddun wuri ne akan siffar spherical—matsala da ke neman binciken wavelet, ba kawai wakilcin Fourier (SH) ko maki (SG) ba. Wannan ya yi daidai da manyan yanayin sarrafa siginar da ke motsawa fiye da tushen mitoci kawai.

Kwararar Ma'ana: Ma'ana tana da kyau. 1) Gano gazawar hanyoyin yankuna biyu na yanzu. 2) Zaɓi kayan aikin lissafi (needlets) waɗanda ke da kaddarorin haɗin gwiwar ƙayyadaddun wuri na asali. 3) Magance matsalar yawa a cikin wannan kayan aiki (rarefi). 4) Ƙirƙirar aikin asara (STL) wanda ke mutunta lissafin kayan aikin da ƙuntatawar sarari na matsalar. Misali ne na littafi na ingantaccen hanyar bincike mai motsa jiki.

Ƙarfi & Kurakurai: Ƙarfinsa shine tushensa na ka'idar kyakkyawa da ingantaccen aikin da aka nuna. Amfani da Sufuri Mafi Kyau don ƙirar asara yana da wayo musamman, yana tunawa da nasararsa a cikin samfuran samarwa kamar WGANs, yana tabbatar da kwatancen lissafi mai ma'ana. Duk da haka, yuwuwar kuskuren takardar shine rikitarwa mai aiki. Farashin lissafi na warware matsalolin OT akan siffar spherical, ko da tare da kusanci kamar jujjuyawar Sinkhorn, ba ƙaramin abu bane idan aka kwatanta da asarar L2. Duk da ba a bincika shi sosai a cikin PDF ba, wannan na iya hana aikace-aikacen ainihin lokaci—mahimmin amfani don sake haskaka AR/VR. Bugu da ƙari, matsakaicin maƙasudi na rarefi $\lambda$ yana buƙatar daidaitawa a hankali; ƙimar da bai dace ba na iya yanke mahimman abubuwan haske masu rauni kamar hasken cike na yanayi.

Fahimta Mai Aiki: Ga masu aiki, wannan aikin ya kafa sabon ma'auni. Lokacin da daidaito ya fi girma fiye da sauri, tsarin NeedleLight ya kamata ya zama farkon farawa. Ga masu bincike, ƙofa yanzu ta buɗe. Aikin gaba dole ne ya mai da hankali kan inganta sawun lissafi na STL—watakila ta hanyar farashin matrices da aka koya ko masu warware OT na jijiyoyi kamar yadda aka gani a cikin ayyukan kwanan nan daga MIT da Google Research. Wata hanya kuma ita ce bincika iyalai daban-daban na wavelet na spherical ko tsare-tsaren matsakaicin maƙasudi. Babban ra'ayin "wakilcin yanki guda + asara mai sanin lissafi" yana da yuwuwar fitarwa zuwa wasu matsalolin koma bayan spherical a cikin hangen nesa, kamar kimanta zurfin 360° ko ƙirar sama.

5. Cikakkun Bayanai na Fasaha & Tsarin Lissafi

Ginin Needlet: Needlets $\psi_{j,k}(\omega)$ ana ayyana su ta hanyar haɗaɗɗun harmonics na spherical tare da aikin taga $b(\cdot)$ da aka zaɓa a hankali wanda ke raguwa a hankali:

$$\psi_{j,k}(\omega) = \sqrt{\lambda_{j,k}} \sum_{l=0}^{\infty} b\left(\frac{l}{B^j}\right) \sum_{m=-l}^{l} Y_{l,m}(\xi_{j,k}) \overline{Y_{l,m}(\omega)}$$

inda $B > 1$ sigar faɗaɗawa ce, $\{\xi_{j,k}\}$ maki ne na quadrature, kuma $\lambda_{j,k}$ ma'auni ne na cubature. Wannan yana tabbatar da ƙayyadaddun wuri da kaddarar firam ɗaure.

Tsarin Sufuri Mafi Kyau: STL yana amfani da nisan Wasserstein-1. A kan siffar spherical da aka raba tare da maki $N$, yana neman shirin sufuri $\mathbf{P} \in \mathbb{R}^{N \times N}_+$ wanda ke rage:

$$\langle \mathbf{C}, \mathbf{P} \rangle_F \quad \text{s.t.} \quad \mathbf{P} \mathbf{1} = \mathbf{a}, \mathbf{P}^T \mathbf{1} = \mathbf{b}$$

inda $\mathbf{C}_{ij}=c(\omega_i, \omega_j)$ matrix farashin geodesic ne, kuma $\mathbf{a}, \mathbf{b}$ su ne rarraba rarraba na $\hat{L}$ da $L$. Ana amfani da algorithm na Sinkhorn mai daidaitawa na entropy yawanci don ingantaccen lissafi.

6. Tsarin Bincike & Misalin Ra'ayi

Yanayi: Kimanta hasken daga hoton ɗaki tare da taga mai rana da fitilar tebur.

Hanyar SH ta Gargajiya: Za ta samar da saitin ƙididdiga masu ƙarancin matsayi (misali, har zuwa band 2 ko 3). Wannan yana haifar da siffar spherical mai santsi, mai watsewa na haske, ya kasa ware kaifi, hasken shugabanci daga taga (mitoci masu yawa, ƙayyadaddun wuri) daga haske mai laushi, na ƙayyadaddun wuri na fitila (tsaka-tsakin mitoci, ƙayyadaddun wuri). Sakamakon shine matsakaicin haske, mara inuwa.

Tsarin NeedleLight:

Rarrabuwar Needlet: Gaskiyar hasken ana tsinkaya shi akan needlets. Needlets masu ƙuduri mai girma kusa da shugabanci na taga suna aiki da ƙarfi don ɗaukar hasken rana mai kaifi. Needlets masu matsakaicin ƙuduri kusa da wurin fitila suna aiki don ɗaukar haskenta. Needlets masu ƙarancin ƙuduri suna ɗaukar hasken yanayi na ɗaki gabaɗaya.
Rarefi: Aikin matsakaicin maƙasudi yana gano waɗannan ƙididdiga masu ƙarfi, masu ma'ana yayin da yake sanya sifili ga waɗanda ba su da mahimmanci daga wuraren duhu na siffar spherical.
Koma Baya & STL: Cibiyar sadarwa tana koyon tsinkaya wannan saitin ƙididdiga na tsinke. STL yana tabbatar da cewa idan hasken taga da aka tsinkaya ya kauce ko da digiri 10 daga ainihin matsayinsa, yana haifar da babban hukunci daidai da nisan siffar spherical, yana jagorantar cibiyar sadarwa zuwa ƙayyadaddun wuri.
Sake Gina: Ana taƙaita ƙididdiga na needlet na tsinke, suna sake gina taswirar haske tare da haske mai haske, mai kaifi na taga, haske na fitila daban-daban, da daidaitaccen inuwa na yanayi—yana ba da damar shigar da abubuwa na roba na gaskiya.

7. Aikace-aikacen Gaba & Hanyoyin Bincike

Ainihin Lokacin AR/VR: Babban aikace-aikace shine sake haskaka ainihin lokaci na hoto don haɗaɗɗun gaskiya. Aikin gaba dole ne ya inganta NeedleLight don na'urorin hannu da na gefe, yuwuwar ta amfani da distillation ilimi cikin cibiyoyin sadarwa masu sauƙi.
Zane-zane na Jijiyoyi & Juyin Juya Hali na Lissafi: Wakilcin hasken NeedleLight za a iya haɗa shi cikin bututun zane-zane na jijiyoyi na ƙarshe-zuwa-ƙarshe kamar NeRF, yana taimakawa wajen rabuwa da kimanta haske daidai daga lissafi da haske.
Samfuran Samarwa don Hasken: Za a iya amfani da sararin samaniya na needlet na tsinke a cikin cibiyoyin sadarwar adawa masu samarwa (GANs) ko samfuran watsewa don haɗa hasken cikin gida/waje mai yuwuwa, iri-iri don horo ko ƙirƙirar abun ciki.
An Ƙara zuwa Bidiyo: Yin amfani da tsarin a cikin lokaci don daidaitaccen kimanta haske a cikin firam ɗin bidiyo, sarrafa hanyoyin haske masu motsi da inuwa mai motsi.
Fiye da RGB: Haɗa wasu bayanan firikwensin (misali, zurfi daga LiDAR ko kyamarori na ToF) a matsayin ƙarin shigar don ƙara takura matsalar mara daidaito.

8. Nassoshi

Zhan, F., Zhang, C., Hu, W., Lu, S., Ma, F., Xie, X., & Shao, L. (2021). Sparse Needlets for Lighting Estimation with Spherical Transport Loss. arXiv preprint arXiv:2106.13090.
Garon, M., Sunkavalli, K., Hadap, S., Carr, N., & Lalonde, J. F. (2019). Fast spatially-varying indoor lighting estimation. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (pp. 6908-6917).
Gardner, M. A., Hold-Geoffroy, Y., Sunkavalli, K., Gagne, C., & Lalonde, J. F. (2019). Deep parametric indoor lighting estimation. In Proceedings of the IEEE/CVF International Conference on Computer Vision (pp. 7175-7183).
Narcowich, F. J., Petrushev, P., & Ward, J. D. (2006). Localized tight frames on spheres. SIAM Journal on Mathematical Analysis, 38(2), 574-594. (Takardar needlet mai mahimmanci)
Arjovsky, M., Chintala, S., & Bottou, L. (2017). Wasserstein generative adversarial networks. In International conference on machine learning (pp. 214-223). PMLR. (Tushen OT don ML)
Mildenhall, B., Srinivasan, P. P., Tancik, M., Barron, J. T., Ramamoorthi, R., & Ng, R. (2020). Nerf: Representing scenes as neural radiance fields for view synthesis. In European conference on computer vision (pp. 405-421). Springer. (Mahallin don juyin juya hali).