Full-colour 3D holographic augmented-reality displays with metasurface waveguides – Nature.com

Azuma, R. T. A survey of augmented reality. Presence: Teleoperators Virtual Environ. 6, 355385 (1997).

Article Google Scholar

Xiong, J., Hsiang, E.-L., He, Z., Zhan, T. & Wu, S.-T. Augmented reality and virtual reality displays: emerging technologies and future perspectives. Light: Sci. Appl. 10, 216 (2021).

Article ADS CAS PubMed Google Scholar

Chang, C., Bang, K., Wetzstein, G., Lee, B. & Gao, L. Toward the next-generation VR/AR optics: a review of holographic near-eye displays from a human-centric perspective. Optica 7, 15631578 (2020).

Article ADS PubMed PubMed Central Google Scholar

Kooi, F. L. & Toet, A. Visual comfort of binocular and 3D displays. Displays 25, 99108 (2004).

Article Google Scholar

Shibata, T., Kim, J., Hoffman, D. M. & Banks, M. S. The zone of comfort: predicting visual discomfort with stereo displays. J. Vis. 11, 11 (2011).

Article PubMed Google Scholar

Cakmakci, O. & Rolland, J. Head-worn displays: a review. J. Disp. Technol. 2, 199216 (2006).

Article ADS Google Scholar

Kress, B. C. & Chatterjee, I. Waveguide combiners for mixed reality headsets: a nanophotonics design perspective. Nanophotonics 10, 4174 (2021).

Article Google Scholar

Gabor, D. A new microscopic principle. Nature 161, 777778 (1949).

Article ADS Google Scholar

Sutherland, I. E. The ultimate display. In Proc. of the IFIP Congress (ed. Kalenich, W. A.) 2, 506508 (Spartan, 1965).

Tay, S. et al. An updatable holographic three-dimensional display. Nature 451, 694698 (2008).

Article ADS CAS PubMed Google Scholar

Blanche, P.-A. et al. Holographic three-dimensional telepresence using large-area photorefractive polymer. Nature 468, 8083 (2010).

Article ADS CAS PubMed Google Scholar

Smalley, D. E., Smithwick, Q., Bove, V., Barabas, J. & Jolly, S. Anisotropic leaky-mode modulator for holographic video displays. Nature 498, 313317 (2013).

Article ADS CAS PubMed Google Scholar

Maimone, A., Georgiou, A. & Kollin, J. S. Holographic near-eye displays for virtual and augmented reality. ACM Trans. Graph. 36, 85 (2017).

Article Google Scholar

Molesky, S. et al. Inverse design in nanophotonics. Nat. Photon.12, 659670 (2018).

Article ADS CAS Google Scholar

Li, Z., Pestourie, R., Lin, Z., Johnson, S. G. & Capasso, F. Empowering metasurfaces with inverse design: principles and applications. ACS Photonics 9, 21782192 (2022).

Article CAS Google Scholar

Jiang, J., Chen, M. & Fan, J. A. Deep neural networks for the evaluation and design of photonic devices. Nat. Rev. Mater. 6, 679700 (2021).

Article ADS Google Scholar

Genevet, P., Capasso, F., Aieta, F., Khorasaninejad, M. & Devlin, R. Recent advances in planar optics: from plasmonic to dielectric metasurfaces. Optica 4, 139152 (2017).

Article ADS CAS Google Scholar

Lee, G.-Y., Sung, J. & Lee, B. Metasurface optics for imaging applications. MRS Bull. 45, 202209 (2020).

Article ADS Google Scholar

Lin, D. et al. Optical metasurfaces for high angle steering at visible wavelengths. Sci. Rep.7, 2286 (2017).

Article ADS PubMed PubMed Central Google Scholar

Song, J.-H., van de Groep, J., Kim, S. J. & Brongersma, M. L. Non-local metasurfaces for spectrally decoupled wavefront manipulation and eye tracking. Nat. Nanotechnol. 16, 12241230 (2021).

Article ADS CAS PubMed Google Scholar

Lawrence, M. et al. High quality factor phase gradient metasurfaces. Nat. Nanotechnol. 15, 956961 (2020).

Article ADS CAS PubMed Google Scholar

Cordaro, A. et al. Solving integral equations in free space with inverse-designed ultrathin optical metagratings. Nat. Nanotechnol. 18, 365372 (2023).

Lee, G.-Y. et al. Metasurface eyepiece for augmented reality. Nat. Commun. 9, 4562 (2018).

Joo, W.-J. & Brongersma, M. L. Creating the ultimate virtual reality display. Science 377, 13761378 (2022).

Article ADS CAS PubMed Google Scholar

Kim, J. et al. Holographic glasses for virtual reality. In ACM SIGGRAPH 2022 Conference Proc. (eds Nandigjav, M. et al.) 33 (ACM, 2022).

Peng, Y., Choi, S., Padmanaban, N. & Wetzstein, G. Neural holography with camera-in-the-loop training. ACM Trans. Graph. 39, 185 (2020).

Article CAS Google Scholar

Shi, L., Li, B., Kim, C., Kellnhofer, P. & Matusik, W. Towards real-time photorealistic 3D holography with deep neural networks. Nature 591, 234239 (2021).

Article ADS CAS PubMed Google Scholar

Peng, Y., Choi, S., Kim, J. & Wetzstein, G. Speckle-free holography with partially coherent light sources and camera-in-the-loop calibration. Sci. Adv. 7, eabg5040 (2021).

Shi, L., Li, B. & Matusik, W. End-to-end learning of 3D phase-only holograms for holographic display. Light Sci. Appl. 11, 247 (2022).

Yeom, H.-J. et al. 3d holographic head mounted display using holographic optical elements with astigmatism aberration compensation. Opt, Express 23, 3202532034 (2015).

Article ADS PubMed Google Scholar

Jeong, J. et al. Holographically customized optical combiner for eye-box extended near-eye display. Opt. Express 27, 3800638018 (2019).

Article ADS PubMed Google Scholar

Yeom, J., Son, Y. & Choi, K. Crosstalk reduction in voxels for a see-through holographic waveguide by using integral imaging with compensated elemental images. Photonics 8, 217 (2021).

Choi, M.-H., Shin, K.-S., Jang, J., Han, W. & Park, J.-H. Waveguide-type Maxwellian near-eye display using a pin-mirror holographic optical element array. Opt. Lett. 47, 405408 (2022).

Article ADS PubMed Google Scholar

Chen, W. T. et al. A broadband achromatic metalens for focusing and imaging in the visible. Nat. Nanotechnol. 13, 220226 (2018).

Article ADS CAS PubMed Google Scholar

Li, Z. et al. Meta-optics achieves RGB-achromatic focusing for virtual reality. Sci. Adv. 7, eabe4458 (2021).

Article ADS CAS PubMed PubMed Central Google Scholar

Kim, C. & Lee, B. Torcwa: GPU-accelerated Fourier modal method and gradient-based optimization for metasurface design. Comput. Phys. Comm. 282, 108552 (2023).

Article CAS Google Scholar

Kingma, D. P. & Ba, J. Adam: A method for stochastic optimization. In Proceedings of the 3rd International Conference on Learning Representations (2015).

Park, J.-S. et al. All-glass, large metalens at visible wavelength using deep-ultraviolet projection lithography. Nano Lett. 19, 86738682 (2019).

Article ADS CAS PubMed Google Scholar

Kim, J. et al. Scalable manufacturing of high-index atomic layerpolymer hybrid metasurfaces for metaphotonics in the visible. Nat. Mater. 22, 474481 (2023).

Article ADS CAS PubMed Google Scholar

Chakravarthula, P., Tseng, E., Srivastava, T., Fuchs, H. & Heide, F. Learned hardware-in-the-loop phase retrieval for holographic near-eye displays. ACM Trans. Graph. 39, 186 (2020).

Article Google Scholar

Choi, S., Gopakumar, M., Peng, Y., Kim, J. & Wetzstein, G. Neural 3D holography: learning accurate wave propagation models for 3D holographic virtual and augmented reality displays. ACM Trans. Graph. 40, 240 (2021).

Choi, S. et al. Time-multiplexed neural holography: a flexible framework for holographic near-eye displays with fast heavily-quantized spatial light modulators. In ACM SIGGRAPH 2022 Conference Proc. (eds Nandigjav, M. et al.) 32 (2022).

Jang, C., Bang, K., Chae, M., Lee, B. & Lanman, D. Waveguide holography for 3D augmented reality glasses. Nat. Commun. 15, 66 (2024).

Hwang, C.-S. et al. 21-2: Invited paper: 1m pixel pitch spatial light modulator panel for digital holography. Dig. Tech. Pap. SID Int. Symp. 51, 297300 (2020).

Article CAS Google Scholar

Park, J., Lee, K. & Park, Y. Ultrathin wide-angle large-area digital 3D holographic display using a non-periodic photon sieve. Nat. Commun. 10, 1304 (2019).

Article ADS PubMed PubMed Central Google Scholar

Kuo, G., Waller, L., Ng, R. & Maimone, A. High resolution tendue expansion for holographic displays. ACM Trans. Graph. 39, 66 (2020).

Article Google Scholar

Jang, C., Bang, K., Li, G. & Lee, B. Holographic near-eye display with expanded eye-box. ACM Trans. Graph. 37, 195 (2018).

Article Google Scholar

Horisaki, R., Takagi, R. & Tanida, J. Deep-learning-generated holography. Appl. Optics 57, 38593863 (2018).

Article ADS Google Scholar

Kim, C., Zimmer, H., Pritch, Y., Sorkine-Hornung, A. & Gross, M. Scene reconstruction from high spatio-angular resolution light fields. ACM Trans. Graph. 32, 73 (2013).

Article Google Scholar

Ronneberger, O., Fischer, P. & Brox, T. U-net: convolutional networks for biomedical image segmentation. In Medical Image Computing and Computer-Assisted Intervention MICCAI 2015 (eds Navab, N., Hornegger, J., Wells, W. & Frangi, A.) 234241 (Springer, 2015).

Ulyanov, D., Vedaldi, A. & Lempitsky, V. Improved texture networks: maximizing quality and diversity in feed-forward stylization and texture synthesis. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition 69246932 (2017).

Excerpt from:

Full-colour 3D holographic augmented-reality displays with metasurface waveguides - Nature.com