Research Hot Spots of Optical See-Through Augmented Reality Glasses

doi:10.16183/j.cnki.jsjtu.2018.10.014

Abstract

Abstract: This paper gives a survey about hot spots of optical see-through augmented reality glasses. General capabilities of optical see-through augmented reality glasses products are summarized, and two problems are found to limit the development of this area: vergence-accommodation conflict and ghosting. Then investigations in last three decades targeting these two problems are described. Finally we concentrate on the tendency of research about these two problems briefly.

Key words: optical see-through, head mounted display, augmented reality, vergence-accommodation conflict, mutual occlusion capability

CLC Number:

TN873
TP37

DENG Nianchen,YANG Xubo. Research Hot Spots of Optical See-Through Augmented Reality Glasses[J]. Journal of Shanghai Jiaotong University, 2018, 52(10): 1255-1266.

References

［1］SIELHORST T, FEUERSTEIN M, NAVAB N. Advanced medical displays: A literature review of augmented reality［J］. Journal of Display Technology, 2008, 4(4): 451-467. ［2］LEE K. Augmented reality in education and training［J］. TechTrends, 2012, 56(2): 13-21. ［3］REITMAYR G, SCHMALSTIEG D. Collaborative augmented reality for outdoor navigation and information browsing［C］//2nd Symposium on Location Based Services and TeleCartography. Vienna, Austria: IMS, 2004: 31-41. ［4］SUTHERLAND I E. A head-mounted three dimensional display［C］//AFIPS’68 (Fall, Part I). New York, NY, USA: ACM, 1968: 757-764. ［5］CAUDELL T P, MIZELL D W. Augmented reality: An application of heads-up display technology to manual manufacturing processes［C］//25th Hawaii International Conference on System Sciences. Kauai, HI, USA: IEEE, 1992: 659-669. ［6］HOFFMAN D M, GIRSHICK A R, AKELEY K, et al. Vergence-accommodation conflicts hinder visual performance and cause visual fatigue［J］. Journal of Vision, 2008, 8(3): 31-33. ［7］LAMBOOIJ M, FORTUIN M, HEYNDERICKX I, et al. Visual discomfort and visual fatigue of stereoscopic displays: A review［J］. Journal of Imaging Science & Technology, 2009, 53(3): 301-302. ［8］BANDO T, IIJIMA A, YANO S. Visual fatigue caused by stereoscopic images and the search for the requirement to prevent them: A review［J］. Displays, 2012, 33(2): 76-83. ［9］KRAMIDA G. Resolving the vergence-accommodation conflict in head-mounted displays［J］. IEEE Transactions on Visualization and Computer Graphics, 2016, 22(7): 1912-1931. ［10］SHIWA S, OMURA K, KISHINO F. Proposal for a 3-D display with accommodative compensation: 3DDAC［J］. Journal of the Society for Information Display, 1996, 4(4): 255-261. ［11］LOCKHART T E, SHI W. Effects of age on dynamic accommodation［J］. Ergonomics, 2010, 53(7): 892-903. ［12］SUGIHARA T, MIYASATO T. System development of fatigue-less HMD system 3DDAC (3D display with accommodative compensation: system implementation of mk.4 in light-weight HMD［J］. ITE Technical Report, 1998, 22(1): 33-36. ［13］SHIBATA T, KAWAI T, OHTA K, et al. Stereoscopic 3-D display with optical correction for the reduction of the discrepancy between accommodation and convergence［J］. Journal of the Society for Information Display, 2005, 13(8): 665-671. ［14］BOS P J, LI L W, BRYANT D, et al. Simple method to reduce accommodation fatigue in virtual reality and augmented reality displays［J］. SID Symposium Digest of Technical Papers, 2016, 47(1): 354-357. ［15］ROLLAND J P, KRUEGER M W, GOON A. Dynamic focusing in head-mounted displays［J］. Proceedings of SPIE, 1999, 3639: 8. ［16］ROLLAND J P, KRUEGER M W, GOON A. Multifocal planes head-mounted displays［J］. Applied Optics, 2000, 39(19): 3209-3215. ［17］SUYAMA S, TAKADA H, UEHIRA K, et al. A novel direct-vision 3-D display using luminance-modulated two 2-D images displayed at different depths［J］. SID Symposium Digest of Technical Papers, 2000, 31(1): 1208-1211. ［18］SUYAMA S, TAKADA H, UEHIRA K, et al. A new method for protruding apparent 3-D images in the dfd (depth-fused 3-D) display［J］. SID Symposium Digest of Technical Papers, 2001, 32(1): 1300-1303. ［19］SUYAMA S, OHTSUKA S, TAKADA H, et al. Apparent 3-D image perceived from luminance-modulated two 2-D images displayed at different depths［J］. Vision Research, 2004, 44(8): 785-793. ［20］TAKADA H, SUYAMA S, DATE M, et al. Protruding apparent 3D images in depth-fused 3D display［J］. IEEE Transactions on Consumer Electronics, 2008, 54(2): 233-239. ［21］AKELEY K, WATT S J, GIRSHICK A R, et al. A stereo display prototype with multiple focal distances［J］. ACM Transactions on Graphics, 2004, 23(3): 804-813. ［22］LIU S, HUA H. A systematic method for designing depth-fused multi-focal plane three-dimensional displays［J］. Optics Express, 2010, 18(11): 11562-11573. ［23］RAVIKUMAR S, AKELEY K, BANKS M S. Creating effective focus cues in multi-plane 3D displays［J］. Optics Express, 2011, 19(21): 20940-20952. ［24］MACKENZIE K J, HOFFMAN D M, WATT S J. Accommodation to multiple-focal-plane displays: Implications for improving stereoscopic displays and for accommodation control［J］. Journal of Vision, 2010, 10(8): 22. ［25］MACKENZIE K J, DICKSON R A, WATT S J. Vergence and accommodation to multiple-image-plane stereoscopic displays: “Real world” responses with practical image-plane separations?［J］. Journal of Electronic Imaging, 2012, 21: 011002. ［26］CHENG D W, WANG Q F, WANG Y T, et al. Lightweight spatial-multiplexed dual focal-plane head-mounted display using two freeform prisms［J］. Chinese Optics Letters, 2013, 11(3): 31201. ［27］CHENG D W, WANG Y T, HUA H, et al. Design of a wide-angle, lightweight head-mounted display using free-form optics tiling［J］. Optics Letters, 2011, 36(11): 2098-2100. ［28］DUNN D, TIPPETS C, TORELL K, et al. Wide field of view varifocal near-eye display using see-through deformable membrane mirrors［J］. IEEE Transactions on Visualization and Computer Graphics, 2017, 23(4): 1322-1331. ［29］MCQUAIDE S C, SEIBEL E J, KELLY J P, et al. A retinal scanning display system that produces multiple focal planes with a deformable membrane mirror［J］. Displays, 2003, 24(2): 65-72. ［30］SCHOWENGERDT B T, SEIBEL E J, KELLY J P, et al. Binocular retinal scanning laser display with integrated focus cues for ocular accommodation［J］. Proceedings of SPIE, 2003. DOI: 10.1117/12.474135. ［31］SCHOWENGERDT B T, SEIBEL E J. True 3-D scanned voxel displays using single or multiple light sources［J］. Journal of the Society for Information Display, 2006, 14(2): 135-143. ［32］HU X D, HUA H. High-resolution optical see-through multi-focal-plane head-mounted display using freeform optics［J］. Optics Express, 2014, 22(11): 13896-13903. ［33］SUYAMA S, DATE M, TAKADA H. Three-dimensional display system with dual-frequency liquid-crystal varifocal lens［J］. Japanese Journal of Applied Physics, 2000, 39(2): 480-484. ［34］LI G Q, MATHINE D L, VALLEY P, et al. Switchable electro-optic diffractive lens with high efficiency for ophthalmic applications［J］. Proceedings of the National Academy of Sciences of the United States of America, 2006, 103(16): 6100-6104. ［35］LIU S, CHENG D W, HUA H. An optical see-through head mounted display with addressable focal planes［C］//7th IEEE/ACM International Symposium on Mixed and Augmented Reality. Cambridge, UK: IEEE, 2008: 33-42. ［36］LIU S, HUA H. Time-multiplexed dual-focal plane head-mounted display with a liquid lens［J］. Optics Letters, 2009, 34(11): 1642-1644. ［37］LIU S, HUA H, CHENG D W. A novel prototype for an optical see-through head-mounted display with addressable focus cues［J］. IEEE Transactions on Visualization and Computer Graphics, 2010, 16(3): 381-393. ［38］LI Y, WU S T. Polarization independent adaptive microlens with a blue-phase liquid crystal［J］. Optics Letters, 2011, 19(9): 8045-8050. ［39］KONRAD R, PADMANABAN N, MOLNER K, et al. Accommodation-invariant computational near-eye displays［J］. ACM Transactions on Graphics, 2017, 36(4): 81-88. ［40］MATSUDA N, FIX A, LANMAN D. Focal surface displays［J］. ACM Transactions on Graphics, 2017, 36(4): 81-86. ［41］SCHOWENGERDT B T, LEE C M, JOHNSTON R S, et al. 1-mm diameter, full-color scanning fiber pico projector［J］. SID Symposium Digest of Technical Papers, 2009, 40(1): 522-525. ［42］SCHOWENGERDT B T, HOFFMAN H G, LEE C M, et al. Near-to-eye display using scanning fiber display engine［J］. SID Symposium Digest of Technical Papers, 2010, 41(1): 848-851. ［43］SCHOWENGERDT B T, MURARI M, SEIBEL E J. Volumetric display using scanned fiber array［J］. SID Symposium Digest of Technical Papers, 2010, 41(1): 653-656. ［44］SCHOWENGERDT B T, JOHNSTON R S, MELVILLE C D, et al. Invited paper: 3D displays using scanning laser projection［J］. SID Symposium Digest of Technical Papers, 2012, 43(1): 640-643. ［45］HONG J, MIN S, LEE B. Integral floating display systems for augmented reality［J］. Applied Optics, 2012, 51(18): 4201-4209. ［46］HUA H, HU X D, GAO C Y. A high-resolution optical see-through head-mounted display with eyetracking capability［J］. Optics Express, 2013, 21(25): 30993-30998. ［47］HUA H, JAVIDI B. A 3D integral imaging optical see-through head-mounted display［J］. Optics Express, 2014, 22(11): 13484-13491. ［48］LANMAN D, LUEBKE D. Near-eye light field displays［C］//SIGGRAPH’13. Anaheim, California, USA: ACM, 2013. ［49］SONG W T, WANG Y T, CHENG D W, et al. Light field head-mounted display with correct focus cue using micro structure array［J］. Chinese Optics Letters, 2014, 12(6): 60010. ［50］MAIMONE A, FUCHS H. Computational augmented reality eyeglasses［C］//IEEE International Symposium on Mixed and Augmented Reality. Adelaide, SA, Australia: IEEE, 2013: 29-38. ［51］HUANG F C, LUEBKE D, WETZSTEIN G. The light field stereoscope［C］//SIGGRAPH’15. Los Angeles, California, USA: ACM, 2015: 24. ［52］JANG C, LEE C, JEONG J, et al. Recent progress in see-through three-dimensional displays using holographic optical elements［J］. Applied Optics, 2016, 55(3): A71-A85. ［53］AKSIT K, LOPES W, KIM J, et al. Near-eye varifocal augmented reality display using see-through screens［J］. ACM Transactions on Graphics, 2017, 36(6): 181-189. ［54］JANG C, BANG K, MOON S, et al. Retinal 3D: Augmented reality near-eye display via pupil-tracked light field projection on retina［J］. ACM Transactions on Graphics, 2017, 36(6): 190-191. ［55］LEE S, JANG C, MOON S, et al. Additive light field displays: Realization of augmented reality with holographic optical elements［J］. ACM Transactions on Graphics, 2016, 35(4): 60-61. ［56］KIM H J, LEE S K, PIAO M L, et al. Three-dimensional holographic head mounted display using holographic optical element［C］//IEEE International Conference on Consumer Electronics. Las Vegas, NV, USA: IEEE, 2015: 132-133. ［57］YEOM H J, KIM H J, KIM S B, et al. Design of holographic head mounted display using holographic optical element［C］//11th Conference on Lasers and Electro-Optics Pacific Rim. Busan, South Korea: IEEE, 2015: 1-2. ［58］YEOM H J, KIM H J, KIM S B, et al. 3D holographic head mounted display using holographic optical elements with astigmatism aberration compensation［J］. Optics Express, 2015, 23(25): 32025-32034. ［59］MAIMONE A, GEORGIOU A, KOLLIN J S. Holographic near-eye displays for virtual and augmented reality［J］. ACM Transactions on Graphics, 2017, 36(4): 81-85. ［60］JOLLY S, SAVIDIS N, DATTA B, et al. Near-to-eye electroholography via guided-wave acousto-optics for augmented reality［J］. Proceedings of SPIE, 2017. DOI: 10.1117/12.2250582. ［61］CHEN Z D, SANG X Z, LIN Q J, et al. A see-through holographic head-mounted display with the large viewing angle［J］. Optics Communications, 2017, 384: 125-129. ［62］ANDO T, YAMASAKI K, OKAMOTO M, et al. Head-mounted display using a holographic optical element［J］. Proceedings of SPIE, 1998. DOI: 10.1117/12.303654. ［63］VON WALDKIRCH M, LUKOWICZ P, TRSTER G. Spectacle-based design of wearable see-through display for accommodation-free viewing［M］//FERSCHA A, MATTERN F. Pervasive computing—Pervasive 2004: Lecture notes in computer science. Berlin, Heidelberg: Springer, 2004: 106-123. ［64］von WALDKIRCH M, LUKOWICZ P, TRSTER G. Oscillating fluid lens in coherent retinal projection displays for extending depth of focus［J］. Optics Communications, 2005, 253(4): 407-418. ［65］YUUKI A, ITOGA K, SATAKE T. A new maxwellian view display for trouble-free accommodation［J］. Journal of the Society for Information Display, 2012, 20(10): 581-588. ［66］MAIMONE A, LANMAN D, RATHINAVEL K, et al. Pinlight displays: Wide field of view augmented reality eyeglasses using defocused point light sources［J］. ACM Transactions on Graphics, 2014, 33(4): No.89. ［67］KIYOKAWA K. Occlusion displays［M］//CHEN J, CRANTON W, FIHN M. Handbook of visual display technology. Berlin, Heidelberg: Springer, 2016: 1-9. ［68］BIMBER O, FROHLICH B. Occlusion shadows: Using projected light to generate realistic occlusion effects for view-dependent optical see-through displays［C］//International Symposium on Mixed and Augmented Reality. Darmstadt, Germany: IEEE, 2002: 186-195. ［69］NODA S, BAN Y, SATO K, et al. An optical see-through mixed reality display with a realtime range finder and an active pattern light source［J］. Transactions of the Virtual Reality Society of Japan, 1999, 4(4): 665-670. ［70］MAIMONE A, YANG X B, DIERK N, et al. General-purpose telepresence with head-worn optical see-through displays and projector-based lighting［C］//IEEE Virtual Reality. Lake Buena Vista, FL, USA: IEEE, 2013: 23-26. ［71］INAMI M, KAWAKAMI N, SEKIGUCHI D, et al. Visuo-haptic display using head-mounted projector［C］//IEEE Virtual Reality. New Brunswick, NJ, USA: IEEE, 2000: 233-240. ［72］KAMEYAMA K. Tangible modeling system［C］//SIGGRAPH’99. Los Angeles, California, USA: ACM, 1999: 279. ［73］KIYOKAWA K, KURATA Y, OHNO H. An optical see-through display for mutual occlusion of real and virtual environments［C］//IEEE and ACM International Symposium on Augmented Reality. Munich, Germany: IEEE, 2000: 60-67. ［74］KIYOKAWA K, KURATA Y, OHNO H. An optical see-through display for mutual occlusion with a real-time stereovision system［J］. Computers & Graphics, 2001, 25(5): 765-779. ［75］KIYOKAWA K, BILLINGHURST M, CAMPBELL B, et al. An occlusion capable optical see-through head mount display for supporting co-located collaboration［C］//2nd IEEE and ACM International Symposium on Mixed and Augmented Reality. Tokyo, Japan: IEEE, 2003: 133-141. ［76］YAMAGUCHI Y, TAKAKI Y. See-through integral imaging display with background occlusion capability［J］. Applied Optics, 2016, 55(3): A144-A149. ［77］UCHIDA T, SATO K, INOKUCHI S. An optical see-through mr display with digital micro-mirror device［J］. Transactions of the Virtual Reality Society of Japan, 2002, 7(2): 151-157. ［78］CAKMAKCI O, HA Y, ROLLAND J P. A compact optical see-through head-worn display with occlusion support［C］//3rd IEEE and ACM International Symposium on Mixed and Augmented Reality. Arlington, VA, USA: IEEE, 2004: 16-25. ［79］ZHOU Y, MA J T, HAO Q, et al. A novel optical see-through head-mounted display with occlusion and intensity matching support［M］//HUI K C, PAN Z G, CHUNG R C, et al. Technologies for E-learning and digital entertainment—Edutainment 2007: Lecture notes in computer science. Berlin, Heidelberg: Springer, 2007: 56-62. ［80］SANTOS P, GIERLINGER T, MACHUI O, et al. The daylight blocking optical stereo see-through HMD［C］//Workshop on Immersive Projection Technologies/Emerging Display Technologiges. Los Angeles, California, USA: ACM, 2008: 4. ［81］HUA H, LIN Y, GAO C. Occlusion capable optical see-through head-mounted display using freeform optics［C］//IEEE International Symposium on Mixed and Augmented Reality. Atlanta, GA, USA: IEEE, 2012: 281-282. ［82］GAO C Y, LIN Y X, HUA H. Optical see-through head-mounted display with occlusion capability［J］. Proceedings of SPIE, 2013. DOI: 10.1117/12.2015937. ［83］HINCAPI-RAMOS J D, IVANCHUK L, SRIDHARAN S K, et al. Smartcolor: Real-time color and contrast correction for optical see-through head-mounted displays［J］. IEEE Transactions on Visualization and Computer Graphics, 2015, 21(12): 1336-1348. ［84］LANGLOTZ T, COOK M, REGENBRECHT H. Real-time radiometric compensation for optical see-through head-mounted displays［J］. IEEE Transactions on Visualization and Computer Graphics, 2016, 22(11): 2385-2394. ［85］ITOH Y, HAMASAKI T, SUGIMOTO M. Occlusion leak compensation for optical see-through displays using a single-layer transmissive spatial light modulator［J］. IEEE Transactions on Visualization and Computer Graphics, 2017, 23(11): 2463-2473.

[1]	LI Yong, ZHANG Mengjun, QIU Dong, FAN Yunfeng . Design and Development of Data-Driven Augmented Reality Electronic Sand Table for Command and Control System [J]. Air & Space Defense, 2021, 4(2): 27-.
[2]	WANG Yue,ZHANG Shusheng,HE Weiping,BAI Xiaoliang. Model-Based Marker-Less 3D Tracking Approach for Augmented Reality [J]. Journal of Shanghai Jiaotong University, 2018, 52(1): 83-89.
[3]	JING Xu1,QIU Shiguang1,YIN Xuyue1,FAN Xiumin1,2,HE Qichang1,2. Stereo Correction of Binocular Images with Optical Axes Divergent Arrangement [J]. Journal of Shanghai Jiaotong University, 2015, 49(02): 141-149.