|
Table of Contents
|
The Origins and History of Augmented Reality
Augmented reality is defined by Merriam Webster as “an enhanced version of reality created by the use of technology to overlay digital information on an image of something being viewed through a device (such as a smartphone camera)1.” Early AR systems superimposed virtual information on the physical environment2. These early systems allowed for simulations that were used for industrial, aviation, and military purposes3.
The general hardware components of an Augmented Reality system include the processor, display, sensors, and input devices4. Various possible display systems that can be used within the larger AR system include (but aren’t limited to): monitors, optical projection systems, head-mounted displays, eyeglasses, contact lenses, heads up display, virtual retina displays, and EyeTap (which substitutes captured rays of light from environment with computer generated ones), spatial augmented reality, and handheld displays5. Some examples of the kinds of sensors used within AR systems are: GPS, compasses, touch recognition, speech recognition, and eye tracking6.
In the present, the capability exists to try things on virtually before buying them. IKEA has an app that allows you to choose IKEA furniture to buy for your home by placing a given piece in your home using the app7. The Museum of London offers an app that allows you to see the London street you are standing on in the present, as it looked long ago, through your phone8. Museums have apps which overlay information about a painting over that piece when you look at it using your smartphone9. Google Translate can translate a sign you see or a phrase you hear in real-time10. Google Sky Map, if you point it at the sky, can help you identify the planets and stars11. GE uses ARKit to enable service workers to visualize industrial equipment and repair it more efficiently12.
American Airlines currently has an AR prototype that overlays real-time information on the surroundings of an airport terminal13. Ubiquity6 is an app which can create a 3D map of a room in about 30 seconds and can recognize walls and furniture well enough to have a digital ball bounce off of them14. You can invite other people into this digital environment, too. Curate is an app created by Sotheby’s International Reality. The idea of the Curate app is that it is expensive for real estate companies to stage furniture in an apartment, so by using the app, they can put in the dimensions of the room and using AR, overlay possible furniture sets to get an idea of the spacing15. Airplane maintenance crews are using AR to help them repair engines, by using the history of other similar planes16. More experienced maintenance teams can also use AR to help the local maintenance team if there is an issue17.
Timeline of Augmented Reality (1901-2017)
| Year | Event/Accomplishment |
|---|---|
| 1901 | Frank Baum, the author of The Wizard of Oz, described a "Character Maker", which was the first recorded reference to AR (electronic glasses mapped data onto people) |
| 1952 | Morton Heilig, a cinematographer, began creating the Sensorama Machine, the world's first virtual reality machine (the viewer experienced visuals, sounds, vibrations, and smell) |
| 1968 | Ivan Sutherland created the first head-mounted display system, which used computer-generated graphics to show simple wireframe drawings to users |
| 1974 | Videoplace, an 'artificial reality lab' was built by Myron Krueger, a computer activist, and researcher. Users were surrounded in an interactive environment by the combined use of video cameras, projectors, and onscreen silhouettes |
| 1982 | Radar (video imaging for weather stations) was created by software engineer Dan Reitan as the first interactive AR system shown on TV |
| 1990 | 'Augmented Reality' was coined as a term by Tom Caudell, a Boeing researcher |
| 1992 | Virtual Fixtures was developed for the Air Force by Louis Rosenberg as one of the first fully-immersive AR systems, allowing for machinery to be guided virtually and remotely. It was a robotic system complex enough to compensate for how slow 3D graphics were processed at the time |
| 1996 | CyberCode was created, and became the model for future marker-based AR systems because it used 2D markers |
| 1998 | A yellow virtual First-Down marker, created by Sportvision, debuted AR during a live NFL game |
| 1999 | NASA used the Hybrid Synthetic Vision system to fly the x-38 by using AR to overlay map data to provide enhanced visual navigation during flight tests |
| 1999 | Naval researchers began working on the Battlefield Augmented Reality System (BARS), which was a wearable device that worked as both a camera and a computer display |
| 2000 | Hirokazu Koto created ARToolKit, the world's first open-source software library, which uses video tracking to overlay computer graphics in a video camera. It is still widely used |
| 2000 | ARQuake, the world's first outdoor AR game was launched |
| 2003 | The first handheld AR system was created for PDA's (personal digital assistants) by Wagner and Schmalsteig |
| 2003 | The NFL's Skycam became capable of inserting the First&Ten line |
| 2008 | AR was first used commercially in a print ad for a model BMW Mini, which appeared on a computer's screen if the print as was held in front of the computer's camera |
| 2009 | FLARToolKit is created and allows developers to display AR content on web browsers |
| 2012 | Google Glass launches (to mixed reviews) |
| 2013 | Car manufacturers begin to make virtual reality service manuals |
| 2014 | Google announces that Google Glass devices would now ship to consumers, beginning the trend of wearable AR |
| 2016 | Investments in Virtual Reality and Augmented Reality reach $1.1 billion |
| 2016 | PokemonGo launched and reached a peak of 45 million daily users |
| 2017 | Apple announces ARKit and Google launches ARCore, and AR-based apps sky-rocket |
Case Study #1 - "Development of Augmented Reality Training Simulator Systems for Neurosurgery Using Model-Driven Software Engineering"
+*20
This study explains how training for common neurosurgical procedures can be assisted through use of AR simulator programs. The researchers argue that if three-dimensional medical images (brain meshes) can be visualized, trainees can practice using these simulators, without running the risk of damaging healthy tissues in the surrounding areas of an actual brain. “In neurosurgery context, AR visualizes spatial and anatomical information about a patient’s case that were not available before the surgery.” The researchers believe trainees will be able to train in the AR simulator for “either full procedures or a specific surgical task within a predetermined scenario.” They suggested one AR system which “will target desktop machine for open surgery simulations, and the second AR system will target mobile machine for immersive surgery simulations.” The idea of evaluation to be used by this AR system is that “once the user begins to perform the tasks, the simulator will be collecting the events performed by the user and also be measuring various metrics such as tool placement, viewpoint, time…the evaluator of the performance can reconstruct the state of the performance at any point within the performance, this allows the evaluator to replay the performance and analyze the performance in a very robust manner.” The four authors of this study wanted to break down tasks into the simplest components possible, in order to increase the applicability of the technology to a wider range of users.
Case Study #2 - "Better Learning Through Augmented Reality: AR and the Classroom"
+*21
This study dives into how Augmented Reality technology can improve learning in a classroom environment. Augmented Reality has a lot of potential to teach a wide variety of academic subject and can be catered to any individual. Experimental trials around America have been implemented into some classroom, where students are taught information through interactions of both physical and virtual space. Rather than reading through a textbook, a student can learn from their own experience. One trial throws students from the University of Wisconsin into a simulation during the 1967 Dow Day riots and teaches through virtual interviews with protesters and real footage. Another adaption was at the University of New Mexico where it puts Spanish students in a scenario of a murder mystery game where they have to solve the mystery through speaking spanish and immersing themselves in the Albuquerque's Hispanic Culture. This new method of teaching can be very effective and allows professors to access their students more accurately.
Case Study #3 - "Enhancing the Tourism Experience through Mobile Augmented Reality: Challenges and Prospects"
+*22
This article by Chris D. Kounavis, Anna E. Kasimati,and Efpraxia D. Zamani focuses on smartphone applications that emphasize superimposed 3-D imaging or digital enhancements layered on top the users' perception of real world (through their smartphone cameras). More specifically, the study focuses on smartphone applications for tourists. These apps would allow users to point their smartphone cameras towards objects, locations, and buildings in order to receive information associated with these targets that would be layered on top of the real-world images their cameras would display. The degree of visualization and the convenient exposure to data makes information about destinations and venues more accessible to users, all while promoting tourism and education without having to compromise the user's awareness of his or her surroundings. Regarding the connection between the system design of such apps to the safety benefits it would provide to users, physical navigation (walking) within and throughout tourist destinations is being greatly facilitated because of the see-through imaging feature of AR apps in addition to a GPS-like feature that lays out a trail to guide users towards specific locations. Additionally, the "scanning" or "pointing" capability would make asking strangers for directions a last-ditch option for tourists, since a language feature would be installed within these apps in order to assist tourists in terms of translating information and directions into their preferred language.
Analysis - Benefits and Drawbacks of Augmented Reality
The versatility of augmented reality-based technologies in terms of their applications to scientific, medical and business tasks, to name a few, point to their potential applications to everyday life in the near future. Among the broad strengths of this technology is giving users the opportunity to visually and physically interact with projects, events, or even people without having to invest in much physical effort in order to do so. The improvement in object visualization would enhance customer service quality for entrepreneurs.23 Entrepreneurs could get a chance to present their products to clients without the need for close physical proximity between the two, as well as to promote more informative and immersive experiences with products. An example of this would be the influence of augmented reality on the cosmetics industry. Customers could request that manufacturers have them try certain makeup products via layered imaging using the client’s face in addition to the makeup products they wish to see themselves in.24 There is also the potential for AR (augmented reality) technology to efficiently training medical students by enabling students to use computerized models of the human body in order to lessen the need for real bodies or organs for which to practice on, instead providing a simulated learning experience with larger margin of error - in other terms it would be more practical and safer for students.25 Engineers could also build, analyze, and re-construct three-dimensional models of their projects or even components or pieces of their greater projects - essentially building a "3-D blueprint." This would allow for more informed, rational decision-making without the need for the old-fashioned pen and paper blueprint.26 AR within engineering would also prevent the depletion of resources necessary for building that is usually a byproduct of the trial and error nature of the job. In terms of the potential use of AR technology by the public, people could use augmented reality to acquire information and data on venues, markets, and businesses they wish to visit.
However, despite the positive implications of augmented reality's potential use in future society, there are some negative aspects of such technology. Seeing as to how many innovative modern-day technologies that are capable of gathering information on users, future technologies are expected to do so as well, and could even be more aggressive in this aspect. Augmented reality gadget/app developers are expected to incorporate image/face—recognition capabilities into their systems.27 The reason that this is of concern to the public is because such capabilities would give users the ability access information about strangers (such as age, mood, name) just by pointing an AR apparatus at the person with whom the user is talking to,28 and such sensitive information could be imported from a combination of their online profiles and bio-metrics. Therefore, considering the potential speed at which information could be accessed, the availability of a person’s information is alarming.
Apart from privacy concerns, the functionality of augmented reality technologies also raises concerns. The first is image/data display, as many would oftentimes argue that the data or images visible to users would have to be well within their point of view and not off to a side or (in a corner) of a person’s field of vision29, and it is likely that this concern was raised in reaction to the design of Google Glass. There is also concern for the quality of the images that are being produced by the first AR technologies being tested today. This refers to the fact that AR glasses/headsets (such as the HoloLens and Project North Star) tend to distort images due to the users’ body movements or movement of objects surrounding the user, since AR glasses would often compensate for physical movements by formatting or accommodating the images to visually fit within the user’s environment.30 Because of a potential deterrence in image quality, people would argue that the image processing systems within AR lenses should correlate better with natural body movements or background activity in order to display accurate imaging and thereby benefit the safety of users. A third concern regarding AR glasses functionality would be the battery life, as such complex systems are expected to negatively impact battery life, therefore rendering AR use as an unsafe option for professional applications such as surgical procedures, at least for now.31
AR gadgets in general (including phone apps) also bring negative implications that affect would affect users financially and socially. The first concern in these aspects would be the price tag of future technologies, specifically the more technologically advanced/sophisticated AR glasses. As demonstrated by the developer of the HoloLens (Microsoft), the commercial price of this apparatus is $3000 USD,32 and it is expected that future HoloLens or non-mobile AR technologies would be expensive to the consumers within the general public. Thus, the use of AR lenses/headsets would be reserved exclusively for business use before it may expand to practical uses in everyday life (for the next 5-10 years, AR apparatuses could be more or so a luxury possession reserved for business, way before AR technologies could start to become accessible to the mainstream public).33 In addition to the price tag, the public is also concerned with the social implications of using AR technologies. Many argue that modern technology has caused social isolation, and future AR technologies would make users develop socially detrimental (antisocial) behaviors due to the immersive nature of AR apparatuses.34
Future of AR
There is massive potential for popular AR use in the future. Augmented reality will disrupt the supply chain drastically, as well as alter the way that building, development, and construction is executed, to touch on just a few aspects that will shift35. It is hard to postulate the exact trajectory of developments in augmented reality. Much of AR requires major amounts of capital, so it takes time for innovations to happen, because the funding must be in place first. There have been such developments in the last fifty years since the first head-mounted system was developed by Ivan Sutherland, that it is hard to predict where AR will be fifty years from now.
As for potential use, augmented reality could be introduced in construction via AR glasses/headsets such as the HoloLens. The use of HoloLens in work zones would improve efficiency, analysis, and planning by engineers and workers alike. Construction workers and engineers can visualize 3-D project models (blueprints) as mentioned beforehand.36 Though applying augmented reality to the construction industry would seem to be plausible, seeing as to how construction workers and engineers alike can produce a visual model of whole projects or certain elements or structures of that project, augmented reality could both help and hinder those within this industry.
In terms of AR’s analytical value and reliability in bringing a blueprint to life, workers can visualize tools, structures, or even whole buildings and gather intricate details about the structures they analyze, and the simplification of planning lessens room for engineering mistakes as well as providing a better understanding of the physical embodiment of a project.37
However there is room for growth, and the following concerns refer to AR glasses/headsets specifically. The first is image processing. As noted earlier, the visuals can be easily distorted by user movement or activity within the physical environment. This is due to the fact that AR glasses have a tendency to produce higher quality images if the system did not have to process constantly changing information and data associated with movement. Therefore, the user would need to lessen the amount and frequency of his/her movement, in addition to the fact that the physical location surrounding the user would need to be static (unchanging) for good quality imaging to be produced.38 This is a problem especially for the construction industry where work-zones always involve movement of objects and people, and this notion of AR inaccuracy in terms of imaging could potentially put the user and others in danger.
The above ties in to safety and health concerns, as AR lenses such as the Project North Star or Ghost lenses have structures that impede peripheral vision. The structure of these lenses/headsets themselves are enough to make prospective users aware of the serious if not lethal implications of AR gadgets. In the case of the construction industry, the work ethic, efficiency, and safety of workers within construction zones would be jeopardized as they would not be able to see incoming vehicles or objects. This concern applies to mainstream usage as well, because there are more than enough incidents in which people unexpectedly get involved in accidents due to the habit of commuting while being distracted by smartphones.
How is Augmented Reality Related to Other Technology?
Mirror Worlds - Augmented reality connects to mirror worlds as potential apps that can exist for the Microsoft Hololens, which can display google maps, the real time weather, or other instances of a mirror world that can simultaneously appear on the screen of the lens.
Life Logging - Like Life Logging, augmented reality systems can be incorporated into wearable gadgets (headsets or glasses such as Google Glass) that are capable of capturing real-time data for the user to see.
Virtual Worlds - Augmented reality and Virtual Worlds have merged as a result of the Medici Effect and has created the virtual reality where a person can use a device to enter a virtual world while remaining in complete control with augmented reality technologies.
Web Analytics - Developers of augmented reality devices and mobile apps use web analytics to improve their products.
Secure Payment -Both the use of augmented reality technologies and cryptocurrency via Secure Payment methods feature issues with privacy, a modern-technology burden that is a significant cause for concern with respect to identity theft.
Bibliography
"Advantages of Augmented Reality Apps For Businesses.” Rishabh Software - An Enterprise Application Development Company, Rishabh Software, 5 Dec. 2017, https://www.rishabhsoft.com/blog/benefits-of-augmented-reality-apps.
Alcántara, Ann-Marie. Brands Are Finally Embracing Augmented Reality, but Not Without Speed Bumps. 5 March 2018. <http://www.adweek.com/digital/brands-are-finally-embracing-augmented-reality-but-with-speed-bumps-along-the-way/>.
Augmented Reality for iOS. n.d. <https://www.apple.com/ios/augmented-reality/>.
“Augmented Reality.” Limitations of AR, About Augmented Reality, <https://aboutaugmentedreality.blogspot.com/p/limitations-of-ar.html.>
Augmented Reality. n.d. <https://www.merriam-webster.com/dictionary/augmented%20reality>.
"Augmented Reality Taking off in MRO Training." Netherlands Aerospace Centre. N.p., n.d. Web. 12 Apr. 2018.
Augmented Reality – The Past, The Present and The Future. n.d. <https://www.interaction-design.org/literature/article/augmented-reality-the-past-the-present-and-the-future>.
Benson, Bruce, Tom, and David A. "How Will Augmented Reality Support the Tourism Experience? - Destination Think! -." Destination Think! N.p., 08 Mar. 2018. Web. 12 Apr. 2018.
Burch, Aaron. Infographics – History of Augmented Reality . 27 May 2016. <https://touchstoneresearch.com/infographic-history-of-augmented-reality/>.
Eagleson, Roy, et al. "Development of Augmented Reality Training Simulator Systems for Neurosurgery Using Model-Driven Software Engineering." 2017 IEEE 30th Canadian Conference on Electrical and Computer Engineering. 2017.
Heimgartner, Jeffrey. “What Is Augmented Reality and How Can Engineers and Designers Use It?” Engineering.com, Engineering.com, 13 Apr. 2016, <https://www.engineering.com/DesignSoftware/DesignSoftwareArticles/ArticleID/11873/What-Is-Augmented-Reality-and-How-Can-Engineers-and-Designers-Use-It.aspx>
HowStuffWorks. "AR 101: The Basics of Augmented Reality." YouTube. YouTube, 16 Dec. 2016. Web. 12 Apr. 2018.
Javornik, Ana. The Mainstreaming of Augmented Reality: A Brief History. 4 October 2016. <https://hbr.org/2016/10/the-mainstreaming-of-augmented-reality-a-brief-history>.
Kounavis, Chris, et al. “Enhancing the Tourism Experience through Mobile Augmented Reality: Challenges and Prospects .” International Journal of Engineering Business Management, SAGE Publications, 23 July 2012, https://journals.sagepub.com/doi/pdf/10.5772/51644.
Mainelli, Tom. “The Challenge and Opportunity of Augmented Reality.” Recode, Recode, 8 May 2016, <https://www.recode.net/2016/5/8/11635348/the-challenge-and-opportunity-of-augmented-reality>.
Manning, Alex; Powers, Russell; and Pedisich, Xan (2012) "Better Learning Though Augmented Reality: AR in the Classroom,"Anthós: Vol. 4: Iss. 2, Article 1.
Mitchell, Taylor. “The Limitations of AR in Surgery.” Augmented Reality in Medicine, AR In Medicine, 8 May 2017, https://arinmed.com/the-limitations-of-ar-in-surgery/.
Nessmann, Carly. AdsReality . 21 November 2017. <http://adsreality.com/history-of-augmented-reality-infographic/>.
Samit, Jay. 5 Ways Augmented Reality Is Disrupting the Supply Chain . 1 March 2018. <http://fortune.com/2018/03/01/5-ways-augmented-reality-is-disrupting-the-supply-chain/>.
Tarmy, James. Augmented Reality Is Coming for the Real Estate Industry. 27 March 2018. <https://www.bloomberg.com/news/articles/2018-03-27/augmented-reality-is-coming-for-the-real-estate-industry>.
“The Pros And Cons Of Augmented Reality Within The Construction Industry.” Animmersion UK, Animmersion UK, https://www.animmersion.co.uk/the-pros-and-cons-of-augmented-reality-within-the-construction-industry/.
Upson, Sandra. This Startup Makes Augmented Reality Social — and Ubiquitous . 28 March 2018. <https://www.wired.com/story/this-startup-makes-augmented-reality-socialand-ubiquitous/>.
Wired. "Augmented Reality - The Future of Building | WIRED." YouTube. YouTube, 04 Nov. 2017. Web. 12 Apr. 2018. <https://www.youtube.com/watch?v=CRcKpl17JgY>
Zweifach, Sarah. “Democratizing Medical Education through Virtual and Augmented Reality.” Tincture, Tincture, 25 Apr. 2017, <https://tincture.io/democratizing-medical-education-through-virtual-and-augmented-reality-2514bd2be1df.>
