Applications of Augmented Reality in Education Using Microsoft Hololens

Quinten Whitaker, Ronald Brooks, David Howard, Tyler Crawley, Trevor BradfordComputer Science Department

College of Computing and Software EngineeringKennesaw State University

ABSTRACTIn this study we look at the history of augmented reality up until where it is today, and then we look into the potential future applications and implications primarily within the realm of education. We explore one particular medical study that surveyed the realism of holographic simulation to visualize internal body parts. We also explore an implementation, loading a hologram into a virtual environment to understand the technology better.

Keywords: Augmented Reality, Hololens

INTRODUCTION/BACKGROUND

Augmented reality is becoming more useful and popular in our modern world. We see devices such as Microsoft’s Hololens, Google Glass, and even mobile apps such as Pokemon Go using similar technology. The basic principle is being able to see what is in front of you as you do normally, but also have some sort of projections or interface that overlays or interacts with it.

This technology has actually been around for longer than many may expect. It started way back in 1968 when Ivan Sutherland developed the first head-mounted display system which simply used computer graphics to show the wearer wireframe drawings in 3d space(Augment.com). The next advancement was in 1974 when Myron Krueger built an AR lab called Videoplace, which combined projectors with video cameras to surround users in an interactive environment(Augment.com). The actual term “Augmented Reality” did not come about until 1990 until Tom Caudell, a researcher at Boeing, coined it. The first development within the military industry is seen in 1992 when Louis Rosenberg developed an exoskeleton that allows the military to control virtually guided machinery from a remote operating space(Augment.com). The first use within the arts is seen in 1994 when Julie Martin creates a Theater production called “Dancing in Cyberspace” where acrobats dance alongside virtual objects on stage(augment.com). Yet another industry is affected by virtual reality as technologies emerge to allow for the 1st and 10 lines for NFL football games are projected onto the field for television in 1998(augment.com). Advancements are continually made in all these industries from here as well as a significant development led by Hirokazu Kato who creates the ARToolKit, which is an open-source software library for AR

development(augment.com). Getting closer to today we see significant advances made in AR, in the hardware, in the market, and the applications. In 2013 car manufacturers began using AR to allow service technicians to visualize the repair process in front of them(augment.com). In 2014 Google announced Google Glass, which is the beginning of consumer wearable AR and a huge leap in popularization of the technology. As of 2015 the total Augmented Reality and Virtual reality investments reached 700 million, and by 2016 they hit 1.1 billion. Bringing us closer to the present, we see the release of Microsoft's Hololens, as well as its developer kits. This is another monumental leap as this technology actual allows for the projection of 3D structures in real time as opposed to Google glass, which only featured a screen overlay.

UNDERSTANDING THE TECHNOLOGY

Because Microsoft’s hololens is currently the apex of wearable AR, it serves as a good example of the technology that we can analyze. It works as a wearable head device pictured above. It is powered by an on board windows 10 OS. It is able to display the objects as if they exist in actual

space via several cameras/sensors. Cameras which take in visual data, and sensors like a gyroscope to keep track of head movements. Through its sensors it can analyze and understand the environment, mapping it out so that the software can build a virtual model to use. Input from the users can be taken in by gestures or by a remote that comes with the device. Some of the software that is already available on the market include HoloTour, HoloAnatomy, and HoloEngine.

There are several games available such as RoboRaid, Fragments, and Young Conker, all utilizing the technology in various ways to create new interactive experiences.

Lastly there are applications such as Skype and Actiongram which simply allow users to interact in new ways. Either by sharing videos of themselves in 3D or sharing homemade holograms just as you would share photos on Instagram.

IMPLEMENTATION

As part of our research we studied different applications for educational use with the Hololens. We first started by testing out the built in holograms on the device, then moved on to importing custom models through Unity and Visual Studio, then tried apps such as HoloAnatomy and HoloEngine to view an anatomical model of the human body and a 3d model of a car engine in 3d space.

DISCUSSION/FUTURE IMPLICATIONSIt is apparent that we have come a long way collectively in developing this technology. But this is only the beginning. Looking at the trends, looking at how the technology has progressed within almost every industry it is not hard to imagine where it may be in 5, 10, 20 years.

Firstly, we will see refinement of technology as is. Computers are already getting faster, smaller, and more efficient, meaning that wearable technology is becoming more feasible and better overall. We can potentially imagine the hololens getting even smaller and more ergonomic in future

versions, as even having an on board computer will not take up that much space. The cameras and sensors will be certainly see improvement. Camera resolution is always getting better, and the technology used to process these images will become more efficient and faster.

Beyond the hardware and software, more time and experimentation will allow for greater depth and breadth of applications. As developers have more time to work and better technology to work with it is certain that the applications we see now will continue to improve in addition to the possibility of entirely new unforeseen applications. We can imagine that many new games and entertainment applications will be developed, as this industry has grown immensely and continues to do so. Especially as this technology grows more popular, and cheaper, the market will grow as well. Within the realm of design, there is already much potential for designers and repairmen to use the technology to interact with machinery and other devices in new ways. As well, medical professionals can use the technology to interact with the human body in new ways, to see what may be causing problems, or simply to understand it better.

TOOLS/TECHNOLOGY● Microsoft Hololens ● Unity 3D● Microsoft Visual Studio● Microsoft Windows 10● HoloAnatomy● HoloEngine

METHOD In one study conducted by E. Z. Barsom, M. Graafland, and M. P. Schijven, they look across a multitude of studies in the field to access the utility of the AR technology for medical purposes and find a positive correlation overall. They interviewed medical students after using the AR technology to perform certain surgeries or medical practices. Some of the research from this study showed promise while others did not. Particularly, one AR simulator was the CAE VIMEDIX ultrasound simulator. This simulator reconstructed images in relation to a manquin. From their study, they concluded 90% of the medical students agreed that the simulator proved to be highly realistic. Other simulations from the same study proved to be not so realistic or helpful only receiving around 50% of approval.

We did a survey to get an idea of how familiar current students are with augmented reality. Thanks to smartphones, augmented reality is more accessible than ever since

popular apps such as Pokemon Go and Snapchat use some form of augmented reality. We also surveyed the usefulness of augmented reality towards their education.

Another

RESULTS

95% of students have used one or more devices/apps that implement augmented reality. The most popular was smartphone apps Pokemon Go and Snapchat, which 63.2% of surveyed students have used. More niche devices like the Microsoft Hololens and Google Glass had fairly low usage, which is due to the high cost. The cheapest Microsoft Hololens is $3000 and the Google Glass was $1500 when it was available. Almost all students have a smartphone with a camera which makes augmented reality available with a simple app download.

Our second question in the survey we conducted helped determine the usefulness of augmented reality to KSU students. 68.4% of students agreed that augmented reality would benefit them in their education. Almost all of these students have use augmented reality so they have hands on experience of what is possible with it.

CONCLUSION

Within the realm of understanding, all together we see the immense potential of the technology to be applied to education. Students can use it to visualize molecules and microscopic structures of all kinds in new ways. And perhaps most exciting off all, just as we see Skype and Actiongram now, we may see augmented reality in general affect social interactions in new ways, especially if we imagine that one day such wearable devices may become as prevalent as cell phones are today. With a multitude of applications allowing for users to interact with one another. We have had fun speculating if one day we will be able to wear some kind of virtual armor or clothing that wearers of AR technology would be able to see, or perhaps we can display information about ourselves regarding what we like and who we are, so suddenly we may no longer just appear as strangers when standing in a large crowd. And we imagine fun RPG type games to emerge where perhaps we can cast some kind of spells or engage in some kind of virtual combat. Needless to say the possibilities are endless and exciting.

REFERENCES

[1] Barsom, E. Z., Graafland, M., & Schijven, M. P. (2016, February 23). Systematic review on the effectiveness of augmented reality applications in medical training. Retrieved October 16, 2017, from https://link.springer.com/article/10.1007/s00464-016-4800-6#citeas

[2] Charara, S. (2017, May 20). Microsoft HoloLens: Everything you need to know about the $3,000 AR headset. Retrieved October 16, 2017, from http://www.wareable.com/microsoft/microsoft-hololens-everything-you-need-to-know-about-the-futuristic-ar-headset-735

[3] II, D. W. (2016, May 13). The History of Augmented Reality (Infographic). Retrieved October 16, 2017, from http://www.huffingtonpost.com/dennis-williams-ii/the-history-of-augmented-_b_9955048.html