VTT TECHNICAL RESEARCH CENTRE OF FINLAND LTD

Holonovel WorkshopIntroductionEAI International Conference on Technology, Innovation, Entrepreneurship and EducationSEPTEMBER 11–12, 2017 | CANTERBURY, UK

Tiina KymäläinenVTT Technical research centre of Finland Ltd.Human Factors, Virtual- and Augmented Reality

211/09/2017 2

Holodeck – Wikipedia definition

§ The holodeck is a plot device used in stories set within the Star Trek universe

§ Permit stories with locations and characters that could not otherwise exist in the Star Trek setting

https://www.youtube.com/watch?v=oZwtVz7z0wM

• Fictional technology – but it has been predicted that similar virtual reality tools "will become a consumer-ready product by 2024”

https://en.wikipedia.org/wiki/Holodeck

311/09/2017 3

Science Inspiration§ The Star Trek Holodeck was inspired by inventor and

holographer, Gene Dolgoff, who is also the inventor of the digital projector

§ His holography laboratory, built in 1964, was the first holography lab in New York City

411/09/2017 4

Episodes in Star TrekStar Trek ®, Star Trek: The Next Generation ®, Star Trek: Deep Space Nine ® and Star Trek: Voyager ® are trademarks of Paramount Pictures registered in the United States Patent and Trademark Office

§ Star Trek animated series: § "The Practical Joker” (1974), Season 2, Episode 3

§ Next Generation:§ "The Big Goodbye” (1988): Season 1, Episode 12§ "Elementary, Dear Data" (1988): Season 2, Episode 3§ "A Fistful of Datas” (1992): Season 6, Episode 8

§ Voyager:§ "Author, Author”, "Flesh and Blood”, "Nothing Human”

§ Star Trek: Enterprise: § “Unexpected” (2001), Season 1, Episode 5

511/09/2017 5

Applications§ Starfleet personnel use holodecks for training, diagnostics and

recreation§ Simulate settings and events for analysis, such as to explore

crime scene or for scientific experimentation§ Holodeck variations of being by the sea, in a forest and being

trapped in a snow-storm§ An example of the holodeck's recreational functions are the

holosuites

611/09/2017 6

“Technology”

§ Holodecks use two major subsystems: Holographic image and the conversion of matter

§ The holodeck is depicted as an enclosed room in which objects and people are simulated by a combination of:

§ Transported matter (Holomatter)§ Tractor beams§ Shaped force fields§ Replicated matter

§ Controlled by voice commands and physical controls§ A hidden computer terminal allows access to the holodeck controls

https://en.wikipedia.org/wiki/Holodeck

711/09/2017 7

How does it really work?Simulation types:§ The walls can generate holographic images which appear to extend for

an unlimited distance § Holograms can be projected into space § Holograms can be augmented with force beams to simulate solid,

tangible objects § Holograms and force beams can be augmented with replicator

technology to provide actual substance § Magnificent: If an object's status changes the simulation type will change

seamlessly. For example, someone sees a distant tree (walls), approaches it (hologram), leans on it (force beams), breaks off a branch (Holodeck-matter), then picks and eats an apple (replicated).

Holodeck and Computers FAQ, Joshua Bell: http://www.calormen.com/star_trek/FAQs/holodeck-faq.htm

811/09/2017 8

Applications§ Microsoft IllumiRoomCreates illusions of the area surrounding TV§ The Void, Utah-based startup: IMAX of VR

experiences, multiplayer VR environment§ University of Illinois at Chicago: CAVE2:

model global weather patterns and help doctors practice surgeries

§ Real Life Holodeck with an Oculus Rift –Computerphile

§ VTT: Hololens approach: shared multiuser MR experience

§ Etc. (growing exponentially)

911/09/2017 9

1011/09/2017 10

Holonovels

§ Writing stories and plotlines for the holodeck is an activity pursued by people known as holonovelists

§ Most holodeck programs shown in the episodes run in first person "subjective mode”

§ User actively interacts with the program and its characters§ Raise interesting philosophical and sociological issues of virtual

realities§ A holonovel "Photons Be Free" authored by the Doctor (Episode:

"Author, Author")

1111/09/2017 11

"Worst Case Scenario"

§ Star Trek: Voyager , Season 3, Episode 25

1211/09/2017 12

HolonovelsDesign Principles

§ Current mixed reality environments: operates between projected reality (VR) augmented reality (AR) – transported matter – and real reality – matter/replicated matter

§ Combines screenwriting, dramaturgy, game design, virtual world construction and e.g. intelligent environment research

§ Occupies space and highlights the role of artefacts§ E.g. bases on sequence construction

§ Instead of linear timeline follows hierarchy (IF-THEN rules)§ Need: Lack of design principles for building

VR/AR/HMD/Hololens applications

1311/09/2017 13

Intelligent Environment (IE) design principles

§ The scene represents combination of triggering event(s), (selected preconditions) and the intended state of IE

§ Computer vision and pattern recognition: Environmental (real world) scene is a place in which we can move and most of the “objects” are within reach (Oliva and Torralba, 2001)

§ The state is the current status of the intelligent environment constructed by the state of an individual device or the aggregate states of a group of devices

Tiina Kymäläinen, Johan Plomp, Timo Tuomisto, Juhani Heinilä and Timo Urhemaa: Designing smart living for ageing Alice – and the persons next door, Intelligent Buildings International, Taylor & Francis, 2015.

1411/09/2017 14

Intention Awareness

Kymäläinen Tiina, Plomp Johan, Heikkilä Päivi and Ailisto Heikki: Intention Awareness: A Vision Declaration and Illustrating Scenarios, Conference proceedings of International Conference on Intelligent Environments, IEEE Computer Society Publications, Shanghai, China, 2014

1511/09/2017 15