smart user interfaces for connected cars · pdf filesmart user interfaces for connected cars...

26
Smart User Interfaces for Connected Cars Dan Henderlong July 27, 2017

Upload: vankhanh

Post on 08-Mar-2018

241 views

Category:

Documents


2 download

TRANSCRIPT

Page 1: Smart User Interfaces for Connected Cars · PDF fileSmart User Interfaces for Connected Cars Dan Henderlong July 27, 2017

Smart User Interfaces for Connected Cars Dan Henderlong

July 27, 2017

Page 2: Smart User Interfaces for Connected Cars · PDF fileSmart User Interfaces for Connected Cars Dan Henderlong July 27, 2017

© Elektrobit (EB) 2017 | Confidential

Smart User Interfaces for Connected Cars

2

Agenda

• User interfaces - Challenges

• EB‘s Complete UI Package

• The In-Car User Platform (ICUP) EB GUIDE Tool

• Augmented Reality Creator

• Summary

Page 3: Smart User Interfaces for Connected Cars · PDF fileSmart User Interfaces for Connected Cars Dan Henderlong July 27, 2017

© Elektrobit (EB) 2017 | Confidential

Gradual Evolution

Smart User Interfaces for Connected Cars

Purely mechanical, rudimentary

Infotainment © Geoffrey C. Fors 2005;

http://www.wb6nvh.com/MTSfiles/Carphone3.htm

First phones Partially automated, touchscreens

Page 4: Smart User Interfaces for Connected Cars · PDF fileSmart User Interfaces for Connected Cars Dan Henderlong July 27, 2017

© Elektrobit (EB) 2017 | Confidential

Smart User Interfaces for Connected Cars

4

Increasing Complexity of User Interfaces

Multiple technologies, disparate systems

Product development cycles

Usability and personalization

OEM brand image

Au

ton

om

ou

s drivin

g

Co

nn

ected cars

Page 5: Smart User Interfaces for Connected Cars · PDF fileSmart User Interfaces for Connected Cars Dan Henderlong July 27, 2017

© Elektrobit (EB) 2017 | Confidential

Smart User Interfaces for Connected Cars

3D cluster instrument + system alerts

Multiple displays + infotainment

Augmented reality + predictive navigation

Demand for New Technologies

Digital cluster instrument + navigation + speech

Extendable | Hybrid | Adaptive | Personalized | Smart

Page 6: Smart User Interfaces for Connected Cars · PDF fileSmart User Interfaces for Connected Cars Dan Henderlong July 27, 2017

© Elektrobit (EB) 2017 | Confidential

Smart User Interfaces for Connected Cars

6

Our Solution: The Complete Package

PLATFORM Reference UI & apps | Navigation | Self-learning HMI | Smart Voice Assistant

TOOLS EB GUIDE Studio | EB GUIDE Speech Extension

FRAMEWORKS EB GUIDE GTF | EB GUIDE STF | AR Creator

Page 7: Smart User Interfaces for Connected Cars · PDF fileSmart User Interfaces for Connected Cars Dan Henderlong July 27, 2017

© Elektrobit (EB) 2017 | Confidential

Smart User Interfaces for Connected Cars

7

In-Car User Platform (ICUP)

• Base platform for in-car infotainment and information displays

• Customizable

• Shorter implementation cycles

Page 8: Smart User Interfaces for Connected Cars · PDF fileSmart User Interfaces for Connected Cars Dan Henderlong July 27, 2017

© Elektrobit (EB) 2017 | Confidential

Smart User Interfaces for Connected Cars

8

In-Car User Platform Architecture

OS (Android)

Libraries & framework (e.g. GTF)

HMI layer

RTOS (e.g. Integrity, QNX)

Application layer

Service layer

Hypervisor or multiple SoCs

Libraries & framework (e.g. GTF)

HMI layer

Application layer

Service layer

Page 9: Smart User Interfaces for Connected Cars · PDF fileSmart User Interfaces for Connected Cars Dan Henderlong July 27, 2017

© Elektrobit (EB) 2017 | Confidential

Smart User Interfaces for Connected Cars

9

Detailed Architecture for One Partition

HM

I lay

er

Ap

p la

yer

Serv

ice

laye

r FW

K &

lib

rari

es

Service Broker Service

Discovery IPC OSAL Persistency

Device abstraction

OpenGL ES

Web kit

GTF

Destination Entry

Map

POI

Route Calculator

Traffic

Guidance

MP3

Spotify

Connectivity

TTS

… ABC

… XYZ

Navigation Media Phone … XYZ Tuner

Navigation Media Phone … XYZ Tuner

STF

Display mgmt.

Notification mgmt.

Layout mgmt.

Page 10: Smart User Interfaces for Connected Cars · PDF fileSmart User Interfaces for Connected Cars Dan Henderlong July 27, 2017

© Elektrobit (EB) 2017 | Confidential

Smart User Interfaces for Connected Cars

10

Reference User Interfaces

• Holistic HMI

• Supports multiple displays and modalities

• Seamless content synchronization between displays

• Customization

Page 11: Smart User Interfaces for Connected Cars · PDF fileSmart User Interfaces for Connected Cars Dan Henderlong July 27, 2017

© Elektrobit (EB) 2017 | Confidential

Smart User Interfaces for Connected Cars

Multimodal Tool

Voice user interfaces

Natural Language Understanding (NLU) | Connected Speech Recognition | Hybrid Speech

Graphic user interfaces

2D | 3D | Composition | Interaction | Animation

Page 12: Smart User Interfaces for Connected Cars · PDF fileSmart User Interfaces for Connected Cars Dan Henderlong July 27, 2017

© Elektrobit (EB) 2017 | Confidential

Smart User Interfaces for Connected Cars

12

EB GUIDE 6

• Multimodal HMI (haptic, touch, speech, gesture) development

• Complex user interfaces • One tool for cross-team collaboration

• WYSIWYG • 3D graphics, animations, effects • Platform agnostic

Page 13: Smart User Interfaces for Connected Cars · PDF fileSmart User Interfaces for Connected Cars Dan Henderlong July 27, 2017

© Elektrobit (EB) 2017 | Confidential

Smart User Interfaces for Connected Cars

13

Multiple Display Support

• Single toolchain (EB GUIDE 6) • Single and distributed multi

System on Chip (SoC) solutions

Hypervisor

SoC

SoC 1 SoC 2

RTOS OS

Apps & HMI Apps & HMI

Apps & HMI Apps & HMI

Single So

C

Mu

lti SoC

Cluster Infotainment

RTOS OS

• Across multiple displays • Model user interfaces • Synchronize content

• Modeling of modality, resolution, and aspect ratio changes

Page 14: Smart User Interfaces for Connected Cars · PDF fileSmart User Interfaces for Connected Cars Dan Henderlong July 27, 2017

© Elektrobit (EB) 2017 | Confidential

Today – One GTF instance/process monolithic HMI using a single model

Smart User Interfaces for Connected Cars

14

Main display state machine

Second display state machine

Speech state machine

Main display

Second display

Speech (in/out)

Data pool

& Events

Page 15: Smart User Interfaces for Connected Cars · PDF fileSmart User Interfaces for Connected Cars Dan Henderlong July 27, 2017

© Elektrobit (EB) 2017 | Confidential

Today – One GTF instance/process monolithic HMI using a single model

Smart User Interfaces for Connected Cars

15

Core

HMI model

State machine State machine State machine State machine

Page 16: Smart User Interfaces for Connected Cars · PDF fileSmart User Interfaces for Connected Cars Dan Henderlong July 27, 2017

© Elektrobit (EB) 2017 | Confidential

Next step – Multiple GTF instances/process non-monolithic HMI using multiple models

Smart User Interfaces for Connected Cars

16

Model service

HMI model

UI

Data

Execution

Extensio

ns

HMI model

UI

Data

Execution

Extensio

ns

HMI model

UI

Data

Execution

Extensio

ns

HMI model

UI

Data

Execution

Extensio

ns

Core

Model service

Core

Model service

Core

Page 17: Smart User Interfaces for Connected Cars · PDF fileSmart User Interfaces for Connected Cars Dan Henderlong July 27, 2017

© Elektrobit (EB) 2017 | Confidential

Goal – Multiple GTF instances/process non-monolithic HMI using multiple models

Smart User Interfaces for Connected Cars

17

Main display state machine

Second display state machine

Speech state machine

Main display

Second display

Speech (in/out)

Data pool

& Events

Data pool

& Events

Data pool

& Events

IPC

IPC

Page 18: Smart User Interfaces for Connected Cars · PDF fileSmart User Interfaces for Connected Cars Dan Henderlong July 27, 2017

© Elektrobit (EB) 2017 | Confidential

EB GUIDE 6 Speech Extension

Smart User Interfaces for Connected Cars

18

• Single toolchain approach • Intuitive system responses and interactive

dialogues • Test directly on the tool

• Cloud-based • Customizable • Platform agnostic

Page 19: Smart User Interfaces for Connected Cars · PDF fileSmart User Interfaces for Connected Cars Dan Henderlong July 27, 2017

© Elektrobit (EB) 2017 | Confidential

Smart User Interfaces for Connected Cars

19

Smart Voice Assistant

The personal assistant in the car – the new travel companion | Driver and passenger centric | Navigation | Alerts and updates

Hello and welcome!

Hello EB GUIDE!

Page 20: Smart User Interfaces for Connected Cars · PDF fileSmart User Interfaces for Connected Cars Dan Henderlong July 27, 2017

© Elektrobit (EB) 2017 | Confidential

Smart User Interfaces for Connected Cars

20

Self-Learning HMI

• Automatically executes non-driving related tasks:

– Usage-sensitive HMI

– User-sensitive HMI

– Context-sensitive HMI

• Minimizes user distraction

• Increases user trust in vehicle

Page 21: Smart User Interfaces for Connected Cars · PDF fileSmart User Interfaces for Connected Cars Dan Henderlong July 27, 2017

© Elektrobit (EB) 2017 | Confidential

Smart User Interfaces for Connected Cars

21

Predictive Navigation

• Self-learning navigation – frequently used routes and destinations

• Real-time guidance and enhanced vehicle control

• Multiple driver profiles, map data providers, data encryption

Page 22: Smart User Interfaces for Connected Cars · PDF fileSmart User Interfaces for Connected Cars Dan Henderlong July 27, 2017

© Elektrobit (EB) 2017 | Confidential

Smart User Interfaces for Connected Cars

Manual takeover

Lane guidance Route guidance

22

Augmented Reality Creator

Cruise control

Page 23: Smart User Interfaces for Connected Cars · PDF fileSmart User Interfaces for Connected Cars Dan Henderlong July 27, 2017

© Elektrobit (EB) 2017 | Confidential

Smart User Interfaces for Connected Cars

23

Augmented Reality Creator Architecture

Guidance Information

EB GUIDE GTF

EB GUIDE Studio

Lane departure view

ACC view

Navigation view

3D Rendering

Data Fusion

HD Positioning

Lane Fusion

Object Fusion

Pre

dictio

n

Sensors

Camera

Radar

Gyro

Odometer

GPS

Accelerometer

Head Up Display Electronic Horizon (ADASIS, PSD …)

Page 24: Smart User Interfaces for Connected Cars · PDF fileSmart User Interfaces for Connected Cars Dan Henderlong July 27, 2017

© Elektrobit (EB) 2017 | Confidential

Smart User Interfaces for Connected Cars

24

Summary

User experience Intelligent UI Safety and trust • Deliver personalized user

experiences with multimodal HMI

• Offer holistic HMI experience and OEM branding

• Learn about user in context

• Adapt to user preferences

• Offer user context-specific information

• Minimize user distraction

• Increase user trust in vehicle

• Provide the required functional safety levels

From human machine interfaces to human machine relationships

Page 25: Smart User Interfaces for Connected Cars · PDF fileSmart User Interfaces for Connected Cars Dan Henderlong July 27, 2017

Get your free copy of EB GUIDE 6 at EB-GUIDE.com

eb-guide.com automotive.elektrobit.com