Secrets of Autonomous Car Design

©2015 Real-Time Innovations, Inc.

The smart machine era will be the most disruptive in the history of IT-- Gartner 2015

The IIoT Disruption

The real value is a common architecture that connects sensor to cloud, interoperates between vendors, and spans industries

Common technology that spans industries brings bold new approaches and enables fast change

A New Freedom: Cars -> Robot on Wheels

• Faster, safer, cheaper, farther, easier• 30% of all US jobs will end or change– Trucking, delivery, traffic control, urban

transport, child & elder care, roadside hotels, restaurants, insurance, auto body, law, real estate, leisure

• 50% of OEM brands may fail

Technical Challenge

• Rapid evolution!• Complex system integration• On & off vehicle communications• Perception and sensing• Decisions• Safety & certification• Software dominates a mechanical world

RTI’s Experience• ~1000 Projects

– Healthcare– Transportation– Communications– Energy– Industrial– Defense

• 15+ Standards & Consortia Efforts– Interoperability– Multi-vendor ecosystems

RTI’s Deep Expertise in Autonomy• Founders from Stanford

Aerospace Robotics Lab• RTI middleware powers

unmanned systems on land, sea, air, and space

• RTI led the US UAS ground station architecture

• RTI-based system will soon allow drones in class-A National Air Space

• RTI Connext DDS was developed for advanced reactive vehicles

Enable UAS Flight in National Air Space

• The Ground Based Sense and Avoid system allows autonomous planes in US National Air Space– Repositioning– Training & testing– Disaster relief– Forest monitoring and fire

suppression• DO178C safety certified• Operational with RTI Connext

DDS in 2016 Management: US Army UAS Project OfficeSystem integrator: SRC, Inc.

©2015 Real-Time Innovations, Inc. Status Feb 2016

©2015 Real-Time Innovations, Inc. Status Feb 2016

Integrate Intelligence• ADAS (level 2)

– The VW Driver Assistance and Integrated Safety system combines radars, proximity sensors, and video to assist safe operation

– It helps avoid obstacles, detect lane departures, track eye activity, and safely negotiate bends

• Autonomy (level 4)– The V-Charge program demoed an

auto-charging and parking vehicle in 2014

http://www.youtube.com/watch?v=7xQfKTAtyNU

DDS Data-Centric Middleware

DDS is Different!

Data-Centric

DDS

Shared Data Model

DataBus

Point-to-Point

TCP Sockets

Client/Server

MQTTXMPPOPCCORBA

BrokeredESB

Daemon

Publish/Subscribe

FieldbusCANbusZeroMQJMS

Queuing

AMQPActive MQ

The Importance of Data Centricity

Data centricity enables interoperation, scale, & integration

Unstructured filesDatabase

Data Centricity Data at Rest

Messaging middleware

DataBus

Data Centricity Data in Motion

The DDS Standard for the IIoT• The Data Distribution Service (DDS) is

the Proven Data Connectivity Standard for the IoT

• OMG: world’s largest systems software standards org– UML, DDS, Industrial Internet Consortium

• DDS: open & cross-vendor– Open Standard & Open Source– 12+ implementations

Interoperability between source written for different vendors

Interoperability between applications running on different implementations

DDS-RTPS ProtocolReal-Time Publish-Subscribe

Distribution Fabric

DDS API

Why Choose DDS?

• Reliability: Severe consequences if offline for 5 minutes?• Performance/scale:

– Measure in ms or µs? – Or scale > 20+ applications or 10+ teams? – Or 10k+ data values?

• Architecture: System lifecycle >3 yrs?

2 or 3 Checks?

Data Centricity Enables Interoperability• Global Data Space

– Automatic discovery– Read & write data in

any OS, language, transport

– Type Aware– Redundant

sources/sinks/nets• No Servers!• QoS control

– Timing, Reliability, Redundancy, Ordering, Filtering, Security

Shared Global Data Space

DDS DataBus

Signalized Intersection

Vehicle Status

Actuation

Perc

eptio

n

SituationPlan

ning

& N

av

Cloud

Offer: Write situation update 100x/sec

Reliable for 10 secs

Contract

Request: Read positions 10x/secIf distance < 200 m && velocity

> 10

QoS Control

• Handles any link– From data and video switches to low-

bandwidth, lossy space communications

• Implements tunable reliability– Balance throughput and latency

• Enforces timing– Priority, deadlines, nanosecond timestamps

Connects Vehicle/Cloud/InfrastructurePhysio-Control supplies emergency response medical equipment to 60% of the world’s emergency vehicles

"Physio-Control is utilizing RTI Connext DDS to exchange critical patient care information throughout the system of care.“

-- Dale Pearson, VP Data SolutionsWe envision a society in which no person dies from acute, treatable medical events

How Does RTI Help Autonomy Development?

• Ensure reliable data availability• Guarantee real-time response• Manage complex data flow and state• Ease system integration• Build security in from the start• Make deployment flexible• Ease safety certification

©2015 Real-Time Innovations, Inc.

Ensure Reliable Data Availability

• What: Continuous availability >> 99.999%• How: Easy redundancy, no servers

Guarantee Real-Time Response

• What: response < 100us, even with load, complex data types, many flows• How: peer-to-peer, multicast, data path optimization

Manage Complex Data Flow and State

• What: Find and deliver the right information to the right place at the right time

• How: Data centric selective source filtering

Data Centricity Definition

a) The interface is the data. b) The infrastructure understands that data. c) The system manages the data and imposes rules

on how applications exchange data.

Data Centricity Ensures Consistent Operation

Message-Centric• A: Can you meet on 1/23@11:30?• BCD: Yes• B: 23rd conflict, how about 2/20?• AC: OK• C: March 6th is better…• AB: OK• A: Can you stay longer?• D: No; start ½ hour earlier?• A: OK, confirmed!

Data-Centric• A: Add: 1/23 @ 11:30A• B: Change: 2/20 @ 11:30A• C: Change: 3/6 @ 11:30A• D: Change: 3/6 @ 11:00A

B: 3/6

A:3/6

D: 1/23

C: 2/20 1/2311:30

A

B

D

C2/2011:303/6

11:303/6

11:00?

Data Centric Systems Manage State

• Things have attributes and characteristics– Meeting is from 11am - 1pm on

3/6 in Fairfax– Car is blue and travelling north

from Sunnyvale at 65 MPH• … whether they exist in the real-

world, the computer, or both• … whether or not we observe or

acknowledge them

“State” (“data”) is a snapshot of those attributes & characteristics

Best practice:Operate on state directly, not on dialogs about state

Ease System Integration

• What: Manage interfaces between teams and modules• How: Explicit interface design, evolution, and enforcement

System Integration

Cloud ServicesDDS Integrates All Components

Sensing

Planning

Radar, LIDAR Vehicle Platform Navigation

Error Management

Visualization

Situation AnalysisSituation Awareness

Vision FusionCameras, LIDAR,

Radar …

Data Fusion

LoggingVehicle Control

Localization

DDS Bus

Traffic Maps

DDS Bus

VW Cargate ECU

• Connect fast Ethernet bus to slower CANbus• Automated data

translation• Simple pub sub

between busses

©2015 Real-Time Innovations, Inc. Status Feb 2016

Build Security In from the Start

• Dataflow-Level Security– Control r,w access to each data item for

each function– Ensures proper dataflow operation

• Complete Protection– Discovery authentication– Data-centric access control– Cryptography– Tagging & logging– Non-repudiation– Secure multicast

• No code changes!• Plugin architecture for advanced uses

CBM AnalysisPMU Control Operator

State Alarms SetPoint

Topic Security model:• PMU: State(w)• CBM: State(r); Alarms(w)• Control: State(r), SetPoint(w)• Operator: *(r), Setpoint(w)

Make Deployment Flexible

• What: Separate development and deployment designs• How: Full location, chip, and OS transparency

Development ProductionBoard A

Board B Board C

ECU A

ECU B

Sensor Fusion

Mapping Route Planning

Vehicle Control

Sensor Fusion Mapping Route

Planning

Vehicle Control

Ease Safety Certification

• Safety certifiable connectivity platform– Stringent SWaP requirements– Complete certification evidence– Full interoperability with DDS implementations

• DO-178C Level A– Flight management systems

• ISO 26262– Road vehicle functional safety

• IEC 60601 class 3– Medical devices

Available

Soon

Soon

Tenets Of Safety-Critical Software

• Reduce code size• Consider testability in design• Design code to be deterministic

05/03/2023 35

Cross Domain Mapping of ASIL

Domain Domain Specific Safety Levels

Automotive (ISO 26262) QM ASIL-A ASIL-B/C ASIL-D -

General (IEC-61508) (SIL-0) SIL-1 SIL-2 SIL-3 SIL-4

Aviation (DO-178/254) DAL-E DAL-D DAL-C DAL-B DAL-A

Railway (CENELEC 50126/128/129) (SIL-0) SIL-1 SIL-2 SIL-3 SIL-4

Certified Middleware Greatly Eases Safety Cert

• Provides non-stop availability– Decentralized architecture– No single point of failure– Support for redundant networks– Automatic failover between redundant publishers– Dynamic upgrades

• No central server or services• Version-independent interoperability protocol

• Supports subsystem isolation and incremental certification• Controls real-time Quality of Service• Makes missed deadlines and presence visible• Proven in thousands of mission critical systems

37

Connext DDS Cert

• Limits size of distributed system– Suits most onboard systems– Reduces ELOC

• Predictable– No dynamic memory allocation– Applications preconfigured– Integrates with Full Connext DDS non-

certified components05/03/2023 38

Software Development Folder (electronic form) (SDF)

NOTE: This information is provided as a set of files on a DVD. They are not maintained as a folder; instead, additional files are generated which allow these materials to be grouped by requirements. The information is presented in a browseable format so that the information may be viewed as a software development folder based on requirement identification.

The Software Development Folder (SDF) includes at a minimum:

Reference to the applicable requirements.

Reference to the implementation (Design & Code).

Evidence of reviews for the requirements, design, code, test procedures, test results, and structural coverage analyses.

Software test procedures.

Software test results.

White Papers.

Artifact Change history (CM System).

Applicable problem reports.

SQA Audit Reports.

Internal Software Conformity Review (provided separate from the certification data package).

CC1 11.9

11.10

11.13

11.14

11.17

11.18

11.19

Full EvidenceProduct Name Product Description Control Category DO-178C

ReferencePlan for Software Aspects of Certification (PSAC)

Provides the certification (approval) authorities an overview of the means of compliance, and insight into the planning aspects for delivery of the product specific to Connext DDS Cert.

CC1 11.1

Software Quality Assurance Plan (SQAP) Defines the SQA process and activities. CC1 11.5Software Configuration Management Plan (SCMP)

Defines the CM and change control processes. CC1 11.4

Software Development Plan (SDP)

Software Requirements Standard (SRStd)

Software Design Standard (SDStd)

Software Coding Standard (SCStd)

Defines the processes used for requirements analysis, development, and test for the software product. Includes the standards for requirements, design, and code.

CC1 11.2

11.6

11.7

11.8

Software Verification Plan (SVP) Defines the test philosophy, test methods, and approach used to verify the software product.

CC1 11.3

Software Test Plan (STP) Documents the project-specific approach to verifying Connext DDS Cert.

CC1 11.3

Tool Qualification Plan Identifies the tools to be qualified under the current project.

CC2 12.2.2

DO-330

10.1.2Software Requirements Specification (SRS) Defines the software requirements applicable to

Connext DDS Cert. CC1 11.9

Software Vulnerability Analysis (SVA) Identifies potential failure conditions in the software, their potential impact, and proposed mitigation for Connext DDS Cert.

CC1 N/A

Design Components, in Program Design Language (PDL)

Describes the design of Connext DDS Cert. CC1 11.10

Software Configuration Index (SCI)

Software Configuration Index (SCI) Tables

Identifies the software components for Connext DDS Cert with version information necessary to support regeneration of the product. Also includes the documents comprising the data package.

CC1 11.16

Software Life Cycle Environment Configuration Index (SECI)

Identifies the tools used to build and test the software for Connext DDS Cert.

CC1 11.15

Technical White Paper:

- Control-Coupling Verification With VerOLink (VerOLinkWP.pdf)

-

Single topic technical paper providing additional information to the certification authorities and users.

CC2 N/A

Requirements Traceability Document (RTD) Provides traceability from the requirements to all related certification life cycle artifacts including design, code, and test materials for the delivered software product.

CC1 11.9

11.21Software Accomplishment Summary (SAS) Documents the actual versus planned (per PSAC)

activities and results for the project. Provides a summary of the means of compliance used for the software. Justifies any deviations from the plans.

CC1 11.20

Sources Provides the Source files for:

- Connext DDS Cert

- Test procedures.

- Build and test scripts.

CC1 11.11

Results Documents the results of the functional and structural coverage analysis. This includes the actual results and any applicable analyses performed including coverage analysis.

CC1 11.14

11.21

11.22Libraries Linkable versions of the “as tested” product

libraries.CC1 11.12

Verification tools Verification tools are identified and described in the Tool Qualification Plan for Connext DDS Cert.

CC2 12.2

940 High-Level Requirements3,680 Low-Level Requirements3,400 test files99.88% code coverage testing

Savings from DDS Certification Evidence (Avionics)

30,000 ELOC 20,000 ELOC 10,000 ELOC

DO-178C DAL A $3,000,000 $2,000,000 $1,000,000

DO-178C DAL B / ASIL-D $2,550,000 $1,700,000 $850,000

DO-178C DAL C / ASIL-B,C $1,800,000 $1,200,000 $600,000

• DDS certification evidence available at fraction of development cost

• Availability at start of project significantly reduces risk

05/03/2023 40

How Does RTI Help Autonomy Development?

• Ensure reliable data availability• Guarantee real-time response• Manage complex data flow and state• Ease system integration• Build security in from the start• Make deployment flexible• Ease safety certification

From Our Autonomous Car Customers…

• Zero configuration• Very reliable• Connects modules, fast or slow, data intensive or not

– Perception– Map & nav– Connect to backend– Decision– Display & visualization– Vehicle control

• Makes it easy to plug in simulated modules• Shift modules around between boards• Offers many platform options• Deployment is much much more scalable• Functional safety cert ISO 26262• Easy to duplicate modules

– Selectively double or triple redundancy– Keep redundancy almost invisible. Deployment easy

• Integrates cameras, lidar, radar, gps, control, errors. Some are very fast (video) Some very frequent (control).

• QoS adapts flows to problem• We use DDS to put our teams together!

– Define profiles & interfaces between modules• Debugging!

– Tools help figure out errors– Much better error handling. Can solve many without

stopping the system– Real-time distributed data logging. We even log videos.– Great display of all the data– DDS makes it easy to expose data. Just put data on the bus,

decide later who will use it. No users => no load.• Bridges to backend & mobile• Handles data fusion & video streams• We tried using VPN & self defined data formats. DDS

makes security easier. • Web integration is much easier. DDS topic can be directly

received by back end service. No middleman translation.

Would have taken us a year to do it, with less functionality

Summary

• An autonomous car is a robot on wheels• The system needs reliable, flexible, real-time,

secure connectivity• DDS supports development, deployment evolution

– Location transparency– Integration with existing protocols– Test and debug

• Proven, standard middleware eases debugging, development and deployment

• Separation middleware makes certification easier and cheaper