Thomas Losert
HRTC Meeting12 September 2002, Vienna
Introduction to the TTA
12 September 2002 / p.2
Thomas Losert
Outlook
• Requirements• Basic Principles
Composability Dense Time versus Sparse Time Communication System Paradigms Temporal Firewall
• Time Triggered Architecture (TTA) • TTP/C protocol• Bus Guardian• Conclusion
12 September 2002 / p.3
Thomas Losert
Outlook
• Requirements• Basic Principles
Composability Dense Time versus Sparse Time Communication System Paradigms Temporal Firewall
• Time Triggered Architecture (TTA) • TTP/C protocol• Bus Guardian• Conclusion
12 September 2002 / p.4
Thomas Losert
Requirement: Small Jitter
Control Model
Sensor Processing Actuator
Control Object (Vehicle)
We must know the exact time difference between observing and acting
12 September 2002 / p.5
Thomas Losert
Requirement: Reduction of Complexity
Design faults have their root in unmanaged complexity.
If the mental effort required to understand a particular system function grows with the system size, there is an inherent limitation to the size of the systems we can build.
Mental Effort (Perceived Complexity)
Human MentalCapability
SystemSize
12 September 2002 / p.6
Thomas Losert
Requirement: Composability• Compose: “to make or form by combining things, parts, or
elements”
• Composition: “the act of combining parts or elements to form a whole” Webster Encyclopedic Dictionary, 1989, p. 302
• Composability: “The ease of forming a whole by combining parts”
• Parts: The component systems• Whole: A system of systems (SOS).• A composition brings into existence new emerging services of
the SOS that are more than the sum of the prior services of the components.
• These emerging services are the result of the integration of the component systems.
12 September 2002 / p.7
Thomas Losert
Requirement: Safety
• Each device will fail sooner or later• Thus an arbitrary single fault must be
tolerated without degradation of service
12 September 2002 / p.8
Thomas Losert
Outlook
• Requirements• Basic Principles
Composability Dense Time versus Sparse Time Communication System Paradigms Temporal Firewall
• Time Triggered Architecture (TTA) • TTP/C protocol• Bus Guardian• Conclusion
12 September 2002 / p.9
Thomas Losert
Composability• We call an architecture composable with respect to a
specified property, if the system integration will not invalidate this property provided it has been established at the subsystem level, e.g.: Timeliness Testability
• System properties should follow from subsystem properties.
Otherwise the system integrator is left with the challenging task to find out why the system does not work, although all subsystems work according to their specifications.
12 September 2002 / p.10
Thomas Losert
How is the “Integration” achieved?
The component systems are integrated by the exchange of messages across the real-time service interfaces.
Our focus is on what are the contents of a message (data) and when a message is sent and received (time).
We abstract from the low-level (physical, coding) aspects of communication.
We assume that all property mismatches of the interacting systems have been resolved by a connection system.
12 September 2002 / p.11
Thomas Losert
The Four Principles of Composability
Independent Development of the Components (Architecture)The message interfaces of the components must be precisely specified in the value domain and in the temporal domain in order that the component systems can be developed in isolation.
Stability of Prior Services (Component Implementation)The prior services of the components must be maintained after the integration and should not fail if a partner fails.
Performability of the Communication System (Comm. System)The communication system transporting the messages must meet the given temporal requirements under all specified operating conditions.
Replica Determinism (Architecture)Replica Determinism is required for the transparent implementation of fault tolerance
12 September 2002 / p.14
Thomas Losert
Outlook
• Requirements• Basic Principles
Composability Dense Time versus Sparse Time Communication System Paradigms Temporal Firewall
• Time Triggered Architecture (TTA) • TTP/C protocol• Bus Guardian• Conclusion
12 September 2002 / p.15
Thomas Losert
Dense Time versus Sparse Time (1)It is impossible to perfectly synchronize the clocks of nodes in a distributed computer system.
In a reasonable set of clocks each clock differs less than 1 granule g from each other clock.
For reasonable clocks the timestamps of one single event can differ at most by 1 clock tick.
0 1 2 3 4
0 1 2 3 4
0 1 2 3 4
R E F C L K
C L K 1
C L K 2
A
5
5
5
g
12 September 2002 / p.16
Thomas Losert
Dense Time versus Sparse Time (2)
The temporal order cannot be established for events with a difference of 1 granule g.
If the duration between two events is at least three granules, the temporal order can be established alwaysbecause the timestamps differ at least by two ticks.
0 1 2 3 4
0 1 2 3 4
0 1 2 3 4
R E F C L K
C L K 1
C L K 2
A
B
C5
5
5
4 -3 = 1
12 September 2002 / p.17
Thomas Losert
Dense Time versus Sparse Time (3)
a s a s a Real Time
a duration of activitys duration of silence
Duration of activity determined by the granularity of the global time
In a sparse time base events occur only at predefined intervals (events occuring in the silence interval are delayed to the next activity interval).
12 September 2002 / p.18
Thomas Losert
Outlook
• Requirements• Basic Principles
Composability Dense Time versus Sparse Time Communication System Paradigms Temporal Firewall
• Time Triggered Architecture (TTA) • TTP/C protocol• Bus Guardian• Conclusion
12 September 2002 / p.19
Thomas Losert
Communication System Paradigms
Event-triggered (ET) communication systems Temporal control signals primarily derived
from non-time events Flexibility High average performance
Time-triggered (TT) communication systems Activities at predetermined points in time Predictability Dependability
12 September 2002 / p.20
Thomas Losert
Flow Control in Unidirectional Data Transfer
• Information push
• Information pull
• Time-triggered
Sender Receiver
Sender Receiver
Control
Data
Sender Receiver
12 September 2002 / p.21
Thomas Losert
Control Flow and Data Flow in the TTA
SenderCNI
MemoryCNI
Memory Rcvr
Information Push
Ideal for Sender
Information Pull
Ideal for Receiver
Time-Triggered
CommunicationSystem
12 September 2002 / p.22
Thomas Losert
Outlook
• Requirements• Basic Principles
Composability Dense Time versus Sparse Time Communication System Paradigms Temporal Firewall
• Time Triggered Architecture (TTA) • TTP/C protocol• Bus Guardian• Conclusion
12 September 2002 / p.23
Thomas Losert
Concept of a Temporal Firewall
• A temporal firewall is a unidirectional data-sharing interface with state-observations in the interface memory where at least one of the interfacing subsystems accesses the temporal firewall according to an a priori known periodic schedule.
• The interface between the host computer and the communication system can be seen as erecting two unidirectional temporal firewalls: an input firewall and an output firewall .
• A temporal firewalls eliminate control error propagation by design.
12 September 2002 / p.24
Thomas Losert
A Temporal Firewall is a Natural Concept
A temporal firewall is a high-level abstract concept. It is a small and stable unidirectional interface that
provides understandable abstractions of the relevant properties of the interfacing subsystems.
Timeliness is an integral part of the temporal firewall concept.
Conceptually, the RT images in the temporal firewall are closely related to the image presented by a sensor of an analog RT entity in the environment.
Temporal firewalls are thus based on an accustomed view of the world.
12 September 2002 / p.25
Thomas Losert
Stable Properties of Temporal Firewalls
The following stable properties of temporal firewalls are known a priori to all interfacing partners: The addresses (names) and the syntactic structure of the data
items in the temporal firewall. A (abstract) model explaining the meaning of the data items
contained in the temporal firewall. The points on the global time base when the data items in
the temporal firewall are accessed by the TT communication system. This information enables the avoidance of race conditions between the producer and the consumer.
The temporal accuracy of the data items in the temporal firewall. This knowledge is important to guide the information consumer about the minimum rate of sampling of the temporal firewall.
12 September 2002 / p.28
Thomas Losert
TTA Interface: Temporal Firewall
A temporal firewall interface is a unidirectional elementary data flow interface for the exchange
of state information. is located in a dual ported RAM of a communication controller--
update-in-place semantics the instants when data is fetched (delivered) from (to) the
communication system are a priori common knowledge to all communicating partners (error detection!)
eliminates control error propagation since no control signal cross the temporal firewall interface
Input Firewall: Assumptions
Output Firewall: Guarantees
12 September 2002 / p.31
Thomas Losert
Temporal Firewalls and Validation
Assume a host that is encapsulated between two temporal firewalls, and input firewall and an output firewall. These two firewalls form the only interfaces of this host to its environment. The stable properties of the input firewall form important
preconditions for the validation of the component under consideration. Many assumptions about the environment are contained in the specification of this input firewall.
The stable properties of the output firewall form important postconditions of the validation.
In the validation process it must be demonstrated that the postconditions, given in the output firewall specification, are always TRUE, provided the preconditions associated with the input firewall hold.
12 September 2002 / p.32
Thomas Losert
Temporal Firewalls and Composability
A composable architecture must support the Independent development of components--relates to
the architecture Stability of prior services--relates to the components Constructive integration of components--relates to
the communication system. Replica determinism--to support transparent
implementation of fault tolerance.
The temporal firewall concept supports these principles of composability.
12 September 2002 / p.33
Thomas Losert
Outlook
• Requirements• Basic Principles
Composability Dense Time versus Sparse Time Communication System Paradigms Temporal Firewall
• Time Triggered Architecture (TTA) • TTP/C protocol• Bus Guardian• Conclusion
12 September 2002 / p.34
Thomas Losert
What is a “Single” Fault in the TTA?
A Fault-containment region in the TTA is a single chip (System-On-a-Chip--SOC--software and hardware) which is at a physical distance from the other fault containment regions.
Byzantine failures of chips are masked by a proper physical interconnection structure.
It is claimed that in a properly configured TTA-star system, every possible failure mode of any single chip (software or hardware) and nearly any possible failure mode of any single wire is tolerated, without a loss of the timely service.
Failures outside the fault-hypothesis (e.g., concurrent multiple chip failures) are detected with a high probability.
12 September 2002 / p.35
Thomas Losert
Priorities in the TTA
Safety without compromises • No single point of failure• Formal analysis of critical functions
Composability: • Building systems out of prevalidated components--
Component reuse• Fully specified interfaces in the temporal domain and
value domain• Two level design methodology
Flexibility• Flexible reuse of existing components
12 September 2002 / p.36
Thomas Losert
Design Principles of the TTA
Provision of a consistent distributed computing base(Membership service)
Unification of Interfaces• Real-Time Service Interface (TT)
• Diagnostic and Management Interface (ET)
• Configuration and Planning Interface (ET)
Temporal Composability Transparent Fault-Tolerance Scalability and Openness
12 September 2002 / p.37
Thomas Losert
The TTA supports the provision of a global time base to all subsystems a predictable temporal behavior that can be analyzed
a priori., the partitioning of a large system into nearly
autonomous composable subsystems by the introduction of stable interfaces.
the independent development and validation of these subsystems, based on these precise interface specification,.
the application transparent implementation of fault-tolerance by active redundancy.
12 September 2002 / p.38
Thomas Losert
TTA
• Services Message transport with
low latency, minimal jitter Fault-tolerant internal clock synchronization Membership service
• Tolerate arbitrary single faults Replicated medium Controller-state agreement Fail silence (bus guardian)
12 September 2002 / p.39
Thomas Losert
Outlook
• Requirements• Basic Principles
Composability Dense Time versus Sparse Time Communication System Paradigms Temporal Firewall
• Time Triggered Architecture (TTA) • TTP/C protocol• Bus Guardian• Conclusion
12 September 2002 / p.40
Thomas Losert
TTP/C
• TT communication system• Periodic transmission of state messages• Two redundant channels with TDMA
Sending slots TDMA rounds
Host
CNI
TTP/C
Host
CNI
TTP/C
Host
CNI
TTP/C
Host
CNI
TTP/C
Communication Network
12 September 2002 / p.41
Thomas Losert
TTP/C Cluster Operation
In terconnection network (rep licated: 2 channels)
N ode (com m unications contro ller & host com puter)
globa l tim eTD M A schem e
T im e M essage
0:00
1:00
Node 1, Msg. 1
Node 2, Msg. 1
TD M A schem eT im e M essage
0:00
1:00
Node 1, Msg. 1
Node 2, Msg. 1
TD M A schem eT im e M essage
0:00
1:00
Node 1, Msg. 1
Node 2, Msg. 1
TD M A schem eT im e M essage
0:00
1:00
Node 1, Msg. 1
Node 2, Msg. 1
12 September 2002 / p.42
Thomas Losert
Time Division Multiple Access
assigned tonode 0
assigned tonode 1
assigned tonode 2
assigned tonode 3
12:00 3:00 5:00 9:00 12:00
assigned tonode 3
assigned tonode 0
slot 0 slot 1 slot 2 slot 3 slot 0slot 3
TDMA round nTDMA round (n-1) TDMA round (n+1)
Real Time
12 September 2002 / p.43
Thomas Losert
TTP/C Protocol Services
• Atomic broadcast and consistent membership• Global time base of known precision• Protection against faulty nodes (fault isolation)
12 September 2002 / p.44
Thomas Losert
Fault Hypothesis
• Fault-Error-Failure• Component types• Correctness of a component• Type of component failures• Frequency of component failures• Number of faulty components & minimum
configuration
12 September 2002 / p.46
Thomas Losert
Component Types in a TTA Network
• Node computer Host computer Communications controller
• Channel of the interconnection network• Component instances fail statistically
independently and as units (component instance fault containment region)
12 September 2002 / p.47
Thomas Losert
Correctness of Nodes
• Correctness of host computer • Correctness of communications controller
Correctness as judged by omniscient observer (and, maybe, as seen by the application)
Correctness as judged by other nodes of the cluster: Correctness at interconnection network interface
12 September 2002 / p.48
Thomas Losert
Correctness of Nodes: Correctness at Network Interface
• A correct frame is received on the respective channel during the sending slot of the node
• A node has two network interfaces• Correct frame
TX starts and ends within slot boundaries Physical line signal obeys line encoding rules CRC check is passed Sender and receiver agree on the distributed state of the
TTP/C protocol (C-state)
• At the TTP/C level a node is considered correct if it is correct on a least one of its network interfaces
12 September 2002 / p.49
Thomas Losert
Correctness of Channels
• Correct channel will deliver identical and authentic copies of a frame received from some node being correct at the network interface to all correct receivers with known delay provided there is only a single sender
• Channel may need a minimum time interval between successive transmissions
12 September 2002 / p.50
Thomas Losert
Types of Node Faults
• A transmission fault is consistent (on a correct channel)
• A node does not send data outside its assigned sending slots on both channels of the network
• A node will never send a correct frame outside its assigned sending slots
• A node will never hide its identity when sending frames
12 September 2002 / p.51
Thomas Losert
Types of Channel Faults
• A channel does not spontaneously create correct frames
• A channel will deliver a frame either within some known maximum delay or never
12 September 2002 / p.52
Thomas Losert
Frequency of Faults
Nodes:• Only one faulty node within the duration of a
TDMA round• A node may become faulty only after any
previously faulty node either has shut down or operates correctly again
Channel:• Only one channel is faulty during a TDMA
slot
12 September 2002 / p.53
Thomas Losert
Number of Faulty Components & Minimum Configuration
• Single faults: At most one component may be faulty during a slot
• Min. three synchronized correct nodes participating in clock synchronization
• I-frame frequency depending on requirements• Correct I-frame sender (to allow for
integration)
12 September 2002 / p.54
Thomas Losert
Outlook
• Requirements• Basic Principles
Composability Dense Time versus Sparse Time Communication System Paradigms Temporal Firewall
• Time Triggered Architecture (TTA) • TTP/C protocol• Bus Guardian• Conclusion
12 September 2002 / p.55
Thomas Losert
The Tasks of the Guardian
• Correct guardian transforms failure modes at the interface of a fault containment region (i.e., component)
• At the interface failure modes of the supervised unit are replaced by failure modes of the guardian
• The goal is to handle arbitrarily faulty nodes, and, thus, to delete the assumptions on faulty nodes
12 September 2002 / p.56
Thomas Losert
The Tasks of the Guardian
Guardian
Fa
ult
Co
nta
inm
en
t R
eg
ion
outputto
system
inputfrom
system interface
12 September 2002 / p.57
Thomas Losert
The Tasks of a Guardian for TTA Networks
• SOS faults w.r.t. the line encoding rules• SOS faults w.r.t. the timing of frame
transmission• Transmission outside the assigned sending slot
(both in startup and synchronized operation)• Masquerading• Transmission of invalid C-state data
12 September 2002 / p.58
Thomas Losert
The Central Guardian Approach: Architecture
Error Containment Region
Fault Containment
Region
12 September 2002 / p.59
Thomas Losert
The Central Guardian Approach: Architecture
• Components of the central guardian• Failure mode transformation units
Reshape unit Transmission timing supervision units (for startup
& synchronous operation)
• TTP/C controller providing Access to the global time base Access to the distributed C-state of the cluster
12 September 2002 / p.60
Thomas Losert
Outlook
• Requirements• Basic Principles
Composability Dense Time versus Sparse Time Communication System Paradigms Temporal Firewall
• Time Triggered Architecture (TTA) • TTP/C protocol• Bus Guardian• Conclusion
12 September 2002 / p.61
Thomas Losert
Conclusion
• The time-triggered architecture provides the requirements regarding composability, security, and scalability
• A central guardian is a both technically and economically promising approach to achieve fault isolation in time-triggered communication
• The concept is realized and available in hardware
• A C1-based hardware prototype is currently tested re-doing fault injection experiments where bus-based clusters suffered fault propagation (IST project FIT)
12 September 2002 / p.62
Thomas Losert
Ongoing Work
• Gigabit TTP/C: TTP/C based on Ethernet, using standard COTS
• Event-Triggered – Time-Triggered: CAN over TTP/C TCP/IP over TTP/C
12 September 2002 / p.63
Thomas Losert
Thank you for your attention!