1 toward target level testing and debugging for embedded software koehnemann, lindquist ritu varma...
Post on 21-Dec-2015
224 views
TRANSCRIPT
1
Toward Target Level Testing Toward Target Level Testing and Debugging for and Debugging for
Embedded SoftwareEmbedded Software
Koehnemann, LindquistKoehnemann, Lindquist
•Ritu Varma•Roshanak Roshandel•Manu Prasanna
2
IntroductionIntroduction• Identifies problems with the test case execution
for embedded systems and propose solutions• Testing is done to reveal errors whereas
debugging is done to locate and correct the cause of error.
• Embedded systems are usually constrained by Concurrent design Real time constraints Distributed hardware architecture Device control dependencies
• Execution visibility and control severely restricted
• Why testing in Embedded Systems necessary?
3
Software TestingSoftware Testing• Testing for embedded systems has four stages:
Module level Testing Integration Testing System Testing Hardware/Software Integration Testing
• Testing Methods which address the problem identified by the embedded systems
Concurrency Real-time constraints Embedded Environment
are of interest to this paper.
4
Testing Concurrent SystemsTesting Concurrent Systems• Concurrency increases testing difficulties:
Large set of execution sequences Subsequent Execution with the same input
different results Valid result as well as how the program arrived at
that result Effect of the environment on testing and its results
• Acceptable output more important than correct output- Control over program execution
• Research: Static analysis: must examine a large set of state
not scalable Dynamic testing: passive watch of execution and
replaying the execution in case of failure Runtime control of concurrent program
5
Non-intrusive TestingNon-intrusive Testing• Control and testing techniques must not interfere with
behavior of test program, timing issues • Why not instrumentation?
Insert probes to watch execution, link users object code with the rest of debugging system, Code interferes with execution behavior of the program
Intrusive, voids execution of system
• Non-intrusive debugger for Ada Separate processor executes testing system and
communicates with the target processor through specialized hardware
Good start but don’t deal with the high level activities (process scheduling ,fault handling) only low level ( monitoring bus activity)
• Other Real time embedded tools: ROM monitors, Emulators, Bus Monitors
6
Impact of Underlying SystemImpact of Underlying System• Problem: dealing with abstraction• Concurrency, IPC at high level and implementation
at complier level (on run-time system)• In Development phase abstraction is fine but in
Testing phase? Complication Without sufficient control over program , there is no way
to ensure that a test case is testing the code. Implementation difference in the host and the target
environment.
• Compilers provide runtime system operation to bridge the gap between language construct and hardware.
7
Current State of Embedded Current State of Embedded TestingTesting
• Concurrency causes Problems for testing and debugging• H/S integration testing reveal problems such as:
Incorrect handling of interrupts, Distributed Communication problems, Incorrect order of concurrent events, etc.
• Goal: reveal errors early as possible Problem: Target level testing tools don’t exist
• Two environments- Host : supports s/w development and tools which give user
control over program execution (high level interface) Target: no support for software development and tools as
minimum cost and space are the goals of the target.
Target level testing occurs late in the development cycle and a
small part is allocated for H/S integration testing. But,Target is the only location where errors can be revealed.
8
Current SolutionsCurrent Solutions• Hardware solutions
Attempts at gaining execution visibility and program control Bus monitors, ROM monitors and in-circuit emulator Minimal effectiveness for software development - low level
machine data gathered only Mapping b/w low level events and entities in the program
• Software solutions Attempts to reduce the cost and time spent testing on target Determining factors in software testing:
Level of Criticality of software module Test platform availability Test Classification
Criteria's that assign program modules to test platforms Type of software, Hardware requirements, Test classification,
Platform availability, Certification Requirements
• Current solution attempts to reduce the amount of target testing
9
Are these Techniques Are these Techniques Effective?Effective?
• Expensive• Low level of functionality• Available too late • Poor test selection criteria• What about new architectures?
10
ExpensiveExpensive
•Custom validation facilities•Little or no reuse•Hardware is late and erroneous Software must be malleable
11
Testing ProblemsTesting Problems• Host: high-level interface, language • Target: machine instruction and
physical address• Translation: Tedious and error prone • Migration to target
Hardware’s errors have to be mitigated by software
Late in the development process – costs
12
More ProblemsMore Problems• Test cases and scenarios depend on
tools and platforms instead of theoretical test criteria
• Guidelines not being followed• Hardware architecture changes rapidly• Solution:
Adding functionality to the underlying system to support debugging
Hardware and run-time system
Specialized Middleware for development and testing of embedded systems?
13
Model Debugging SystemModel Debugging SystemAdd functionality to targetAdd functionality to target
• Ada Semantic Interface Spec toolkit• Two communicating processor:
Running the test program Running the debugger
• Target processor can: Execute and debug code only at a break point Run the debugger as a separate processor Provide a separate execution unit to execute
the debugger
• Interface between internal and external debugger
14
Changes in HW ArchitectureChanges in HW Architecture
• Hardware partitioning of memory• Computational facilities for
debugger• Hardware break points• Dedicated bus
15
Memory PartitioningMemory Partitioning
• Software is infinitely malleable Any change requires several tests and
retests
• Late changes is expensive• Hardware should bound software• SW is usually partitioned based on critical
level instead of design factors
Hardware partitioning – protected address space for processes – restriction on access
16
Computational Facilities for Computational Facilities for DebuggerDebugger
• External debugger vs. Internal debugger • Internal debugger: located on target, executes from the
target without interfering with the application Runs as a regular process on the processor
Low-level or periodic high level process Architecture provides separate facility for debugger execution
Target processor provides computational facility
• Computation Facility utilize architecture to dump some representation of instruction being executed
Info must be parsed to create history. Implement state machines for messages sent by HW.
Program the (stateful) functional unit within the processor. At boot time, the internal debugger code gets loaded and reside there. Messages are passed between internal and external debuggers.
17
Hardware BreakpointsHardware Breakpoints• Software and conditional
breakpoints are not feasible for real time systems
• Breakpoint registers – data and instruction types
• State of a processor – all internal registers, pipeline and cache information must be saved when a break is encountered
• Conditional breakpoints – how are they dealt with?
18
Architecture Support for Architecture Support for AbstractionsAbstractions
• Utilities must be built into architecture to provide basis for emulation, testing and program visibility
• A new level of abstraction is needed: Basic type (int, float) Process, memory management, semaphore, etc
• Hardware should be more aware of programming elements (software)
• Awareness of the programming environment is important
19
Dedicated BusDedicated Bus• An interface between the processor and the
external world• Protocol used across the connection
data rate, volume, direction, etc.
• Master-slave relationship Internal and external debuggers Bus and debugger
• Determination of active element within the processor Processor is active – sends messages to debugger Use special debugger portion of RTS – both debugger
and RTS are active Perhaps a combination of the two
20
Runtime System AdditionsRuntime System Additions• Goal: to minimize data and computational
requirements of the internal debugger along with communication with the external debugger
• Standardize services that provide visibility into RTS abstractions
• Processes: concurrency is a common abstraction Need to support modifying task state,
communication and synchronization, and process scheduling
• Interrupt management, Time management, Memory management, Exception Handling
21
SummarySummary• Embedded system testing techniques are ill-
equipped• Architectural and RTS changes have to be made• Evaluate the changes and check feasibility• Definition of Embedded Systems??
What is the border between HW/SW? Software techniques more suitable for less resource
constraint system Need for classification of embedded system??
• Need for specialized middleware for both design and testing of embedded systems
• Need for software development techniques although not discussed in the paper