You will find the figures mentioned in this article in the German issue of ATZ 07-08I2007 beginning on page 648.

Effizientes Testen in der Automobilelektronik –

Von der Simulation bis zur Diagnose

Efficient Testing in

Automotive Electronics

From Simulation to Diagnostics

Authors:Thomas Riegraf, Siegfried Beeh and Stefan Krauß

Hardly an expert denies today the important role, which the topic testing plays in the development process of automo-tive electronics. Nevertheless there is the feeling that up to now unused potential can be released in this area. Suitable strategies, ideas and tools are looked for. Vector Informatik analyzes the state of the art, clarifies problematic interac-tions occurring in practice, and demonstrates that tools are already available today for solving concrete project tasks related to testing in an elegant and efficient way.

1 Introduction

Over the past ten years, the status of auto-motive electronics has changed fundamen-tally. At the beginning, just a few ECUs were used in the automobile but now more than 60 are being installed in some luxury class cars. Additional electronic systems of-fer improvements in the areas of safety, convenience and energy-saving operation. Today, more innovations are based on elec-tronics, and increasingly, a large share of this functionality is based upon software.

Growing complexity has made extensive and effective tests more important than ever before. The widespread use of numer-ous electronic components has caused the number of potential error sources to rise disproportionately. Tests are indispensable in all phases of ECU development to detect and correct errors early, keeping costs as low as possible. Some weaknesses of a total

system do not manifest themselves until components are integrated in the vehicle under actual and real-time conditions. This has turned testing into a cross-departmen-tal and cross-manufacturer discipline.

The enormous electronic problems that occurred in initial years shows what hap-pens when these facts are not considered and systematic tests are neglected. The later problems are identified in the develop-ment process, the more serious impact there is to the increase in cost. This be-comes clear in an extreme way when cor-rection of errors leads to costly recall ac-tions after product has been shipped. Par-ticipants in the automotive industry have learned their lessons and now attach great importance to the topic, but it is possible to further increase efficiency by consistent application of available resources.

Costs for tests represent a considerable share of a project budget, but proper func-

ATZ 07-08I2007 Volume 10916

tionality of the ECU must be assured. There-fore, it is important to achieve a maximum of test quality and test depth with transpar-ent concepts, for example by replacing in-sufficiently automated test steps by mod-ern methods and tools.

2 Tool for Analysis, Simulation and Testing

The networking of ECUs represents the backbone of motor vehicle electronics. In this context, the method of remaining bus simulation provides an important founda-tion in performing ECU tests. Without at least a rudimentary simulation of the ECU environment, most ECUs cannot be put in-to operation meaningfully. For example, many ECUs only operate properly if they serve network management functions.

CANoe from Vector Informatik is a wide-ly used tool for analyzing, simulating and testing distributed, embedded systems, Figure 1. It is used widely for remaining bus simulation and supports all important bus systems – in particular CAN, LIN, MOST and FlexRay – for which Vector Informatik also supplies suitable PC interfaces. Commercial-ly available interface cards can be used to address the I/O lines of ECUs from CANoe. Moreover, Vector has announced an I/O hardware product that supplements these general capabilities with test-specific func-tions such as switching additional loads and short circuits directly to the ECU terminals.

The various analysis capabilities, simu-lation components, and test sequences rely on models integrated in the tool in the form of databases. These might be the com-munication matrices in DBC format for CAN, Fibex for FlexRay, XML function cata-logues for MOST or LDF files for LIN. Simi-larly, CDD and ODX descriptions may be used to describe the diagnostic capabilities of an ECU. Besides containing essential in-formation on the system, test descriptions also contain symbolic names for signals, messages, diagnostic services, etc. This sim-plifies the work of the test user and test developer and creates an abstraction layer between the test and communication de-scription.

Any simple workstation PC running un-der the operating system Windows can be used to run CANoe. More powerful test sta-tions with improved real-time capabilities can be set up in a real-time configuration as dual-computer operation. This is done by executing the remaining bus simulation and the actual test execution on a dedicat-ed computer (real-time tester) running un-

der a real-time operating system (Windows CE), while a separate PC (GUI) is used as the graphical user interface and for evaluation purposes, Figure 2. The two computers com-municate via TCP/IP with one another. In this way, the system can also serve as a test execution environment for a component HIL tester.

3 Integration of Testing and Development

Today’s development models call for tests in various phases of development, Figure 3. Generally, the individual tests are self-con-tained, separate activities that are per-formed by specialized personnel at suitably equipped, dedicated workstations using special tools, languages and methods. In this context, test creation is often organ-ized as an independent task, detached from other development activities.

This segmented work approach results from distribution of the many different tasks of the development process to special-ized working groups. However, if this sepa-ration of tasks is followed too strictly, the numerous contact points between various development and testing tasks will most likely not be utilized optimally. For exam-ple, only good coordination between com-ponent testing and system testing can pre-vent expensive redundant development of test cases that are identical in content. When compatible tools are used, test cases that have already been developed once can be used as a basis for other developments in various areas. This avoidance of redun-dant developments frees up resources that could be used, for example, to profitably invest in the validation of existing test cas-es and their advanced development. Com-prehensive test management supplies a solid foundation for cooperation and, ap-plying the same resources, optimizes the depth and breadth of testing. Coordination also helps to detect and fill gaps in testing.

Besides linking the different test phases, development and test activities must also be interrelated and adapted to one another. Testing must be understood as an integral component of development that requires comprehensive support using proper meth-ods and tools. This must be guaranteed in addition to the procedural and organiza-tional integration. What is important here is to make tests available in conjunction with development, not just in the required formal verification phases. Ideally, it is pos-sible to perform tests directly at the ECU developer’s workstation with the resources existing there.

For this purpose, CANoe offers a run-time environment for test execution that can be used in parallel to the remaining bus simulation and analysis functions. The process is very easy to set up, especially if developers are already using CANoe for re-maining bus simulation and analysis of bus communication.

CANoe’s test component enables manu-al, semiautomatic, and fully automatic ex-ecution of tests. The developer can begin with simple tests and later expand and complete them. In general, the process of creating complex tests is a task of valida-tion departments that build their tests up-on the developer’s tests.

An important foundation for such tests is the remaining bus simulation, which ideally is not set up manually, but rather is auto-matically generated and parameterized from the databases of the system description. The actual work is performed by so-called mode-ling DLLs (for example Interaction Layer, Net-work Management, etc.), which are supplied with the tool or which Vector puts together as OEM-specific modeling packets. The sig-nals that the remaining bus simulation sup-plies to the simulated nodes may be acquired directly from test scripts, or may be stimu-lated or added manually.

In contrast to the systematically planned, executed and documented activities of the test phases, formal execution and documen-tation are generally omitted in tests accom-panying development. Nevertheless, these tests make substantial contributions toward overall quality, because they give the devel-oper the ability to deliver a more stable product to the subsequent testing phase.

4 Maturity Level Assessment and Error Analysis

To assess the maturity level of an ECU dur-ing development, a comprehensive evalua-tion of all executed tests is necessary. The quality of individual test results with re-gard to reliability and relevance must be considered, but more important is broad coverage of the required properties by suit-able tests. Therefore, the results of less for-mally executed tests are also helpful in ma-turity analysis. A prerequisite for this – be-sides keeping records of test execution – is the consistent use of configuration man-agement. This is also indispensable from the perspective of achieving quality-orient-ed, structured development processes.

A test record is produced whenever a test is executed using CANoe, whether in the test laboratory or at a development worksta-

ATZ 07-08I2007 Volume 109 17

DEVELOPMENTMeasurement Techniques

tion, and is created without intervention by the user or test case developer. It is then available for tests accompanying develop-ment without requiring additional effort. The XML format used for the test records is an open format thus other tools can be used for further processing of the results. For ex-ample, a test management system might be used to evaluate the test records in the con-text of a maturity level analysis.

Essential in this effort is a mapping of test results to test cases, and of test cases to requirements. This is easy to achieve by the use of unique identifiers (for example a re-quirement ID), which the test case devel-oper references in individual test cases. CANoe automatically copies this identifier to the test record so that all test cases, test results and requirements are clearly inter-related, Figure 4.

At least as important as recording and evaluating test results, is the analysis of the causes of the errors that actually occur. Most test tools do not provide any such ca-pabilities or provide just rudimentary ca-pabilities. One important reason is that er-ror analysis is often considered as a com-pletely independent task for developers. First, they are faced with the problem of understanding errors detected in the test and tracing their causes. In particular, when errors are reported by test laborato-ries, the developer usually does not even have access to the systems used in testing.

Therefore, at the test bench it is manda-tory to precisely record the test procedure and log every interaction with the test can-didate, especially the bus communication. During the role of analysis, CANoe enables replay and analysis of any desired record-ings (log records). It is thus possible and beneficial to have the same type of test sys-tem at the development workstation as that of the test bench, so that the developer can reproduce error producing test cases independently. In many cases it is possible to execute the relevant test cases even if many simplifications are necessary, for ex-ample to avoid addressing nonexistent measurement hardware.

5 Signal Abstraction and Diagnostics

Abstraction is a commonly used and impor-tant method for handling complexity in software development and system design. This can also be applied to the handling of tests. Growing functionality in the ECUs not only increases the complexity of sys-tems, but also requires tests that are more extensive and complex. The choice of the

correct abstraction layer in composing tests not only affects the effort required to cre-ate test cases, and therefore costs, but also the quality of test cases. Like all other soft-ware components, the test cases themselves may contain errors as well and should be checked before use. Another aspect is the necessary maintenance tasks, for example making adaptations to modified require-ments and correcting test cases.

Abstraction on the signal level is a com-mon way to test ECU functionality, and this is why in the ECU the actual application is generally based on a signal abstraction, Figure 5. For a CAN bus, for example, an In-teraction Layer in the ECU provides the sig-nal abstraction. That is exactly how CANoe uses an Interaction Layer; it parameterizes itself from information contained in the network descriptions, which also serve to create the ECU software. This ensures that ECU and test environment utilize the same abstraction layer and are therefore opti-mally tuned to one another.

Simultaneously, signal abstraction also represents – at least on the protocol level – the remaining bus simulation. For exam-ple, it ensures that periodic signals are ac-tually transmitted periodically. In testing, the ECU is represented in a realistic envi-ronment regarding bus communication. Moreover, when a change is made to the system’s communication matrix, it is usu-ally possible to continue to use the test cases unmodified. With the same applica-tion, the abstraction enables reuse of test cases in similar projects.

In testing ECUs, it is not only the signal interface that is important. Many ECU func-tions can only be tested meaningfully if deeper access to the ECU is possible. Such accesses are provided by the diagnostic and calibration interfaces, which are accessed via an ECU’s existing bus interfaces. It does not make sense to address these interfaces by simple message sequences, since defined protocols underlie the communication process. It is more convenient and reliable to have appropriate abstractions for diag-nostics and calibration.

In CANoe, either description files from the CANdela tool or ODX description files are responsible for parameterizing the di-agnostic access layer. If no description is available for the actual diagnostic capabili-ties of an ECU, a generic description for KWP2000 and for UDS supplied with CANoe may be used. Either the generic de-scription or a diagnostic description file customized for the ECU will offer conven-ient access to the diagnostic services de-fined there. It is possible to obtain the same

abstractions and advantages as in the sig-nal abstraction described before.

The CCP and XCP measurement and calibration protocols can be used to access internal ECU variables via test scripts in CANoe. The measurement and calibration tool CANape handles these protocols and the ECU description files (A2L) that are needed. CANoe controls CANape via the COM interface. This accomplishes the same goals as in the signal and diagnostic ab-straction described before.

6 Efficient Test Generation

A precise study of an ECU’s test cases will reveal that many of the test cases can be derived from just a few recurring patterns. This is quite evident in gateway ECUs: A majority of the test cases serve to check the routing of signals and messages. Finally, the only reason for the large number of test cases is the large number of possible input and output data. But the same types of pat-terns are also found in other types of ECUs. Expressed abstractly, this means that many functions are tested by first putting the ECU in a specific state using a suitable stimulus and then checking the state that is reached. The recurring pattern of these test cases is: Set signals (stimulation), wait for the maximum allowable reaction time, and then check the signals of the new ECU state. With some experience in the use of test patterns, users will likely recognize a few additional run-time patterns from which many test cases can be derived.

These patterns represent an opportunity for further optimization of test case genera-tion. CANoe, in addition to offering classic programming of test cases, also lets the us-er define test cases based on test patterns. It is no longer necessary to program the pat-tern contents, since the procedures are al-ready known and permanently integrated in the patterns that are supplied, Figure 6. Test case generation is reduced to defining the target behavior, including any supple-mental data needed, such as the settling times to be toleranced.

To the extent that it is sensible, the sup-plied test patterns themselves are placed on the signal abstraction described and en-able symbolic access to signals, messages, etc., via the associated databases. The use of diagnostic services or I/O signals is also pos-sible. In short: The entire testing infrastruc-ture of CANoe can be used with test pat-terns. If there are requirements that extend beyond these capabilities, the option of programming the test cases still exists.

DEVELOPMENT

ATZ 07-08I2007 Volume 10918

Measurement Techniques

(Automatic) generation of test cases is another method for creating tests efficient-ly, if suitable sources of information are available. The contents of generated tests are by necessity limited to the description levels and depths of the sources. Neverthe-less, generation offers valuable support, primarily when it comes to covering the formally defined basic properties of an ECU by tests. The relatively low effort required to generate test results in quicker availabil-ity thereby making it possible for earlier detection of undesirable trends.

The tool chain from Vector utilizes such test generator approaches. Description files such as the DBC database or CANdela defini-tions are used as the source for the genera-tor, Figure 7. The generator uses them to gen-erate test cases, which CANoe then executes. Since test scripts may make use of the entire tool infrastructure, the test generators are often designed to be quite simple. For exam-ple, with just a little effort a generator can create suitable test cases from a customer-specific gateway description (for example in the form of a database or Excel spreadsheet). Thanks to the test patterns described above, this just requires a simple transformation of the customer-specific data into the format of the test patterns. Users can create such a gen-erator in a straightforward manner. Vector offers further support in the form of project-related services.

7 Summary

The only way for automotive OEMs and sup-pliers to deal with the growing require-ments for ECU tests is by efficient test crea-tion and automatic test execution. The pre-sented testing tool CANoe of Vector Informa-tik offers a proven solution for implement-ing testing tasks in automotive electronics with signal abstraction, integration of diag-nostic, calibration and I/O interfaces, the concept of test patterns, and test case gen-erators. CANoe is a high-performance runt-ime environment for testing ECUs and net-works. The tool enables early creation and execution of tests with little effort, right at the developer’s workstation.

The open interfaces of CANoe facilitate seamless integration in a comprehensive test-ing strategy and tool-supported test manage-ment. Although some users might still imag-ine it to be a futuristic vision, with suitable integration of CANoe it is already possible to determine maturity levels today. Vector is continuously developing CANoe for use in these areas, thereby supporting users with a modern and efficient test platform.

IMPRINT

ATZ W O R L D W I D E

www.all4engineers.com

07-08|2007 · July/August 2007 · Volume 109 Vieweg Verlag | GWV Fachverlage GmbHP.O. Box 15 46 · D-65173 Wiesbaden · Germany Abraham-Lincoln-Straße 46 · D-65189 Wiesbaden · Germany

Managing Directors Andreas Kösters, Dr. Ralf Birkelbach, Albrecht SchirmacherSenior Advertising Thomas Werner Senior Production Ingo EichelSenior Sales Gabriel Göttlinger

EDITORS-IN-CHARGE

Dr.-Ing. E. h. Richard van Basshuysen

Wolfgang Siebenpfeiffer

EDITORIAL STAFF

Editor-in-ChiefJohannes Winterhagen (win) Tel. +49 611 7878-342 · Fax +49 611 7878-462 E-Mail: johannes.winterhagen@vieweg.de

Vice-Editor-in-ChiefDipl.-Ing. Michael Reichenbach (rei) Tel. +49 611 7878-341 · Fax +49 611 7878-462 E-Mail: michael.reichenbach@vieweg.de

Chief-on-DutyKirsten Beckmann M. A. (kb) Tel. +49 611 7878-343 · Fax +49 611 7878-462 E-Mail: kirsten.beckmann@vieweg.de

Editors Ruben Danisch (rd) Tel. +49 611 7878-393 · Fax +49 611 7878-462 E-Mail: ruben.danisch@vieweg.de

Dipl.-Ing. (FH) Moritz-York von Hohenthal (mvh) Tel. +49 611 7878-278 · Fax +49 611 7878-462 E-Mail: moritz.von.hohenthal@vieweg.de

Dipl.-Ing. Ulrich Knorra (kno) Tel. +49 611 78 78-314 · Fax +49 611 7878-462 E-Mail: ulrich.knorra@vieweg.de

Permanent Contributors Christian Bartsch (cb), Prof. Dr.-Ing. Peter Boy (bo), Prof. Dr.-Ing. Stefan Breuer (sb), Jens Büchling (jb), Jörg Christoffel (jc), Prof. Dr.-Ing. Manfred Feiler (fe), Jürgen Grandel (gl), Erich Hoepke (ho), Thomas Jungmann (tj), Prof. Dr.-Ing. Fred Schäfer (fs),Caterina Schröder (cs)

AssistantsEllen-Susanne Klabunde, Martina Schraad Tel. +49 611 7878-244 · Fax +49 611 7878-462 E-Mail: atz-mtz@vieweg.de

AddressPostfach 15 46, D-65173 Wiesbaden, Tel. +49 611 7878-244 · Fax +49 611 7878-462

MARKETING | OFFPRINTS

Product Management AutomediaSabrina Brokopp Tel. +49 611 7878-192 · Fax +49 611 7878-407 E-Mail: sabrina.brokopp@vieweg.de

OffprintsMartin Leopold Tel. +49 228 6907-87 · Fax +49 228-6907-88E-Mail: leopold@medien-kontor.de

ADVERTISING | GWV MEDIA

Ad Manager Nicole Kraus Tel. +49 611 7878-323 · Fax +49 611 7878-140 E-Mail: nicole.kraus@gwv-media.de

Key Account Manager Elisabeth Maßfeller Tel. +49 611 7878-399 · Fax +49 611 7878-140 E-Mail: elisabeth.massfeller@gwv-media.de

Ad Sales Andreas Bausch Tel. +49 611 7878-333 · Fax +49 611 7878-140 E-Mail: andreas.bausch@gwv-media.de

Display Ad Manager Sandra Reisinger Tel. +49 611 7878-147 · Fax +49 611 7878-443 E-Mail: sandra.reisinger@gwv-media.de

Ad PricesPrice List No. 50

SUBSCRIPTIONS

ServiceVVA-Zeitschriftenservice, Abt. D6 F6, ATZPostfach 77 77, 33310 GüterslohRenate ViesTel. +49 5241 80-1692 · Fax +49 5241 80-9620E-Mail: vieweg@abo-service.info

PRODUCTION | LAYOUTKerstin Gollarz Tel. +49 611 7878-173 · Fax +49 611 7878-464 E-Mail: kerstin.gollarz@gwv-fachverlage.de

PRINT | PROCESSINGImprimerie Centrale Luxemburg. Printed in Europe.

SUBSCRIPTION CONDITIONSThe journal ATZ appears 11 times a year (with at least 5 additional special editions) at an annual subscription rate of 209 €. The price for an annual subscription including the English text supplement ATZ Worldwide is 259 €. Special rate for students on proof of status in the form of current registration certificate 81 €. Special rate for students including the English text supplement ATZ Worldwide 120 €. Special rate for VDI/VKS members on proof of status in the form of current member certificate 153 €. Special rate for studying VDI/ÖVK members on proof of status in the form of current registration and member certificate 45 €. Price per copy 23 €. All prices exclude mailing (annual subscription: inland 21 €; foreign countries 35 €; AirMail 109 €; annual subscription including the English text supplement ATZ Worldwide: inland 22 €; foreign countries 44 €; AirMail 119 €. Cancellation of subscriptions in writing at least six weeks before the end of the subscription year.

© Friedr. Vieweg & Sohn Verlag | GWV Fachverlage GmbH, Wiesbaden 2007

The Vieweg Verlag is a company of Springer Science+Business Media.

The journal and all articles and figures are protected by copyright. Any utilisation beyond the strict limits of the copyright law without permission of the publisher is illegal. This applies particularly to duplications, translations, microfilming and storage and processing in electronic systems.

HINTS FOR AUTHORSAll manuscripts should be sent directly to the editors. By submitting photographs and drawings the sender releases the publishers from claims by third parties. Only works not yet published in Germany or abroad can generally be accepted for publication. The manuscripts must not be offered for publication to other journals simultaneously. In accepting the manuscript the publisher acquires the right to produce royalty-free offprints. The journal and all articles and figures are protected by copyright. Any utilisation beyond the strict limits of the copyright law without permission of the publisher is illegal. This applies particularly to duplications, translations, microfilming and storage and processing in electronic systems.