Contact
QR code for the current URL

Story Box-ID: 436376

iSYSTEM AG Carl-Zeiss-Str. 1 85247 Schwabhausen, Germany http://www.isystem.com
Contact Mr Erol Simsek +49 8138 697156
Company logo of iSYSTEM AG
iSYSTEM AG

100% Code Coverage is not equal 100% Code Coverage

iSYSTEM technical paper on how reliable is code coverage information shown in source code

(PresseBox) (Munich, )
Today iSYSTEM AG presented a new technical paper which discusses that a code coverage measurement on source code level is not equal to code coverage measurement on object code level. The article provides source and object code samples showing that difference.

Code coverage is a method to assess the quality of test runs. Based on this evaluation new test cases are deduced, redundant test cases are eliminated and inefficient test cases are changed or replaced. At the same time, code coverage is a measure to find code that is never executed. This "dead code" represents undesired overhead when assigning memory resources and is also a potential hazard if this code was not executed during testing.

Code coverage measurement (mainly on source code level) is recommended for certification in avionic, medical, automotive and nuclear standards like DO-178B, DO248D, IEC 62304, ISO 26262 and SC45A.

According to Beizer [2] 100% statement coverage at source code level might cover 75% or even fewer statements at object code level [3].

Programs written in high level programming languages like C, C++, ADA etc. are not running on a processor or controller without being converted to object code. There is a chain of transformations that produces the object code running on the actual target. For C programs usually these steps are: preprocessing, compilation, optimization, assembly, linking and conversion. These steps are not bijective: information is added and removed in every step.

These modifications directly affect code coverage analysis on source code level because

- often compiler or library calls add object code and
- it is not easy to detect when object code is added by a compiler or a library call and
- the added object code may not be 100% covered by your tests and
- it is very complex to establish which object code correlates to which source code line (usually code coverage analysis is shown in source code).

The examples in the article show that the usage of complex instructions (e.g. for loop), complex conditions (e.g. if statement) or library/operating system functions add object code that is not directly traceable to source code statements.

Here one example discussed in the article - Compiler Optimizations:

Compilers may

- merge several source code lines (e.g. merge string constant expressions, expression simplification,...),
- deem unnecessary code lines (e.g. dead code elimination, removal of unused symbols, common subexpression elimination,...) and/or
- reorder the execution flow (e.g. loop inversion, loop fusion,...).

!! When measuring code coverage, verify what source code lines generate object code and if not, check why. Any code change and recompilation may change the compiler optimization and could lead to untested code paths.

iSYSTEM tools take great care to show accurately what is actually going on. In iSYSTEM's winIDEA source code view every line that generates object code has a code coverage legend aligned to the left. Note that some "{" and "}" are not creating object code and therefore have no code coverage legend aligned.

Conclusion

The requirements for code coverage analysis are constantly increasing in almost every market. Code coverage based on object code is a convenient method to test as close as possible to the final product.

To achieve a high level of code coverage (some claim 100% is the right figure), review the object code and find test cases to cover the (not yet) tested code.

iSYSTEM tools provide code coverage analysis within the same tool that is used for code development. This way development costs and time are optimally used to detect and fix software bugs as soon as possible within the development cycle.

More Information

[1] How reliable is Code Coverage Information shown in Source Code?Here you find the complete article with all examples

[2] Beizer, Boris: Software Testing Techniques, Second edition, Von Nostrand Reinhold Company, Inc., 1990

[3] Hayhurst, K. J., Veerhusen D. S., Chilenski J. J., Rierson, L. K.: NASA / TM-2001-210876 A Practical Tutuorial on Modified Condition/Decsion Coverage, NASA Center for Aerospace Information (CASI)

[4] Difference between Source and Object code Coverage

[5] Offline versus Realtime Execution Coverage

[6] Code Coverage: Relation between Source and Object Code

iSYSTEM AG

iSYSTEM manufactures specialized tools for embedded software development and test, supports customers with their embedded projects and provides professional support. Our vision is to easily enable developers and testers to use embedded development and test tools within the complete development process.

iSYSTEM products are best suited for development and test of security-sensitive applications. Commonly used are these tools in the automotive, avionic and medical areas. To achieve an optimal integration of development tools, iSYSTEM maintains close relationships with semiconductor, compiler, RTOS and test tool vendors.

The majority of iSYSTEM products present a link between the embedded system and the host PC. Dependent on the CPU architecture an in-circuit emulator, an on-chip debugger, an on-chip debugger plus trace tool or an external bus analyzer can be used. The modular design allows the subsequent use of iSYSTEM products in other configurations. iSYSTEM tools support more than 50 different CPU architectures, 2500 microcontrollers and 150 compilers.

Links to iSYSTEM

www.isystem.com
www.twitter.com/isystemag
Blog: www.embedded.typepad.com

The publisher indicated in each case (see company info by clicking on image/title or company info in the right-hand column) is solely responsible for the stories above, the event or job offer shown and for the image and audio material displayed. As a rule, the publisher is also the author of the texts and the attached image, audio and information material. The use of information published here is generally free of charge for personal information and editorial processing. Please clarify any copyright issues with the stated publisher before further use. In case of publication, please send a specimen copy to service@pressebox.de.
Important note:

Systematic data storage as well as the use of even parts of this database are only permitted with the written consent of unn | UNITED NEWS NETWORK GmbH.

unn | UNITED NEWS NETWORK GmbH 2002–2024, All rights reserved

The publisher indicated in each case (see company info by clicking on image/title or company info in the right-hand column) is solely responsible for the stories above, the event or job offer shown and for the image and audio material displayed. As a rule, the publisher is also the author of the texts and the attached image, audio and information material. The use of information published here is generally free of charge for personal information and editorial processing. Please clarify any copyright issues with the stated publisher before further use. In case of publication, please send a specimen copy to service@pressebox.de.