Code Coverage Testing - Related to White box testing

[Archana Kannouj]

Hi All,

Its just out of my curiosity, i wanted to know how do we test "code coverage testing".

Any tool available for that? And how it works?
Any Idea??


Regards,
Archana

[friendster7]

Code coverage is a measure used in software testing. It describes the degree to which the source code of a program has been tested. It is a form of testing that looks at the code directly and as such comes under the heading of white box testing. Currently, the use of code coverage is extended to the field of digital hardware, the contemporary design methodology of which relies on Hardware description languages (HDL's).
Code coverage techniques were amongst the first techniques invented for systematic software testing. The first published reference was by Miller and Maloney in Communications of the ACM in 1963.

[Archana Kannouj]

Thanks for the Reply friendster7.


I am more interested in the details how any tool for white box testing actually works.

Regards,
Archana

[friendster7]

One solution to this problem is to use white-box testing in addition to black-box testing. White-box testing strategies include designing tests such that every line of source code is executed at least once, or requiring every function to be individually tested.

Very few white-box tests can be done without modifying the program, changing values to force different execution paths, or to generate a full range of inputs to test a particular function. Traditionally, this modification has been done using interactive debuggers, or by actually changing the source code. While this may be adequate for small programs, it does not scale well to larger applications. Traditional debuggers greatly affect the timing, sometimes enough so that a large application will not run without major modifications. Changing the source code is also unwieldy on a large program that runs in a test bed environment.

There are a number of testing tools that let you perform white-box testing on executables, without modifying the source and without incurring the high overhead of an interactive debugger. Advantages of this approach include:

• These tools speed testing and debugging because there is no need to wait for test support code to be inserted into the program. In commercial software development, with many developers and a separate department in charge of testing and integration, this can save significant time.
• Best use can be made of a test environment. For example, running a 747 simulator requires a specific hardware setup. If each tester changes the loaded software on the simulator configuration, then each test takes much longer to set up and each testing scenario is much more error prone. A 747 test environment is expensive to maintain. The faster testing can occur, the cheaper the cost of the testing process.
• Source code may not be available for all of the software. It is common to use third-party products or software delivered from another organization as part of any commercial software effort. These typically do not ship with source code and certainly do not ship with the ability to rebuild them after changing the source code.
• It is better to test the actual executable that will be delivered, rather than a special testing executable. For one thing, it eliminates extra error-prone steps in the testing process. It also lets the testing scenario be more easily repeated on demand, rather than the single shot effort that typically occurs with source code modification.

Most white-box testing tools change the executable in one way or another, or check for certain classes of failures. Rational's Purify (IBM Rational Software), for instance, modifies the compiled code so each load or store initiates a check of the memory location against specific criteria. BugTrapper from Mutek (Application Development Optimization: AppSight Application Problem Resolution System from Identify Software, part of BMC Software) alters the executable to record all of the system calls. Applied Microsystems LiveCODE (http:// www.applied-microsystems.com/) adds trace commands to embedded systems so execution paths can be analyzed. Aprobe from OC Systems (my company; OC Systems - Improving quality, reliability and availability of critical applications.) allows you to modify the executable in whatever way you might choose.

Typically, white-box testing tools such as these provide a high-level programming interface for writing code that patches the executable and performs a specific function or obtains a particular type of information. Other features might include patches that execute as part of the application, at full machine speeds, which makes them noninvasive and useful for experimentation. Also, the tools might be able to automatically mangle/demangle names so users can use source code names even though the executable contains mangled object code names.

To illustrate, I'll present examples that use code for Aprobe. Aprobe uses ANSI C (with a few keywords added) as the base language in which the patches are specified, so it is straightforward to read/write code.

The aprobe.h include file contains a number of functions useful for writing patches for testing, debugging, and benchmarking. All of these functions have descriptive names and start with "ap_." These functions are designed to support the testing process. While adding arbitrary source code to an executable is advantageous, much of the power of such tools lies in these support functions.

One thing to note is the shortness of the code sequences. With this type of testing tool, most of the patches are extremely short -- much shorter than the corresponding source code modifications.

[friendster7]

note:-source for above post is Dr. Dobb's | White-Box Testing | March 1, 2000

i have a pdf link too the link is..

agile.csc.ncsu.edu/SEMaterials/WhiteBox.pdf

hope this helps..

[Archana Kannouj]

Thanks alot for providing me the above information. I was really interested in knowing the above details.

Regards,
Archana

[maheshkrishnan]

Hi,
You can get many tools for code testing if you search in the web.
Some are sharewares or some are free.
I have used in my previous company a tool called "Hindsight" which is not a shareware one.(I dont kno the cost)

For example., If you have a c code, The testing is done not by entire code units, It is by MOdule by module, called UNIT testing.
we will check each and every functionality to make prove that the code doesnt work ...(This is testing,,, proving code works properly is not a real testing...haha). You have to create each test cases for each functionality minimal. Let in a Module , you have many logical operation, or Math functions, You have to check the each variables data type, how they suites . There range, there muliplication factor... ...etc and create test cases to prove that it is clearing all test cases u have created....

[Archana Kannouj]

Ok Thanks for the Information...

Can i have some free ware code testing tool(just names)

~Archana