A static code analysis is commonly understood to be an automatic check by a tool. This is only a recent development. Only 10 years ago these tools were not as good as they are today and they were not available in such a variety as today. Static code analysis is simply part of what so far was called a static white-box test. This is a test which is performed on the source code without executing it. Thus it is called "static". The expression "white-box" stands for the kind of test which is concerned with the implementation of the code i.e. its structure and details, calculations and data operations. The overall functionality of the software is not in focus, but the details, quality and safety of its programming. A part of the test can be done automatically using analysis tools as e.g. Lint or the Safer-C toolset. But this should not be all it has to be followed by a manual code inspection, which is an inspection or review of the source code by specialists, according to defined rules. The output of the test is a proper test report and respective inputs for a defect data collection for statistical and process improvement purposes.

There are approximately 700 possible static problems or faults in C programming. E.g. it may be regarded as a possible problem if you have a code line as follows:

if (a = b)

It is perfectly valid C. It assigns the content of b to the variable a and then evaluates the contents of a to see if it is other than zero. In this case the if condition would be fulfilled. However what the programmer probably meant was:

if (a == b)

A good automatic checker would make a comment to this line because it is unclean programming. A good programmer would write:

a = b;
if (a)

This makes the line more clear and a code checker would not stumble over this line. Of course there are many other possible static faults, like the precedence of operators or order of evaluation problems. Modern code checkers can find the most important one of these faults. In total this will be approximately 350 to 400 fault conditions which can be detected by current state of the art static code analysis tools.

Types of Code Checkers

Not all the code checkers are doing the same kind of analysis. Again here I will focus on the code analysis tools which are suitable for C and the microcontroller world. There are varoius checkers available. They can be grouped into 4 main categories. So far I did not find an official naming for the categories, so I made up my own.

1. Pitfalls and Traps Checkers

There is a commonly agreed area of C programming which is considered as dangerous programming. The outlines of this kind of programming can be found in my course description on safe programming in C. You will find similar information in Andrew Koenig's book "C Traps and Pitfalls" and a number of other publications. Last but not least there is a so called MISRA standard which addresses the problem. There are a number of checkers which try to find as much as possible of these language transgressions. One of the most widely used is PC-Lint. However there are other similar ones, e.g. the Safer-C tool of Prof. Les Hatton or a tool called QA/C made by a company with the same name, and finally a tool called Logiscope made by Telelogic.

Some of these tools have a basic version for the more commonly agreed fault modes in programming and an extended version which offers additional MISRA checks. The MISRA standard is very strict and not all of their recommendations are commonly accepted. Therefore it makes sense to separate the basic and the MISRA version of the tools. However, you have to be aware that not all of the MISRA rules can be found by the checkers. Some of the MISRA rules are doubtful anyway and some are redundant to other rules. Then there is a good amount of rules of which the transgressions can be hardly detected in an automatic way and they have to be enforced by reviews or inspections. Finally it has to be said that not all of the tools have the same performance. Some of them come up with really stupid warnings about non existing problems, other ones are much better. If you want to have more information on this subject or a set of test cases to check the performance of these tools, feel free to contact me.

2. Configurable Checkers

Although the simple traps and pitfalls checkers are configurable e.g. regarding the bit size of variables and the rules which have to be checked or omitted, the term "configurable" means something else here. Configurable means that the tool offers an interface, i.e. in most cases a script language, which can be used to write additional scripts to check additional proprietary rules. You may have own rules and design guidelines which you want to enforce. E.g. this could be a naming convention. Imagine you have a coding rule which says that you have to encode the data type of a variable in the variable name. By the way this is a very good style! An encoding I have seen is e.g. to add a suffix to the variable name with a shortage of the data type. E.g. MyVariable_us would stand for an "unsigned short". MyVariable_uc for an "unsigned char" and so on. A script can easily search all your variable names for the existence of one of the defined suffixes (_uc, _us, etc. etc.). A script could even do further checks and search the source code to see if I defined my variables correctly i.e. matching the suffixes. E.g. it would search for a variable definition "unsigned short MyVariable_us;" in case it would find "unsigned char MyVariable_us;" it comes up with an error message warning me of a mismatch of the naming and the definition.

This was only one of the more simple things a script can check in your source code. It takes some time to set up the scripts, but once they are done they can be of great benefit and save a lot of inspection and review time. Why waste manpower if a tool can do it much faster and more exact. Configurable checkers are QA/C and Logiscope, at least the ones I know. On top of having a script support these tools also offer added value in the form of metric generation. Since they analyze the code anyway they generate source code metrics on the fly. Usually this feature is also configurable and you can set your own limits regarding the decision if a metric value is good or bad. These metrics do anything from a simple comments per source line ratio to the analysis of the cyclomatic complexity according to Mac Cabe.

3. Sophisticated Checkers

During the recent years some promissing attempts have been made to dig with static checkers into the domain of so far dynamic checks. One of the tools in this field is Polyspace. These checkers attempt to find runtime problems before the SW is executed. Imagine a code line like "unsigned short = unsigned short * unsigned short". For runtime and other resource reasons the programmer choose to use an unsigned short as variable for the result of the multiplication. Now this is potentially very dangerous. Depending on the values which have to be multiplied the calcualtion will overflow or not. Sopisticated checkers will attempt to find out and calculate the worst case inputs into such an operation. Depending on these worst case inputs they will issue a warning or not. The sophisticated checkers are usually very expensive. However not the price is the problem. These tools will usually come with a threefold result for critical code lines. They will give you a "green" in case everything is fine. They will give you a "red" in case they detected a real problem, and they will give you a "yellow" which basically means "I don't know you have to check by hand". Personally I would be grateful to use such a tool if they leave me with 10% or 20% of yellows. Then I have a big benefit. But if I get 80% to 90% yellow, I can not see the benefit. However, there is a future for these tools and as time goes on I am sure the benefit will rise.

4. Supporting Tools

There are also some supporting tools like SNIFF+ or CodeSurfer. They were made for re-engineerig of software and better project overview. If you are confronted with an unknown source code for review or maintenance these tools make it much easier to dig into it and understand foreign code. Instead of only focusing on individual C expressions they visualize the logic and structure of a program. You will get call graphs and data flow charts of of these little helpers. They can not be considered as true automatic checkers but since I said in the introduction to this page that automatic checks are only part of the bigger subject of static white-box tests, the supporting tools would be the bridge between automatic checks and manual checks, supplementing them with tool support.

Things to observe when you do automatic code checks

Automatic code checks are part of proper software testing. Basically the same rules as for the other test activities also apply for automatic checks. E.g. they have to be specified and documented.

You have to define a standard configuration for PC-Lint (or your other checker). Lint offers a wide range to customize the checks, but it makes no sense to switch off too many checks. The best solution is to define a standard set for the company or organizational unit. This set has then to be applied for every project. The only legitimate deviations may be due to compiler or controller related items as e.g. data size of "int" or the necessity to define own data types as e.g. "bit". However this has to be considered very carefully.

The automatic checks have to be executed, and in case of detected errors they have to be reported. This has to be followed by a bug fix by the developer. This loop has to be repeated until no further PC-Lint or MISRA errors are found. The End of Test Criteria is to be 100% error free on the defined Lint checks or MISRA related checks if applicable. In case this can not be reached and errors are remaining, an explanation has to be documented why the errors can not be fixed and that they do not constitute a risk.

After identifying all errors they need to be eliminated by changing the code to comply with the Lint rules. It is a good working style that the tester supplies a proposal on how to fix the Lint errors as a sample code, but he should not fix the errors himself on a code which will be released without further checks. This has to be done by the developer, who after a bug fix has to check again the functionality of the overall software.

Since automatic code checks can be done easily it is good working style if the developer checks his code by himself and immediately fixes his bugs. Some companies even have to rule that only Lint error free code can be checked in into the version management archives. If you adhere to this hint the testing activity using automatic code checks is only a quick and more formal activity. Verifying that the good working style was adhered to. The automatic check should be the first testing activity. It is the entry criteria for further testing. A code has to be rejected until it is "clean" and further testing as e.g. code inspections, module tests, etc. have to be done only on Lint error free code.

You should know what your automatic code checker really checks and what is left to be checked by other tests. This is most important! Otherwise you may have gaps or you waste time in inspecting code for things which were already checked by the tool.

Imprint