Writing good quality code is the goal of all the software developers.
However, writing very good quality code following complete rules at the first
attempt is difficult. It is essential to use code quality tools, unit test
execution and coverage to identity the main vulnerabilities, coverage of the
code and fix before release for quality testing. The complexity analysis and
quality metrics are important for developers, architects and managers to shape
the project to deliverable quality.
There are many open source and commercial code quality tools available
in the market. Code quality tools can have categorized into three main categories
1.
Static and Dynamic analysis tool
2.
Code coverage tools
3.
Unit test framework
Static and Dynamic analysis
tools:
Static code analysis aims to scan the entire project,
directories, files and provides the detailed report about code quality and
potential issues such as memory leak, unused variables, dereferencing null pointers,
boundary validation etc. It also provides on the fly feedback to the developers
about the potential issues and can also provide run time errors and warnings.
The tools also generate unique metrics for easy visualization and understanding
about the code qualities. Few standard static analysis tools already incorporated
some of the Coding standard compliant mechanism such as MISRA C/C++, CWE etc.
I have used many opensource tools in my career for
analysing C, C++, Java and JavaScript code. I am going to list out few open
source static analysis tools and demonstrate usage one or two familiar tools
with an example. I have captured and listed the information from my own study
and experimentation. Some information’s may not be 100% accurate or missing 😊.
Static Code analysis – Open Source
|
Tool
|
Features
|
Metrics/Format
|
Comments/Links
|
|
Cppcheck
|
• Static
Code analysis
• Detect
undefined behaviors
• Memory
leak
• MISRA
compliance check
• Bounds
checking
• Function
usage
• Available
as Eclipse and Jenkins plugin
• Use custom
config file
• Custom
rules using regular expression
|
• Results
categorized as Error, Warning, Performance and Informative
• Report in XML
Format
• Can
convert from XML to HTML
|
List of
checks:
Plugin
available for
Eclipse -
Cppcheclipse
Jenkins -
Cppcheck Plugin
|
|
Sonarcube
and SonarLint
|
• Open
source platform for continuous inspection
• Uses
pattern matching, dataflow analysis
• SonarLint
is an eclipse plugin to connect with SonarQube
• SonarLint
needs C/C++ plugin installation in SonarQube for C/C++ code analysis.
• Find code
smells, bugs and security vulnerabilities
• parser
supporting C89, C99, C11, C++03, C++11, C++14 and C++17 standards
• Buffer check,
memory leak, conditions, pointer check.
• Analysis
is automatically triggered
• can
configure rules
• import Unit
Tests Execution Reports
• import Coverage
Results
• Activate
multithreaded code scan
• CWE
Compatible
|
• Cognitive
Complexity
• Visualize
the history of a project
• Enforce
Quality Gate
• Lines of
code
• Duplicates
• Issues/category
• Integrate
Unit test coverage
• Import
GCOV Coverage
• Integrate
CPPUnit for unit testing
• Bugs
• Vulnerabilities
• Coverage
• Issues
categorized as Error, Warning, Performance and Informative
• Memory
Leak
• Dead Code
• Logic Flow
Error
• Coding
convention
• Error
Handling
|
https://www.sonarqube.org/
|
|
Clang
|
• can be run
either as a standalone tool or within Xcode
• Fast and
light-weight
• built on
top of Clang and LLVM
|
• Core
Checkers
• C++
Checkers
• check for
unused code
• Nullability
Checkers
• insecure
API usage and perform checks based on the CERT Secure Coding Standards
• Use of
Unix and POSIX APIs
|
|
|
SonarQube
C++ plugin
|
• Adds C++
support to SonarQube with the focus on integration of existing C++ tools
• SonarCFamily
for C/C++ - Commercial version
• SonarQube
C++ - Community plugin for free
|
• Support to
integrate CppCheck for code analysis
• Support to
integrate CppUnit for executing unit tests
• Support to
integrate Gcov / gcovr for coverage reports
|
|
|
Valgrind
|
• Valgrind
is an instrumentation framework for building dynamic analysis tools. There
are Valgrind tools that can automatically detect many memory management and
threading bugs, and profile your programs in detail. You can also use
Valgrind to build new tools.
• The
Valgrind distribution currently includes six production-quality tools: a
memory error detector, two thread error detectors, a cache and
branch-prediction profiler, a call-graph generating cache and
branch-prediction profiler, and a heap profiler.
• It also
includes three experimental tools: a stack/global array overrun detector, a
second heap profiler that examines how heap blocks are used, and a SimPoint
basic block vector generator.
|
• six
production-quality tools:
• a memory error detector
• two thread
error detectors
• a cache
and branch-prediction profiler
• a
call-graph generating cache
• branch-prediction
profiler
• heap
profiler.
• It also includes
three experimental tools:
• stack/global
array overrun detector
• heap
profiler that examines how heap blocks are used
• SimPoint
basic block vector generator
•
|
|
Code coverage Tools
|
Tool
|
Features
|
Metrics/Format
|
Comments/Links
|
|
Covtool
|
• Open source coverage tool for code
analysis
• Dynamic code analysis
• Not maintained for long time
|
• Total number of (instrumented)
lines of code,
• Total number that were executed
|
|
|
Gcov / gcovr
|
• Code coverage analysis and
statement-by-statement profiling tool
• Use in conjunction with GCC to
test code coverage
• use gcov along with the other
profiling tool, gprof
|
• Text output with total lines,
coverage statistics indicated with summary statistics and lists of uncovered
line
• Branch coverage
• XML output that is compatible with
the Cobertura code coverage utility.
• HTML output with coverage rates
indicated using colored bar graphs
|
Unit test framework
|
Tool
|
Features
|
Metrics/Format
|
Comments/Links
|
|
CppUnit
|
• JUnit framework for unit testing
• very familiar for developers who
have used JUnit or similar testing tools
|
• XML output compatible
with continuous integration reporting systems
• Automatic testing –XML output and
GUI based for supervised tests
|
|
|
Bandit
|
• Specifically developed for C++11
|
|
|
|
CppTest
|
• portable and powerful, yet simple,
unit testing framework for handling automated tests in C++
• Focuses on usability &
extensibility
• Hierarchical test suits
• Simplicity in use, use one include
file and single Suite class in TEST name space
• Rich assertion messages
|
• Test::TextOutput. This is the
simplest of all output handlers. The display mode could be either verbose or
terse.
• Test::CompilerOutput. The
output is generated in a manner resembling a compiler build log.
• Test::HtmlOutput. Fancy HTML
output.
|
|
|
Google Test
|
• Supports automatic test discovery,
a rich set of assertions, user-defined assertions, death tests, fatal and
non-fatal failures, various options for running the tests, and XML test
report generation.
• Does not support C++11 move
semantics
|
• Test discovery.
• A rich set of assertions.
• User-defined assertions.
• Death tests.
• Fatal and non-fatal failures.
• Value-parameterized tests.
• Type-parameterized tests.
• Various options for running the
tests.
• XML test report generation.
|
|
|
CppUTest
|
• unit xUnit test framework for unit
testing and for test-driving your code
|
• Allow to run tests JUnit style
• Available as Eclipse plugin
|
|
|
Catch
|
• Fast test setup, ease of use, and
detailed reporting
• No external dependencies. As long
as you can compile C++11 and have a C++ standard library available.
• Write test cases as,
self-registering, functions (or methods, if you prefer).
|
• JUnit xml output is supported for
integration with third-party tools, such as CI servers
|
|
|
Boost Test Library
|
• Handles exceptions and crashes
very well
|
• Simplify writing test cases by
using various testing tools.
• Organize test cases into a
test tree.
• Relieve you from messy error
detection, reporting duties and framework runtime parameters processing.
|
|
The below section describes the setup and usage of SonarQube.
SonarQube :
SonarQube is an open source platform for continuous code quality
inspection and perform automatic inspection of code for code check, bug detection,
find vulnerabilities. It helps to perform checks on more than 20 programming
languages. SonarQube is the server for report hosting and interpreting reports
,metrics generation using appropriate scanner pluing for each language. The
following are the key metrics provides by the SonarQube for C/C++ code
analysis.
•
Bugs and
Vulnerabilities
•
Code Smells
•
Coverage
•
Duplications
•
Issues:
•
Major
•
Critical
•
Minor
•
Blocker
•
Directory level
issues count
•
File level issues
count
•
Reliability
•
Security
•
Maintainability
•
Coverage
–
Lines to cover
–
Uncovered lines
–
line coverage
•
Duplications
–
Lines
–
Files
•
Size
–
lines of code
–
lines,
–
statements,
–
functions,
–
classes,
–
files,
–
directories,
–
comment lines,
–
comment %
•
Complexity
–
Cyclometric complexity
(overall project, component wise and files)
•
Issues
–
Bugs
–
Vulnerabilities
–
Code smells
•
Quality Gate Set
–
Coverage on New
Code
–
Duplicated Lines
on New Code (%)
–
Maintainability
Rating on New Code
–
Reliability Rating
on New Code
–
Security Rating on
New Code
Setup and Run SonarQube Server:
2. The pre-requstie
for SonarQube installation and execution are listed in https://docs.sonarqube.org/display/SONAR/Requirements . Make sure the
appropriate version of open JRE/Open JDK installed.
3. Use the below
command in ubuntu to install open JDK and Open JRE version 8.
sudo apt-get install openjdk-8-jdk
sudo apt-get install openjdk-8-jre
4. set up the Java 8 environment variables and
check Java version
sudo apt-get install oracle-java8-set-default
java -version
5. Configure the
parameters as described in https://docs.sonarqube.org/display/SONAR/Requirements
6. Install SonarQube
in /etc folder and run using the command:
sudo
./etc/sonarqube-x.x/bin/linux-x86-32/sonar.sh
console
The below message shows the SonarQube server started
successfully
jvm 1 | 2018.04.23 11:44:46 INFO app[][o.s.a.SchedulerImpl] Process[web] is up
jvm 1 | 2018.04.23 11:44:46 INFO app[][o.s.a.p.ProcessLauncherImpl] Launch
process[[key='ce', ipcIndex=3, logFilenamePrefix=ce]] from
[/etc/sonarqube-6.7.1]: /usr/lib/jvm/java-8-oracle/jre/bin/java
-Djava.awt.headless=true -Dfile.encoding=UTF-8
-Djava.io.tmpdir=/etc/sonarqube-6.7.1/temp -Xmx512m -Xms128m
-XX:+HeapDumpOnOutOfMemoryError -cp ./lib/common/*:./lib/server/*:./lib/ce/*:/etc/sonarqube-6.7.1/lib/jdbc/h2/h2-1.3.176.jar
org.sonar.ce.app.CeServer
/etc/sonarqube-6.7.1/temp/sq-process8900001930010179462properties
jvm 1 | 2018.04.23 11:44:52 INFO app[][o.s.a.SchedulerImpl] Process[ce] is up
jvm 1 | 2018.04.23 11:44:52 INFO app[][o.s.a.SchedulerImpl] SonarQube is up
8. Login SonarQube
Dashboard using default username:admin
Password:admin
9. Generate new token
from http://localhost:9000/account/security/ and the same token should be used from clients
(scanners) to establish connection.
Install C/C++ plugin and Scanners for Static Analysis:
1.
Commercial plugin-
Download and install sonarCFamily plugin (http://www.sonarsource.com/products/plugins/languages/cpp/and
copy SonarQube Plugin Directory (/etc/sonarqube-x.x/extensions/plugins/) . It
requires valid license.
2.
For Community plugin , download
community sonar-cxx plugin from (https://github.com/SonarOpenCommunity/sonar-cxx ) and copy SonarQube Plugin Directory (/etc/sonarqube-x.x/extensions/plugins/)
. It does not require any license
3.
The C++ plugin
won't execute any test code or run coverage tool or run static code checkers in
SonarQube . It basically helps to interpret the provided reports from the Scanner,
generate metrics and loads in the server.
4.
Note that the
implementation of sonar-cxx supports multiple sensors such as CPPCheck,
PCLint,Valgrind etc.. The complete list of supported sensors implementation can
be seen inside plugin sonar-cxx/org/sonar/plugins/cxx/ . It means the project
generates report from the tools such as CPPCheck, PCLint,Valgrind etc.., push
report to server and the reports are interpreted , validated and metrics
generated using sonar-cxx plugin at the server side.
5.
It is the
application or project responsibility to generate reports and push it to the
SonarQube Server. Application or Project can use Command Line tool (sonarscanner)or
IDE plugin (sonarLint for Eclipse) to generate report connect with SonarQube
and push .
6.
SonarLint will not
work with Community sonar-Cxx Plugin. It will work only with sonarCFamily
plugin for C/C++. So SonarScanner is the option for open source
7.
Install SonarScanner Command line interface
from https://docs.sonarqube.org/display/SCAN/Analyzing+with+SonarQube+Scanner , Sonar scanner used to scan, connect with sonarqube
,push the report and display the analysis result.
8.
Configure sonar.host.url
in sonar-scanner.properties (sonar-scanner-x.x/conf)
#-----
Default SonarQube server
sonar.host.url=http://localhost:9000
9.
Update the PATH
variable with sonar-scanner installation path.
The setup is fully
ready to from the scanner to connect with server and SonarQube to report
interpretation, analysis and publishing. But Need report to analysis. How to
generate the report? It is required to generate static analysis report at the
project level. Now we will see how to generate static analysis report using
multiple tools in the project and send to server for analysis.
Integrate Static Analysis, Code coverage and Unit test
framework in Project and Generate report:
Here I want to
discuss how to integrate few standalone static analysis tools in project ,
define rules and generate reports.
1.
Capture Compiler Warnings
First , let us start
with capturing compiler warning report . Most of the time compiler warning is
not taken into account. However less compiler warning defined good code
quality. To include all the compiler warnings in the static analysis report,
the warning flag of the compiler enabled and all warnings are captured in a
file.
1.
Enable -fdiagnostics-show-option flag in GCC
build option
2.
Redirect all
warning into file (warning.log)
3.
In
sonar-scanner.properties file define the following properties
sonar.cxx.compiler.parser=GCC
sonar.cxx.compiler.reportPath=warning.log
The options and procedure is listed in the link https://github.com/SonarOpenCommunity/sonar-cxx/wiki/Compilers
While running analysis, sonar scanner fetches the file
and push to server for analysis
2.
Run CPPCHECK and Capture Static Analysis report
Cppcheck
is general purpose static checking tool . It detects many real-time issues with
few false positives. It also used for MISRA C compliance check. The complete information on installation and
configuration details are available in http://cppcheck.sourceforge.net/
B.
Run $cppcheck in command line displays the options and
information of the tool
C.
To enable all the
checks and store the output in the XML, use the below command.
cppcheck -v --enable=all --xml -Isrc src 2> output/cppcheckreport.xml
It
will analyze the src folder and store the XML result in output folder. Enable
options for additional checks. The values are performance, portability, information, style,all
. Enaling ‘all’ to include all the checks.
3.
Run VERA and Capture Static Analysis report
#
Run vera: static code checker focusing on code style issues
sonar_vera:
bash -c 'find src -regex
".*\.cc\|.*\.hh" | vera++ - -showrules -nodup |&
vera++Report2checkstyleReport.perl > $(BUILD_DIR)/vera++-report.xml'
4.
RATS to perform security check and analysis
RATS
perform rough analysis on the security problems such as common security related
programming errors such as buffer overflows and TOCTOU (Time Of Check, Time Of
Use) race conditions. The tool downloaded and installed in the system .The RATS
report can be captured in XML using
rats -w 3 --xml src >
output/rats-report.xml
For more information
on refer https://code.google.com/archive/p/rough-auditing-tool-for-security/
5.
VALGRIND for memory analysis in program
Valgrind
can be used for memory debugging, memory leak detection and profiling. The tool
first installed in the system and use hte below command to get valgrind report
from program in XML format.
-valgrind --xml=yes
--xml-file=output/valgrind-report.xml output/program
6.
Unit Test and Coverage
Unit
test and coverage is more critical to detect error during development phase and
get the code coverage. Google Test framework is widely used unit test tool for
C/C++. The procedure to illustrate the usage of gtest and gcovr are given below
A.
Google Test
Setup
and build:
Gtest
enables writing independent, reusable and portable C++ unit test cases.
2.
Build using CMake as mentioned here https://github.com/google/googletest/blob/master/googletest/README.md
3.
Simple build without CMake by coping
src directory to the local working directory and use make file to compile as
below
libgtest.a:
g++ -I
/gtest/include -I /gtest/src
-c src/gtest-all.cc -o
gtest-all.o
g++ /gtest/include -I /gtest/src -g -c src/gtest_main.cc -o gtest_main.o
ar -r libgtest.a gtest-all.o gtest_main.o
4.
Now gstest library is compiled and
ready to write test cases .
Writing
Test cases and Execution:
The
below example shows the simple example to write test case for simple class
methods
Compoment1.cc
has simple class and method
//Class
class
Test1 {
public:
int Check(int ,int);
};
// return
Sum of two int
int Test1::Check(int
a ,int b){
return (a + b);
}
The
test cases are written for the method
libcomponents.a:
components.o
ar -r $@ components.o
#include
<gtest/gtest.h>
#include
<component1.cc>
namespace
{
class Component1Test : public
::testing::Test {
protected:
Test1 test1;
};
The source component is build using
TEST_F(Component1Test,
PositiveNos) {
ASSERT_EQ(3, test1.Check(1, 2));
ASSERT_EQ(0, test1.Check(0,0));
ASSERT_NE(10, test1.Check(10,0));
ASSERT_NE(21, test1.Check(9,19));
}
TEST_F(Component1Test,
NegativeNos) {
ASSERT_EQ(-8, test1.Check(-8,0));
ASSERT_EQ(0, test1.Check(5,-5));
ASSERT_NE(-3, test1.Check(2,-5));
ASSERT_EQ(-5, test1.Check(-4,-1));
ASSERT_EQ(-2, test1.Check(-6,2));
}
TEST() and
TEST_F() registers the test cases with google test
RUN_ALL_TESTS()
registers and runs all the tests in the unit . It can be called from main()
int
main(int argc, char **argv) {
::testing::InitGoogleTest(&argc, argv);
return RUN_ALL_TESTS();
}
test_component1: libcomponents.a
test_ Component1Test.o g++ -lpthread
libgtest.a libcomponents.a -pthread – coverage -o $@
Run the unit test cases and capture report in XML using
test_component1 --gtest_output=xml:xunit-report.xml
B. gcovr
(GCC Code Coverage Report)
Follow
instructions to install gcovr from https://pypi.org/project/gcovr/
Collect the coverage data and store XML format
using the command
gcovr -x -r . > gcovr-report.xml
Retrieve reports, rules loading, and report analysis
Now we have the
cppcheck ,rats, vera,gtest and gcovr report in the form of XML. Now running
SonarScanner to push all the reports to the SonarQube server and analyse the
report using CommunityC/C++ plugin in the server . The Plugin in SonarQube
interprets the reports and generate possible metrics.
1. Create sonar-project.properties file in the root
directory of the project to be scanned.
2. Add sonar.projectKey=<project_name> and sonar.login=<Generated-SonarQube-securitytoken>
if forceauthentication enabled in
SonarQube Server settings.
3. The relevant properties (sonar.cxx.cppcheck.reportPath, sonar.cxx.vera.reportPath etc.)
for the reports should be defined in the sonar-scanner.properties file.
# required metadata
sonar.projectKey=CxxPlugin:Sample
sonar.projectName=Sample
sonar.projectVersion=0.0.1
sonar.language=c++
# paths to the reports
sonar.cxx.cppcheck.reportPath=build/cppcheck-report.xml
sonar.cxx.coverage.reportPath=build/gcovr-report*.xml
sonar.cxx.coverage.itReportPath=build/gcovr-report*.xml
sonar.cxx.coverage.overallReportPath=build/gcovr-report*.xml
sonar.cxx.valgrind.reportPath=build/valgrind-report.xml
sonar.cxx.vera.reportPath=build/vera++-report.xml
sonar.cxx.rats.reportPath=build/rats-report.xml
sonar.cxx.xunit.reportPath=build/xunit-report.xml
4. To launch analysis and push the report, run sonar-scanner
command from the project root.
5. On successful analysis, EXECUTION SUCCESS message
displayed.
6. Open http://localhost:9000 to access the SonarQube Server dashboard and view the
results. The generated report from sample project is described as below
PCLint report is not included.
Summer Training In Noida
ReplyDelete