Unit testing means running automated tests for the certain block of the code. These tests are run either by scripts or unit testing framework. The result is a list os passed and/or failed tests.
Unit testing framework used with WinMerge development is the Google Test, available from http://code.google.com/p/googletest/. Other frameworks can be used when need arises. However we don't want to use every known framework so the reason for adding new framework needs to be good, like new technology or language used that current framework does not support.
Earlier we tried to start using CppUnit but unfortunately even compiling CppUnit seemed to be too hard for many people. And creating tests is just a mess. So CppUnit usage is discouraged nowadays.
Existing unit tests must be run without errors. For every error there must be a known explanation. It is a good practice to document errors into unit tests so others know what is going on. It is also recommended to submit bug reports of failing tests if the fix is not imminent.
When changing the code which has unit tests, test must be always run before the patch is submitted to patch tracker and before the patch is committed to SVN. Possible known failures must be explained. Failing unit can be a blocker preventing committing the patch.
Failing unit tests may be (and most likely are) blockers for the beta and stable releases. Experimental releases can be released with known errors.
Unit tests by nature rarely are and can be complete. Adding more tests and improving current tests is highly encouraged. It is the easiest way to improve WinMerge quality!
There must be some care when altering existing tests though. It may be that there was a reason the existing test were done as it was. If that reason is not obvious, it is better to create new improved test and leave the old test untouched.
Fixing a failing unit testing by modifying the test is highly questionable. The test must be buggy for this to be allowed.
Google Test is an external library/framework that must be compiled before using with unit tests.
Unzip the Google Test sources to some folder. Pick a folder you can keep around (not some temp folder) because you'll need the header files every time you compile unit tests.
Open solution file msvc\gtest.sln
to the Visual Studio.
Depending on Visual Studio version the project/solution file may need to be
updated for later version before usage. Visual Studio asks about the conversion
when opening the solution file.
Build both Debug and Release targets of gtest project. You don't need to build other projects. Indeed, rebuild all for the solution does not work. Some of the projects in the solution are unit tests for the framework itself.
Copy the compiled gtest.lib
and gtestd.lib
files
from msvc\Release
and msvc\Debug
subfolders to
Testing\GoogleTest
- folder in WinMerge source tree.
As mentioned earlier Google Test header files are needed to compile the tests. As the Google Test was unzipped to separate folder we must first tell Visual Studio where to find those headers.
Add the gtest-[version]\include
-folder to Visual Studio's
include file folders:
gtest-[version]\include
-folder as LAST folder in
that listOpen the testing project to Visual Studio (VS 2003 and later work) and compile Debug or Release -target of the project.
Open the Command Prompt and CD to folder where executable were build,
usually [projectname]/Debug
or [projectname]/Release
.
While the testing runs it prints progress information and info about passed/failed tests.
Initial creation of new test project takes some effort and time for the first time. It is still pretty straightforward so nobody should have any real problems with it. If somebody has, please tell Kimmo about it and we'll improve the documentation!
Open the Visual Studio and select creating new project. Select Win32 Console Project as the project type. You can create some other type too and then just fix the properties. But console project selects many properties correctly for you.
Note: If you are creating tests for class depending on MFC, then remember to include MFC (libraries) to the project.
Add new test projects into subfolders to folder Testing\GoogleTest
. E.g. Testing\GoogleTest\CommandLine
. The project (folder)
name should somehow relate the to the code/functionality/class you are testing.
Several project settings need to be changed to make test project compile (properly). There might be better choises and practices, but this is the best setup we currently know. If you have ideas for improving, let us know!
Set the C/C++ properties according list below:
Set the Linker properties according list below:
gtest.lib
(note: you
may use debug version gtestd.lib
in debug target)
Next you need to add new C++ code (.cpp
) file to the project.
Name the file as [projectname]_test.cpp
and make sure the file
name does not overlap with the C++ files you are testing! _test
as
part of the filen name helps.
Copy/paste the beginning of existing test file to the new .cpp
-file. Copy e.g. from Testing\GoogleTest\Paths\paths_test.cpp
. Copy the content to the first test function (TEST_F()
).
Don't copy the test functions, but copy the namespace closing brace and
main()
function from the end of the file.
Rename the test class to desribe your tests (e.g. CommandLineTest
). Remember to rename constructor/destructor methods too!
Now you can start adding tests. Look Google Test documentation and existing unit tests for help on that.
The simple test method looks like this:
TEST_F(PathTest, Create_abspath1) { EXPECT_TRUE(paths_CreateIfNeeded("c:\\Temp")); }
The TEST_F
macro defines a new test function, having test class
(PathTest
) and test name (Create_abspath1
) as
parameters. Then there is a function call done inside test macro. The framework
defines several macros for the tests. The EXPECT_TRUE
macro makes
the test succeed if the parameter (funtion return value) is true, otherwise the
test fails.
GoogleTests allows splitting tests to several cases and several files.
Adding new test case is easy: just add new file and create new test class
there. Don't copy main()
function. Eatch test project should have
only one main()
.