In this blog I will be introducing FitNesse tool, an automated testing tool for user acceptance testing. This blog will give an overview of the tool which is a Wiki built on top of Fit framework and used for automating Acceptance test cases.
Existing User Acceptance Testing Processes
- Manual Acceptance testing: User exercises the system manually using his system knowledge.
- Acceptance testing with “GUI Test Drivers” (at the GUI level): These tools help the developer perform functional/acceptance testing through user interface such as a native GUI or web interface.
- Table-based Approach for acceptance testing: Starting from a user story (or use case or textual requirement), the customer enters in a table (spreadsheet application, html, etc) the expectations of the program’s behavior. The customer can manually insert inputs in the system and compare outputs with expected results.
The Existing Manual User Acceptance Test process as described above is time consuming, prone to errors and incurs cost. Manual testing becomes exhaustive because of the large test data and numerous test cases. This process can be improved by automating the test cases for acceptance testing phase. FitNesse is the only Open source automation tool available for automating the Acceptance test cases/scenarios. This tool is Simple and has Robust Test Frameworks and also provides simple and user friendly result reporting.
What is FitNesse?
- FitNesse is an Open source integrated testing framework for automating the User Acceptance Testing and Regression Testing process.
- FitNesse creates a feedback loop between customers and programmers and allows customers and testers to use tools like MS Office Excel to give examples on how a program should behave.
- FitNesse automatically checks those examples against the actual program, thus building a simple and powerful bridge between the business and software engineering worlds.
- FitNesse enables customers, testers and programmers to learn what their software should do, and to automatically compare that to what it actually does.
- FitNesse is used by agile web teams to create comprehensive test suites that make system-level assertions to test in the TDD process.
FitNesse is basically
- Open source User accepting Testing tool written in Java 1.4
- Tests written in the form of HTML tables in wiki pages
- Makes use of two frameworks – Fit and Slim frameworks
- Test tables references fixture classes that connects to the business logic, Fixture classes/code are written in either – Java, .Net, Ruby, Python, C#
Benefits of FitNesse
Reusability of Test Cases based on dataset – Same Set of Test Cases can be used to test positive as well as negative scenarios with different dataset.
Reusability of Framework and Fixtures – As the Fixture classes are developed using Java, they can be reused for similar scenarios and fixture tables. Frameworks are also reusable for similar types of scenarios and functionalities.
Unit Testing and Regression Testing – FitNesse can be used and implemented for Unit Testing as well as Regression testing. Generation of fixture classes and test table can be incorporated as a process in the development life cycle. This gives advantage of using the same test cases for unit testing which will save time required for next levels of testing by finding the defects at early stages which will reduce testing cost.
Script Maintenance Activity – The developer, who works on change request can also modify the test cases in Fitnesse as a part of process. This will help in upgrading the test scripts quickly as per the new features in the application.
Multiple Test Suites – Once all the test cases are created in Fitnesse, different Test Suite can be created based on the selection of test cases that are required for execution. This will help in creating multiple test suites like- Unit Test, Regression Test and user Acceptance Test.
Independent Test Cases – The Test Cases in Fitnesse are independent of each other. During script execution, if a specific test case is not correctly executed it will not affect other test cases in the suite.
Reliability – Automation Process is always more reliable than the Manual Process
FitNesse Testing Process/Framework
- Write unit/regression/acceptance tests: These tests are written in a form of executable tables with Wiki syntax.
- Write fixture code: After the customers define the acceptance tests in wiki tables, the developers work on implementing the test cases with fixtures which communicates with business logic.
- Run acceptance tests: Acceptance tests are executed through the FitNesse Test runner. The Test runner parses the test tables to find the key words and link them together to construct the class name of the fixture code which communicates with the business logic and returns the actual data.
- View test results: After running the acceptance tests on the FitNesse server, the results are shown in the form of the original test table with the appropriate value cells colors that represent the acceptance testing results.
The Table styles
There are two types of test-systems/frameworks available in FitNesse- Fit and Slim. The test/wiki pages have wiki text written in wiki syntax, the input and expected output is written in the form of tables. The Fit and Slim Test System has it’s own particular kind of Test Table styles. First Row- first cell always refers Fixture Name. Subsequent cells are used to pass the input parameters and expected values.
The Most Common Fit Table/Fixture Styles
|Column Fixture||This is the style you may end up using most: rows of data represent inputs and expected outputs.|
|Row Fixture||This is good for testing queries that should return an exact set of values
|Action Fixture||This style allows you write a script that emulates a series of events
(Such as controls manipulated on a user interface).
|Comment Tables||Sometimes you want a tabular comment that is not executed as a test.|
The Slim Table styles
The first cell of a slim table tells you what kind of table it is. Here are the table types so far:
|Decision Table||This is similar to Fit Column Fixture which supplies inputs and outputs for decision.|
|Query Table||Supplies the expected results of a query. This is similar to the Fit Row Fixture|
|Subset Query Table||Supplies a subset of the expected results of a query.|
|Ordered query Table||Supplies the expected results of a query. The rows are expected to be in order. This is similar to the Fit Row Fixture|
|Script Table||A series of actions and checks. Similar to Do Fixture.|
|Table Table||The design which you want it to be!|
|Import||Add a path to the fixture search path.|
|Comment||A table that does nothing.|
|Scenario Table||A table that can be called from other tables.|
|Library Table||A table that installs fixtures available for all test pages|
Advantages of FitNesse
- Fitnesse is webserver – requires no extra configuration or Setup
- FitNesse is a wiki –You can easily create and edit the testcases on wiki pages
- Tests can be written before the code so that this approach can support TDD-Agile approach
- Requirements Engineering –
- Easy to describe the Requirements themselves for existing frameworks using tables
- Good tool for requirements engineering
- Easy for the developer to glue the requirements with business logic using Fixtures
- FitNesse is light weight and Open Source framework that makes it easy for software teams to:
- Collaboratively define Acceptance Tests- wiki with table of inputs and expected outputs
- Write the Fixture code to communicate this tests with Business logic
- Run those tests and analyze the results
- FitNesse has features adapted to system tests, like:
- CommandLineFixture available to execute special cases
- DoFixture and Ordered Query table executing some methods in order
- FitNesse is perfectly suited for regression testing
- The biggest advantage Fitnesse has over other integration testing frameworks is that the entry point into the code stack is not the (web) frontend – fixtures are written in java/C# and call out to whatever part of the codebase you are testing. This allows us to test separate layers as well as testing the entire code stack working together.
Limitations of FitNesse
- No capture & replay possible
- Cannot explicitly test a web frontend
- Web interface can make setting up expectations time consuming
- Learning curve to understand the framework is steeper if we have to modify the framework.
This was a brief introduction to the FitNesse tool, in the subsequent blogs we shall see how to use FitNesse tool to Create Test pages/wiki pages, Write Fixture classes, etc along with code samples