Behavior Driven Development with SpecFlow:
Introduction:
This white paper explains how to use SpecFlow & WatiN in Behavior Driven Development (BDD). BDD has been embraced by many developers helping them keep focused on the business value of the software they create while at the same time covering their code with tests. This has been a challenge with .NET - the tools haven't been that expressive. That's changing with SpecFlow
BDD is all about implementing an application by describing it from the point of view of its stakeholders.
Behavior-driven development is an “outside-in” methodology. It starts with at the outside by identifying business outcomes, and then drills down into the feature set that will achieve those outcomes. Each feature is captured as a “story”, which defines the scope of the feature along with its acceptance criteria
Objective:
This article describes the way to address the gap between business users and developers by doing development which directly translates the business expectations
Using SpecFlow we can bridge gap between business users and developers. Business users will write the specifications/feature which will be further implemented by developers in the form of test cases. Developers will use these test cases to test the application code
WatiN was used to test the User Interface. Using this we can test controls values in controls and validations. WatiN is developed in C# and aims to bring us an easy way to automate our tests with Internet Explorer and Firefox using .Net. It supports AJAX testing as well
Scope:
This whitepaper will cover two scenarios on how to use SpecFlow and WatiN to implement BDD.
Acronyms and Definitions:
Definition/Acronym
|
Explanation
|
SME
|
Subject Matter Expert
|
SpecFlow
|
SpecFlow is a .Net library that allows us to describe user expectations in a consistent Domain-Specific Language that can be wired for automatic execution. By writing executable tests in a human readable manner, these tests can serve as a bridge between the users expectations and the code we produce to meet them.
|
WatiN
|
WatiN stands for Web Application testing in .NET, and it's a framework that enables web application testing through Internet Explorer and other browsers. WatiN is developed in C# and aims to bring an easy way to automate your tests with Internet Explorer and Firefox using .Net.
|
User Story
|
A User story is a high level description of system behavior, used as a planning tool. Stories are vertical slices of system functionality that are deliverable independently. Each story should specify the value it brings to one or more stakeholders, the stakeholders that care about the story and the scope of the intended work.
|
Feature File
|
A Feature file describes a feature or a part of a feature with representative examples of expected outcomes. SpecFlow uses these files to validate some system functionality against its specifications.
|
Scenario
|
A Scenario captures one key example for a feature in the feature file. It represents a way that the system delivers some value to a stakeholder. Scenarios are units of specifications for SpecFlow.
|
Steps
|
Steps are domain language phrases that we can combine to write scenarios. They can either refer to the context of a scenario, an action that the scenario describes or a way to validate the action.
|
Business Challenges:
Due to disconnect between the developers & SME’s because of less documentation of functional requirements, business was not able to understand and approve the unit tests written by developers as they were purely technical. SME wanted tests which can co-relate with the functionality of the application.
Solution:
Technology Landscape
Platforms / Environments
|
Visual Studio 2010
|
|
Framework/IDE
|
.Net Framework 4.0
|
|
UI
|
ASP.Net MVC
|
|
Application Server
|
IIS
|
|
Tools
|
SpecFlow, WatiN
|
|
Automation Testing
|
MSUnit
|
|
Browsers
|
IE, Firefox
|
|
Design Approach
Business User/stakeholders will represent the business functionality in the form of Feature and Scenarios. Business user will share the feature file with developers. Developers will understands and implement domain functionality for that feature.
Business Users will execute these scenarios and makes sure all features and scenarios are passed. Below is high level flow chart which explain the flow from business users and how it translates till the developer
Figure: High-level flow from business users to testers
Implementation Approach
BDD based unit test were implemented using SpecFlow and WatiN which translates the business functionalities in the form of unit tests.
Figure: Feature-to-unit Test Process
Let us take two business scenarios to understand SpecFlow and WatiN.
First scenario, create test scenario for login page including negative test scenario also.
Second scenario, get the Partner details based on LenderId. Partner details should be loaded from excel file before executing the scenario. Data should be deleted after execution of test cases.
First Scenario:
Create a new ASP.NET MVC application with a unit test project. In the MSUnit project, add a feature file to the unit test project.
Figure: Add new feature file
Feature file contains Feature and a sample Scenario. Modify the feature file as per requirements. Notice that the file is created with a .feature extension and that Visual Studio recognizes this as a supported file. You may also notice that your feature file has a related .cs code-behind file. Each time you save a .feature file, SpecFlow parses the file and converts the text in that file into a test fixture. The code in the associated .cs file represents that test fixture, which is the code that’s actually executed each time you run your test suite.
Figure: Users functionality feature which contains multiple scenarios
Create a class (.cs) file with same name as feature file and write the test cases. Mention Given, When, Then and And attributes before each test case to map test case with scenario in feature file.
Use WatiN to create IE object and invoke the browser. Using IE object, add test data into the controls and test the scenarios.
Figure: Code for Users Feature
Execute the test cases from the “Test View” window. You can choose to “Debug” or “Run” the selected test case(s). 
Figure: Browser is opened and executing test case
“Test Results” windows will display the status of executed test cases. 
Figure: Test result window displaying the result of executed test case(s)
Second scenario:
Business Challenge: Retrieve the Partner details based on LenderId. LenderId value tends to change. Whenever LenderId record is deleted from database, then test case will fail.
Solution: Manually add the partner details and lender details in excel sheet. Read excel sheet data and insert records into the related tables before execution of test case. After completion of test case, delete the inserted records from the table.
Write a method to read excel sheet and mention BeforeScenario attribute. We can use various attributes like BeforeScenarioBlock, BeforeStep, BeforeFeature attributes to run method before test case executes.
Write a method to read the data from database. Pass the LenderId and get the partner details. Write a method to delete the records from database and use AfterScenario attribute. This will run after test case execution.
Report Generation:
To generate report, we need to use command-line tool of SpecFlow.
Command: specflow.exe generateall SpecFlowDemo.SpecFlowMVCTest.csproj
Salient Features of the solution
Involves Business users/stakeholders in the process. This makes sure that requirements/features are clearly defined.
Constantly getting feedback from stakeholders reduces development time. This process makes sure that what we are developing is correct.
Helps us in catching regression issues when adding new functionality.
Uses SpecFlow and WatiN tools to create and execute test cases, both tools are easy to install and free of cost.
Both tools are integrated with Visual Studio. Able to debug feature from Visual Studio.
Setup & Configuration:
Install
-
Download and install WatiN from this URL.
Add References
By default, SpecFlow uses NUnit as its test-runner, but it also supports MSTest with a simple configuration change.
Business Benefits:
Involving the test team and SpecFlow early in the planning process has a number of advantages.
First, having test involved in the planning stages could help give a different perspective. Are there key stories missing? What happens in the scenario is changed by doing X? Why do we need this story if we can already do this with the Y feature? A lot of these are questions that already may come up, but the extra diversity may get the team thinking about things earlier.
Next having test involved early potentially compresses the development time. Test can get a better understanding of the behavior, stub out tests, and then implement automation in parallel with development thus reducing the amount of time between dev code complete and story acceptance. Quality may increase since test will have better understand early and has more contexts around what expected behavior should look like (and also have more time to ask clarifying questions). Once SpecFlow gets mixed into the process, now the entire team can get good visibility into what the acceptance tests actually look like and give feedback to help increase the coverage in a language that everyone understands.
That last bit is the important part –anyone can easily understand what a particular test case is actually doing without having to bring up MSDN or decipher curly braces.
Pros and Cons:
Advantages of SpecFlow
Scenarios are basically UnitTests and can be run with your existing infrastructure.
Scenarios and steps are easily debuggable out of VisualStudio.
Integration in your continuous build should be a breeze, since the infrastructure to run unit-tests is most certainly there already.
Business specifications mapped to test names automatically (auto-generated)
Low entry cost, easy to get setup and going
Disadvantages of SpecFlow
Limited ability to share and collaborate with product owners and stakeholders since specifications is held in Visual Studio.
Specifications must be written in Gherkin language (given/when/then)
Specification maintenance must be done by staff comfortable with Visual Studio
It does not support as many languages as Cucumber
SpecFlow depends on a Unit-Test framework, and currently only NUnit and MSTest are supported.
Reporting in SpecFlow is not very sophisticated yet. Need to use command-line to generate the report.
Advantages of WatiN
Disadvantages of WatiN
Best Practices:
Feature Files Should Actually be Features, Not Entire Portions of an App
Organize Your Features and Scenarios with the Same Thought You Give to Organizing Your Code
Feature should be self-explanatory. Use descriptive title or tags to describe the feature.
Don’t get trapped in user interface details.
Make Scenarios Independent and Deterministic. Avoid writing features that are tightly coupled with code.
Write Scenarios for the Non-Happy-Path Cases As Well
Revisit, Refactor, and Improve Your Scenarios and Steps
Use Compound Steps to Build Up Your Feature
Avoid Using Conjunctive Steps: Each step should do one thing. You should not generally have step patterns containing “and.”
Don’t over specify examples in a single scenario.
References
CodeProject
