Behavior Driven Development with SpecFlow

Behavior Driven Development with SpecFlow:

Introduction:

This white paper explains how to use SpecFlow & WatiN in Behavior Driven Development (BDD). BDD has been embraced by many developers helping them keep focused on the business value of the software they create while at the same time covering their code with tests. This has been a challenge with .NET - the tools haven't been that expressive. That's changing with SpecFlow

BDD is all about implementing an application by describing it from the point of view of its stakeholders.

Behavior-driven development is an “outside-in” methodology. It starts with at the outside by identifying business outcomes, and then drills down into the feature set that will achieve those outcomes. Each feature is captured as a “story”, which defines the scope of the feature along with its acceptance criteria

Objective:

This article describes the way to address the gap between business users and developers by doing development which directly translates the business expectations

Using SpecFlow we can bridge gap between business users and developers. Business users will write the specifications/feature which will be further implemented by developers in the form of test cases. Developers will use these test cases to test the application code

WatiN was used to test the User Interface. Using this we can test controls values in controls and validations. WatiN is developed in C# and aims to bring us an easy way to automate our tests with Internet Explorer and Firefox using .Net. It supports AJAX testing as well

Scope:

This whitepaper will cover two scenarios on how to use SpecFlow and WatiN to implement BDD.

Acronyms and Definitions:

Definition/Acronym	Explanation
SME	Subject Matter Expert
SpecFlow	SpecFlow is a .Net library that allows us to describe user expectations in a consistent Domain-Specific Language that can be wired for automatic execution. By writing executable tests in a human readable manner, these tests can serve as a bridge between the users expectations and the code we produce to meet them.
WatiN	WatiN stands for Web Application testing in .NET, and it's a framework that enables web application testing through Internet Explorer and other browsers. WatiN is developed in C# and aims to bring an easy way to automate your tests with Internet Explorer and Firefox using .Net.
User Story	A User story is a high level description of system behavior, used as a planning tool. Stories are vertical slices of system functionality that are deliverable independently. Each story should specify the value it brings to one or more stakeholders, the stakeholders that care about the story and the scope of the intended work.
Feature File	A Feature file describes a feature or a part of a feature with representative examples of expected outcomes. SpecFlow uses these files to validate some system functionality against its specifications.
Scenario	A Scenario captures one key example for a feature in the feature file. It represents a way that the system delivers some value to a stakeholder. Scenarios are units of specifications for SpecFlow.
Steps	Steps are domain language phrases that we can combine to write scenarios. They can either refer to the context of a scenario, an action that the scenario describes or a way to validate the action.

Business Challenges:

Due to disconnect between the developers & SME’s because of less documentation of functional requirements, business was not able to understand and approve the unit tests written by developers as they were purely technical. SME wanted tests which can co-relate with the functionality of the application.

Solution:

Technology Landscape

Platforms / Environments	Visual Studio 2010
Framework/IDE	.Net Framework 4.0
UI	ASP.Net MVC
Application Server	IIS
Tools	SpecFlow, WatiN
Automation Testing	MSUnit
Browsers	IE, Firefox

Design Approach

Business User/stakeholders will represent the business functionality in the form of Feature and Scenarios. Business user will share the feature file with developers. Developers will understands and implement domain functionality for that feature.

Business Users will execute these scenarios and makes sure all features and scenarios are passed. Below is high level flow chart which explain the flow from business users and how it translates till the developer

                       

Figure: High-level flow from business users to testers

Implementation Approach

BDD based unit test were implemented using SpecFlow and WatiN which translates the business functionalities in the form of unit tests.

Figure: Feature-to-unit Test Process

Let us take two business scenarios to understand SpecFlow and WatiN.

First scenario, create test scenario for login page including negative test scenario also.

Second scenario, get the Partner details based on LenderId. Partner details should be loaded from excel file before executing the scenario. Data should be deleted after execution of test cases.

First Scenario:

Create a new ASP.NET MVC application with a unit test project. In the MSUnit project, add a feature file to the unit test project.

Figure: Add new feature file

Feature file contains Feature and a sample Scenario. Modify the feature file as per requirements. Notice that the file is created with a .feature extension and that Visual Studio recognizes this as a supported file. You may also notice that your feature file has a related .cs code-behind file. Each time you save a .feature file, SpecFlow parses the file and converts the text in that file into a test fixture. The code in the associated .cs file represents that test fixture, which is the code that’s actually executed each time you run your test suite.

Figure: Users functionality feature which contains multiple scenarios

Create a class (.cs) file with same name as feature file and write the test cases. Mention Given, When, Then and And attributes before each test case to map test case with scenario in feature file.

Use WatiN to create IE object and invoke the browser. Using IE object, add test data into the controls and test the scenarios.

Figure: Code for Users Feature

Execute the test cases from the “Test View” window. You can choose to “Debug” or “Run” the selected test case(s).

Figure: Browser is opened and executing test case

“Test Results” windows will display the status of executed test cases.

Figure: Test result window displaying the result of executed test case(s)

Second scenario:

Business Challenge: Retrieve the Partner details based on LenderId. LenderId value tends to change. Whenever LenderId record is deleted from database, then test case will fail.

Solution: Manually add the partner details and lender details in excel sheet. Read excel sheet data and insert records into the related tables before execution of test case. After completion of test case, delete the inserted records from the table.

Write a method to read excel sheet and mention BeforeScenario attribute. We can use various attributes like BeforeScenarioBlock, BeforeStep, BeforeFeature attributes to run method before test case executes.

Write a method to read the data from database. Pass the LenderId and get the partner details. Write a method to delete the records from database and use AfterScenario attribute. This will run after test case execution.

Report Generation:

To generate report, we need to use command-line tool of SpecFlow.

---

Command: specflow.exe generateall SpecFlowDemo.SpecFlowMVCTest.csproj

---

Salient Features of the solution

• Involves Business users/stakeholders in the process. This makes sure that requirements/features are clearly defined.
• Constantly getting feedback from stakeholders reduces development time. This process makes sure that what we are developing is correct.
• Helps us in catching regression issues when adding new functionality.
• Uses SpecFlow and WatiN tools to create and execute test cases, both tools are easy to install and free of cost.
• Both tools are integrated with Visual Studio. Able to debug feature from Visual Studio.

Setup & Configuration:

Install

• Download and install the SpecFlow from this URL.
• Download and install WatiN from this URL.

Add References

• Add reference of WatiN.Core.dll to unit test project
• Add reference of TechTalk.SpecFlow.dll to unit test project

By default, SpecFlow uses NUnit as its test-runner, but it also supports MSTest with a simple configuration change.

Business Benefits:

Involving the test team and SpecFlow early in the planning process has a number of advantages.

First, having test involved in the planning stages could help give a different perspective. Are there key stories missing? What happens in the scenario is changed by doing X? Why do we need this story if we can already do this with the Y feature? A lot of these are questions that already may come up, but the extra diversity may get the team thinking about things earlier.

Next having test involved early potentially compresses the development time. Test can get a better understanding of the behavior, stub out tests, and then implement automation in parallel with development thus reducing the amount of time between dev code complete and story acceptance. Quality may increase since test will have better understand early and has more contexts around what expected behavior should look like (and also have more time to ask clarifying questions). Once SpecFlow gets mixed into the process, now the entire team can get good visibility into what the acceptance tests actually look like and give feedback to help increase the coverage in a language that everyone understands.

That last bit is the important part –anyone can easily understand what a particular test case is actually doing without having to bring up MSDN or decipher curly braces.

Pros and Cons:

Advantages of SpecFlow

• Scenarios are basically UnitTests and can be run with your existing infrastructure.
• Scenarios and steps are easily debuggable out of VisualStudio.
• Integration in your continuous build should be a breeze, since the infrastructure to run unit-tests is most certainly there already.
• Business specifications mapped to test names automatically (auto-generated)
• Low entry cost, easy to get setup and going

Disadvantages of SpecFlow

• Limited ability to share and collaborate with product owners and stakeholders since specifications is held in Visual Studio.
• Specifications must be written in Gherkin language (given/when/then)
• Specification maintenance must be done by staff comfortable with Visual Studio
• It does not support as many languages as Cucumber
• SpecFlow depends on a Unit-Test framework, and currently only NUnit and MSTest are supported.
• Reporting in SpecFlow is not very sophisticated yet. Need to use command-line to generate the report.

Advantages of WatiN

• Fast execution
• Very good IE support. Can Attach and Detach from running instances. Can access native window handles etc.

Disadvantages of WatiN

• Very little documentation.
• No native XPath support
• No out-of-the-box remote execution/grid based execution

Best Practices:

• Feature Files Should Actually be Features, Not Entire Portions of an App
• Organize Your Features and Scenarios with the Same Thought You Give to Organizing Your Code
• Feature should be self-explanatory. Use descriptive title or tags to describe the feature.
• Don’t get trapped in user interface details.
• Make Scenarios Independent and Deterministic. Avoid writing features that are tightly coupled with code.
• Write Scenarios for the Non-Happy-Path Cases As Well
• Revisit, Refactor, and Improve Your Scenarios and Steps
• Use Compound Steps to Build Up Your Feature
• Avoid Using Conjunctive Steps: Each step should do one thing. You should not generally have step patterns containing “and.”
• Don’t over specify examples in a single scenario.

References

• SpecFlow: http://www.specflow.org/
• TechTalk – SpecFlow: https://github.com/techtalk/SpecFlow
• BDD with SpecFlow http://blog.stevensanderson.com/2010/03/03/behavior-driven-development-bdd-with-specflow-and-aspnet-mvc/
• WatiN: http://watin.org/
• Started with SpecFlow and WatiN: http://ardalis.com/everything-you-need-to-get-started-with-specflow-and-watin
• Other Tools:
• NSpec: http://nspec.org/
• Cucumber: http://cukes.info/
• StoryQ: http://storyq.codeplex.com/

CodeProject