Specflow Test Execution Report Enhancement

“Specflow”….. you lost my Tags from the feature file !!!

This post should help you get back your tags from the feature file into the HTML Specflow Report.

But before that, let us take a quick look at what is Specflow ?

“Specflow”, to simply speak, is a tool that can help you automate your test cases aka scenarios by writing them in Gherkin language(don’t worry it’s more English and not like C, C++, java, C# etc) in a file viz “Feature” which your Business Analysts / Customers/ Domain Experts can understand. This same feature file will be hooked to the underlying implementations by your .Net developers. Thereby narrowing down the gap between the domain experts and your .Net developers. For more details on Specflow, I recommend reading its quick introduction page at http://www.specflow.org/getting-started/

With that quick background let’s start with today’s topic i.e. enhancement to the existing Specflow Test Execution Report (STER).

Below is my feature file where I am testing a calculator for addition of 2 numbers. Please note the tags
@RequirementNumber-1  and @RequirementNumber-2

Sample Feature File

Sample Feature File


Now, Let’s run the test using Ms Test and generate a Test Results(.trx) file. The command to generate the html STER from a Ms Test Results file can be found @ http://www.specflow.org/documentation/Reporting/. The STER looks as below for our calculator

Specflow Default Test Execution Report

Specflow Default Test Execution Report

Notice that the tags we added in the feature file are not shown in the HTML report, this happens because of 2 reasons

  1.  When internally Specflow convert the Ms Test Report to a n-unit test report (xml) through an internal xslt file “MSTestToNUnit.xslt”, the tags/test category node is excluded.
  2. The default internal xslt file “NUnitExecutionReport.xslt” for converting the n-unit test report to HTML file does not include styling instructions for the nunit-categories node.

Now, the documentation of STER states that we can pass our own customized style sheet as a parameter to the Specflow.exe command line interface, however while generating the report it is observed that the customized xslt is applied to the n-unit xml file and not to the original Ms Test results file therefore if any information such as “TestCategory” is lost in the first transformation step itself then it will never appear in the final html STER.

To address this issue and if you want to run your test through Ms Test with Test Categories included in the report then you can do the following

1. Download the Specflow code from the github repository

2. Update the MSTestToNUnit.xslt file with the one that I have uploaded here

3. Compile the Specflow solution or update the  TechTalk.SpecFlow.Reporting.dll in your reference folders.

Running the Specflow command through command prompt with the same Ms Test Results (.trx) file will now generate a node for “TestCategory” from the trx file in the n-unit test xml as follows.

NUnit Report XML

NUnit Report XML


This enables us to include the Test Category aka Tags in our HTML report as shown in the snapshot below with a customized xslt (download link) thus making the Automated Test execution report complete.

Specflow Test Execution Report_Enhanced

Specflow Test Execution Report_Enhanced

Hope that helps and makes each user story & its acceptance criteria traceable to its automated test scenario.

DISCLAIMER : The views, opinions and information compiled in this blog are completely mine and has nothing to do with any of my previous, current or future employers