Monday, February 22, 2016

What is the Business Value of Unit Testing? Part 2

Part 2 you are here
Part 1

In the previous post, we started on a business application with the requirement:

> As a user, I need a way to enter a number. Once the number has been entered, I need it to be printed back to me.

The customer loves the app!  But has a few new requirements for us:

> As a user, I need a way to enter a number. Once the number has been entered, I need it to be printed back to me.
> If a number evenly divisible by 3 is entered, the user should receive back "Fizz" rather than the number
> If a number evenly divisible by 5 is entered, the user should receive back "Buzz" rather than the number

The requirements now are equivalent to the children's problem and/or code kata FizzBuzz

Our code previously only had a single branch of logic.  Get number, return number.  Based on the new requirements, we can see there will be a few more branches:

  1. number is not evenly divisible by 3 or 5
  2. number is evenly divisible by 3
  3. number is evenly divisible by 5
  4. (not explicitly stated but) number is evenly divisible by both 3 and 5.
The 4th branch was not stated by the requirements, but seem like something that should be asked of the business owner, as it might not have been considered, or could have even been assumed.

Our original method which looked like:

public string ReturnNumberAsString(int numberToReturn)
{
    return numberToReturn.ToString();
}

Will be updated to now look like:

        public string ReturnNumberAsString(int numberToReturn)
        {
            if (numberToReturn % 3 == 1 && numberToReturn % 5 == 1)
                return "FizzBuzz";
            else if (numberToReturn % 3 == 1)
                return "Fizz";
            else if (numberToReturn % 5 == 1)
                return "Buzz";
 
            return numberToReturn.ToString();
        }

Now be aware, all of our unit tests from last round, continue to pass, as the data that was being used to test the method continues to pass with our new implementation.  This is often something that is brought up as a potential pitfall of unit testing and requirements changing - and it is seemingly a valid concern!  Our unit tests are continuing to pass, when the requirements are much more complex than they were previously.

This is why it's so important to take into account *both* unit tests (and their asserts) as well as code coverage.  There is always the possibility that the unit tests *won't* break as a result of new branches in your code. BUT, if you were to look at code coverage, specifically the code coverage as it applies to our method, you'll see that not all branches of code are covered by unit tests.


As you can see from the above screenshot, our code coverage as indicated by the percent, and the purple and yellowish text has gone down.   The 3 new branches within our code is not currently being covered by unit tests.  Here's the repo as of updating the method, but without the unit tests to cover the requirements: https://github.com/Kritner/UnitTestingBusinessValue/tree/bb1f9bda9250fbdb85a8737c0c006f06e6daa788

Now to write a few unit tests:

/// <summary>
        /// number mod 3 and 5 returns FizzBuzz
        /// number mod 3 returns Fizz
        /// number mod 5 returns Buzz
        /// </summary>
        [TestMethod]
        public void NumberReturner_ReturnNumberAsString_SpecialCasesReturnValid()
        {
            // Arrange
            NumberReturner rt = new NumberReturner();
            int modThree = 9;
            int modFive = 10;
            int modThreeAndFive = 15;
 
            // Act
            var resultsModThree = rt.ReturnNumberAsString(modThree);
            var resultsModFive = rt.ReturnNumberAsString(modFive);
            var resultsModThreeAndFIve = rt.ReturnNumberAsString(modThreeAndFive);
 
            // Assert
            Assert.AreEqual("Fizz", resultsModThree, nameof(resultsModThree));
            Assert.AreEqual("Buzz", resultsModFive, nameof(resultsModFive));
            Assert.AreEqual("FizzBuzz", resultsModThreeAndFIve, nameof(resultsModThreeAndFIve));
        }

Hmm. We currently have a failing test.  Fizz is not being returned from resultsModThree, but 9 instead.  Let's see what's going on here.

Oh. Looks like I've inadvertently created a bug in my implementation of requirement #2.

if (numberToReturn % 3 == 1 && numberToReturn % 5 == 1)
    return "FizzBuzz";
else if (numberToReturn % 3 == 1)
    return "Fizz";
else if (numberToReturn % 5 == 1)
    return "Buzz";

Should have been:

if (numberToReturn % 3 == 0 && numberToReturn % 5 == 0)
    return "FizzBuzz";
else if (numberToReturn % 3 == 0)
    return "Fizz";
else if (numberToReturn % 5 == 0)
    return "Buzz";

Now that we've corrected the code, our new unit test passes. But our original unit test:

// Arrange
int expected = 42;
NumberReturner biz = new NumberReturner();
 
// Act
var results = biz.ReturnNumberAsString(expected);
 
// Assert
Assert.AreEqual(expected.ToString(), results);

Is now failing. Of course it is - 42 % 3 is 0, so we actually received a Fizz for 42. Updating that test to have an expected value of 7 instead.

What does all of this mean? Our unit tests both helped us and hurt us in this scenario.  They helped us because they helped us determine I had a logic error in my implementation of a requirement.  They hurt us because we had a "false positive" pass.  This is why it's so important that unit test Asserts are relevant, and code coverage stays high.  Without a combination of both of these, the business value of the tests is less significant.  The updated implementation and logic:  https://github.com/Kritner/UnitTestingBusinessValue/tree/78f03b8550593b9576f28e8608561f4add989879

In a non unit testing scenario, it is likely that our business logic would only be testable through the UI.  UI testing is much clunkier, slower, and harder to reproduce consistently.  Imagine after each change of our logic, we had to test all branches over and over again through the UI.  This probably means a compile, a launch, an application log in, navigate to your logic to test, etc.  Oh, and then do it three more times (due to this **simple** application's logic).  This is another reason unit testing is so powerful.  As we keep making changes to our code, we can help ensure the changes we're making are not impacting the system in ways we're not expecting.  And we're doing it in a fraction of the time that it would take to do the same thing through manual UI testing.

Hopefully this post and the previous help show how a good unit testing suite can really help you not only have less bugs in code, but get your code tested much faster.

Sunday, February 21, 2016

What is the Business Value of Unit Testing? Part 1

Part 2
Part 1 you are here

We've been doing a lot more concentration on unit testing at work lately, and a question has come up. What value do automated unit tests provide?

The text book(ish) answer is: if you're unit testing your code with relevant asserts, with good enough code coverage, the less likely there will be bugs. Sounds great! Why isn't everyone doing it? Unit testing does require a certain style of coding, loose dependencies, and a certain amount of planning. Here's a good SO answer that goes into some detail on how/why unit testing is great - but I like examples.

Requirements change. It's a fact of (programming) life. Something that held true today, might not be true months or years down the road. One of the great things about unit tests and code coverage is that when considered together, you can really get a feel for if your code is working correctly, even create requirements based on your unit tests! On to the example - we're going to build a super important piece of business logic based on this requirement:

>  As a user, I need a way to enter a number.  Once the number has been entered,  I need it to be printed back to me.

Well that sounds easy.  Going to start a new github repo to track progress.

So based on our requirement.  I'm going to create a console application that takes a user's entry, and then prints it back to them.. This is probably the most useful business logic in the history of the universe.

Based on the requirement, I've created a class and method:

    /// <summary>
    /// This class is used to return a number.
    /// </summary>
    public class NumberReturner
    {
 
        /// <summary>
        /// Return the provided number as a string
        /// </summary>
        /// <param name="numberToReturn">The number to return</param>
        /// <returns>The number as string</returns>
        public string ReturnNumberAsString(int numberToReturn)
        {
            return numberToReturn.ToString();
        }
 
    }

The above method is extremely easy to test as there is only a single branch.  It should be no problem getting 100% code coverage, with a completely relevant assert.

/// <summary>
    /// Tests for NumberReturner
    /// </summary>
    [TestClass]
    public class NumberReturnerTests
    {
 
        /// <summary>
        /// Ensure ReturnNumberAsString has appropriate return type
        /// </summary>
        [TestMethod]
        public void NumberReturner_ReturnNumberAsString_CorrectReturnTypeIsString()
        {
            // Arrange
            int expected = 42;
            NumberReturner biz = new NumberReturner();
 
            // Act
            var results = biz.ReturnNumberAsString(expected);
 
            // Assert
            Assert.IsInstanceOfType(results, typeof(string));
        }
 
        /// <summary>
        /// When ReturnNumberAsString is provided a number, the number is returned as a string
        /// </summary>
        [TestMethod]
        public void NumberReturner_ReturnNumberAsString_ReturnsNumberThatWasProvided()
        {
            // Arrange
            int expected = 42;
            NumberReturner biz = new NumberReturner();
 
            // Act
            var results = biz.ReturnNumberAsString(expected);
 
            // Assert
            Assert.AreEqual(expected.ToString(), results);
        }
 
    }

And our code coverage:


With the above test and code coverage, we can safely say we have thoroughly tested our code.  Our requirement is extremely simple as of now, but next time we'll expand our requirements by a bit, while still keeping focus on our unit tests and code coverage.  Here's the repo as of this post: https://github.com/Kritner/UnitTestingBusinessValue/tree/f8b21a5bde31635c2f37d530130d8bd393eee23e

On to part 2