Friday, September 30, 2016

xml2rfc oh my jeez


Ugh.  Been having crazy trouble with getting started with Python for windows, all the way through getting xml2rfc installed.  Background is I'm trying to regenerate some RFC documents based on JSON edits in the appropriate XML files for a project.

SO, here's the steps I (think) were taken in the end to get it working - in case I ever need to do it again.


  1. Install Python 3.5 
    1. URL 
      1. https://www.python.org/downloads/release/python-352/
    2. Exact link I used:
      1. https://www.python.org/ftp/python/3.5.2/python-3.5.2-amd64-webinstall.exe
  2. (Fails) pip install xml2rfc
    1. Error related to lxml, for some reason exception being thrown from a VS14 directory?
  3. (Fails) pip install lxml
    1. Ok... so here's what not to do I guess
  4. pip install wheel
    1. After trolling through threads, found that "wheel" can be used to install lxml
  5. python -m pip install lxml-3.6.4-cp35-cp35m-win_amd64.whl
    1. From http://stackoverflow.com/a/30494745/2312877
    2. Grab .whl for lxml (not sure why pip install lxml does not work)
    3. Put in a directory
    4. run command from base
  6. pip install xml2rfc
Hooray!  That took forever... apparently stuff just needs to be done in a weird order, and not one that (at least I would) expect.  Oh how I miss npm compared to this package manager...

Monday, February 22, 2016

What is the Business Value of Unit Testing? Part 2

Part 2 you are here
Part 1

In the previous post, we started on a business application with the requirement:

> As a user, I need a way to enter a number. Once the number has been entered, I need it to be printed back to me.

The customer loves the app!  But has a few new requirements for us:

> As a user, I need a way to enter a number. Once the number has been entered, I need it to be printed back to me.
> If a number evenly divisible by 3 is entered, the user should receive back "Fizz" rather than the number
> If a number evenly divisible by 5 is entered, the user should receive back "Buzz" rather than the number

The requirements now are equivalent to the children's problem and/or code kata FizzBuzz

Our code previously only had a single branch of logic.  Get number, return number.  Based on the new requirements, we can see there will be a few more branches:

  1. number is not evenly divisible by 3 or 5
  2. number is evenly divisible by 3
  3. number is evenly divisible by 5
  4. (not explicitly stated but) number is evenly divisible by both 3 and 5.
The 4th branch was not stated by the requirements, but seem like something that should be asked of the business owner, as it might not have been considered, or could have even been assumed.

Our original method which looked like:

public string ReturnNumberAsString(int numberToReturn)
{
    return numberToReturn.ToString();
}

Will be updated to now look like:

        public string ReturnNumberAsString(int numberToReturn)
        {
            if (numberToReturn % 3 == 1 && numberToReturn % 5 == 1)
                return "FizzBuzz";
            else if (numberToReturn % 3 == 1)
                return "Fizz";
            else if (numberToReturn % 5 == 1)
                return "Buzz";
 
            return numberToReturn.ToString();
        }

Now be aware, all of our unit tests from last round, continue to pass, as the data that was being used to test the method continues to pass with our new implementation.  This is often something that is brought up as a potential pitfall of unit testing and requirements changing - and it is seemingly a valid concern!  Our unit tests are continuing to pass, when the requirements are much more complex than they were previously.

This is why it's so important to take into account *both* unit tests (and their asserts) as well as code coverage.  There is always the possibility that the unit tests *won't* break as a result of new branches in your code. BUT, if you were to look at code coverage, specifically the code coverage as it applies to our method, you'll see that not all branches of code are covered by unit tests.


As you can see from the above screenshot, our code coverage as indicated by the percent, and the purple and yellowish text has gone down.   The 3 new branches within our code is not currently being covered by unit tests.  Here's the repo as of updating the method, but without the unit tests to cover the requirements: https://github.com/Kritner/UnitTestingBusinessValue/tree/bb1f9bda9250fbdb85a8737c0c006f06e6daa788

Now to write a few unit tests:

/// <summary>
        /// number mod 3 and 5 returns FizzBuzz
        /// number mod 3 returns Fizz
        /// number mod 5 returns Buzz
        /// </summary>
        [TestMethod]
        public void NumberReturner_ReturnNumberAsString_SpecialCasesReturnValid()
        {
            // Arrange
            NumberReturner rt = new NumberReturner();
            int modThree = 9;
            int modFive = 10;
            int modThreeAndFive = 15;
 
            // Act
            var resultsModThree = rt.ReturnNumberAsString(modThree);
            var resultsModFive = rt.ReturnNumberAsString(modFive);
            var resultsModThreeAndFIve = rt.ReturnNumberAsString(modThreeAndFive);
 
            // Assert
            Assert.AreEqual("Fizz", resultsModThree, nameof(resultsModThree));
            Assert.AreEqual("Buzz", resultsModFive, nameof(resultsModFive));
            Assert.AreEqual("FizzBuzz", resultsModThreeAndFIve, nameof(resultsModThreeAndFIve));
        }

Hmm. We currently have a failing test.  Fizz is not being returned from resultsModThree, but 9 instead.  Let's see what's going on here.

Oh. Looks like I've inadvertently created a bug in my implementation of requirement #2.

if (numberToReturn % 3 == 1 && numberToReturn % 5 == 1)
    return "FizzBuzz";
else if (numberToReturn % 3 == 1)
    return "Fizz";
else if (numberToReturn % 5 == 1)
    return "Buzz";

Should have been:

if (numberToReturn % 3 == 0 && numberToReturn % 5 == 0)
    return "FizzBuzz";
else if (numberToReturn % 3 == 0)
    return "Fizz";
else if (numberToReturn % 5 == 0)
    return "Buzz";

Now that we've corrected the code, our new unit test passes. But our original unit test:

// Arrange
int expected = 42;
NumberReturner biz = new NumberReturner();
 
// Act
var results = biz.ReturnNumberAsString(expected);
 
// Assert
Assert.AreEqual(expected.ToString(), results);

Is now failing. Of course it is - 42 % 3 is 0, so we actually received a Fizz for 42. Updating that test to have an expected value of 7 instead.

What does all of this mean? Our unit tests both helped us and hurt us in this scenario.  They helped us because they helped us determine I had a logic error in my implementation of a requirement.  They hurt us because we had a "false positive" pass.  This is why it's so important that unit test Asserts are relevant, and code coverage stays high.  Without a combination of both of these, the business value of the tests is less significant.  The updated implementation and logic:  https://github.com/Kritner/UnitTestingBusinessValue/tree/78f03b8550593b9576f28e8608561f4add989879

In a non unit testing scenario, it is likely that our business logic would only be testable through the UI.  UI testing is much clunkier, slower, and harder to reproduce consistently.  Imagine after each change of our logic, we had to test all branches over and over again through the UI.  This probably means a compile, a launch, an application log in, navigate to your logic to test, etc.  Oh, and then do it three more times (due to this **simple** application's logic).  This is another reason unit testing is so powerful.  As we keep making changes to our code, we can help ensure the changes we're making are not impacting the system in ways we're not expecting.  And we're doing it in a fraction of the time that it would take to do the same thing through manual UI testing.

Hopefully this post and the previous help show how a good unit testing suite can really help you not only have less bugs in code, but get your code tested much faster.

Sunday, February 21, 2016

What is the Business Value of Unit Testing? Part 1

Part 2
Part 1 you are here

We've been doing a lot more concentration on unit testing at work lately, and a question has come up. What value do automated unit tests provide?

The text book(ish) answer is: if you're unit testing your code with relevant asserts, with good enough code coverage, the less likely there will be bugs. Sounds great! Why isn't everyone doing it? Unit testing does require a certain style of coding, loose dependencies, and a certain amount of planning. Here's a good SO answer that goes into some detail on how/why unit testing is great - but I like examples.

Requirements change. It's a fact of (programming) life. Something that held true today, might not be true months or years down the road. One of the great things about unit tests and code coverage is that when considered together, you can really get a feel for if your code is working correctly, even create requirements based on your unit tests! On to the example - we're going to build a super important piece of business logic based on this requirement:

>  As a user, I need a way to enter a number.  Once the number has been entered,  I need it to be printed back to me.

Well that sounds easy.  Going to start a new github repo to track progress.

So based on our requirement.  I'm going to create a console application that takes a user's entry, and then prints it back to them.. This is probably the most useful business logic in the history of the universe.

Based on the requirement, I've created a class and method:

    /// <summary>
    /// This class is used to return a number.
    /// </summary>
    public class NumberReturner
    {
 
        /// <summary>
        /// Return the provided number as a string
        /// </summary>
        /// <param name="numberToReturn">The number to return</param>
        /// <returns>The number as string</returns>
        public string ReturnNumberAsString(int numberToReturn)
        {
            return numberToReturn.ToString();
        }
 
    }

The above method is extremely easy to test as there is only a single branch.  It should be no problem getting 100% code coverage, with a completely relevant assert.

/// <summary>
    /// Tests for NumberReturner
    /// </summary>
    [TestClass]
    public class NumberReturnerTests
    {
 
        /// <summary>
        /// Ensure ReturnNumberAsString has appropriate return type
        /// </summary>
        [TestMethod]
        public void NumberReturner_ReturnNumberAsString_CorrectReturnTypeIsString()
        {
            // Arrange
            int expected = 42;
            NumberReturner biz = new NumberReturner();
 
            // Act
            var results = biz.ReturnNumberAsString(expected);
 
            // Assert
            Assert.IsInstanceOfType(results, typeof(string));
        }
 
        /// <summary>
        /// When ReturnNumberAsString is provided a number, the number is returned as a string
        /// </summary>
        [TestMethod]
        public void NumberReturner_ReturnNumberAsString_ReturnsNumberThatWasProvided()
        {
            // Arrange
            int expected = 42;
            NumberReturner biz = new NumberReturner();
 
            // Act
            var results = biz.ReturnNumberAsString(expected);
 
            // Assert
            Assert.AreEqual(expected.ToString(), results);
        }
 
    }

And our code coverage:


With the above test and code coverage, we can safely say we have thoroughly tested our code.  Our requirement is extremely simple as of now, but next time we'll expand our requirements by a bit, while still keeping focus on our unit tests and code coverage.  Here's the repo as of this post: https://github.com/Kritner/UnitTestingBusinessValue/tree/f8b21a5bde31635c2f37d530130d8bd393eee23e

On to part 2

Monday, January 4, 2016

Getting started with Unit Testing and Moq - Part 4

Part 1
Part 2
Part 3
Part 4 you are here

In the previous post, we had setup our basic WCF project to play around with for unit testing. Since then, I have pulled out the WCF service reference in the console application, and placed it in our business project. Now that the business project has the WCF service reference, I have added a new class that handles the newing up of the WCF client.
Additionally, I have updated the console application to use the new business object wrapper of the WCF client.
Both of those classes look like this:

Program.cs

namespace RussUnitTestSample
{
  class Program
  {

      #region consts
      const string CONNECTION_STRING = "Data Source=192.168.50.4,1515;Initial Catalog=MBES;Persist Security Info=True;Integrated Security=true;";
      #endregion consts

      #region Entry

      static void Main(string[] args)
      {
          GetNumbersAndAddThem obj = new GetNumbersAndAddThem(
              new DbGetSomeNumbers(new BaseDbConnection(CONNECTION_STRING)),
              new NumberFunctions()
          );

          Console.WriteLine("\n");
          Console.WriteLine(obj.Execute());
          Console.WriteLine("\n");

          Business.WCF.Service1 service = new Business.WCF.Service1();

          Console.WriteLine("\n");
          Console.WriteLine("{0}", service.GetData(42));
          Console.WriteLine("\n");

      }

      #endregion Entry
  }

}

WCF.Service1

namespace RussUnitTestSample.Business.WCF
{

    /// <summary>
    /// Communication with the WCF Service1
    /// </summary>
    public class Service1
    {

        #region Private
        private ServiceReference1.Service1Client _service;
        #endregion Private

        public Service1()
        {
            this._service = new ServiceReference1.Service1Client();
        }

        public string GetData(int value)
        {
            return this._service.GetData(value);
        }

    }
}

I moved the WCF service and newing up of that client from the console application to make it easier to unit test. We are still not at a point that WCF.Service1 can be unit tested, though the service itself can.
I’ve added a new RussUnitTestSample.WCF.Tests project to my solution, and added the following tests for my Service1.svc class (the implementation of IService1).

As a reminder the IService1.cs was defined as:

// NOTE: You can use the "Rename" command on the "Refactor" menu to change the interface name "IService1" in both code and config file together.
[ServiceContract]
public interface IService1
{

    [OperationContract]
    string GetData(int value);

    [OperationContract]
    CompositeType GetDataUsingDataContract(CompositeType composite);

    // TODO: Add your service operations here
}

I have added the following unit tests based on the implementation in

Service1.svc:

namespace RussUnitTestSample.Wcf.Tests
{

    /// <summary>
    /// Unit tests for service1
    /// </summary>
    [TestClass]
    [ExcludeFromCodeCoverage]
    public class Service1Tests
    {

        /// <summary>
        /// Get data works as expected with a positive number
        /// </summary>
        [TestMethod]
        public void Service1_GetData_PositiveNumber()
        {
            // Arrange
            Wcf.Service1 service = new Wcf.Service1();
            int num = 55;
            var expected = string.Format("You entered: {0}", num);

            // Act
            var result = service.GetData(num);

            // Assert
            Assert.AreEqual(expected, result);
        }

        /// <summary>
        /// Get data works as expected with a negative number
        /// </summary>
        [TestMethod]
        public void Service1_GetData_NegativeNumber()
        {
            // Arrange
            Wcf.Service1 service = new Wcf.Service1();
            int num = -42;
            var expected = string.Format("You entered: {0}", num);

            // Act
            var result = service.GetData(num);

            // Assert
            Assert.AreEqual(expected, result);
        }

        /// <summary>
        /// Get data works as expected with zero
        /// </summary>
        [TestMethod]
        public void Service1_GetData_Zero()
        {
            // Arrange
            Wcf.Service1 service = new Wcf.Service1();
            int num = 0;
            var expected = string.Format("You entered: {0}", num);

            // Act
            var result = service.GetData(num);

            // Assert
            Assert.AreEqual(expected, result);
        }

        /// <summary>
        /// An exception is thrown when the CompositeType is null
        /// </summary>
        [TestMethod]
        [ExpectedException(typeof(ArgumentNullException))]
        public void Service1_GetDataUsingDataContract_ExceptionThrownWhenCompositeTypeNull()
        {
            // Arrange
            Wcf.Service1 service = new Service1();

            // Act
            var result = service.GetDataUsingDataContract(null);
        }

        /// <summary>
        /// When BoolValue is false, do not append "Suffix" to StringValue
        /// </summary>
        [TestMethod]
        public void Service1_GetDataUsingDataContract_CompositTypeBoolValueFalse_DoNotAppendSuffix()
        {
            // Arrange
            Wcf.Service1 service = new Service1();
            string testString = "Test";
            CompositeType ct = new CompositeType()
            {
                BoolValue = false,
                StringValue = testString
            };

            // Act
            var result = service.GetDataUsingDataContract(ct);

            // Assert
            Assert.AreEqual(testString, result.StringValue);
        }

        /// <summary>
        /// When BoolValue is true, append "Suffix" to StringValue
        /// </summary>
        [TestMethod]
        public void Service1_GetDataUsingDataContract_CompositTypeBoolValueTrue_AppendSuffix()
        {
            // Arrange
            Wcf.Service1 service = new Service1();
            string testString = "Test";
            CompositeType ct = new CompositeType()
            {
                BoolValue = true,
                StringValue = testString
            };

            var expected = testString + "Suffix";

            // Act
            var result = service.GetDataUsingDataContract(ct);

            // Assert
            Assert.AreEqual(expected, result.StringValue);
        }

    }
}

Code Coverage:



Taking a look at our code coverage, you can see that currently we have 100% coverage for our RussUnitTestSample.Wcf project, but our coverage of RussUnitTestSample.Business has gone from 100, to 54.21. This is expected of course, as we have added a Wcf Service reference, as well as a wrapper of the WCF client. I think we could technically unit test the Service Reference code, but it is auto generated, so I think I’m going to ignore it for now. Wonder if I can exclude it from Code Coverage.

So now let’s look into how to go about testing our Business.Wcf client wrapper.

WCF.Service1

namespace RussUnitTestSample.Business.WCF
{

    /// <summary>
    /// Communication with the WCF Service1
    /// </summary>
    public class Service1
    {

        #region Private
        private ServiceReference1.Service1Client _service;
        #endregion Private

        public Service1()
        {
            this._service = new ServiceReference1.Service1Client();
        }

        public string GetData(int value)
        {
            return this._service.GetData(value);
        }

    }
}

As this class currently stands, we’re working with a Service1Client and not an interface, so it’s difficult to unit test. Let’s do a little refactoring. Instead of newing up the Service1Client, let’s take in the interface of said client. After updating our class looks like:

namespace RussUnitTestSample.Business.WCF
{

    /// <summary>
    /// Communication with the WCF Service1
    /// </summary>
    public class Service1
    {

        #region Private
        private IService1 _service;
        #endregion Private

        #region ctor

        /// <summary>
        /// Constructor - new up IService1 with client
        /// </summary>
        public Service1()
        {
            this._service = new Service1Client();
        }

        /// <summary>
        /// Constructor - takes in implementation of IService1
        /// </summary>
        /// <param name="service">The IService1 implementation
        public Service1(IService1 service)
        {
            if (service == null)
                throw new ArgumentNullException(nameof(service));

            this._service = service;
        }

        #endregion ctor

        #region Public methods

        /// <summary>
        /// Call service GetData
        /// </summary>
        /// <param name="value">The value to pass to the WCF service
        /// <returns>The returned value from the WCF service call</returns>
        public string GetData(int value)
        {
            return this._service.GetData(value);
        }

        #endregion Public methods

    }
}

Now that we’re taking in an interface of the service, we can write some unit tests:

RussUnitTestSample.Business.Tests.Wcf.Service1Tests.cs

namespace RussUnitTestSample.Business.Tests.WCF
{

    /// <summary>
    /// Unit tests for Service1
    /// </summary>
    [TestClass]
    [ExcludeFromCodeCoverage]
    public class Service1Tests
    {

        #region Private
        private Mock<iservice1> _service;
        #endregion Private

        #region Public methods
        /// <summary>
        /// initialize the mocks
        /// </summary>
        [TestInitialize]
        public void Setup()
        {
            this._service = new Mock<iservice1>();
        }

        /// <summary>
        /// Exception thrown when IService implementation is not provided
        /// </summary>
        [TestMethod]
        [ExpectedException(typeof(ArgumentNullException))]
        public void Service1_NullIService1InConstructor_ThrowsException()
        {
            // Arrange / Act
            Business.WCF.Service1 service = new Business.WCF.Service1(null);
        }

        /// <summary>
        /// Object properly constructed when implementation of IService1 provided
        /// </summary>
        [TestMethod]
        public void Service1_ConstructorWithProvidedIService1_NewsCorrectly()
        {
            // Arrange / Act
            Business.WCF.Service1 service = new Business.WCF.Service1(_service.Object);

            // Assert
            Assert.IsInstanceOfType(service, typeof(Business.WCF.Service1));
        }

        /// <summary>
        /// Ensure that a string is returned from Service1 when calling GetData
        /// </summary>
        [TestMethod]
        public void Service1_GetDataTest()
        {
            // Arrange
            this._service.Setup(s => s.GetData(It.IsAny<int>())).Returns("test");
            Business.WCF.Service1 service = new Business.WCF.Service1(_service.Object);

            // Act
            var result = service.GetData(It.IsAny<int>());

            // Assert
            Assert.IsInstanceOfType(result, typeof(string));
        }

        #endregion Public methods

    }
}

And our new code coverage:



Now we’ve hit everything except the default constructor used for Service1. Guess I’ll have to figure out how to accomplish that later. Also I added a .runsettings file to exclude “Service Reference” folders from code coverage.

Latest code as of post:

https://github.com/Kritner/RussUnitTestSample/tree/b9c2f329adbc700688fb69943cc4b7b28ffd87c4