Skip to main content

An Occasional Unit Test Failure Caused by Dates

In honor of the Leap Day of February 29th, let me tell a story about an unusual date problem I encountered while unit-testing.

One project I worked on had made a small effort at writing unit tests, using Java and jUnit. Unfortunately, the team culture was not to run the test suite frequently while developing. Rather, developers were sort of willing to run their own tests, but let the nightly job run the full suite.
When a test would fail, someone would track down and lean on whoever had broken it.

Once in a blue moon, a certain couple of Unit tests would fail. This is the story of why they were breaking, why it was hard to duplicate the issue, and how I fixed it once I found it.

These tests failed one night in the spring, and again one night in the summer.
But next morning, no one could reproduce the failure. So they were chalked up to disturbances in the Force or cosmic rays, one of "those things" that happen "sometimes" and nothing was done.


They failed again in late summer. Following that fail, I took it upon myself to solve the mystery. Let's take a peak at why these tests would fail.

Here is one of the occasionally-failing tests:
@Test
public void testGetAge_OneFutureMonth_ExpectedSingularString()
{
int monthsDiff = 1;
Calendar date = Calendar.getInstance();

 DBRecord rec = makeEmptyDBRecord();
rec.setDateYear(date.get(Calendar.YEAR));
rec.setDateMonth(1 + date.get(Calendar.MONTH) + monthsDiff);
rec.setDateDay(date.get(Calendar.DAY_OF_MONTH));

 assertEquals("" + monthsDiff + " month future", DateUtilities.getAge(rec));
}

Sure enough, as with previous times, it passed when run the next day. So let's code-review it and ask: What exactly is this test doing?
It is testing a getAge() method, which takes a DBRecord data structure object as input and returns a string that describes the difference between today's date and the date in the data structure.
The data structure stores the date with day, month and year separated and stored as integers. The month value is incremented by 1 because this application uses 1-based months and Calendar uses 0-based months.
Of course, this means that invalid dates are possible, e.g. January 99, or a negative year such as -2016.

Since it's a Date-related test, maybe the fact that yesterday was a different date than today is a hint?
The current date when I did this review was September 1st. That meant that the test failed during the nightly run on August 31st.
Thinking it through, it means that the test run on the 1st sets the DBRecord object to October 1st, one month in the future. But the test run the previous night, Aug 31st, set the future date to an invalid September 31st. The DBRecord structure permits that.
But the Calendar class does not.

Here's part of the static method that our test is testing:
public static String getAge(DBRecord record)
{
// skip some lines...
int year = record.getDateYear();
int month = record.getDateMonth();
int day = record.getDateDay();
Calendar today = Calendar.getInstance();
Calendar date = Calendar.getInstance();
if (year != 0)
date.set(Calendar.YEAR, year);
if (month != 0)
date.set(Calendar.MONTH, month - 1);
if (day != 0)
date.set(Calendar.DAY_OF_MONTH, day);
// skip some more lines...

When we get to the getAge() method's lines that set the year, month and day, Calendar normalizes the date from Sept 31st to Oct 1st.
So the call to getMonthsDifference() returns 2 months between August (the "current date" when the failed test ran) and October (the future date).
Since the test is only expecting a difference of 1 month, it fails.

Aha! The mystery of why it fails occasionally is solved!
Now how to fix it so that it runs correctly, regardless of the date the test runs? Lots of possible fixes come to mind.

For one, this would have been discovered much sooner had the team - myself included - followed the best-practice of running the entire test suite before committing our changes. Had we been doing that, this test would have failed on all of our work-stations all day on the 31st and not produced the overnight-fail mystery.

Another very good fix would be to introduce some kind of Date Provider or other class, something that will provide the current date in Production, but that our tests can manipulate to return a test-controlled date, rather than a calendar-controlled date.

I like that solution, but I did not choose that fix back in September. It's been a while but my thought process was probably that the bug was in the test code, not the production code, and that creating and introducing some form of Date Provider would require changes to the production code.

Instead, I tweaked the tests to fix the bug in them. I created a helper method to ensure a valid date in the DBRecord, one that does not allow the day of the month to be greater than 28.
But..

private DBRecord getDateOffsetBy(int monthsDiff, int yearsDiff)
{
Calendar date = Calendar.getInstance();

 if (date.get(Calendar.DAY_OF_MONTH) > 28)
date.set(Calendar.DAY_OF_MONTH, 28);

 date.set(Calendar.MONTH, date.get(Calendar.MONTH) + monthsDiff);
date.set(Calendar.YEAR, date.get(Calendar.YEAR) + yearsDiff);

 DBRecord rec = makeEmptyDBRecord();
rec.setDateYear(date.get(Calendar.YEAR));
rec.setDateMonth(1 + date.get(Calendar.MONTH) + monthsDiff);
rec.setDateDay(date.get(Calendar.DAY_OF_MONTH));

 return rec;
}

I then changed the failing tests to variations of:
@Test
public void testGetAge_OneFutureMonth_ExpectedSingularString()
{
int monthsDiff = 1;
DBRecord rec = getDateOffsetBy(monthsDiff, 0);

 assertEquals("" + monthsDiff + " month future", DateUtilities.getAge(rec));
}

Labels:

Popular posts from this blog

Git Reset in Eclipse

Using Git and the Eclipse IDE, you have a series of commits in your branch history, but need to back up to an earlier version. The Git Reset feature is a powerful tool with just a whiff of danger, and is accessible with just a couple clicks in Eclipse. In Eclipse, switch to the History view. In my example it shows a series of 3 changes, 3 separate committed versions of the Person file. After commit 6d5ef3e, the HEAD (shown), Index, and Working Directory all have the same version, Person 3.0.
Read more

Scala Collections: A Group of groupBy() Examples

Scala provides a rich Collections API. Let's look at the useful groupBy() function. What does groupBy() do? It takes a collection, assesses each item in that collection against a discriminator function, and returns a Map data structure. Each key in the returned map is a distinct result of the discriminator function, and the key's corresponding value is another collection which contains all elements of the original one that evaluate the same way against the discriminator function. So, for example, here is a collection of Strings: val sports = Seq ("baseball", "ice hockey", "football", "basketball", "110m hurdles", "field hockey") Running it through the Scala interpreter produces this output showing our value's definition: sports: Seq[String] = List(baseball, ice hockey, football, basketball, 110m hurdles, field hockey) We can group those sports names by, say, their first letter. To do so, we need a disc
Read more

Java 8: Rewrite For-loops using Stream API

Java 8 Tip: Anytime you write a Java For-loop, ask yourself if you can rewrite it with the Streams API. Now that I have moved to Java 8 in my work and home development, whenever I want to use a For-loop, I write it and then see if I can rewrite it using the Stream API. For example: I have an object called myThing, some Collection-like data structure which contains an arbitrary number of Fields. Something has happened, and I want to set all of the fields to some common state, in my case "Hidden"
Read more

How to do Git Rebase in Eclipse

This is an abbreviated version of a fuller post about Git Rebase in Eclipse. See the longer one here : One side-effect of merging Git branches is that it leaves a Merge commit. This can create a history view something like: The clutter of parallel lines shows the life spans of those local branches, and extra commits (nine in the above screen-shot, marked by the green arrows icon). Check out this extreme-case history:  http://agentdero.cachefly.net/unethicalblogger.com/images/branch_madness.jpeg Merge Commits show all the gory details of how the code base evolved. For some teams, that’s what they want or need, all the time. Others may find it unnecessarily long and cluttered. They prefer the history to tell the bigger story, and not dwell on tiny details like every trivial Merge-commit. Git Rebase offers us 2 benefits over Git Merge: First, Rebase allows us to clean up a set of local commits before pushing them to the shared, central repository. For this
Read more

Code Coverage in C#.NET Unit Tests - Setting up OpenCover

The purpose of this post is to be a brain-dump for how we set up and used OpenCover and ReportGenerator command-line tools for code coverage analysis and reporting in our projects. The documentation made some assumptions that took some digging to fully understand, so to save my (and maybe others') time and effort in the future, here are my notes. Our project, which I will call CEP for short, includes a handful of sub-projects within the same solution. They are a mix of Web APIs, ASP MVC applications and Class libraries. For Unit Tests, we chose to write them using the MSTest framework, along with the Moq mocking framework. As the various sub-projects evolved, we needed to know more about the coverage of our automated tests. What classes, methods and instructions had tests exercising them, and what ones did not? Code Coverage tools are conveniently built-in for Visual Studio 2017 Enterprise Edition, but not for our Professional Edition installations. Much less for any Commun
Read more