Given my last few posts, you'd be forgiven for thinking this blog has an underlying testing theme.
As unintended as it may be, since I've written yet another lightweight unit testing framework that works significantly different to my previous
specification structured approach, I think it qualifies for another post.
given-fixture is intended to support writing really concise but readable tests built from many small,
fluent extension methods that configure a fixture for taking care of all that nasty boiler plate.
Me and the rest of my team have written hundreds of unit tests with this. We seem to be having great success and continue to refine it regularly.
So please take a look, it's available on GitHub and NuGet.
I've been writing specification structured unit tests in C# for a few years now and other than a few occasions, mainly when testing over ambitious services who's behaviour tree had grown totally out of control, I've had a great experience with them. I like how I can read service behaviours in test logs clearly and identify failure causes within a few lines of code. However, my current team have not been quite so enthused with the extra investment in complexity required to write them. I've started noticing developers ignoring my examples entirely and falling back into free-form tests, sometimes with framework abuse, other times dropping the framework entirely. I can't really blame them as, I've come to realise that specification structured testing is over complicated.
It has taken me writing three testing themed posts to do so but I now believe that I write tests in more of an opinionated style than other developers. I think that this has been caused by some unforgettable past experiences of working with tens of thousands of lines of very badly written free-form tests that reliably violated fundamental programming concepts such as DRY. In my opinion, when fixing unit tests takes orders of magnitude longer than writing application code then your technical debt has reached a critical status. I remember getting away with deleting and disabling entire test classes in order to make my life easier. No-one reads the tests in pull requests right?
From this, I think I value test structure, readability and maintainability to a far higher degree than some of my peers, who seem to often consider testing as an afterthought. This has led me to believe that with specification structured testing, I may have engineered too much complexity for it to succeed as a framework. It needed more of a balance between capability, complexity and... as ridiculous as it sounds... beauty. Whilst specifications look great in logs, in code, specification inheritance can get so out of control that they become extremely difficult to read. I think developers prefer to write code that looks nice, feels cool or seems like a productive use of their time.
I recently introduced my team to a new fluent fixture based integration testing framework to very positive feedback. Admittingly, we were using Postman based tests before so anything would seem like a revolution, but in comparison to attempts coming from other teams, we were definitely on to a good thing. A fixture based approach was a perfect fit for integration testing as tests written for a particular API or entity type, will follow a very similar pattern.
Configure the test server
Configure the database, including adding any entities required for the test
Make an HTTP request
Try to deserialize the HTTP response as JSON
Assert against the response
Assert against changes in the database
The fixture reduces the boiler plate involved with bootstrapping and calling the test server, whilst the fluent extension based approach allows easy integration with libraries specific to each database technology used by each of our microservices. Most of all, it feels productive to use. With tiny but powerful extension methods like HavingDatabaseWithEntity, WhenMakingRestRequest and ShouldReturnJson, tests are readable and verbose without violating DRY.
Fluent fixtures work great for integration tests but how about unit tests?
The structure of unit tests is not quite as standard as integration tests but there are still patterns that we can observe for writing a fluent process.
Generate test data
Configure mocks with the test data
Optionally construct a subject from the mocks
Call a method on the subject or a static method
Assert features of the result returned or exception thrown
Assert expected mock actions
Using this flow as a base, I have developed given-fixture, a really simple library that provides a fixture to configure and assumes a fluent style of writing tests.
To develop a successful fluent style, we need a convention for using fluent verbs. For given-fixture I went with:
Having for arranging e.g. Having some data, Having some configured mock.
When for acting e.g. When we use a subject to call a method, When we call a static method.
Should for asserting e.g. Should return a specific value, Should throw a specific exception.
The name of each fluent extension should begin with one of these verbs. Since given-fixture is configured with a fixture, there is no strict ordering of fluent extension methods, but to keep a clean conversation flow we should always conform to the order "Having", "When", "Should" i.e. arrange, act, assert.
Some test data must conform to a particular structure in order for our tests to be valid e.g. the test subject might expect URL's or email addresses to be parsed or validated. However, the vast majority of all test data is characterized by us not really caring about the value itself, just that the test subject used it in the expected way to produce the expected result.
For structured data, the excellent Bogus library is perfect. The test fixture in given-fixture provides a Faker instance so that we can wrap it fluently like so.
Given.Fixture
.HavingFake(f => f.Internet.Url(), out var url)
.HavingFake(f => f.Internet.Email(), out var email)
.HavingFake(f => f.Random.AlphaNumeric(16), out var knownLength)
.HavingFake(f => f.Random.Int(1, 10), out var knownRange);
Given.Fixture
.HavingFake(f => f.Internet.Url(), out var url)
.HavingFake(f => f.Internet.Email(), out var email)
.HavingFake(f => f.Random.AlphaNumeric(16), out var knownLength)
.HavingFake(f => f.Random.Int(1, 10), out var knownRange);
The Bogus library really fits in with our fluent style.
"Given fixture", "having fake internet URL" and "having fake random integer"
If a test is often generating data of a particular structure, we may want to wrap it in a new extension method, for example.
public static ITestFixture HavingUrlWithPath(this ITestFixture fixture,
out string url) =>
fixture.HavingFake(f => f.Internet.UrlWithPath(), out url);
public static ITestFixture HavingUrlWithPath(this ITestFixture fixture,
out string url) =>
fixture.HavingFake(f => f.Internet.UrlWithPath(), out url);
For unstructured data we can use AutoFixture. The test fixture in given-fixture provides an IFixture instance so that we can wrap it fluently like so.
Given.Fixture
.HavingModel(out SomeModel model)
.HavingModels(out ICollection<SomeModel> models)
.HavingModel(out Guid guid);
Given.Fixture
.HavingModel(out SomeModel model)
.HavingModels(out ICollection<SomeModel> models)
.HavingModel(out Guid guid);
These methods support the composer functionality of AutoFixture.
Given.Fixture
.HavingModel(out SomeModel model1, c => c.Without(c => c.SomeProperty))
.HavingModel(out SomeModel model2, c => c.With(c => c.SomeProperty, "some value"));
Given.Fixture
.HavingModel(out SomeModel model1, c => c.Without(c => c.SomeProperty))
.HavingModel(out SomeModel model2, c => c.With(c => c.SomeProperty, "some value"));
With extensions provided in the library, you can also combine Bogus with AutoFixture to generate structured data in POCO's.
Given.Fixture
.HavingModel(out SomeModel model,
c => c.With(c => c.Email, f => f.Internet.Email())
.With(x => x.Price, f => f.Random.Decimal(10, 20)));
Given.Fixture
.HavingModel(out SomeModel model,
c => c.With(c => c.Email, f => f.Internet.Email())
.With(x => x.Price, f => f.Random.Decimal(10, 20)));
Again we may want to generate the same sort of data in multiple tests, in which case we would extract a new reusable extension method.
public static ITestFixture HavingSomeModelWithPrice(this ITestFixture fixture,
out SomeModel model,
decimal price) =>
fixture.HavingModel(out model, c => c.With(x => x.Price, price));
public static ITestFixture HavingSomeModelWithPrice(this ITestFixture fixture,
out SomeModel model,
decimal price) =>
fixture.HavingModel(out model, c => c.With(x => x.Price, price));
For mocking dependencies, I have used Moq with the repository from Autofac.Extras.Moq. The test fixture in given-fixture provides an AutoMock instance so that we can wrap it fluently like so.
I noticed whilst writing tests using these methods that most of the time I was mocking methods to return models or throw exceptions that I had just generated with AutoFixture. So I have included a set of extension methods that cover most of these common cases. Using these extension methods, the above example becomes more concise.
Given.Fixture
.HavingMocked<ISomeService, SomeModel>(x => x.SomeMethodAsync(), out model)
Each of these methods calls the Verifiable(string because) method on the mock object. This is a good practice as it asserts all mock actions were actually completed i.e. the test is actually testing the subject behaviour that we expect. After result and exception assertions have completed, the test fixture automatically calls VerifyAll() on each configured mock.
Once we have all test data ready and all dependent calls mocked out, we need a subject to test. With given-fixture, you can use Autofac.Extras.Moq to automatically construct a subject using it's mock repository and the Autofac IoC container. If you are testing instance methods, this is the preferred approach as changes to the constructor signature will not automatically break all tests. The fixture also allows testing without a subject in order to test static methods.
For asserting features of the result, the fixture provides ShouldReturn and for the thrown exception, ShouldThrow. The library also provides a selection of common assertion extension methods using the excellent FluentAssertions library.
Given.Fixture
.ShouldReturnEquivalent(new { Name = "some name", Price = 10.0m });
Given.Fixture
.ShouldReturnEquivalent(new { Name = "some name", Price = 10.0m });
These also include common exception assertions for example.
Up to this point, we have only been configuring the fixture. We must call either Run or RunAsync to run the fixture depending on whether the act step is asynchronous.
First of all, each test will share common test fixture configuration steps, such as configuring the subject and method to call. To avoid repeating ourselves in code we should wrap these in extension methods.
internal static class BreakfastTestExtensions
{
/// <summary>
/// Configures the mock breakfast item repository to return a relevant breakfast item
/// when called with the specified breakfast item type.
/// </summary>
public static ITestFixture HavingBreakfastItem(this ITestFixture fixture, BreakfastItemType type, out BreakfastItem item) =>
public Task When_attempting_to_get_breakfast_with_no_items() =>
Given.Fixture
.WhenGettingBreakfast()
.ShouldThrowArgumentException("request")
.RunAsync();
Next we should test the happy path.
[Fact]
public Task When_getting_bacon_egg_and_sausage() =>
Given.Fixture
.HavingBreakfastItem(BreakfastItemType.Bacon, out var bacon)
.HavingBreakfastItem(BreakfastItemType.Egg, out var egg)
.HavingBreakfastItem(BreakfastItemType.Sausage, out var sausage)
.WhenGettingBreakfast(BreakfastItemType.Bacon,
BreakfastItemType.Egg,
BreakfastItemType.Sausage)
.ShouldReturnBreakfastWithCorrectNameAndPrice("Bacon, Egg and Sausage",
bacon, egg, sausage)
.RunAsync();
[Fact]
public Task When_getting_bacon_egg_and_sausage() =>
Given.Fixture
.HavingBreakfastItem(BreakfastItemType.Bacon, out var bacon)
.HavingBreakfastItem(BreakfastItemType.Egg, out var egg)
.HavingBreakfastItem(BreakfastItemType.Sausage, out var sausage)
.WhenGettingBreakfast(BreakfastItemType.Bacon,
BreakfastItemType.Egg,
BreakfastItemType.Sausage)
.ShouldReturnBreakfastWithCorrectNameAndPrice("Bacon, Egg and Sausage",
bacon, egg, sausage)
.RunAsync();
Finally we should write tests for behaviours that are not represented by the happy path i.e. code in the service that has not been called by any of the above tests. Firstly we need to test the failure when one of the breakfast items cannot be retrieved from the repository.
[Fact]
public Task When_attempting_to_get_breakfast_with_missing_item() =>
Next we have the case where all breakfast items are requested, then we should get a special case of a "Full English Breakfast".
[Fact]
public Task When_getting_full_english_breakfast() =>
Given.Fixture
.HavingBreakfastItem(BreakfastItemType.Bacon, out var bacon)
.HavingBreakfastItem(BreakfastItemType.Egg, out var egg)
.HavingBreakfastItem(BreakfastItemType.Sausage, out var sausage)
.HavingBreakfastItem(BreakfastItemType.Toast, out var toast)
.WhenGettingBreakfast(BreakfastItemType.Bacon,
BreakfastItemType.Egg,
BreakfastItemType.Sausage,
BreakfastItemType.Toast)
.ShouldReturnBreakfastWithCorrectNameAndPrice("Full English Breakfast",
bacon, egg, sausage, toast)
.RunAsync();
[Fact]
public Task When_getting_full_english_breakfast() =>
Given.Fixture
.HavingBreakfastItem(BreakfastItemType.Bacon, out var bacon)
.HavingBreakfastItem(BreakfastItemType.Egg, out var egg)
.HavingBreakfastItem(BreakfastItemType.Sausage, out var sausage)
.HavingBreakfastItem(BreakfastItemType.Toast, out var toast)
.WhenGettingBreakfast(BreakfastItemType.Bacon,
BreakfastItemType.Egg,
BreakfastItemType.Sausage,
BreakfastItemType.Toast)
.ShouldReturnBreakfastWithCorrectNameAndPrice("Full English Breakfast",
bacon, egg, sausage, toast)
.RunAsync();
Next we have the case where not all Full English Breakfast items are requested but the selection includes toast, then we should get a special case of "{items} on Toast".
[Fact]
public Task When_getting_bacon_and_egg_on_toast() =>
Given.Fixture
.HavingBreakfastItem(BreakfastItemType.Bacon, out var bacon)
.HavingBreakfastItem(BreakfastItemType.Egg, out var egg)
.HavingBreakfastItem(BreakfastItemType.Toast, out var toast)
.WhenGettingBreakfast(BreakfastItemType.Bacon,
BreakfastItemType.Egg,
BreakfastItemType.Toast)
.ShouldReturnBreakfastWithCorrectNameAndPrice("Bacon and Egg on Toast",
bacon, egg, toast)
.RunAsync();
[Fact]
public Task When_getting_bacon_and_egg_on_toast() =>
Given.Fixture
.HavingBreakfastItem(BreakfastItemType.Bacon, out var bacon)
.HavingBreakfastItem(BreakfastItemType.Egg, out var egg)
.HavingBreakfastItem(BreakfastItemType.Toast, out var toast)
.WhenGettingBreakfast(BreakfastItemType.Bacon,
BreakfastItemType.Egg,
BreakfastItemType.Toast)
.ShouldReturnBreakfastWithCorrectNameAndPrice("Bacon and Egg on Toast",
bacon, egg, toast)
.RunAsync();
Finally, we need to make sure that the service is reliably deduplicating breakfast items. TODO: support multiple items e.g. 2x Bacon.
[Fact]
public Task When_getting_duplicate_bacon() =>
Given.Fixture
.HavingBreakfastItem(BreakfastItemType.Bacon, out var bacon)
So what do you think? Your testing opinions may not align perfectly with mine but surely you consider the idea of using a fixture to fluently test behaviours like this to be damn cool. I think a lot of developers already think in this fluent style, I've even seen methods named with fluent verbs being used half baked in free-form unit tests, so maybe providing a common structure such as given-fixture will reliably help you and your team to write clean, reusable and highly maintainable tests.