TWiStErRob

How to use it to help you write clean, focused tests

Practical JFixture

Magic one-liner with powerful behaviour

Tests need data, let’s see how can we create them using JFixture. In this article we’ll go through some practical usage patterns of JFixture, and compare those to some other alternatives.

Introduction

JFixture is a Java library to assist in the writing of Unit Tests, particularly when following Test Driven Development. It generates types based on the concept of ‘constrained non-determinism’, which is an implementation of the Generated Value xUnit test pattern. JFixture README.md

In practice this means, is that JFixture can create any data object with very little developer effort, here’s an example:

private val fixture = JFixture()

@Test fun test() {
    val fixtJourney: Journey = fixture();

    val result = sut.process(fixtJourney);

    assertNotNull(result);
}

Compare this to the classic approach where we would need to create each object individually, filling in the properties with dummy values, and potentially creating constants for these values. No matter how deep or complicated it gets, JFixture usually finds a way to create an instance with all the data filled in.

val journey = Journey(
    "",
    listOf(
        Leg(
            Stop("", ""),
            LocalDateTime.now(),
            TransportMode.TRAIN,
            Stop("", ""),
            LocalDataTime.now()
        ),
        Leg(
            Stop("", ""),
            LocalDateTime.now(),
            TransportMode.TRAIN,
            Stop("", ""),
            LocalDataTime.now()
        )
    )
)

Note: In this article I use JUnit Jupiter and Mockito + Mockito Kotlin as the testing frameworks; and Kotlin as the language, because of its conciseness. While JFixture was written for the JVM with Java in mind it works for Kotlin too due to Kotlin’s awesome interoperability. Everything you see here applies to Java as well (unless it’s about using some Kotlin language feature).

The need for JFixture

In this section, we’ll go through developing and testing a fictitious component. Starting simply, and getting more complicated, we’ll find solutions to problems that come up on the way.

From the ground up

Let’s imagine we’re writing a user interface for displaying some info based on a journey. Journey data is coming from a data source, and we transform it to display it on the UI.

Let’s start with a simple data structure:

data class Journey(
    val id: String,
    val origin: String,
    val destination: String
)

data class Model(/*...*/)

Additionally, let’s say this is the class that orchestrates the loading and displaying:

class JourneyPresenter(
    private val view: JourneyView,
    private val dataSource: DataSource<Journey>,
    private val mapper: (Journey) -> Model
) {
    fun load(journeyId: String) {
        dataSource
            .getById(journeyId)
            .map(mapper)
            .subscribe { model -> view.show(model) }
    }
}

Here are the collaborator interfaces for completeness (using RxJava’s Single):

interface JourneyView {
    fun show(model: Model)
}
interface DataSource<T> {
    fun getById(id: String): Single<T>
}

To test this, we will need to mock the collaborators and stub their inputs and return values. Mind you, we’re testing the data flow here: dataSourcemapperview. At this point, we don’t really care what the data is, as long as it’s of the right type:

private val mockView: JourneyView = mock()
private val mockDataSource: DataSource<Journey> = mock()
private val mockMapper: (Journey) -> Model = mock()

private val sut = JourneyPresenter(mockView, mockDataSource, mockMapper)

@Test fun `loads journey and presents it to view`() {
    whenever(mockDataSource.getById(fixtJourneyId)).thenReturn(fixtJourney)
    whenever(mockMapper.invoke(fixtJourney)).thenReturn(fixtModel)

    sut.load(fixtJourneyId)

    verify(mockView).show(fixtModel)
}

You might notice, some of the declarations are missing, namely fixtJourneyId, fixtJourney and fixtModel. These are left out, as they will be the focus of the next sections.

Classic approach

Usually when we’re faced with a problem of creating an instance of a class, we use instantiation (the good old new in Java). The simplest way to fill in the missing pieces above is this:

private val fixtJourneyId = ""
private val fixtJourney = Journey("", "", "")
private val fixtModel = Model("", 0, 0)

Here, we filled in the values, so that it compiles. At the same time, we don’t really care about what the values are, so we mostly use default values such as use empty strings, false, or 0s.

More complex model

Let’s expand our very simple model to a more realistic example:

data class Journey(
    val id: String,
    val legs: List<Leg>
)

data class Leg(
    val origin: Stop,
    val departure: LocalDateTime,
    val mode: TransportMode,
    val destination: Stop,
    val arrival: LocalDateTime
)

enum class TransportMode { WALK, TRAIN, TAXI }

data class Stop(
    val id: String,
    val name: String
)

Now let’s update our classic approach:

private val fixtJourney = Journey("", emptyList())

OK… I took a shortcut here, but what will happen if we can’t assume to have an empty list there? Say, the production code needs to loop through the list and we need to check the outcome based on that. Let’s go with the assumption that we need at least 2 legs:

private val fixtJourney = Journey(
    "",
    listOf(
        Leg(
            Stop("", ""),
            LocalDateTime.now(),
            TransportMode.WALK,
            Stop("", ""),
            LocalDataTime.now()
        ),
        Leg(
            Stop("", ""),
            LocalDateTime.now(),
            TransportMode.WALK,
            Stop("", ""),
            LocalDataTime.now()
        )
    )
)

What can we observe so far? Instantiate one Leg, instantiate multiple Stops, pick an TransportMode to use; and then instantiate another leg, and so forth. It’s simply too much to write, read, and maintain; and when we scale it, there’s repetition.

Enter factories

What do we do when we see repeated code? We extract methods/classes. So we go on and refactor our code to introduce a factory method for leg creation:

private val fixtJourney = Journey("", listOf(createLeg(), createLeg()))

private fun createLeg() = Leg(
    Stop("", ""),
    LocalDateTime.now(),
    TransportMode.WALK,
    Stop("", ""),
    LocalDataTime.now()
)

Cool, it works! Using code looks clean, but is that scalable? How many factory methods should we create to cover all the model types. Is it easily maintainable? How many variations of the same factory method should we have to satisfy all the test scenarios? Okay, we could parameterise each factory method by passing in the parameters that we are interested in; we could even leverage Kotlin’s default parameters for functions. But…, what happens if it’s a complex model? How many parameters and/or overloads should we have?

Enter builders

So then, let’s move on to the next solution… builders! The perfect tool to solve all our issues in terms of maintainability, is it not? It alleviates the repetition and allows dynamism for creating objects; for example something like this would help with Journeys:

class JourneyBuilder(
    private var id: String = "",
    private val legs: MutableList<Leg> = mutableListOf()
) {
    fun build() = Journey(id, legs)

    fun setId(id: String): JourneyBuilder = this.apply { this.id = id }
    fun addLeg(leg: Leg): JourneyBuilder = this.apply { legs.add(leg) }
}

class LegBuilder(
    private var origin: Stop = Stop("", ""),
    private var departure: LocalDateTime = LocalDateTime.now(),
    private var mode: TransportMode = TransportMode.WALK,
    private var destination: Stop = Stop("", ""),
    private var arrival: LocalDateTime = LocalDateTime.now()
) {
    fun build() = Leg(origin, departure, mode, destination, arrival)

    fun setOrigin(origin: Stop): LegBuilder = this.apply { this.origin = origin }
    fun setMode(mode: TransportMode): LegBuilder = this.apply { this.mode = mode }
    // ...
}

Phew, that was a lot of code to type — and I didn’t even write out all the properties (e.g. destination), and sub-builders (e.g. Stop), nor did I use Kotlin DSL and lambdas.

Anyway, let’s use this builder to replace the factory:

private val fixtJourney = JourneyBuilder()
    .addLeg(LegBuilder().build())
    .addLeg(LegBuilder().build())
    .build()

Nice! Easy to read, we can create our own domain specific language. But I think it’s easy to see how the builder approach might get complicated really fast as the system grows and more data types, properties are added and combined together in various ways.

JFixture to the rescue

Remember JourneyBuilder we created just now? Turns out a very similar effect can be achieved by using the JFixture library:

private val fixtJourney: Journey = fixture.create(Journey::class.java)

… but what does this do? and what are the benefits? That’s what we’ll explore next.

How does JFixture create an object?

When we call fixture.create(Class), JFixture tries numerous ways to create our objects:

  • built-in types ( see Default Supported Types and DefaultEngineParts )
  • customise() (if any)
    • lazyInstance
    • sameInstance
    • useSubType
  • public constructor with the least number of arguments
  • public factory method with the least number of arguments
  • package visible constructor with the least number of arguments

Pro tip: public constructor with the least number of arguments can be changed to most when a class has overloaded constructors:

fixture.customise(GreedyConstructorCustomisation(SomeType::class.java))

When it succeeds, after creation it’ll also use:

  • customise()
    • propertyOf
    • intercept
  • setters for mutable properties
  • write visible mutable fields

Notice that while this may seems like a lot to digest at first, we’re usually only using 1 or 2 of these.

Benefits: No builder required

JFixture is a builder by itself, except it adapts to the shape of your code automatically.

less code to write

We can save a lot of code that would just create our objects by using the library.

less code to maintain

Every time a new property is added all the usages of the class have to be revised and the constructor calls/builders adjusted. Most of the time these are unrelated places and the new field doesn’t affect behaviour of the tests, yet we still have to modify them. With builders the situation is much better than the classic or factory approach, but maintenance is still needed.

customisation of objects is still possible

The flexibility of the builder is that we can pick which parts to be set and how. For example a TRAIN-only journey with multiple legs; this is still easily possible.

Benefits: Data is randomised

Don’t worry, it’s the good type of random.

no conflicts from random data

Consider this simple thing:

 val journeys = setOf(
     JourneyBuilder().addLeg(LegBuilder().build()).build(),
     JourneyBuilder().addLeg(LegBuilder().build()).build()
 )
 assertThat(journeys, hasSize(2))

With the hand-built builders we would have to write randomising code ourselves to get this to pass. What’s even worse: sometimes it would pass, sometimes it wouldn’t; depending on what time it is now and how fast our computer is.

more confidence in verification

Let’s imagine we have a bug, and the production code is using the wrong leg to calculate the origin station’s name. With the current builder implementation each leg’s station names are all the same. This means that we could easily end up with a false positive verification. JFixture puts different data everywhere, so it would catch the problem: With the current builder implementation

 // same as   fixtJourney.legs[i].origin.name
 assertEquals(fixtJourney.legs[0].origin.name, model.origin)

A note on Kotlin

Writing fixture.create(Journey::class.java) over and over again gets old really fast. In Kotlin we can do better:

inline fun <reified T> JFixture.build(): T = create(T::class.java)
inline operator fun <reified T> JFixture.invoke(): T = create(T::class.java)

The first one gives us val fixtString: String = fixture.build() with inferred type arguments. The second one enables val fixtString: String = fixture() similar to mock() (given val fixture: JFixture in scope of course). It’s up to you which way you go, based on how much Kotlin magic is acceptable. I recommend writing extension methods for most of the customise() methods using KClass, or inline + reified so they feel more idiomatic.

I’ll use operator invoke in the rest of the article.

Data-oriented testing

Most of the examples in this section demonstrate features that are also listed on the JFixture cheat sheet.

Until now the focus was on the presenter, and data flow; let’s move on to the JourneyMapper’s tests to demonstrate how JFixture can help.

Controlling collections

Let’s say the model has a field which represents how many changes passengers have to make on a journey:

data class Model(
    // ...,
    val changeCount: Int
)

this is of course calculated as follows:

override fun invoke(journey: Journey) = Model(
    // ...,
    changeCount = journey.legs.size - 1
)

If we were to use the classic approach, tests for this would get unwieldy. Builders are a bit better, but testing for multiple counts in a parameterised test still requires some form of looping, or a custom factory just to create a certain amount of legs. With JFixture this is supported out of the box for any type:

@CsvSource("1, 0", "2, 1", "3, 2", "4, 3")
@ParameterizedTest
fun `changeCount is mapped correctly`(legCount: Int, expectedChanges: Int) {
    fixture.customise().repeatCount(legCount)
    val fixtJourney: Journey = fixture()

    val result = sut.invoke(fixtJourney)

    assertThat(result.changeCount, equalTo(expectedChanges))
}

repeatCount is a built-in method that changes how many elements each fixtured collection will contain; the default is 3.

Warning: This actually does not scale well, but good enough for simple tests.
More on this in “Fine-grained repetition”.

Property customisation

Let’s look at how we can calculate if the journey is train-only:

override fun invoke(journey: Journey) = Model(
    // ...,
    trainOnly = journey.legs.all { it.mode == TRAIN }
)

The tests for this involve setting all the leg modes to TRAIN. With the builders approach we could use LegBuilder.setMode, with JFixture we just say: all transport modes are TRAIN:

@Test fun `trainOnly is true for TRAIN-only journey`() {
    fixture.customise().sameInstance(TransportMode::class.java, TRAIN)
    val fixtJourney: Journey = fixture()

    val result = sut.invoke(fixtJourney)

    assertThat(result.trainOnly, equalTo(true))
}

sameInstance is a built-in method that can be used to make sure a type is always resolved to the same instance.

We need to test another scenario though: when there are no TRAIN legs. In this case each leg should have a mode of anything but TRAIN. Here’s a way to do this:

@Test fun `trainOnly is false for a TRAIN-less journey`() {
    fixture.customise().lazyInstance(TransportMode::class.java) {
        fixture.create().fromList(*Enum.valuesExcluding(TRAIN))
    }
    val fixtJourney: Journey = fixture()

    val result = sut.invoke(fixtJourney)

    assertThat(result.trainOnly, equalTo(false))
}

lazyInstance will be called each time a Leg is being created to fill its mode.
fromList will pick an item from the array.

Enum.valuesExcluding is not in a library, it’s one of the few utilities we use to make parameterised tests and fixturing nicer:

inline fun <reified E : Enum<E>>
Enum.Companion.valuesExcluding(vararg excluded: E): Array<E> =
    (enumValues<E>().toList() - excluded).toTypedArray()

You may notice that this lazyInstance + fromList + valuesExcluding combination has potential for re-use, and you’re right: it is possible to extract customisation logic via SpecimenSupplier and/or Customisation interfaces.

Mutating the immutable

I warn you, things are going to get controversial. You may have noticed that the only var I used is in the builder: we use a lot of immutable data types. These data types still need to be created, and we’re using JFixture as our builders. This is where this very useful small utility comes into play:

fun Any.setField(name: String, value: Any?) {
    this::class.java.getDeclaredField(name).apply {
        isAccessible = true
        set(this@setField, value)
    }
}

An example usage would look like this:

@Test fun `duration returns time for whole journey`() {
    val expectedMinutes: Int = fixture()
    val fixtJourney: Journey = fixture()
    fixtJourney.legs.last().setField("arrival", fixtJourney.legs.first().departure.plusMinutes(expectedMinutes))

    val result = sut.invoke(fixtJourney)

    assertThat(result.length, equalTo(Duration.of(expectedMinutes, ChronoUnit.MINUTES)))
}

You can see that it gives us the benefit of naturally mutating objects that are otherwise immutable. Additionally it really focuses on the bits that we care about and leaves everything else up to JFixture. The reflection may look counter-productive, but in the rare instance we rename properties or change types of properties the tests will fail at run-time; which is slower to detect than a compile error, but we’re still protected from false positives.

Sometimes each object has to contain self-consistent data, for example: each leg has to be a non-zero length travel. In this case intercept comes in handy combined with setField:

fixture.customise().intercept(Leg::class.java) {
    it.setField("arrival", it.departure.plusMinutes(fixture<Long>()))
}

intercept is a built-in method that lets you mutate the created object further after creation. It will be called once for each instance.

JFixture has a propertyOf customisation which we could use instead of setField, but it’s a no-go when using immutable classes. It will only work on vars, not vals.

Fine-grained property customisation

Here’s the thing about lazyInstance: it affects every single object created from the given fixture factory. This may be unwanted, and needs to be resolved. Let’s consider what happens if there are Passengers in the system:

data class Passenger(
    val preferredMode: TransportMode
)

In this case, creating a Journey and a Passenger in the same test would mean we would be always creating them with the same TransportMode when using sameInstance. To make this more explicit we can use:

fixture.customise().intercept(Leg::class.java) {
    it.setField("mode", TransportMode.TRAIN)
}
fixture.customise().intercept(Passenger::class.java) {
    it.setField("preferredMode", TransportMode.WALK)
}

Fine-grained repetition

Similarly repeatCount is a tricky one. It changes each collection’s size at once, which is probably not what we want. For example we may need to test 2 passengers over 3 legs:

data class Journey(
    // ...,
    val legs: List<Leg>,
    val passengers: List<Passenger>
)
val fixtJourney: Journey = fixture()
fixtJourney.setField("passengers", fixture.createList<Passenger>(size = 2))

createList is not a library function, it is one of the extensions we added to help us create lists in a very simple way. JFixture — being designed for Java — has some quirks in Kotlin, so hiding them in utility functions helps to write nicer code:

@Suppress("UNCHECKED_CAST") // can't have List<T>::class literal, so need to cast
inline fun <reified T : Any> JFixture.createList(size: Int = 3): List<T> =
    this.collections().createCollection(List::class.java as Class<List<T>>, T::class.java, size)

Best practices

Now that we have the tools, here are some tips to prevent hurting ourselves.

Shared JFixture

It might be tempting to create globals, but try not to. It’s not worth the hours of debugging when you accidentally end up with the wrong fixtured data because another test customised it in a counter-productive way.

val fixture = JFixture()
// or
object fixture : JFixture() { ... }

@TestMethodOrder(Random::class)
class SharedStateTest {
    @Test fun customiser() {
        fixture.customise().sameInstance(String::class.java, "")

        assertThat(fixture<String>(), emptyString())
    }

    @Test fun annoyedObserver() {
        assertThat(fixture<String>(), not(emptyString()))
    }
}

In the above example the fixture global property outlives the test’s lifecycle and leaks the "" customisation to the other test, which expects JFixture’s default behaviour. This expectation is usually implicit when writing fixtured tests.

As seen above, sharing state between a single test class’ methods is risky. We always use non-static JFixture instance set up in @BeforeEach to prevent cross-test customisations:

private lateinit var fixture: JFixture
@BeforeEach fun setUp() {
    fixture = JFixture()
}

If we share fixture like this and put customisations into the setUp method, we could end up with not fully self-contained tests. Your mileage may vary on how much shared setup is acceptable in your project. This is a similar decision to how/where mock() variables are set up and stubbed.

Note that having private val fixture = JFixture() inside the class may not be enough, because the test runner may create an instance per test class, not per test method: Google @TestInstance(PER_CLASS) for more information on when this could break down.

Sharing data setup

That being said, sharing setup is a good idea. For example, if all the tests create objects which have CharSequence fields, the following setup makes sure that is possible in each test:

@BeforeEach fun setUp() {
    fixture = JFixture()
    fixture.apply {
        customise().useSubType(CharSequence::class.java, String::class.java)
    }
}

Notice that it’s only the code being shared, not the run-time objects.

Centralized Fixture

Taking it a step further, it’s very likely we’ll end up having very common customisations. CharSequence → String is a good example; it could occur often in Android. It is recommended to create a custom type that builds on JFixture:

class MyFixture : JFixture() {
    init {
        customise().useSubType(CharSequence::class.java, String::class.java)
    }
}

It helps:

  • sharing customisations
  • sharing fixture related utility methods
    (not so beneficial when we can use Kotlin extension methods, but it is in Java)
  • creating a better API fitting your style
    (if you go with encapsulation over inheritance)

That said, we are not using this pattern, because we prefer easily discoverable and focused test setups.

Stubbing fixture behaviour

Let’s see what happens if our data classes have calculated properties, for example:

data class Journey(
    // ...
    val legs: List<Leg>,
) {
    val changeCount get() = legs.size - 1
}

This is very simple, but it can easily get complicated, in which case setting up the fixture to be exactly correct for the calculated property could be tricky. At this point you might be thinking: let’s mock the logic to isolate it, but it’s on a JFixture generated class. Mockito spies come to the rescue:

val spyJourney = spy(fixture<Journey>())
doReturn(6).whenever(spyJourney).changeCount

Beware: when using spys, we must always use the doReturn(result).when(spy).method(args) pattern, otherwise the real method gets called, which can cause problems.

Tip: even when using annotated setup, it’s possible to combine the two frameworks:

@Mock lateinit var mockView: JourneyView
@Mock lateinit var mockDataSource: DataSource<Journey>
@Mock lateinit var mockMapper: (Journey) -> Model

@Fixture lateinit var fixtJourneyId: String
@Spy @Fixture lateinit var spyJourney: Journey
@Fixture lateinit var fixtModel: Model

private lateinit var fixture: JFixture

private lateinit var sut: JourneyPresenter

@BeforeEach fun setUp() {
    fixture = JFixture()
    FixtureAnnotations.initFixtures(this, fixture)
    MockitoAnnotations.initMocks(this)

    sut = JourneyPresenter(mockView, mockDataSource, mockMapper)
}

The important thing here is to have initMocks after initFixtures.

Pitfalls

No tool is without its drawbacks — here are some interesting problems we have encountered while using JFixture.

Kotlin generics interoperability

Kotlin makes every usage of a generic out T (e.g. kotlin.collections.List<T>) automatically appear in the class files as ? extends T. JFixture doesn’t like this, because there’s no right choice for it to pick when creating test data. The workaround we use is adding @JvmSuppressWildcards on the field declarations like so:

@Fixture lateinit var fixtJourneys: List<@JvmSuppressWildcards Journey>

Customise the right JFixture

When using the annotated setup, it’s quite easy to end up using the wrong JFixture instance and wonder why none of our customisations work:

@Fixture lateinit var fixtModel: Model

private lateinit var fixture: JFixture

@BeforeEach fun setUp() {
    fixture = JFixture()
    fixture.customise()....
    FixtureAnnotations.initFixtures(this)
}

The problem here is using the wrong initFixtures overload. When we create and customise a JFixture instance, we need to call initFixtures(this, fixture); otherwise initFixtures will create a new non-customised JFixture instance.

Beware of overloads

fixture.customise().useSubType(CharSequence::class.java, String::class.java)
fixture.customise().sameInstance(CharSequence::class.java, String::class.java)

Let’s say we wanted to use useSubType, but accidentally used sameInstance. This makes a subtle difference when it’s one of the hundreds of lines of test code. Sadly, this compiles, and when JFixture tries to create an instance of CharSequence, it throws a ClassCastException: cannot cast Class<String> to CharSequence. This is because sameInstance has an overload that matches the arguments:

public interface FluentCustomisation {
    <T> FluentCustomisation sameInstance(Class<T> clazz, T instance);
    <T> FluentCustomisation sameInstance(Type type, T instance);
}

Class<*> implements Type so when the instance argument is not a subclass of T in clazz, the overload resolution finds the Type overload and T becomes Class<String>.

Flaky tests

One of the main benefits of JFixture is reproducibility through constrained non-determinism. In practice this means that if tests fail on one computer (e.g. CI) they’ll fail the same way on another (e.g. dev machine). Mind you, this failure may not be immediate, but it’s eventually reproducible. Don’t fret though, it’s not as bad as the infinite monkey theorem states. In practice, we observed that if we see a flaky test on the CI, we can simply run the test a 100 times and we’ll see some failures:

Run/Debug Configurations > JUnit > Configuration > Repeat > N Times

Here’s an example flaky test that will pass 90% of the time:

enum class Type { Other, Type1, Type2, Type3, Type4, Type5, Type6, Type7, Type8, Type9 }

fun Type.isValidType() = this != Other
@Test fun flaky90() {
    val fixture = JFixture()
    val fixtType: Type = fixture()

    val result = fixtType.isValidType()

    assertTrue(result)
}

This could mean that if you run the test a few times, and the code reviewer runs it, and the CI runs it; it’s still going to pass all those times, but then when you merge to master, it fails. The above example is over-simplified, but even the most complex flaky tests we had of this type weren’t that hard to debug and fix. Notice that the reason for failure is assuming what the fixture will generate. If this isValidType is used in an if, that behaviour will trigger 90% of the time. To fix this test, we need to use valuesExcluding technique described in the Property Customisation section, and write a separate test for Other specifically. Bear in mind that in some cases we need to consider that the isValidType’s input should be parameterised to test all possibilities (not just a random one), but that’s a topic for another time.

Conclusion

Introducing JFixture to an existing project is a big leap, but we think it’s worth the effort and the learning curve is not that bad. There will be times when you scratch your heads, but you’ll learn something new each time, until you develop a solid stable usage pattern. I personally found it very useful to debug into JFixture when something is wrong. Give it a whirl and let me know how you find the framework.

References

Go to top