Discussion: data driven testing

[xhd2015]

Never depend on the implementation directly. Abstract a layer between the implementation and the intent.

For example, when testing a service with external dependencies, we want to mock all of them.

But we should not directly mock their request or response, instead, we abstract a data structure called TestingContext.

This TestingContext is declared as a universal data fake storage:

type TestingContext struct {
    Users []*User
    Checkouts []*Checkout
    Config *Config
}

Then each dependency read data from this central data storage, and construct corresponding response based on the data.

And to reduce duplication of repeating declaring TestingContext, different tests can share a basic TestingContext, adding their custom overlay to the context.

[xhd2015]

Two important concepts:

• fake data storage
• data overlay

[xhd2015]

Data driven requires the user to have a basic knowledge of the system's models as well as APIs to interact with these models.

[xhd2015]

But data drive testing is already taken, also known as table driven testing. Wiki: https://en.m.wikipedia.org/wiki/Data-driven_testing.

However, given that table driven is widely used and has gained a relative stable shape(a table), then this method can still be referred as Data driven testing.

[xhd2015]

Data driven requires the user to have a basic knowledge of the system's models as well as APIs to interact with these models.

To do data driven testing, following these 4 steps:

• recognize entities in the system,
• define their ids and properties ,
• mock the API behavior based on the mocking data
  • this can be done by comparing the querying key with the data context, and retrieve matching result from the context. panic out if no data found
• construct base data context, and create overlays to repeat the test for different logic paths

(draw a diagram to illustrate the relation of data and APIs, APIs are fall into 2 categories: internal and external like database.)

[xhd2015]

Data driven requires the user to have a basic knowledge of the system's models as well as APIs to interact with these models.

To do data driven testing, following these 4 steps:

• recognize entities in the system,

• define their ids and properties ,

• mock the API behavior based on the mocking data

  • this can be done by comparing the querying key with the data context, and retrieve matching result from the context. panic out if no data found

• construct base data context, and create overlays to repeat the test for different logic paths

(draw a diagram to illustrate the relation of data and APIs, APIs are fall into 2 categories: internal and external like database.)

The reverse process, convert real API response to data context, is also valuable. Compared to that directly replay real traffic, we actually split that into 2 phase:

• extract
• replay with mock

With this, unit test can be more easily constructed and maintained from live traffic.