A/B Testing

[satishviswanathan]

Just checking if there is any plans to add A/B testing capability.

[GeorgeMac]

Hey @satishviswanathan 👋

Technically speaking, Flipt's multi-variant capabilities can be leveraged to build A/B style testing capabilities.

You can setup rules and distributions, which choose a particular variant to be returned:
https://www.flipt.io/docs/concepts#rules

Distributions allow you to describe how variants are (psuedo)-randomly distributed within a segment.
For example, you can have 70% receive variant A and 30% variant B.

What Flipt doesn't support is tracking of conversion metrics. I don't suspect we will put metrics like this in Flipts storage. This kind of data is more appropriate for a timeseries database or an OLAP database.

Maybe one day we can add a kind of conversion endpoint, so that it is all one client.
Though we would need to support some kind of analytics backend and create an experience around that.
Lots to think about there.

In the meantime, our clients return the matched variant key. If you have a system for tracking timeseries data or an OLAP DB (even Google Analytics) then this variant key can be added to a conversion event/metric being tracked in those systems.
Then Flipt can be used to configure and control the distributions, but measurement data can live in your existing analytics tool.

Would you find a guide to setting something like this up with Flipt useful?