I have to say that of all the tools and techniques at your disposal to help you improve your product; my favorite is by far the concept of A/B testing.
We all want to build the perfect technology product, things that impact people, as well as high quality in all its components, its process, interface, design, etc.
Unfortunately, there is no way to effectively do this without user feedback. Some even advocate that there is no limit to the benefit provided by user feedback, the more user feedback your product receives, the better the product will be.
One of the most popular ways to capture user feedback is through a well-tailored survey. While this is definitely a viable option, maintaining a survey can be tedious, not to mention that some users simply don’t have the time or interest in filling surveys.
Wouldn’t it be better if we could obtain their feedback by analyzing their behavior and interaction with our product? As opposed to bothering the user with surveys they might not want to answer at all.
That is precisely what we can accomplish with A/B testing.
As you probably know, A/B testing is an iterative, user centered and data driven method to obtain user feedback that we can then use for make decisions regarding our products and services. It is completely transparent to the user and they don’t know they are providing this data while they are (aside from mentioning this in your privacy policy and disclaimers).
You are presented one random variation of a human facing component from two (or more) options (Version “A”, the control version vs version “B”, the variation. Multiple variations can be included)
With proper experiment design we can identify the highest performing variation and modify our product accordingly. This process can be performed as many times as needed.
A complete framework for an A/B test follows these steps:
- Define a research question
- Refine the question with user interviews
- Formulate a hypothesis, identify appropriate tools, and define test metrics
- Set up and run experiment
- Collect data and analyze results
- Share results and make decision
Should we put that optional second form in a modal or should we put it right next to the first one? A/B test it. It seems users are rarely clicking at our products, should we show a large image with minimal information or a smaller image with more product information? A/B test it.
If you think about it, A/B testing is pretty much like natural selection where the more effective version of your product survives.
Even if you take a component of your product to a desirable point of effectiveness it’s not a bad idea to try the experiment again a year or two later. Trends in product design are ever changing, so what is very effective to your users today might not be effective at all several years later.
Also, people’s way of thinking and tech savviness vary depending on the audience your product is aimed to so if your audience changes over time you might want to run more A/B tests.
Since A/B testing requires effort and resources that are not infinite, it should be done in the most important parts of the product or the ones that generate more value. But really nothing is stopping you from doing it to any component of your product.