A/B Tetsing - in 5 steps
Written by - Millan Kaul
So waht is A/B tetsing ?
โA/B testingโ is a shorthand for a simple randomized controlled experiment, in which a number of samples (e.g. A and B) of a single vector-variable are compared ~ from Wiki
Also known as bucket testing, split-run testing, or split testing. It is a user experience research methodology where randomized experiment usually involves two variants (A and B).
GOLA ๐ฏ Determining which of the variants is more effective.
I will explain to you A/B testing process in 5 steps - for continuous testing and improvement.
Follow these 5 steps in sequence :
- ๐๐ฒ๐ณ๐ถ๐ป๐ฒ g๐ผ๐ฎ๐น
- ๐๐ฟ๐ฎ๐ถ๐ป๐๐๐ผ๐ฟ๐บ/g๐ฒ๐ป๐ฒ๐ฟ๐ฎ๐๐ฒ I๐ฑ๐ฒ๐ฎ
- ๐๐บ๐ฝ๐น๐ฒ๐บ๐ฒ๐ป๐ c๐ต๐ฎ๐ป๐ด๐ฒ๐
- ๐๐ฟ๐ผ๐๐ป ๐๐ต๐ฒ n๐ฒ๐ c๐ต๐ฎ๐บ๐ฝ๐ถ๐ผ๐ป
- ๐ง๐ฒ๐๐ it
๐ท ๐ฌ๐ญ ๐๐ฒ๐ณ๐ถ๐ป๐ฒ g๐ผ๐ฎ๐น (๐ฐ๐ผ๐ป๐๐ฒ๐ฟ๐๐ถ๐ผ๐ป) ๐นโฆ ๐ฏ
Before your create a test, your need to know what, exactly, youโre hoping to accomplish.
A gola can be :
โธ Increasing click-through rates
โธ Boosting sales
โธImproving user engagement
A well-defined goal means focus and direction for your experimentation thatโs Aaa-Bee test.
Real life example ๐ฏ : Goal to increase product page conversion rate from 2% to 5% within 3 months.
๐ท ๐ฌ๐ฎ ๐๐ฟ๐ฎ๐ถ๐ป๐๐๐ผ๐ฟ๐บ/g๐ฒ๐ป๐ฒ๐ฟ๐ฎ๐๐ฒ !๐ฑ๐ฒ๐ฎ ๐ค๐ฏ๏ธ
Once youโve determined a goal you want to accomplish, youโll need to generate ideas by use of qualitative and quantitative data to identify potential areas for improvement or innovation.
Real life example ๐ฏ : Analyzing user feedback and heatmaps to identify friction points in the checkout process.
๐ท ๐ฌ๐ฏ ๐๐บ๐ฝ๐น๐ฒ๐บ๐ฒ๐ป๐ c๐ต๐ฎ๐ป๐ด๐ฒ๐ ๐
You will be need to implement changes with highest potential, intuition and best practices. Maybe develop and execute variations of the original element, such as different headlines, layouts, or calls-to-action.
Real life example ๐ฏ : Creating two versions of a websiteโs homepage with different hero images and CTAs to test user engagement.
๐ท ๐ฌ๐ฐ ๐๐ฟ๐ผ๐๐ป ๐๐ต๐ฒ n๐ฒ๐ c๐ต๐ฎ๐บ๐ฝ๐ถ๐ผ๐ป ๐
Once youโve launched your test, you need to let it run for a long enough period of time. Basically compare the performance of the control and variant using statistical analysis to determine the winner. Here you can use tools such as LaunchDarkley, two test environments, free vs preemium user account.
Real life example ๐ฏ : After running an A/B test on a e-commerce website, the variant with a prominent โBuy Nowโ button outperformed the control in terms of both click-through rate and conversion rate. The new design is implemented as the winner based on statistical significance.
๐ท ๐ฌ๐ฑ ๐ง๐ฒ๐๐ it !
๐ง๐ฒ๐๐ u๐ป๐๐ถ๐น s๐๐ฎ๐๐ถ๐๐๐ถ๐ฐ๐ฎ๐น๐น๐ s๐ถ๐ด๐ป๐ถ๐ณ๐ถ๐ฐ๐ฎ๐ป๐
By this point, youโve set a goal, select a page and determine what you want to test. However I would suggest to keep running the test until you achieve statistically significant results to ensure reliable conclusions or reach the goal you would have set above (#1)
Real life exampleโs ๐ฏ :
โธ Examlpe 1 : A websiteโs A/B test showed inconclusive results initially. After increasing the sample size, the variantโs higher click-through rate became statistically significant.
โธ Examlpe 2 : A social media platform with two versions of a user sign-up form. The version with simplified form fields and clearer instructions resulted in a 15% increase in completed sign-ups over a month.
โธ Examlpe 3 : A mobile app with two variations of its onboarding process. The version with a step-by-step tutorial had a 25% higher user retention rate after one week.
โธ Examlpe 4 : An online news website with two variations of its headline font size. The version with larger font size increased the time users spent on the page by 10%.
โธ Examlpe 5 : A website with two versions of its online forms, one with enhanced color contrast and larger fonts. The version with improved accessibility features resulted in a 15% decrease in form abandonment and a higher completion rate among users with visual impairments.
Hope these examples and overall blog would have provided you some insights into A/B testing. You can also tailor your experimentation strategy to your unique goals in your projects.
Want to learn moreโ Follow my hashtag.#QualityWithMillan on or just follow me on linkedIn
