Maximizing technology investment in the real world requires continuous optimization.
A/B testing is often used to determine if technology changes are beneficial. The concept is simple: try a new approach with a treatment group and compare the outcome with a control group. If the treatment group outcome is better, then apply the approach to all groups. If not, then try something else.
While this appears logical and scientific under “lab” conditions, the real world of digital is more complex. A/B testing in the wild frequently fails to deliver reliable results.
Multi-Dimensional Problem
The digital world is rarely simple enough for only one variable or dimension to be changed whilst all others stay the same.
Changing just one dimension whilst forcing the others to stay the same is simply not practical in the real world of business. Time of day, age of content, performance of the internet, device, geography, dependent parties making their own changes (e.g. search engines or demand partners), and the duration of an experiment all impact results.
Publishing and Advertising
For publishers the nature of the content and the current demand for advertising are significant factors. Demand side partners will be changing their approaches in parallel with any changes being made by the publisher. Traditional A/B testing becomes inconclusive at best.
Continuous Optimization
The solution is continuous optimization where the system modifies configurations in near real time to identify and build on emerging positive outcomes and minimize negative outcomes.
Importantly the system can’t assume a positive outcome will persist indefinitely. Continuous modification is needed to seek out better options and stop using ones that are no longer useful.
With continuous optimization A/B testing becomes irrelevant as positive outcomes are automatically identified and maximized.
Artificial Intelligence
Whilst it would be impossible for humans to handle the sheer volume of known and implied dimensions and variables that impact outcomes, AIs have no such problem. Feeding as much data as possible to an AI to maximize outcomes is no longer the stuff of science fiction. It can be done today.
Advertising Auctions
Auctions help sellers discover the true market price. Competitive bidding among buyers reveals the price each participant is willing to pay. The final price is the item’s actual value.
Auctions between publishers and advertisers for advertising are critical to the digital economy. Publishers wish to maximize the revenue for their advertising opportunities.
However, like any auction, information asymmetries exist. No one lists a car for sale and emphasizes worn tires or the state of the cam belt!
Knowing how much information to make available to buyers and to what granularity is critical to establishing value. This requires continuous optimization.
Publisher Examples
A publisher that doesn’t present misleading information, such as a geographic location that might be incorrect, is likely to gain a better reputation with buyers than one that is known to mislead (even unwittingly). Publishers can use continuous optimization to determine the granularity and method of establishing the location approach that drives the best outcomes.
A publisher that takes steps to reduce fraud such that they become known for only offering advertising opportunities from humans will likely gain a better reputation and attract higher prices. However, such steps can be intrusive and drive humans away. In any case it’s not possible to eliminate fraud entirely from an eco-system. Bad actors will always be trying new approaches. Continuous optimization can be used to establish the fraud prevention methods that work best.
Advertiser Example
An advertiser that can verify the accuracy of the information being provided by the publisher can gain an advantage in an auction. Sometimes the cost of verification might be high and not practical to apply in real time in all situations. Continuous optimization establishes where the balance is set of each publisher.
Conclusion
The concept is simple. With the right technology partners, it’s possible to achieve.
We’re not going to divulge all our hard learned lessons publicly. But if we’ve tweaked your interest in the subject, and you want to discuss further, then give us a little context to help us understand your challenges via the contact us form and we can talk.