Marketers rarely share a/b test results. With AI that becomes a bigger problem and opportunities. AI can result in important victories, but it can also stumble in ways that harm campaigns and brands.
Most of what we learn remains hidden in individual campaigns, which leads to duplicated efforts, repeated errors and slower progress. It is time to learn from AI’s Boosts and Blunders and to share the results.
When shiny new technology meets reality
I have always been a bit of a nerd. I like to try out the latest technology, especially when it saves time or helps me to create something better. I also learned that shiny new tools do not always live up to the hype.
Some take longer to control than the time they have to save. Others are unreliable or rough on the edges to years of updates. And sometimes they never live up to their promise.
AI is the newest shiny tool marketers. It promises enormous benefits – from faster copywriting to smarter scores for lead – and it is already clear that AI can help us do more in less time. But although it yields quantity, can it really deliver quality?
Before you stop reading and you propose that I am determined an anti -ii luddite to maintain a traditional, handmade profession, you should know: I am a huge believer in AI. It has demonstrably improved the work that we do for our agency and our customers.
Of course it has also abandoned us more than once. But let’s talk about where AI has delivered.
Where AI delivers
AI has achieved a number of impressive marketing victories. Heinz used it to generate ketchup images, and Nike simulated the tennis competitions of Serena Williams. The Digital Marketing Discovery Group Digitaldefynd even follows this and Other striking AI campaigns.
But most of these successes come from extensive, expensive efforts that produce huge amounts of content. That is not the daily reality for most marketers. What is important to them is knowing when AI starts with the provision of incremental improvements – and when it starts to stumble and becomes liability.
At the moment that is difficult to quantify without major financial resources. To paraphrase John Wanamaker: “Half of the money I spend on AI is wasted; the problem is, I don’t know which half.”
Dig deeper: your AI strategy is stuck in the past – here is how you can repair it
Where AI stumbles
You probably already use AI in many Martech tools, such as Google ads and the responsive advertisement format. The idea is simple: you make newspaper heads and descriptions and the AI ​​system quickly works the most effective combinations. Google will even prepare those annoying headlines and descriptions for you.
But without human controls on brand standards and legal requirements, AI van de Riem can have an adverse effect.
It is also unrealistic to test A/B, the responsive advertisement format at any possible combination to confirm the best results. Most people assume that it works well – until they write a murderous head or description shorter than the others, just to see it rarely and quickly displayed.
In those cases you cannot convince me that there are enough data to be statistically significant. Whether AI gambles, or it works under a line that says you should use as many characters as possible – or else.
AI also stumbles on personalization. We have all seen Cringeworthy AI-generated e-mails that scrape the website of a company and somehow make 2 + 2 = 27. These e-mails:
- Are formal.
- Written in a style that was clearly generated by the machine.
- Deliver often self -assured but completely false statements.
Many senders do not have time to assess the dozens or hundreds of thousands of e -mails that an LLM produces for them, but they have to taste at least enough to know if those messages are quietly damaging their brand.
Diger Diger: How you can use generative AI in copywriting for an A/B test program
Why AI errors can hurt
Nobody is perfect – including AI. We all accept a certain level of hallucination (or, more honest, errors) in AI -output. It is impossible to avoid, but with careful input AI usually gets it good.
We recently performed a simple test: we asked AI to list the top three markets for every company we e -mail. We only wanted four words for every e -mail (one of them is always “and”).
The results? About half was great. A handful was completely wrong. The rest was just okay. A person would have produced lists that were clearer and more transparent.
Balancing AI’s Boosts and Blunders
In general, the boost of the use of AI for personalization was considerable. But although the number of screws was relatively small, the potential damage was high.
In this case we were marketing for a very small, very well -defined audience. If the mistakes of the AI ​​were not checked and corrected, we could have seriously harm our brand with a financially important market segment.
When we call the boost in ROI to use AI “B” and the percentage of the “S” screw, the math suggests that everything looks great as long as B is larger than S. and AI can usually erase that beam.
But this analysis ignores something critical: the long -term impact of brand damage. Errors are cumulative.
At the moment, people are enthusiastic about the immediate improvements that AI entails. But we also have to concentrate on minimizing the errors.
The easiest way is to prevent AI when it is likely to hallucinate. With some basic training, marketers can learn to recognize those risks before they become problems.
Dig deeper: AI’s big bang effect means that marketing must evolve or die
Let’s open the largest A/B test in the world
As a rule, marketers do not share the results of their A/B tests. Some Martentch tools try to collect results, but if “blue” turns out to be the winning color for one campaign, that does not mean that every campaign suddenly has to turn blue. That is the kind of Overgeneralization AI is susceptible to make.
However, AI is different. There are many ways in which we all use it in comparable contexts. When writing Google advertisements, for example, AI is great in filling an empty box with options. The suggestions are solid about 95% of the time.
But there is a big reservation: AI often struggles when they work with deep technical customers. The generation of text that transfers very technical information – or compress functions in benefits within 30 characters – is none of the strengths of AI.
The real chance is the pool of what we learn. If marketers contribute their AI results, we can build the world’s largest open-source A/B test.
Fuel with free marketing insights.
Controlling authors are invited to make content for Martech and their expertise and contribution to the Martech community are chosen. Our contributors work under the supervision of editorial employees and contributions are checked for quality and relevance for our readers. Martech is owned by Semus. Contributor was not asked to make direct or indirect entries Semus. The opinions they express are own.
#Sharing #victories #failed #save #marketeers #repeating #errors #Farmer