Expert Strategies for Navigating The Dynamic Landscape of Conversion Optimization

Hi. Thank you for having me.

I was trained as a computer programmer, so I was a CS major in college. But when I graduated, I went into sales, semiconductor sales, they called Texas Instruments. That worked for about three years until I rose to my highest level of incompetence there and was invited to leave. I then went back into software programming for a small company here in Austin. I wasn’t that good a programmer anymore, so I got promoted to marketing and ran their marketing. During that time, I developed my first analytics package, trying to track what was going on with emails and which landing pages were working. But I left with the director of engineering there and started my first company in the ’90s doing web development. In the 2000s, some of our clients got hit by the dot-com bust, so we closed that company. I spent the first part of this century building lead-generation websites for a number of technology companies. But I had a lot of ideas of my own. I went to this very interesting business school here in Austin called Wizard Academy. Brian and Jeffrey Eisenberg introduced me to what would become called conversion optimization.

I was like, This is what I want to do. With my background, what else could you do? Become a conversion scientist. I put on my lab coat and hung a shingle and spent the next easily four years just trying to educate the market on the importance of being able to design this way, to be able to develop websites with data so that you’re not just creating websites and launching them and hoping that they’re working, but you’re redesigning them slowly over time in a deliberate way with the scientific method, which is our method. That was in 2007 that I started doing that, and we’ve been going ever since.

The biggest change, of course, is that when you decide to start using experimentation and data while you’re designing, it’s very humbling. It’s not unusual for brilliant designs that are objectively better, that are going to improve things for your visitors to fail an A/B test and not be better than the control that you had before. I think the most important thing is understanding just how different every audience is. Things that we’ve done that worked for a company don’t work for a company in the same industry because they bring different visitors. They have different value propositions. I think that’s the biggest thing. For me, understanding that the scientific method that has been used by the Pernicus focuses, and Newton and Einstein over the years does work in web design and web development. I think those were the biggest understandings, the biggest shifts For me.

Probably the biggest mistake is focusing on the tests instead of the ideas. The reason we’re good at what we do is that we’re good at picking the thing that we’re going to take to an A/B test. A/b testing is the best data we can collect, and controls for all the variables. When we run an A/B test, we have high confidence that the results of the test are going to map onto the rest of the population going forward. But you’ve got to pick the right things to test, and that requires a lot of upfront research. So using analytics, user testing, surveys, or anything you can get your hands on to increase the confidence that an idea is worth testing, or sometimes you even prove that you just need to implement it or cross it off the list because there’s not good evidence. So I think when we’re brought into companies that already have CRO programs, we inevitably learn that they’re having weekly brainstormings to see what they’re going to test next. And so they always start from zero. The ideas of the highest-paid person in the room typically are the ones that they run with rather than collecting those ideas, evaluating them, and just picking those that are going to have the biggest impact or teach you something about your visitors that you didn’t know before, which is important.

It’s this exploration versus exploitation. Sometimes you just want an idea that’s going to improve your conversion rate, and sometimes you want to learn something about what your visitors want.

I’ve always talked about when I first started, a lot of our work was around personas. So understanding the visitors that are coming to your site, understanding the biggest segments of visitors that are coming to your site, and then rafting a series of recommendations, which are essentially ideas or hypotheses for what these visitors want. The hard part has always been crafting different messages for different kinds of visitors. So the question of writing very differently, most companies create their copy internally. It’s always better to have somebody on the outside asking questions and doing discovery and writing that copy for you. But it becomes difficult to write different kinds of messages because as you have different ways of approaching your value proposition that appeal to different kinds of visitors, where do you put them on the page? How do you express those? How do you understand which of those audiences you’re talking to at any time? The new generative, large language model, the AI models, are fantastic at rewriting copy in a voice that appeals to different visitors. I think understanding that you need to focus on different parts of your value proposition for different visitors, use AI to help you rewrite those things, because it’s We have three pounds of seething biases between our ears which make us write things that we like, things that have worked in the past, avoiding things that have not worked in the past.

These are all shortcuts. We’re using stereotypes to figure out how we want to write for women versus men, people from different countries, all driven by stereotypes. This is not a good way to appeal to your visitors. The language models can very easily drop themselves into a different mode and say, Okay, now I’m going to write differently. You end up with very different messages and very different ways of conveying your value proposition. It’s very important to be good at this. This creates a ceiling in conversion optimization if you aren’t hitting the basics of your message. Then you can either A/B test those, or alternate them to see which one is most effective. Essentially, what you’re doing is finding out which of your segments is the biggest that are coming to your website. Or there are strategies that we have for putting them on a website, multiple different ways of approaching the message on a web page or a landing page and appealing to broader and broader segments that are coming to your site. We’re in a golden age. On one side, we’ve got great applications for gathering data. We have great services for doing user testing so that we can put our copy in front of people and find out where it’s confusing, and where it’s not clear.

Now we have this fantastic co-writing partner in the generative AI that lets us get out of our brain and come up with these different messages. I’m very excited about the future of data-driven message selection and implementation. That makes sense.

Let’s frame this. Conversion optimization is redesigning your website. When you employ a conversion scientist, you’re asking them to redesign your website. Only instead of writing a creative brief and doing some research on the front end and then changing everything and launching the results, we’re going to take a bit more disciplined approach, a more rigorous approach, and we’re going to redesign it based on specific ideas that we have, hypotheses that we have, that we’ve taken from the data to change a little bit at a time over time. As we do that, we learn about what the visitors want, so that influences our design decisions down the road. The best place to test your redesign ideas is on the existing site. When you find something that works, when you find an idea that’s going to increase your conversion rate, raise your revenue, raise your lead generation, you leave it on there. You don’t have to wait 6-12 months for the design to be completed to get the conversion rates, crossing your fingers hoping that they’re going to be higher, but you’re improving the conversion rate on the site as you’re experimenting to inform the design decisions that you’re making.

From my point of view, why would you design any other way? The only exceptions, I think, are if you’re changing platforms and you have to completely change the website, or you’re doing a brand refresh where the brand and the messaging are changing. But if you think about launching new products, if you think about adding new features, each of those is an opportunity to come up with the right message and present it on the page in a way that is compelling to your visitors. Testing all of the pieces of that slowly, really makes sure that you’re putting exactly what they want on your website.

Yes. The issue with personalization is that as you personalize more and more for segments, you essentially have multiple versions of your website, and your sample sizes get smaller, so it becomes more difficult to do A/B testing. You rely more on the research from user testing and so on. Just as an example, sometimes we’ll see a mobile test that is inconclusive. Our idea did not improve things over the control. But when we drill in, we find out that the iOS people coming on Apple devices loved the new idea. People coming in on Android did not like it. So there’s an opportunity for personalization. If somebody’s coming in on an iOS device, they see the variation, and the people coming in on an Android device will see the control unless I swap those. But you get the idea. But what that means is if I want to do a test on mobile next, I’m going to test separately on the Android segment and the iOS segment. Our tests have to run longer to do that. But personalization, I think, has suffered because the tools allow us to target almost anybody we want. But then they say, Okay, marketer, you decide what needs to be on the website or let’s say that this person has come back for the third time.

Maybe they’re ready to buy. Marketer, you decide, do you want to use a discount or something like that to entice them, versus somebody who’s coming for the first time and was probably doing more research, so you provide them with more information, more links to explore your product and the solution. Is that the best solution? There’s no way to know unless you do a controlled test like an A/B test. I prefer personalization rather than, Okay, here’s a segment that’s coming. Come up with some idea that will appeal to I prefer that you do an A/B test, you go in post-test analysis, figure out what the different segments are that are responding differently to that idea, and then personalize to hit those because you’ve got data that is telling you exactly what different segments like and don’t like. That’s, I think, a better way to personalize.

We tend to use analytics as our database of truth. We work with a number of A/B testing tools, and they’ll give you a top-level analysis as to whether we’ve reached statistical significance on a particular hypothesis, a particular test, and if we should vote up or down, whether this is a good way to continue going. But beyond that, we do spend most of our time in analytics looking at not only the segment that we targeted but was there these subsegments that behaved differently. That’s our favorite tool. If that answers your question, was there more to your question?

It’s rather slimming, isn’t it?

Yeah. If you’re getting into digital marketing, you’re going to have to be able to use these tools. You’re going to have to know how to divvy up a design into ideas and hypotheses, how to research those hypotheses and analytics to give them support, and how to do user testing to narrow down. It’s not unusual for a creative agency to come to you and say, Hey, we have three mockups for this page. Pick the one that you like the best, and that’s the one we’ll do all the design on. The proper answer is, Why don’t you collect some data for me and tell me which one of those I should pick? Which one of those is going to be verbed by my customers, not by me? You need to be able, if you’re not doing the analysis yourself, you still need to be conversant in helping them understand what you mean because most creative agencies are going to look at you and go, Can we collect some data on that? Can we do that? And you say, Yes, here are the user testing services that will bring a panel of people to look at it that you can ask some questions about to see which one is most effective.

I think integrating AI with messaging is a set of skills that you’re going to need. The learning curve on those is incredibly small because the language models understand English very well. Ultimately, moving away from best practices, figuring out how not only to integrate this experimentation into the design process, because you’re moving risk from post-launch into the design process, and understanding how to scope those projects so that there is room in them for this testing piece. When somebody’s doing a redesign of a website, they’re very eager to get it out and launch it. Understanding how to integrate data into that process is going to be a very important skill, I think.

Okay.

In my last Google Search?

What was my last Google search? I would have to look that up. My last Google search, I don’t even remember. The last search that I did, AI Search, was how to write some copy for an NF Myers-Briggs type. I was picking some copy and I said, I want to write this in a more humanist way for people that are interested in connection and interested in who they’re going to be doing business with. It rewrote that for me very nicely.

The first paycheck of my life I bought an expensive car.

It was good at the time.

If I’m lucky, I am meeting with other entrepreneurs and artists and just interesting people for dinner, or they come over here for a fire and talk about life and living and business. I love the philosophical conversations.

No, thank you very much for having me.