You Ask, I Answer: Why do Recommendation Engines Fail?

In today's episode, Oz asks, why is some consumer AI so bad? Instagram, for example, senses that I like bright colored clothes, then it shows me ads for bright colored clothes that are also cheap crap that I never buy. What is the perspective of these tech companies? Is it does that work great for most people? We just need to get this right for five people.

It covers the cost. We know it generally sucks, but it's better than nothing. So it was a good question. The answer has a lot to do with how recommendation engine technology works. Recommendation technologies um take in data sets and essentially try and find uh patterns in those data sets to uh to predict outcomes, right?

So if we if you like these certain things, we're gonna predict uh certain other things. Recommendation engines can use a lot of very different algorithms under the hood. And one of the challenges we have, uh and it's a challenge in the industry overall, is that a lot of these companies don't reveal what is in their algorithm, what algorithm they're using. Are they using something as simple as like a naive Bayesian classifier? Are they using uh something as complex as you know a many, many layer deep neural network?

Uh are they using you know K nearest to neighbor clustering? We don't know. We don't know what's under the hood, and so we don't we can't necessarily offer input as to why uh some things behave the way they do. But there's two general considerations, well, three general considerations as to why some of these algorithms don't uh spit out useful stuff. Uh the first and by far the the most uh likely is computational cost.

The more complex the algorithm, the more it costs to run it. And uh the cost here is in compute com computational capacity. Uh how fast can you get the result? With a lot of ad systems, for example, you were talking about millisecond response times, particularly when you're doing stuff like header bidding and things where there is a real-time auction going on and ad systems have to match and generate results extremely quickly. And as a result, they have to pick algorithms that are super super fast, even if the accuracy is leaves a little something to be desired.

I mean it's better than nothing. For those who remember the early days of digital marketing, you'd be browsing on a website and you have like a Medicare wheelchair program being displayed to someone who's you know 22 and healthy, uh it's like no, that's completely incorrectly targeted. Um speaking of which there is always the potential for uh advertisers themselves simply being so bad at advertising that they they have blanket targeting uh and you know the machines cannot override a user's preferences. If the the advertiser says, hey, I want to advertise to every living person within the boundaries of this nation, okay. As long as you've got the budget to support it, uh it's gonna do that.

But computational cost is a big thing. Second thing is um the what data you have going in, uh the data that goes into the system uh may not be robust enough to offer uh anything that has true predictive power, uh, especially if, and this is important, especially if uh companies are correctly implementing um ethical, unbiased AI. You may not, for example, in a lot of cases, uh judge somebody uh and and you know, tune your ads on a protected class, or you shouldn't be, let's put it that way. And so uh if the advertising that comes out is incorrectly targeted because you uh a back end you know ethical check has said hey you can't use race as as a targeting criteria for this ads okay so now you're gonna get you know sham wow right uh even if that's not necessarily something that you want because there may be some balancing happening behind the scenes to ensure that the uh protected class is not being used um a third part is objective optimization and this is where uh this is where advertisers should be a little bit concerned objective optimization and compensation models dictate uh how advertising networks work what does the ad network get paid for do they get paid for the impression do they get paid for the click do they get paid for the outcome advertisers have been pushing to very little success over the last 20 years uh with digital marketing to have uh avert to have uh action based or outcome based advertising where you get paid for the lead generated you get paid for the form filled out uh you get paid for the the shopping cart filled and understandably the big ad networks have absolutely zero interest in doing this because it means much more rigorous computation on the back end it means much more in-depth tracking there may be um substantial risks to the ad network because yet you could potentially uh inadvertently or intentionally be collecting uh sensitive protected information and frankly most ad networks realize that behind the scenes ad performance across the board is pretty crappy I mean if you think about it uh when you look at like the click through rates on some of these ads and you look at these campaigns, you know, when people celebrate like crazy when they get like a five percent click through rate, which when you think about it, means you wasted ninety five percent of your budget, right? If you didn't get more than five percent of the clicks, from the advertiser perspective, you're like, well, what did I pay for?

If these systems were tuned to results only um advertising, that'd be a very different and much worse calculus for the ad networks because they wouldn't get paid unless they got the result. Is there a possibility that companies could pivot that way? Potentially, um, but right now, everything in advertising is effectively cost per impression. When you look at the back end reporting and you see all these you know metrics in in like Facebook and stuff, effective cost per click. No, what you're really doing is you're st you're still doing all your bidding by impressions.

Um and you're still doing all your optimization on that. And as a result, it doesn't really matter to the ad network whether or not you click on the thing beyond a you know, with a reasonable doubt, but it for the most part doesn't matter because they're getting paid on the impression, they're not getting paid on a click for the most part, they're definitely not getting paid on the on the action uh that was taken. Now, if advertisers forced ad networks to uh to pivot and said, Look, we're not gonna pay you unless you deliver results, that would drastically change the machine learning outcomes that a lot of these systems are tuned on. It would make them computationally much more expensive because you would have to be uh you you wouldn't be able to do simple stuff like K nearest neighbor clustering, uh just on on impressions, right? Um, you would have to collect a lot more data, you'd have to collect a ton more data, uh, and that would make for very, very different optimizations.

When you look at how, for example, LinkedIn works versus how Facebook works for this advertising, LinkedIn stuff works very differently because they have as one of their major outcomes. We need to keep people on this professional network so that our enterprise talent management software, which is 40% of their revenue, um, can draw useful data from people's profiles and recommend it to recruiters. Right. So there's a there's that's an example of a system that is much more outcome-based, and as a result, you see a very different culture on LinkedIn. You see very different advertising on LinkedIn.

Whereas Facebook is like, show all the ads, show every ad possible, see what people click on. Cool, great, whatever. Same with Instagram. We get paid on the view, so who cares what the result is. So that's why consumer AI is so sometimes untuned.

There's a bunch of different reasons, and there's no way to clearly tell without independent third-party audits what's going on behind the scenes, um, how it's working. I would love for companies like Facebook, for example, to reveal hey, this is how we do the thing. Um, these are the inputs, this is how the system is optimized. But that is literally their secret sauce. So it's unlikely that they would ever reveal that, even if they could, because in a lot of cases, some of these in Facebook's case, some of their neural networks are so complex, I doubt there's any single human could that could even interpret what's going on behind the scenes.

The models are just that big. So, really good questions. There's a lot to unpack in here about how these algorithms work, how they're tuned, and what's going on underneath the hood. Um, hopefully, as time goes on, we will see uh advertising itself pivot more towards results-based advertising as well. Uh if you have follow-up questions, leave them in the comments box below.

Subscribe to the YouTube channel and the newsletter. We'll talk to you soon. Take care. Want help solving your company's data analytics and digital marketing problems? Visit TrustInsights.ai today and let us know how we can help you.