You Ask, I Answer: Choosing Marketing Data Science Variables?

In today's episode, Jessica asks, as a data scientist for marketing, how do you decide which variables are important? So variable importance, uh also known as feature selection, predictor importance, is a set of techniques and algorithms that you use to essentially try to figure out which of the variables that you have in a data set have a uh relationship with the outcome that you care about. Uh so this is typically regression analysis, although it can uh there can be uh variance of for classification, but fundamentally it's a regression analysis to figure out is there a mathematical relationship between an outcome and all the data that you have with it? And this is something that we've been doing for a very, very long time, right? If you've ever run a basic correlation in an Excel spreadsheet, you're technically doing a type of uh variable importance or variable selection.

What's different today from doing it in Excel, for example, is that you can use machine technology to look at every possible combination of variables, which you we would call multiple regression or multiple regression subset analysis, and have machines try and pick the algorithm that would be best suited for that data set, because there are some uh algorithms that are better suited for uh looking at categorical or non-number uh data. There's some algorithms that are good at number data, there's some algorithms that are good at both, but not as good as either one. And so using machine learning technology allows us to identify those relationships in a much more robust way and in front quite frankly, just a faster way than trying to do it by hand. Now, what you get with a lot of feature selection techniques is a correlation, right? Regression analysis leads to a correlation.

And that's important to know because when you have a correlation or an association, you have not proved causation. Stats 101, correlation is not causation. So you would use machine learning technology to first do a first pass at what are the features that we think are important, and then ideally use uh the scientific method to prove that this has a relationship with the outcome. Now, that's if you find the relationship isn't spurious. Uh sometimes you will get what's called spurious correlations, correlations that make no sense at all.

They're they're variables that have no relationship, uh, but the machine sees a pattern even though it's not valid. There's actually a great blog by Tyler Vegan called spurious correlations. If you go Google spurious correlations, it's hilarious. It's uh all these things that have strong correlations but clearly no relationship to each other, like the number of people who uh died from drowning and the number of movies Nicolas Cage has been in, right? Have no relationship to each other, but there's a mathematical relationship, and that's why you need the scientific method to be able to prove that what A causes B.

This is also why you have to know your data set really well as a subject matter expert. Part of data science is having that subject matter expertise so that you can look at the variables that a machine would say these correlate and go, no, they don't really correlate. They're, I mean, they're they they mathematically do have a relationship, but it's not a valid relationship. And the worst case scenario with a lot of these tools is that you get a whole bunch of nothing, right? You get a whole bunch of uh inconclusive answers that then tell you you don't have enough data, or there's data missing, or there are relationships missing in your data that you then have to go and either uh augment by bringing in more data, or engineer by creating new data from the data you already have.

So let's look at an example of this. I'm gonna bring this up here. This is IBM Watson Studio, and what I've done is I've taken my lead scoring uh data from my marketing automation system, and I've fed it in here, and I've said, tell me, feature importance-wise, of all the data that I'm collecting in my marketing automation system, what has the highest mathical map mathematical relationship to the outcome I care about, which in this case is the points. If you've ever worked with a marketing automation or uh CRM system, uh lead score or points uh is one of the indicators that says, hey, this is a high quality lead, or this is a low quality lead. In this case, we see a very strong relationship between uh when a a contact was last active and their uh lead score.

And this makes total sense. The more active you are, and the more frequent you're actually active you are, of course, the higher the points you're probably gonna have, right? Somebody who is was active once four years ago, they're not a very good lead. The second relationship, which is much, much weaker, I would actually say it's there's not a relationship here, uh, is activity on Twitter. Um so this is an example of you had a very good indicator, which is activity, and then you got some indicators that are not so good, and then you you know you go into the suburbs here, and there's a whole bunch of data that has no relationship whatsoever.

So now we have a relationship. The question is could we prove that this relationship leads to a higher lead score? Well, we know intuitively that that probably is the case, but we would want to scientifically prove that. To do that, we could do things like send more emails or run retargeting and remarketing ads to see if we can get people to be active who are not active. So I would take my data set, take everybody who's been active in the last 30 days, put them out of the data set.

Um actually don't put them in a uh control group, take everybody who's older than 30 days, put them in the experiment group, maybe randomize mix and match like 20% of each, and then run the same ads to both, saying, Hey, come read today's email, right? And what we'd want to see is would we see the points increase on the experiment group substantially to prove that activity date last active actually does increase lead scores? Now, this is a very simple, straightforward way to prove this. Here's the catch. This is where subject matter expertise comes in.

If my lead scoring uh algorithm, the the way that I've chosen to assign points in my marketing automation system is flawed or makes no sense, then I could be testing and proving something that doesn't matter, right? We would want to, for example, analyze taking a step back. Does lead score have a relationship with people who actually bought something? If it doesn't, then the lead score itself is broken, and then this analysis doesn't matter. So you get a sense of when it comes to how to decide what variable is important.

There is a lot of technology, but there's also a lot of business sense. There's also a lot of common sense. Is there a relationship here? Does that relationship matter? So these are the questions that you would need to take as you do this kind of analysis.

Really good question. Challenging question because again, there are so many layers to the onion that you're going to end up peeling back that you'll realize at some point things may be more broken than you think. It's always a challenging place to be in. If you have follow-up questions, leave them in the comments box below. Subscribe to the YouTube channel and the newsletter.

I'll talk to you soon. Take care. Want help solving your company's data analytics and digital marketing problems? Visit TrustInsights.ai today and let us know how we can help you.