You Ask, I Answer: Marketing Data Science Hypothesis Creation?

In today's episode, Jessica asks, How will a data scientist create my model or hypothesis if they don't know my business? It's an excellent question, and actually, one of the big problems facing data science, machine learning, artificial intelligence. Uh the short answer is you can't. There is no way to create a hypothesis, no way to create a model, no way to create a piece of code without knowing the business. Remember that data science is fundamentally four different skills, right?

It is business skills and subject matter expertise or domain experience. It is scientific skills, it is technical skills, and as mathematical skills, statistical skills. A lot of what people think of as data science tend to be overly heavy on the technical skills by a large margin, and the mathematical skills. And those are important, right? You can't do data science without those two at the more complicated levels.

But the scientific skills, the hypothesis creation, uh tends to not be taught a whole lot or taught very well. And the domain expertise and the subject matter expertise are something you really can't teach effectively in a generic data science course or degree or whatever. And I know there's a ton of them out there, you know, get your data science certificate in six weeks. And what they're teaching mostly is coding. But that's a quarter of the puzzle, right?

Hypothesis testing, when you're creating a hypothesis, it there's the three steps to it, right? There is you have a question you need an answer to, you define the parameters of the question, what data you'll need, what information you'll need, and then you create a hypothesis, which is a verifiably true or false statement on a single condition. Example, you're about to do uh website A B test experiment, and you have a hypothesis. Well, you have a question like what would make my website convert better? Well, that's a pretty broad question.

What are all the things that go into conversions? Do your customer journey mapping, do your path analysis, um, do your audience segmentation and demographics, and now you're starting to define the problem, and you come down to we think that we've got a different audience from one channel than another. I think our Twitter audience is different than our LinkedIn audience. Okay, that's a that's a good definition of the problem. Now, how are you gonna test that?

What will you test to prove true or false that statement? Um the question would be, or the hypothesis would be our Twitter audience converts better than our LinkedIn audience. That's a really good hypothesis, because you can prove it true or false, and it's only testing one thing. So you go to a landing page on your website, you would optimize as best you could, and then with great tracking codes, you would send your Twitter audience to that page, you would send your LinkedIn audience to that page. You would run the test uh for a sufficient amount of time, which you would determine by uh sample size calculation, and then after that, you would analyze the results and draw a conclusion that either one audience does perform better than the other, neither audience performs better than the other, or that uh you messed up somewhere and you have to start over again.

But you can see how if you didn't have marketing domain expertise, marketing uh skills, you might leap to the most naive conclusion. Well, let's try changing the button color. Well, no. In this example, maybe you identify the audience was the problem. Remember, we're going back to Bob Stone's 1968 direct marketing framework.

List uh offer creation, list is your audience. And you have to test these you have to figure out where the problem is in these different areas. And so that's where that domain knowledge is so important and why you can't do data science without it. If you try, you're going to come up with either things that are very surface and they might make incremental improvements, right? If you change the button color on your website, it might perform a little bit better.

It might even be statistically relevant, but it's not solving the big problem that that you brought to it, the the big question, which is how do we make our website perform better? So there is no substitute for domain knowledge and subject matter expertise. Now, does your data scientist need that? A data science project needs that, but the individual person may not necessarily have to. If you are a or you are or you have a skilled project manager, um, they can identify where the gaps are in those four areas and then bring in extra people on the project, assuming you have the budget for it, to supplement where your data scientist scientist or science team might be lacking.

So you might, in the in the marketing case, you might have a data scientist who is technically skilled, mathematically skilled. Uh you might have a project manager who is scientifically skilled who can think about how do we how are we going to set up this project to be statistically valid? And then you bring in a subject matter expertise. You bring in your VP of marketing or your marketing director and say, Hey, what uh help us identify a question and define the problem deeply so that we know what to test. And what will happen over time as long as everybody gets along is that you will see cross pollination of skills.

You will get the project manager to think even more statistically or more technically. You'll get the data scientist to think more business or more subject matter expertise. You will get the marketer to think about scientific uh testing and technical stuff, and that's what you want. You want that team to become a cohesive unit to fulfill all four areas of data science. If you don't have all that accessible, then your data science projects are not going to work.

You need to have all four areas fulfilled. Not necessarily in the same person, but you need to have it in all four areas so that the project doesn't go off the rails. So you don't waste a lot of time and a lot of money. One of the reasons why data science projects, machine learning projects, artificial intelligence projects go off the rails is that you're missing a key area, and you end up with a result that's not usable. You end up with a conclusion that's like, well, you know, we tested and we think, you know, poop emojis on Tuesdays work best.

Well, okay. The statistical side will say, show me the statistical significance of that. The technical side would be like, okay, make sure we did the implementation correctly. But the business and domain knowledge side would say, pretty sure that has almost no relevance to our business, unless you know you're like Sharman. Um you can't substitute it.

And you cannot expect the machines to do it for you. There is no AI out there in the present day that has broad general knowledge that can provide domain expertise and subject matter expertise. None. Not even the most advanced ones. You can give them very specific narrow tasks and provide a lot of background data to those tasks, but the orchestration, the coordination, the keeping the machine on the rails, that is still very much the provenance and the domain of human beings.

So, good question. Be careful, be careful as you do these data science projects. Make sure that you have a great project plan. Make sure that you have all the necessary skills in order to make it successful. If you have follow up questions, leave them in the box below.

Subscribe to the YouTube channel and the newsletter. I'll talk to you soon. Take care. Want help solving your company's data analytics and digital marketing problems? Visit TrustInsights.ai today and let us know how we can help you.