You Ask, I Answer: What is the Marketing Data Science Lifecycle?

In today's episode, Manina asks, what steps make up a data science life cycle? Where do you begin? So the short answer to this question is that the data science lifecycle is essentially a series of processes that we use to make data science work. It begins with defining the problem, the question, the hypothesis, the early steps of the scientific method. And then we move into things like exploring your data, testing your data, building a model, validating it, um, deploying it, and observing it.

And this overlaps pretty well with the scientific method, as it should, hence data science. But each of these stages is composed of multiple substeps. There's a lot more to unpack in each of these. So uh let's actually bring this up here. So what you see here is the data science lifecycle.

The red part, defining the problem and your hypothesis is probably the most important part of this entire thing. Because without great problem definition and a provably true or false statement for the hypothesis, the rest of this stuff doesn't matter. This part also in the problem definition takes time to figure out what data you'll be needing in order to do the rest of the process. So the red part there, most important. Then you get to five steps in preparation of data.

So ingesting the data, getting it from all the different systems it's in, analyzing it, just to not for what the data says, but just making sure that the data is in good working condition. How much is missing? How much is uh how many anomalies are there? Is there a possibility of bias? Is there corruption in the data?

All those things go into the data analyze stage. After that, if you have to repairing any of the data, things that are broken, cleaning it up, uh normalizing it, uh, if you need to, putting it into the proper data structures, uh, and then after the cleaning is preparing the data, so rating it reading it for analysis. Uh this can be things like encoding, declaring variables categorical or uh continuous. All this stuff is probably the most laborious stage of data science, but it's also one of the most important besides problem definition because again, we're doing data science, which means we're doing science with data. And if our data is corrupted, then we can't do good science.

After that, you get to the yellow stages. This is where we start doing what's called exploratory data analysis, and that is a whole cycle in and of itself, but fundamentally we're looking to do we need to augment our data with new external data. We do full exploration and we do comparison, uh looking inside of our data to see what potential answers it has. We have not actually tested our hypothesis yet. We are just still in the the data verification stage to make sure that our data is going to do what we want it to do.

That's when we get to the green stage, the hypothesis testing, where we make that uh prediction, is our hypothesis true or false? Uh what should we do about it? And then we we build a a model, a theory. It's not fully a theory until it's proven, uh, but a model of our hypothesis with our data. Then get to the blue part, hypothesis testing, validating that model.

Does our data and our hypothesis work together to to m answer that provably true or false statement? For example, uh, you could say in in Google Analytics, our hypothesis is that website traffic will always be lower on the weekends. That is a provably true or false statement. Uh it's a singular condition, and we would bring in our data, analyze it, repair, clean, prepare if you know Google Analytics wasn't working for us one or two days, augment, com explore, compare. Our prediction uh is that this is a true statement.

Uh and if it is true, you might want to think about what to do about it. We build that model uh of very simple, you know, uh we in the uh augmenting stage, we might have augmented days of week in the data set, right? Because Google doesn't give you that out of its out of Google Analytics, it'll give you the the numerical date, but it won't give you the day of week. And then you validate, you know, when you do an average of all the Saturdays and uh average all the Sundays, and then you have an average of all the weekdays. Is your hypothesis true or false?

You validate it. And if it's false, you have to refine it, or start over, or throw it away. And if it's true, you might want to restart uh and augment, you know, hop back to augment and get more data. Maybe you looked at a year, maybe you should go two years, three years, five years. Maybe look at any other sites that you have uh legitimate access to, whatever the case may be, you would refine that that hypothesis, and then once you've got a working model that you've essentially proven, you you deploy it.

Now, in the case of uh the insight that your website traffic's lower on the weekends, um that deployment would simply be telling your marketing team, hey, if we want more traffic on the weekends, we've got to run some ads. Or if we don't care about um weekend traffic because no one in the office is around to answer sales questions, cut our ad spend on the weekends. So that deploy stage is really about taking our prescribe from the green section and rolling it out once we've proven that our hypothesis is true or false. And then we observe it. Make sure that, yep, our our model is working as intended, and and we have proven true for ourselves whatever our hypothesis was.

That's the data science lifecycle as a whole. And again, there's things to unpack in each of these stages. Every even here in this more detailed model, just taking something like repairing your data can be a whole series of you know 10, 15, 20 steps, doing things like you know, missing value imputation, determining if you're if you have missing data, is it missing at random, is it not missing at random? There's all sorts of things that you can do at each of these stages. And that's one of the reasons why data science is so complex.

Because at each of these stages, there are mathematical principles at work, there are technical principles at work, there are business principles at work, there's domain expertise at work. So there's all of these things that you have to unpack and be able to do in a data set in order to be able to execute the scientific method and develop that working model that is reliable, that is repeatable, and that is uh defendable. Uh you know, you want to do something like peer review, or at the very least, colleague review to make sure that your model is in fact valid. So as you start your journey, one of the things I would recommend you do is take this model and then start with very simple data sets. Again, the Google Analytics answer example is a is a good one because it is compact, it is mostly clean most of the time, more or less.

Um, and it allows you to test your knowledge of each of these steps without having massive, massive mathematical and technical hurdles uh at each stage. You start with super simple, and then as you get comfortable running through this life cycle, you can then work with more and more complex data, build harder to test hypotheses, and ultimately be able to use this on a regular basis. But really good question. If you have follow-up questions, leave them in the comments box below. Subscribe to the YouTube channel and the newsletter.

We'll talk to you soon. Take care. Want help solving your company's data analytics and digital marketing problems? Visit TrustInsights.ai today and let us know how we can help you.