So What? How do I clean and prep my data for analysis – Part 3

Well, hey, we are live. This is So What, uh, the Marketing and Analytics Insights show. Uh today we are on part three. How do I clean and prep my data for analysis? So the past two uh two episodes, the past two weeks, what we've covered is um really going through that exploratory data analysis process, uh so that you can start to get organized, do those business requirements.

Last week we covered actually extracting the data, figuring out what questions you need to ask, um, so that you can go back and see where my was my hypothesis right? Am I answering the question being asked? And this week, what we want to cover is actually getting your data into some sort of machine learning analysis. So we're gonna cover more advanced feature engineering, um, some obvious and non-obvious errors in your data, and then choosing an analytic approach. Uh you're probably thinking, like, what do you mean an analytic approach?

Like you just analyze the data. Well, Chris is going to cover what that actually means. Um, so Chris, John, take it away. All right. So as Katie said, we've been looking at the exploratory data analysis process.

And one of the things that we highlighted in previous weeks is that at any point in the process, if you are doing stuff and you realize, oh, this is not working. You actually have can you know jump back a step, jump back several steps. Sometimes you have to jump back all the way to the beginning. And so this week we're going to be looking at the data we've been working on for the last two weeks, which is our our Twitter uh analytics data set. And we're gonna find uh spoiler alert, some problems with the data and stuff and see what we can do with this.

So, to recap, what we've done so far is we have gotten the data from Twitter itself, from analytics.twitter.com. And you've done uh some initials just cleaning, making sure we know what's in the data. So let's take a look at it just very quickly here. Uh inside the data, we have the tweet ID, the permalink, the text of the two of my tweets, um, the time, the number of impressions, engagements, the engagement rate, retweets, replies, so on and so forth. All the fun of uh what comes out of Twitter.

From there, we've done some basic uh feature engineering. So things like identifying, like making sure all of our text is in lowercase because when you're doing any kind of text analysis, um, most uh algorithms and things are case sensitive. So even something like that is just forcing your text to lowercase is a good thing to do. And then we know generally what I tweet about. So we've put together some counters to see uh like how many times do I in a tweet do I reference SEO or data and data science that it creates a in the table more columns for those numbers.

Now, here's where things get interesting. Let's go ahead and start running um just a quick topic check just to see how many times am I talking about each of these things and what are the the frequencies of these words. When we run this, and it really is just a word count, right? We see a lot of things like a book a consultation for free, right? Well, one of the things that we do in our in our Twitter scheduling is have repeated uh calls to action stuff.

We try to do that anywhere between 10 and 20 percent of our social media content to make sure that we're we're getting people's attention. This is good for business, but it's bad for analysis because that doesn't really help us understand what we're talking about, right? So, one of the things that we should probably do is take that down to just one, so instead of having every single promotional tweet, which is not a great idea. What if we take that and turn it into just uh one of each of the promotional tweets? So let's go ahead and do that.

We're gonna take we call this a tweet text. Now, Chris, while you're doing that, I just I apologize if I missed it, but I have a question. So you said that you're doing this word count so that you can try to figure out what you tweet about the most. Now, I know with text mining, um, a lot of times you have to provide a dictionary of sorts. So you have to like pre-bake in the words that it's looking for.

Did you do that in the sense you said, I want to look for SEO, I want to look for data science, or is this script actually just finding and counting all of the similar words? That's what exactly what this step is is to look at those the frequency of those terms. And now, for example, the book of consultation stuff has largely disappeared, right? Um, but my question was my question was did you give it a dictionary beforehand to tell it what to look for? Got it.

Okay, so that is the difference here. You're not doing text mining, you're actually just doing frequency. That that's right. We're we're well, it is and it's a form of text mining, it's the simplest possible form. Um, this is just simple classification, it's a question of like what's in the box, you know, what words and phrases are in the box.

And going through this now, we can see that our instincts and our our judgment about the things, the topics that we've all pre-tagged, we're on target, right? Marketing and data and analytics and data science. It makes logical sense. That you know, uh obviously I would hope that I would know what I'm tweeting about, but I don't. That's that's a different strategic problem.

Um but this is one of those things where that's a non-obvious problem. You may look at your data set and that initial just peak of the data, you look at it, go, oh, this looks fine, right? These are all my tweets, and I would assume there's no problems with this data. But when you then do even just that little bit of an analysis, you go, huh? That's a problem, right?

That that that's something that is is kind of messy. So we've now gone through and we've cleaned that up. Uh, and we should have let's just make sure I am where I think I am. Yep, we should have 59 different variables that we could use for modeling. Now, again, one of the things that and this is why it's so important to have some experience or to grow some experience in data science, is that very folks who are early on in their journey will rush right into okay.

Now let's feed this to you know, IBM Watson and the auto AI feature and see what comes out of the box, right? And just kind of rush in and and do stuff. And it's a really bad idea here because this this data set is still not ready for prime time. There's a lot of things, you know. If data is an ingredient, there's a lot of extra stuff in here that we don't necessarily want in our in our final recipe, right?

You know, this is like plastic spoons like in in the cake mix, like somebody left the plastic spoons in the box. So we gotta fish all that stuff. We gotta fish all that stuff out. So this is now where we start getting to more advanced uh types of cleaning. So let's take a look at the first one.

Um in numeric data, there's this concept of the near zero variable. And this is a variable which doesn't change a whole lot. It either is zero or it's very close to zero and doesn't fluctuate very much. Um, and that's not really helpful, it doesn't really predict anything, right? If you know if you have uh in my case, number of promoted tweets, zero.

I don't spend any money on Twitter, right? So having those fields in here doesn't make any sense, but we have to know that they're there and clean them out. So in our next set of steps, we're gonna say, okay, let's get rid of uh all those fields that are our you know zero fields, right? Because that's a just a bad thing to have. And what we end up with is out of the 59 fields we started with, it's down to 20, right?

We we just chopped away 39 uh columns in our spreadsheet. Just cut them off because they're full of zeros, and that includes some of the topics too, like in this corpus of tweets, it looks like I have not really tweeted about SEO, because it's just it's just gone, right? So at this point, we have to go back to our requirements, right? And say, is this a data problem? Like I could have sworn I tweeted about SEO or something, right?

Along in the last month, is this a data problem? Is this a um processing problem? What kind of problem is this? If you know that something's not there that you thought was supposed to be there, and go, huh? So let me ask you this question because this this might be part of uh the challenge that you're running into.

So obviously, if I'm understanding correctly, this data it's only reading the tweet itself. So if you are tweeting about SEO, but the title of the article or the tweet itself doesn't contain the term SEO, it's going to appear as if you're not actually tweeting about SEO. So there might be a little bit of a mismatch in the copy that you're coming up with to put on social media, even though you're the article or the blog post or whatever it is is the right topic. Is that is that right? That's correct.

And that it really goes into this section here on feature engineering. If we think there's a problem and we validate the data itself is correct, we have to do augmentation, right? We would have to say, okay, if I'm sharing links, and I know that they're like, you know, search engine journal, right? Uh, which is exclusively about SEO, and it's not, and I'm not tweeting about it because I'm pulling in the article titles, and the article title doesn't have you know common terms like structured data, SEO ranking, SERP, etc. Then maybe I need to augment my data set with the copy of the actual article so that we can check the articles themselves for those terms.

Um, we're not going to do that today because that would take a lot longer than we have uh on this show. Um, but that is the kind of question that we need to be asking is to say, okay, when you're doing data analysis, it's it's the old Rumsfeld thing, right? Uh uh, you know what you know. Yeah, there are known knowns, known unknowns, unknown knowns, and unknown unknowns, right? The sort of the two by two matrix.

And this is a case where you know that something's supposed to be there and it's missing because it's a known unknown. Like, why is this happening? Uh and it'd be one of the things we wanted to try and fix. So that's what we've done just in this stage is clean up as near zero variables, then quality check, look at the data set and go, yeah, that doesn't seem right. Uh it seems like something's off.

I do remember this is one of those like weird moments. I do remember exactly where I was the first time I heard you explain that two by two matrix, the known known, the known unknowns, the unknown knowns and the unknown unknowns, if probably got that wrong. Because I just remember sitting there wide-eyed saying, What the heck did he just like what is happening? This is all gibberish. I don't know what this means, but now getting more familiar with it, I understand what you're talking about.

But I I've I remember very distinctly exactly where I was, what conference room I was sitting in, what chair I was sitting in, because my like I was that little emoji of like my brain just what just happened. Um, I do think, you know, if if anyone has questions about you know that two by two matrix, feel free to uh stick them in uh the comment section. But you know, I feel like maybe at some in a future episode we should probably cover that two by two matrix because it is until you're really into it and can like really break it down. That's a tough concept. Isn't it the Rumsfeld uh Rumsfeld the first guy to throw that out?

Yeah, that's I don't know if he's the first guy, but he was the one who's most prominent for for the interesting explanation of it for you Saturday Night Lifetime. Before we move on, and this is something that we had talked about at the end of the show last week. We do have a comment. Um it says, you know, Chris, put this stuff in Excel. More more people will understand what all of that is.

And so I do think that, you know, while this seems a little tongue-in-cheek, it's probably worth mentioning that you know, Chris, you can definitely um put some of this in Excel and do it with more advanced Excel commands. However, it will take you longer and the room for error is higher. And so a lot of what you're doing is automating those processes. But you know, there is a bit of this that you can do in Excel. You don't have to do this in R.

Yes. And in Excel, if you're going to do that, you would probably end up having to write some custom functions yourself, which you can do. Excel has a really good built-in programming language. So as long as you're comfortable coding in inside of Excel's environment, you'll be able to do uh a fair bit of this. You will we will reach a point fairly soon where you can't, where you do need some kind of statistical language.

Um and that would be now. Uh and that is now. Exactly. Um the next check we want to do anytime um we're dealing with numeric data is look for things uh especially when we when we know we're trying to find something, right? We're trying to find an answer.

In this case, we're trying to find what gets me traffic from Twitter, right? What drives Twitter clicks is the question that we've been working on the last two weeks. There's this concept called linear combinations, right? Uh linear dependencies where a a piece of data is embedded within another piece of data. Um so in this case, engagements, for example, combines retweets and likes and comments and so on and so forth.

And so if you're building any kind of model uh where you're gonna you want to figure out what is is driving this outcome, you do not want linear combinations because it will totally hose your model. And so in this case, we ran a quick check on it. And we see that uh there are none that the software is able to detect. And then the last uh checkpoint is to ask ourselves: are there any um too high correlates? Um things with a correlation, uh a spearment correlation above 0.95, which means the two values march in lockstep, they're essentially the same thing.

You see this a lot with Facebook data in particular. Facebook will give you things like post views, total views, organic views, uh, and stuff. And those are like 90, yeah, well, a 0.99% correlation because they're essentially all the same thing. Um, there are variations on you know some Facebook accounts, but for the most part, they're pretty much the same thing, like organic post-reach and post-reach. Uh, unless you're spending a crap ton of money on Facebook, your numbers are probably going to be fairly close.

If you were doing only organic social media on Facebook, uh, those numbers will be identical. And so we want to eliminate those correlates because again, those that's one of those things that will screw up your model. So, Chris, I know that we're getting to the point where you're going to then put this into some sort of machine learning software. Uh uh, if I recall correctly, a little while back, IBM had basically some sort of a software where you could put in your social media data, and it would in broad strokes sort of predict what would lead to engagement. But you had to know what you were doing.

You had to know, you know, what was contained within your data. You couldn't just throw raw Twitter data in there and say what leads to engagement. That's correct. That's IBM's auto AI. And uh everything that we've done up to this point uh creates a clean data set for prediction.

So you do have the option at this point to be able to say, do I want to do it in a coding environment? If so, you would continue on. Do I want to change it? Languages? Once you've made the data set, at this point you can say, okay, I'm ready to take this data set and do something else with it.

Um, that includes tossing it into IBM Watson, the auto AI software. Because the auto AI software, and this is true of all the software on the market, not just IBMs, is still very naive in the sense that it can do some basic feature engineering, try and figure out what's going on and work towards your objective. But it's it it lacks that human touch. It lacks knowing, for example, which correlates to get rid of. It may highlight, hey, you've got some tightly strung correlates or you've got near zero variables here, but it won't delete it for you because the software is does you know it that the designers have said, yeah, we don't want to go messing with your data uh without your permission.

Um so it can highlight problems, but it can't fix them. Um and so you still need to go back and do all this tuning no matter which platform you you use. If you just try and throw your raw Twitter data in, like you said, it's not going to get anywhere. And particularly the feature engineering, like where we're trying to find topics, like what am I tweeting about? It can't do that at all.

So that's something that still has to be done. So there's still a long way to go for automated AI. Got it. Now, so you're using social media data, and I feel like social media data is that currency that a lot of people understand because it has those general terms of likes, engagements and reshares and tweets and all those things. Um, you know, I'm thinking about other data sets that you could potentially use to do this kind of prediction.

So, you know, for example, you and John have the marketing over coffee podcast. Um, could you do something similar? Um to find out like what is likely to uh cause someone to download an episode or subscribe to the newsletter. Could you do something, you know, along the lines of what you're describing with this Twitter data? Absolutely you could.

So if you were to if you were to go over to um marketing overcoffee.com and you look at uh just things like the show titles, you could even easily detect, like you know, shows, you know, there typically is a you know a topic, a wid, and then a proper noun. Um, and those are the shows with guests. So you could easily differentiate guest show versus non-guest show. It would be interesting to do that classification uh and see like does having a guest predict more traffic or less traffic on marketing over coffee. It would be a good strategic question to ask.

You have the show uh descriptions, right? So you can obviously do the same type of topic tagging uh to see, you know, is this a show about sales? Is this a show about brand? Uh things like that. Um, you have all the things you'd expect, like date, uh, the amount of traffic that the individual post gets, the number of downloads each episode gets.

Uh, and so there's a lot of different ways to do um that kind of extraction. You just have to figure out again, going back to the our exploratory data analysis process, what is the thing that we're trying to find and what are the requirements we need to find it? Makes sense. So kudos, John, for being consistent in your naming conventions. Um that I mean, but in all seriousness, that probably helps this type of analysis go a long way if you're being consistent.

And so if you're thinking about using this on your own content or you know, uh something similar, you know, set yourself up for success when you're developing these things and try to stay consistent with your naming conventions. Because when you try to do this kind of analysis, it's going to be harder to get to the root of the answer because you have to clean out all the junk and normalize everything. Exactly. And again, even with with marketing over coffee, even with podcast data, there's going to be external data you may want to bring in. You may want to bring in, you know, for example, um, with a lot of podcasts, particularly longer running podcasts like marketing over coffee, people aren't going to go to the website.

They're going to go to Apple, they're going to go to Google Podcasts and stuff like that. They're going to search for the show, the show's name in search results. And they're not going to click on the website. They're going to go to the recommended, you know, platform like Google what Google will highlight. Uh, in fact, let's let's try that because I'm I'm curious.

Let's do marketing over coffee podcast. And see, you can actually, you can even see like there's you can play it right from the search, right? So you don't have to go anywhere. Um, Google has already done that. So in a case like this, um, you'd want to augment with uh Google Search Console data just to see how many people are searching for uh the brand name and then how often does the site come up, but you also would want to use an SEO tool of some kind like HREFs just to catch the the number of searches for the brand where your site doesn't come up, right?

If your podcast is named something pretty obvious, like you know, the marketing podcast or whatever, um you're gonna want to track that branded search because your site may not come up at all. And if it doesn't come up, it won't show up in Search Console. So you want that search volume from the uh a more uh broad looking SEO tool. And so you again have to augment some of that data. Which makes sense.

So, John, I hope you were taking copious notes. I expect this report on my desk tomorrow. Um but Chris, to your point of to go back to the Twitter data that you were starting with, there might come a point where the tweet itself isn't enough information. So you then have to augment it with the actual text of the content that you're linking to as well, provided that's what you're doing. If you are only sending tweets with no links and you're not including those keywords in your tweets, well, that's a whole different issue.

Yep. Likewise, um you could also bring in if you have enough of the data, you could be bringing in things like the opens and clicks from your newsletters, right? Because you're I have a newsletter promoting show. Uh, I know John has been doing uh promotion of the show through text messaging, right? So bringing in some of that data, whatever data you can get your hands on, you'll want to bring in and then run these initial analyses, the near zero variables and stuff, get rid of obviously the things that don't matter, um, so that you have a clean data set of all the things that could predict whether or not um you're what predicts your final outcome.

So where do we go from here? We go straight into running in machine learning. Actually, no, that's not true. Um where we go next is a really important part, and is the analytic approach, right? And this is where this is where subject matter expertise and domain expertise within data science really matters because at this point, um, we've done pretty much all this stuff, right?

We have to decide what kind of of model we want to build, right? We know this is a numeric data. Um, so there's you know, four fundamental types of models you can build, right? There's a regression model for numeric, there's classification uh for non-numeric, uh, there is dimension reduction for non-numeric, unsupervised, uh, and then there's just a hot mess of you know of other like smaller models. But fundamentally, we want to know whether we have a supervised uh learning model, we and it's numeric.

So we're gonna run some kind of regression model. Inside that landscape, there's probably a dozen different ways to do uh regression modeling, right? There's ridge and lasso and straight linear logistic, um, there's gradient uh modeling, gradient descent, uh gradient boosted. Which one do you choose? Which one is right for the data?

The answer to that question is where a tool like IBM uh Watson Auto AI will really come in handy because you can and people do uh test this out manually. Uh you run, you build a model against you know 15 or 16 different versions, and you say, okay, I'm gonna try every single model and see which one, at least from the the just the first pass, uh, is able to give me uh something that vaguely resembles an answer. That's pretty inefficient, um, all things considered. It it takes a long time to do that. Um, what you want to do is have machines do at least the first part of that research to say, okay, machine, you go through and at least just tell me which of these 50 models uh models is the best general fit for my data.

Um so if we were to look in in what in Watson Auto AI. Let's go ahead and oh, go away. I don't want to move that tour. I I know how to use it. If you don't know how to use it, then taking the tour is probably a good idea.

It is, yes.27. There's some new features here, I'm sure. But you know, I think Chris, what you're continuing to reiterate is, you know, people want to jump ahead to like the quote unquote fun stuff, which is running the machine learning model, but there's so much prep ahead. And that's really what we're trying to help the audience understand is that yeah, you can jump right to the machine learning model, but you're gonna get, you know, garbage in, garbage out. If you take the time up front, you can continue to run these models over and over again way more efficiently and more cost effectively.

Exactly. So in this case, um, all that prep that we did creates an output file, you know, from our environment. We can then put that cleaned up, ready to go file in any system, right? Python, um, if you want to try and do some of these regression analyses in Excel, you could do that. Um, continue with it in R.

Uh, in this case, uh, I've stuck it in IBM auto, uh, the Watson Auto AI system, and I have, okay, what do I want to predict? Well, we remember from a while back we said we want to predict URL clicks. And so it gives you the option if you want to do additional tuning here. Or you could say, okay, go, and then you go and get a sandwich. Um, and a little while later, it'll it will come back and say, okay, I've here's all the things I've tested.

Here's every possible variation. I'm gonna you know, try and do some new numeric feature optimization. I'm gonna try and do hyperparameter optimization. I'm gonna choose uh a dozen different algorithms, and it will eventually spit out something. I do this with a data set just to see which algorithms it thinks in general are the best fit.

Um, and then there's some additional tuning that I want to do that I know that the platform isn't designed to do right now. One of the things you can do with uh Watson Auto AI is you can then say, okay, send this back to a Jupyter notebook where you can tune the code yourself. Unfortunately it does it in Python and my Python skills are terrible. So I will use this as a diagnostic tool and then go back into R, which is the language I choose to work in, and take its recommendations and use them in R. So I'm gonna I'm gonna stop for a second because you just threw out a lot of terms that I just want to make sure that you know everyone is on the same page about.

So what is a Jupyter notebook? A Jupyter notebook is an interactive environment where you can write and run Python code right within a page. And Python is a programming language similar to R. It's just a matter of preference. They both roughly do the same thing.

No, Python is a much bigger language. Python's a general purpose programming language. So you can like build video games and stuff like that. Whereas R is a statistical computing language. It is is optimized and tuned for stats and machine learning only.

You cannot build a video game at R, right? You can absolutely do that in Python. Python is a much more flexible language. It is easier for beginners to pick up the the challenge I have with it is I don't like its syntax. I am old and I grew up in uh C and C and Python syntax does not resemble that in any way, shape or form.

That was going to be my next question is what is it most similar to in terms of programming? So C and C. If you're familiar with programming languages, then that reference will make sense to you. It is definitely not worth us digging into the details of what all of those things mean. But I just wanted to make sure that there was a couple of terms that were explained that might be newer.

But you're seeing on the screen, Chris, that there's definitely some terms that you've already talked about, such as feature engineering was just on the screen, relationship map. That to me, if I had no idea what that meant, I would say, oh, that might mean the relationship between variables and which variables are likely to influence one another, for example. I might be wrong, but I think that those are the kinds of things that if you go through the tour of auto AI, those things will be explained. And so definitely don't just jump into uh using Watson or any other uh AI machine learning software without really knowing what it is you're looking at. Exactly.

Uh that's and that's the really is the critical point is that it's gonna do a lot of stuff here for you, but you still have to have some general idea of what it's doing. Um, so right now it's picking algorithms, right? So here's here's one that's an extra trees regressor, which is a uh decision trees, uh, and here it's it's got another one of random force, the top two uh algorithms uh that it was going to use to try and classify this data. It's then gonna create pipelines, which are just ways of saying, okay, I'm gonna try making different sets of the data and running tests on those sets to see what they come up with. And then from there, it tries all these different uh techniques to do the feature engineering.

So uh sigmoids, transforms, uh cubes, uh cubes and cube roots and stuff to try and figure out is there some mathematical combination of of mixing and matching the data together that gets to the best possible outcome based on the objective we've given it and the measurement of accuracy that we care about. And again, this is why this tools like this are so valuable, because this would literally take you 10 hours to do by hand, right? Uh and we here we are at only four minutes in. It's already it's already built now on the eighth uh pipeline, and it's doing uh hyperparameter optimization on half of these pipelines. So far, number six is the is the winner so far.

Let's take a look at number six and dig into it. So this is a random forest. Um we have our all of our different uh measures, and then we have our variables, right? And so from this analysis, uh Watson is saying the things that predict URL clicks the best so far are impressions, you know how many people have seen my tweets, um, detail expanse, how many people clicked on piece of media in the tweet, and then URL pro user profile clicks, people who clicked on my profile. Now, here's the problem.

This is on a scale of zero to one, right? It's basically it's almost like a percentage. And like most forms of uh regression, you know, anything below 0.25 is kind of noise. 0.25 to 0.5 is moderate correlation. Um, and 0.5 and above is strong as a strong signal.

All of these are way below 0.25. So what this is telling us is that there isn't anything in this data, even with all the feature engineering we've done so far, there isn't anything in here that's a strong enough predictor that would say yes, you can make a decision based on this data. So we've done a lot of work, but the answer's not there. Right. And so this is something that as you do data science, this is one of those things you have to be aware of.

Sometimes you're gonna you're gonna you know, swing a miss, you're gonna strike out. And in this case, with this data set as it stands, we have struck out. These you know, none of this is is good enough to take to the bank. Man, I mean, three weeks of this build-up, and then this is what we get. So the question is, Chris, and this probably happens more often than not, even if you don't take it as far as bringing it into machine learning, the data that you have doesn't answer the question.

So where do you go from here? What do you do now? We start over. So the qu the the key is this we the goal in the strategy is still good, right? We know we're ultimately trying to figure out what drive what makes uh people click on tweets, right?

So we have a we have a clear goal, we have a clear strategy. We've collected a reasonable amount of data, we've done a classification, we've done our initial analysis, and what we're finding now is that we don't have the data that we need, right? It's there's nothing in this data set that fulfills it. So our requirements uh means we have to go back to data collection and say, okay, what other data could we bring in? Or what other data could we engineer that would answer these questions?

So earlier we were talking about what about the text of the articles themselves? Maybe if if even if the the tweet doesn't have our keywords in it, maybe it's the you know something in the articles themselves. There's things like we appoint parts of speech that we have not brought into this at all. So, for example, how long is the tweet? How many words does it use adjectives or adverbs more than other parts of speech or things we could engineer into this data set from the data that we have?

You know, for example, are there if if you tend to use a lot of uh adverbs and adjectives, you might be using more descriptive language. So does descriptive language matter? Um, we all know those wonderful clickbait headlines, right? You know, five things that uh predict this or it's only one you know one thing will drive him crazy or whatever. It's all language is decomposable.

So we could we could uh extract that information and build that in as well as well. But we also have to acknowledge the possible that the the possibility that this may not be an answerable question, right? At least not with the tools and techniques we have right now. There's possible that there may not be an answer that we can get to today. And after a few more attempts at this if we keep doing this we may find yeah we can answer that question today.

Not with not without you know some some some things that uh data we don't have or data we're not willing to invest in. You could for example uh spend a lot of money and run surveys, ongoing surveys to your Twitter audience say like you know what kinds of topic do you want me to tweet about and stuff. I'm not going to do that for this project because it's not worth it. But uh if it was mission critical we absolutely would do that. So for at the end of a process like this where you completely and totally fail um you have to figure out what's what are the next steps.

And there's also a point from a goal and strategy perspective and this is where you know Katie someone like you as an executive would be involved is say okay guys stop it's a waste of resources right you know at this point you're hitting diminished you're hitting not like diminishing returns you're hitting no returns yeah you are now just wasting money. Exactly well and I think you know, especially if you're looking at social media um data to try to figure out what's working, the thing that you don't have in your data set is how the algorithm of that particular platform works as well. And so is it serving it up based on you know your previous number of engagements? Is it serving it up based on how many followers you have? Is it serving up based on time of day?

Is it randomly serving it up? Is it, you know, like we talked about on our podcast earlier in the week? Is it excluding your information based on some random bias in the algorithm? So that's also an unknown piece of information. So, you know, as you continue to like bang your head against the wall trying to figure out social media in particular, keep in mind that you're never going to have the whole story.

Yep. And this is true of marketing data in general. Unless until the day comes where we have, you know, chips embedded in people's heads or or functioning telepaths, we will not know why somebody made the decisions they made. We'll not be able to reach in their heads. They have to tell us, right?

Um, so again, surveying focus groups, customer advisory boards are all super important. Uh, they're part of the the journey to AI because you need them, you need that qualitative information to fill the gaps. You cannot get that from from what we've been talking about here. And and we can see clearly in in our experiment, there is no answer why in any of this data. We can get to what and we can get to strong associations, but there is no why.

So this sort of wraps up the technical portion of our process. And so in our next steps is we have to figure out how do we move on from here. We have to figure out the so what. In this case, there is no so what. Well, so John, what's your takeaway after all of this?

Yeah, well, it goes back to so many business problems that we have, and that the only way to find out what's going on is to dig in and invest. You know, there's kind of this cultural thing of like never fail with any project. But the reality is there's no substitute for actually digging in and finding the data. And there's value in knowing that you never need to go down this path again. You know, just saying that, okay, we know it's not just we don't want to go to Fiverr and buy 50 million views because that's not going to, you know, increase engagement.

We know that with our best quality audience, it it's not making a difference. So yeah, you know, it's definitely you don't get the accolades, or there's not going to be um an award that you're going to get from some random publishing company that nobody really cares about, but you at least do know, you know, one less place to go hunting. Yep. Um so before we wrap up, Chris, there is one more comment that I think is worth uh noting. Yep.

Uh so might be worth trying dummy variables to classify tweets. What do you think about this approach? So this is this is part of where we were going with that idea about using things like parts of speech. Um so you can decompose. So a dummy variable is when you take something that is a categorical or non-numeric variable and transform it into a numeric variable of some kind.

Um the the normal technique in machine learning is called one hot encoding. Um but in this case, you know, there are multiple ways to get at that. So you we've we tried the topic approach, and that really didn't yield very much the way it results. We absolutely could do things like even just number of nouns, number of verbs, things like that, and and really blow out just the sheer number of variables we could create um from this and see what comes up, right? Um, just to look for ways to to to do that segmentation.

Um, and if you find something, you could also then change analytical approaches, right? So what we've done is regression based, uh, which is we're looking at you know what has a correlation to URL clicks. We could take an approach and do a classification approach where it'd say take the top 10% of tweets or whatever, you know, if you would do a graph of the number of tweets and find the inflection point on the declining curve, you'd classify everything above that point as like top performing tweets, like maybe top 5%, top 3%, whatever the number is, and you classify those as top performers, and you classify everything else as not top performers, and then run a classification algorithm instead of a regression algorithm and say, okay, what do all of the top performers have in common that are is not in common with the bottom performer. So that's a good secondary approach that you could take to try and again classify uh what's going on. But I would go that definitely go do the dummy variable or the the imputation route first.

Okay, what else? How much more can we squeeze out of the language that we've got going here, right? Um because this is my Twitter account. I could look, I could even bring in the number of URL clicks on the individual URLs themselves, right? Because we have our the Trust Insights link shortener and bring some of that data into.

So there is still more we could do to bring in and squeeze out more data. But definitely uh it's a good approach and a and a great comment. I've I think I know who that is, but unfortunately it just says LinkedIn user, but I think I know who said that. We can find out afterwards. We know all.

And I think on that note. The Oracle says it's over. All right. So next week we're gonna tackle the so what? Uh so stay tuned for that.

And then after that, I think we'll probably uh move on to a different topic. So thanks for uh popping in today, folks, and we will talk to you next time. Thanks for watching today. Be sure to subscribe to our show wherever you're watching it. For more resources and to learn more, check out the Trust Insights Podcast at Trustinsights.ai slash TI podcast and our weekly email newsletter at TrustInsights.ai slash newsletter.

Got questions about what you saw in today's episode? Join our free Analytics for Marketers Slack group at TrustInsights.ai slash analytics for marketers. See you next time.