So What? How do I clean and prep my data for analysis – part 2

Well, hi, and welcome to So What, the Marketing Analytics and Insights Live Show. Uh this week we are covering part two of how do I clean and prep my data for analysis. If you missed part one, uh, which we covered last week, which are the basics of um preparing for data analysis, getting organized, you can find that on our YouTube channel at trustinsights.ai slash YouTube. It is in the so what playlist. Today in part two, um, we are covering the data itself.

So we're gonna be walking through an actual data set that Chris has been working on and sort of going through what to look for in your data set, what kinds of questions you should be asking, and based on how you got organized before you even pulled the data, what additional organizational steps you should be taking in order to set yourself up for success. So that's what we're covering today. Um Chris, John, anything else before we dive into the data? John's like, let's go. Take it away.

All right. So as you recall last week, uh, one of the first steps in the process is figuring out what our goals are, right? What what is it we're trying to prove? Um so I was uh actually writing a uh blog post about uh someone was asking how we do social media strategy planning. And the first thing came up to mind was okay, well, we kind of need to know like what's working.

So before we talk about data prep and data collection and and some social media data, we gotta ask ourselves, what should we be caring about? So a while ago, um, we ran some customer journey analyses. These are attribution models. Um this is on my personal website. I wanted to compare 2020 versus 2021, looking at social media.

Now, obviously, Google and email, far and away the the things that drive a lot of conversion on my website. So we we can give that a miss. But in 2020, um Twitter was sort of you know the fourth most useful source, followed by Medium, then Facebook, uh, and then going down here, uh, you have YouTube down the middle and LinkedIn. This, by the way, is why I stopped doing a daily um YouTube show. Because when we look at 2021, uh it's still you know generating almost no conversion.

So even though it's fun, you know, uh it didn't really move the needle for conversion. What we saw in 2021 is uh if you look, Twitter generated about nine conversions uh in January of last year and 11 this year. So even though it is proportionally a smaller piece of the pie because email is doing so much better this year, uh it's still generating good conversions. Meanwhile, Facebook and LinkedIn are down in the middle of the pack. Like, uh, well, this isn't really working for me.

Um so the question that I had and the the data I want to dig into is well, okay, Twitter's at least doing something, right? It's uh it it it's in there. What what could I be doing on Twitter that's better? Uh how could I improve it? So Katie, when you're facing a situation like this, and you look at data like this, what comes to mind?

How are you how do you start thinking about framing out the problem of how do I make my Twitter better? How do I make my Twitter better? Um, I like the alliteration. So the first thing I you know need to look at is what the heck am I currently doing? So because you want to take a snapshot of the current state, uh, because in order to measure what you're doing better, you need to know what you're changing, and that gives you an opportunity to, you know, change one thing at a time.

I think that's one of the mistakes that you know a lot of people make that we've made historically is trying to change too many things all at once. Then you that how you measure kind of gets away from you because you don't know what exactly it is that's working. So if you're looking specifically at your Twitter account, Chris, the first thing I would ask you is how active are you on Twitter? You know, how many times a day are you posting? Is it consistent?

Is it random? Do you have a current plan that you could look at to say, okay, this is what I'm currently doing. Where in this could I be doing better? Have I measured my engagements? Am I growing consistently with followers?

You know, what are people doing? What things am I posting? What specific posts are actually converting? So there's a lot to dig into before you even try to start changing things. Exactly.

And so when we start talking about those questions, we're really talking about requirements gathering. What kinds of data do we need to be able to answer these questions? So when we go into uh the data, uh, let's go ahead and pull the actual actually pull up Twitter first. So when you go to your Twitter account, if you go to analytics.twitter.com, you can export all the data that you see here. And what you get out of that is a pretty okay, somewhat helpful spreadsheet.

You have the ID of the tweet, you have the tweet link, you have what you said in the tweet, and you can see this whole bunches of stuff in here, the time of the date, the number of impressions, which is a number, by the way, it does not show up in other social media monitoring tools. You have to get it directly from Twitter itself. Uh, your engagements, your engagement rate, and then you have all of Twitter's appended data. So the number of retweets to that tweet, replies, likes, user profile clicks, URL clicks. So people who click the link in the tweet itself, detail expands.

So if there's a piece of media attached to the tweet, like a photo or a video, someone clicks on it to look at it in greater depth. Permalink clicks, uh app opens, app, and so if you have app-based tweets, emails, if someone's emailed a tweet. If there's a video attached, you can see the number of media views, the number of media engagements uh on it. And then uh if you've been paying for it, you get all the same data for paid. Now, here's the challenge I run into with this.

And this is where we start getting into a data prep. There's a there's a bunch of things that you just talked about, Katie, that aren't in here, right? So we have the tweets, but we don't really have a change in followers per se. Um we do have uh all the different types of engagement. We have the time, we have the impressions, um, but I want to know how to make Twitter convert better for me, right?

So ultimately, the the measure I'm gonna have to go off of is URL clicks. Because if I tweeted something like a link to my website, I want to know if you clicked on it, right? So that's sort of my objective. So one of the first things we have to figure out anytime we're looking at a data set is what's our response variable? Like what's the the outcome that we're looking for?

And if the outcome's not in here, you can't really do an analysis. Well, and Chris, I know that one of the strategies that you employ is you tweet actually tweet out, and you know, we all do a lot of uh third-party content, so content that isn't yours. So as you're going through this, would you need to exclude those tweets and only be looking at the tweets that go to your web properties? It'd be nice to do that. You can't do that though with this data, uh, at least in its current form, because as uh folks who know who use Twitter, Twitter runs everything through its own link shorteners.

So every link is going to be the t.co links uh in here. So that's something that to keep in mind of stuff we'd want to engineer. But there's other stuff that uh we'd want to create as well. That again, Twitter doesn't include in the data set. We have to do what's called feature engineering.

We have to take existing data and turn it into other kinds of data. So uh looking at the data, what things could we engineer that might help us understand? Yeah, actually, that you know, that might be a factor that that could boost you know, clicks versus not clicks. Um, you know, one of the I think one of the quote-unquote easier things to feature engineer, and again, feature engineering is not something that should be scary and unapproachable. It really just means, you know, taking your existing stuff and sort of picking it apart to smaller uh piles of information, at least in this context.

So I would probably start with some of the date and time data, you know, and break it down a little bit more granular, you know, because I can look at the dates, but then I also then have to look at a calendar and say, what day of the week was that? Does that make a difference? Or what time of day was that? Does that make a difference? And if that all of that data is contained in a string, then I would want to pull that out into individual pieces of data.

Yep. Um, other things that'd be pretty straightforward. Uh, how many handles haven't mentioned? Like here I've been, you know, uh at mentioning ring central. Uh, here I've mentioned our our friend Jeremy Aoyang.

This is a chat that uh a Twitter chat that was happening. Uh can I engineer out the number of hashtags used in a tweet? Absolutely. You can just do raw counts, how many hashtags were used in this. And again, something that we might be able to say does the number of hashtags used in a tweet helpfully predict the like that somebody clicks on on that tweet.

Um if you look at the syntax, any tweet that begins with uh an at symbol is a reply. So we could code a tweet as to whether it's a reply or not and be able to differentiate uh that. And I think critically for a lot of social media managers who are struggling with their content strategy, uh, the big question is what content are you sharing? What content should you be sharing? So if we look at this list here, uh, and this is just uh one month of tweets.

I I tweet a lot. So there's a lot of data to go to. And you can see there's different topics in here. We've got you know, top three podcasts from marketing majors, uh, AI algorithm using the heart rate uh and motion to predict age and and gender, uh promotional tweets uh for for trust insights on 2021 marketing plans. There's a lot of information in here.

One of the things we want to do is we want to boil this down uh from text into into numbers. So one of the steps that we would take to do this is to basically take all of the the copy of all the tweets and get a list of the the number of words used, right? Uh which individual words uh have been used in those tweets. So let me show you an example of how that would look. Let's go here.

So let's go ahead and take our tweets. We're gonna bring them all into uh a big data frame. Now, this is in the programming language R. You don't need to use this. I use it because it's convenient.

Um, it's it's easier for me to process the data. You could do this in Excel and then with some third party things on the web. But we're going to go here and take all those tweets. I'm just going to make a frequency list. And this frequency list is just the number the number of times any given word appears.

So I use marketing 161 times in this pile of tweets. Use the word data 112 times. Shows up 45 times. We see some of my US guy answer stuff. We have business, so we have data science showing up, podcasting.

And so this gives me a good starting point to try and figure out how many different topics are there that I'm tweeting about. And so what I've done is I've gone ahead and said, okay, I want to count how many times did I mention social media and all these different words in social media like Twitter, Facebook, uh, TikTok, Instagram, et cetera. How many times do I mention SEO stuff? How many times did I mention data, data science, trust insights, podcasting, Google Analytics? Uh, and then engineering how many was there a URL in there, uh, uh an ad handle, uh, is it a reply?

Uh, how many hashtags? And so this process is the is the data preparation process to go through and understand what are the things that I might want to predict on, what are the things that I might want to run an analysis on and just even doing something as simple as getting raw counts, like, oh, okay, I you know, tweet about marketing way more than data. If I did that and did those counts, I could say, huh, is that what I meant to do? Like, did I actually mean to be tweeting so much about uh you know marketing technology when I'd like to be known for data science? They're they're different things.

Let me ask you this question, Chris. And maybe it's just a matter of depending on what your goal is. But so you're looking at raw counts based on frequency of words used in the text of those tweets. Would you recommend that someone look at you know a different metric first, like number of engagements or number of retweets, pull that information, and then look at what the context of those tweets were. So that goes uh back to the original data science lifecycle, uh, the data prep lifecycle, where we have to know what uh the what our strategy is, what is what what method are we going to be using to judge this outcome.

In this case, um, we've already selected URL clicks as our response variable, and we know we're going to be doing a regression analysis to say, okay, what things uh have the most the strongest mathematical relationship to this outcome we care about. If we didn't have that outcome, then yeah, you could actually do a whole bunch of mixing and matching and explore uh what are some other possibilities. It might turn out, for example, that uh in my if you were to do it um either with oh, you could do with the regression analysis. Um engagements might matter a whole lot uh to URL clicks. We don't know yet.

We haven't we haven't figured that out yet because we haven't run that part of the analysis. Um right now we're just trying to prepare the data so that it can even can be used in analysis. And that's that's it sometimes it feels like you're putting the cart before the horse, but it's one of those things that's very iterative. Well, and I think that you know it goes back to to your point of what we were talking about last week with that lifecycle, is the one of the first things Chris you pointed out on this live stream was that I asked a bunch of questions that the data can't answer because that data does not exist in that data set. And so those are the types of things that you would want to sort of write down and figure out what data do I need in order to answer those questions.

Because if you know, I say, Chris, I want to know the number of followers month over month that we grew, and you hand me, you know, the export from that Twitter, I'd say, I can't answer that question. And there's this, you know, that's then just sort of wasted time and wasted resources. You know, there might be a different kind of data to look at or a different way to capture that information. Um, so it really does, you know, go back to all of the planning up front, which might feel daunting, is going to save you a lot of time, especially when you start getting into um expensive resources who know how to use programming language languages such as R. Um, that's not an inexpensive ask.

It's definitely not. And the techniques that you need to do the analysis, um, in addition to the computational costs, you have the expertise costs. Um, because in order to run this particular model the way that I've set it up, you have to, you know, if we'll just a quick scroll down here shows you just how much more goes into this before you actually get to the answer. You know, this is not a small amount of coding and programming to pull this off. So there is absolutely a very large cost to not doing the prep work and then building a model and then finding out the model sucks, um, and then having to redo it over and over and over again.

So, John, you you know, you don't actually use Twitter a lot. Um, I know because I have access to your Twitter accounts. Um, because as a CEO, that's what I get to do is I get to see everything. Um, but if you were, you know, just hypothetically for yourself, if you were to start tweeting more or setting up some sort of a strategy, what kinds of things would you particularly want to be known for aside from you know your partnership with Trust Insights? Yeah, I think you know, the biggest we we always joke about the the cobblers kids have no shoes, you know, the the gap that I'm missing is there's all this marketing over cut coffee content, which could just be recycled every week.

You know, there's no reason the featured episode from two years ago just couldn't get plugged uh and fill that all up. So yeah, you know, one of these days I'll get around to actually doing at least the baseline adequate marketing. And then after that, I you know, I don't know, the channel doesn't seem to fit that well because you know, LinkedIn and Facebook are a little bit better because they give better previews of podcasts or of links that you're sharing. Um, you know, the content it's just it kind of falls to the bottom of the list. The other thing though, and this I'd be interested in what you guys think about this, is that you know, is it worth going to the effort of really doing a drive to get more followers to try and get in front of more people?

Because that's always been a question, you know, if I were to actually do a full on press and campaign to see how much action I could just get happening in that channel, and it would feed upon itself, you know, as you get more people, you start to publish more times per day. Is that a kind of is would it be worthwhile to build that kind of flywheel? Um it depends. It depends on a whole bunch of things. We know though that social networks in particular really do focus uh on engagement.

Their algorithms are tuned towards getting eyeballs to be sticky. And so one of the challenges, and this is this is something that has come back to bite a lot of brands hard, um, is they've spent enormous sums of money building huge followings, and then those followings were not engaged. And so it actually cratered their their Facebook pages, um, Twitter, you know, very similarly, because you can't you can't have you know 10,000 followers, have one person engaged, you look terrible versus having 10 followers and one of them engaged is you know 10% engagement rate. And you know, again, we don't know the back end scoring mechanism, but we can see through uh empirical evidence that the less engagement you have, the less well you do. So uh just on a campaign, just on a basic campaign basis, I would say no, you probably shouldn't be buying followers.

Yeah, and well, it's funny, you know, because I noticed on Twitter there was a tweet from Twitter that was being retweeted like crazy, and it just said uh four likes is a lot. And you and when you think about out among billions of tweets, yeah, actually, if you get four likes, like that you're doing well. That's a really good point. And I think you know, that engagement piece, and that's something that we'll look into with this data, is important because it's not just about people liking it, but if someone asks a question and it goes ignored, you know, are you do you have the time and availability to then be responding to people on Twitter to keep that conversation going on one single tweet or one thread? And so engagement is more than just someone clicking a like button.

Like you have to do work as well. Exactly. So we're at a point now where we've prepared the data, I think probably as well as as it's going to get. Um, because we're now at a point where we don't know what we don't know. Right.

We know we've got this data, we've engineered, I think a decent number of features out of it. But we're kind of stuck now as to what should we be doing next. So in terms of what to do next, if we go back to the life cycle and try to pull this up here, probably should have had this up and running in the first place. Uh let's see, where is my data science lifecycle? Here we are.

We've collected our data. We've taken a look at it to see like what's in here, what dimensions are what metrics, what data types, right? Um, and we've done some very initial analysis. So now we're kind of at the requirements gathering stage. We've actually had to do some feature engineering up front because we talked about it early on, but we know that in the initial data, the some of the required features were s were simply not there.

So the next thing to do in the process is just do a quick quality check, like how much data is the data in good condition or not. So let's go ahead and run this. Well, and I think that again a good PSA. Um, if you've only tweeted three times in the past month and you're looking at trying to do this kind of analysis, you probably don't have enough data. You have to have, you know, there's no magic number, but if you've only tweeted three times, for example, like that's really not gonna tell you a whole lot.

Exactly. There's no magic number, but it's way bigger than four. It's bigger than three, but it's not magic. Here I am, I'll be here all day talking, you know, sample sizes. Oh boy.

This is why you hang out with us, Chris. Exactly. So this is um a type of exploratory data analysis. This is just very simple univariate analysis to look at what's in the box, right? So we're we're taking our our data and asking about what's in the box.

Um, there are some tweets here, for example, that have occur occurred uh more than a few times. They're on repeat, right? So we would want to know how much repetition is there in the data set, and there's actually a fair amount in here. We have our our time stamps, we have our impressions, how many impressions? So we have a minimum number of four, we have a median of 1200 and a maximum of 14,000.

That's a pretty wide range in terms of impressions. John, to your question, uh about like, you know, should you be focusing on uh building awareness. You might want to first just take a look at your data that exists and say, what is the range of reasonable impressions on the tweets that you're sharing? Same for engagements, right? Uh minimum zero, no surprise.

The median six and a maximum of 343. Now, bear in mind, this is on a Twitter account with 95,000 followers. 343 engagements is the maximum, kind of sucks, right? Uh, it definitely goes to show that just because you have a lot of followers doesn't necessarily mean it's a good thing. Uh engagement rate, the median zero.

Um it makes all of us feel better, Chris. Oh, yeah. Retweets, replies, and things like that, the likes. And then we start getting into the stuff that we started engineering. So uh looking at how many, how often is it, you know, talking about topic marking?

Most of these uh topics um have zeros, right? So they're not frequent. So there's there's not a lot in here. So that would make me start to question the predictive power of some of these topics. If there's not enough to even be, you know, the median topic, then A, my Twitter feed is probably not as focused as it could be.

Um, but B, it it might not be a lot of predictive power uh when we run an actual algorithm. Um we look at the number of times we use handles. Um I've one tweet apparently I had 19 handle stuff in it. That must have been a fun tweet to read. Uh I've seen those come through.

Those are those are intense. Uh up to five hashtags and so on and so forth. So even just this very simple uh univariate analysis, just looking across the board at the data set tells us a lot about what's in what's in the data. Um the things that are not in here that I'm glad to see aren't in here uh are there's no missing data, right? So looking at this, there's no file, you know, big piles of NA's.

There's uh if you look in this column here, uh valid, you got all 100%, which is looking good, uh makes me happy. If you had like you know, 20% missing, 30% missing, you probably can't go on a whole lot, right? There's you you're gonna get stuck because you've got big chunks of missing data. And at that point, you have to say, do we need to supplement the data? Do we need to uh do imputation to to try and repair it?

Um do we exclude the missing data? What do we do with it? But in this case, because it's you know a a clean data source, it came out real and nicely well ordered. Question for you, Chris. Would uh so I often see a lot of people tweeting in different languages.

They have keyboards that handle special characters that the standard uh you know English-based keyboard doesn't have. Um would that throw an error into your, you know, the script that you just ran? Or are those things that you would have to account for ahead of time if let's say you were fluent in a different language and so you were constantly tweeting in both kinds of languages? So that's actually a data engineering question. And the short answer, at least in the code that I'm writing, is that everything gets um transliterated down to the ASCII character set.

So you would want to change character sets for your tweets. If you use UTF-8, UTF 16, something that'll accommodate that, and then you um the code would then function exactly the same. But you would need to know that going into it. So if you were doing this on, say a customer service data set where there's the possibility you don't know what languages are in there, then you'd want to probably use something like UTF-80, UTF-16 to make it as maximally permissible as possible for all the different languages. One of which um UTF 8 uh is UTF-8 MB4 is one of the formats that uh is an extended multi byte character set that accommodates emoji.

Um if you use standard UTF 8, emoji don't work. They come in as these weird strings of characters. You have to use that other character set. Uh and again, knowing you're looking at social media data, um, that might be something you want to plug in here. Now we're looking at my tweets.

I don't use emoji in tweets, so we're okay there. Um but if someone are mentioning you, they might. Exactly. If I was looking at social media monitoring data from like TalkWalker, for example, then like, ah, I need to use a different character set. And again, that's an engineering question.

Well, it's an engineering question, but it also goes back to prepping your data for analysis, things that you should probably be aware of if you're looking at social media data, there is likely to be an emoji or two uh thrown in there. Exactly. So we've asked questions of the data set. We've you know obviously dug around, looked at the raw data itself and seen what what comes in the box. And we even started doing some of the the preparations for analysis, engineering new fields, doing quality checks and things like that.

Um Katie, what's coming up next week in this topic? Um next week, we are actually you know looking at um, is the data ready for more heavy-duty machine learning? What do we need to do to get the data uh to that predictive space to you know do more of that uh sentiment analysis, for example, that you can do some of that out of the box, but it's not as good as if you're um doing you know more heavy-duty uh machine learning. You know, Chris, you had mentioned predictive power, those kinds of things. You know, what is it that you need to do to get your data ready for those things?

Yep. One of the things that you have to do to know when to, and I'm don't know if I'm phrasing that correctly or not, is you've got to be able to at least run some quick tests on the data to see if what you what you've got to work with is is even useful. Um and this is part of that requirements verification stage, um, but it's also part of modeling and insight. So uh, like I said, I was working on this data for a blog post, and I've got two different results here of basically how good is the data at answering the question I wanted to ask answer. And there's two different fields here.

Um this one is uh one set of parameters for testing, this is another set. And looking at the root mean squared error, which a lower number is better. It means that the data is less noisy. Um, and then the R squared uh error range, which again a closer to one is better because it means that the data fits better. So far, just looking at these initial results from an initial test model run, they kind of suck.

I the root mean squared error is good. I mean, that's it's it's a pretty low number. But the R squared error is is way below uh one, which means that the data doesn't really answer the question, right? It I asked it what explains URL clicks, and of all the stuff that we've put in so far, none of it's a particularly good fit. Now there's two things we could do at this point.

Number one is we have to question, and this goes back to our goal and strategy. Are we using the right algorithm? Uh we may not be. Uh, and two, if this isn't the case, uh, if we are pretty confident in the algorithm, then we have to ask ourselves are we missing some data? Is there some data that is is not in here?

Like, for example, uh, you were pointing out earlier, followers change uh or the number of followers that would assist the model in building uh a prediction about this. Right now, these numbers are so low, I would not put this in front of a client. I would say, you know what, this is this needs some more work in the lab, because neither of these results I would would to me would be acceptable enough to say yes, bet your strategy on these results. Well, the the R squared, if that gets above 0.5, would you go with that, or you need it to be all the way up to one? Um I would be looking for above 0.5.

Um, more than anything though, uh, because one of the things you uh I can show you what the actual you know files look like. There's all these different variations, so I'd run probably 35 or 40 different variations of all the ways to test the data and then look at the the cohort of them and find okay do any of these uh parameters you know show up as as a as a solid test um and you can see I've done six unit tests so far none of them none of them are are are are reaching the finish line yet and that's all with the same data set all the same data set uh it's at this point it's just it's called what's called it's called hyperparameter optimization it's like uh easiest way to explain is you've got the same ingredients now you're tweaking all the dials on the oven to see like you know different temperature how long you cook it for and stuff like that to try and make a good batch of cookies so far I've made uh six batches of bricks well and this this goes back to sort of the point of this particular episode is what questions sort of should you be asking when you have the data. And so the first question I'm gonna ask you Chris is are you looking at the right data to answer the question. It sounds like you're not yeah something's missing something's missing uh or it may turn out that URL clicks is just the wrong thing to to be optimizing on. Now the from a a a logic and common sense perspective there's really only a couple things that you could re in Twitter data that you could optimize on uh one of which would be um you know the the engagements the impressions or the clicks you know essentially sort of the actions that people take to validate that you would actually have to set up a different experiment.

So you want to take the the Twitter data to your website and then the different numbers inside Twitter and do a regression analysis on that to say, okay, what number most closely recogniz looks like Twitter traffic and then use that as a benchmark. Knowing how I tweet and knowing what I'm doing with my data, I'm fairly confident that in fact uh URL clicks is the correct objective for this particular test, but it could be something we could test and see if there's you know a different possibility. What I I think is the bigger problem is that most of my tweets that have URLs go to the Trust Insights website and stuff. And so you know that test would fail essentially because I'd be benchmarking off my personal website, and I should be using the Trust Insights website as the as the regression analysis. Okay.

So let me see if there's a way to simplify this a little bit because we've talked a lot about regression analysis. We've talked a lot about cleaning the data, feature engineering, bringing it into a system like R. What if someone doesn't have access to those things or doesn't have that skill set? What are some of the things that they could do with, you know, obviously anybody can export that data from analytics.twitter.com. What could they do with just that spreadsheet to try to answer some of these questions?

Of course, you know, going into the analysis, knowing this is what I have available to me. Uh, we covered in last week's episode what tools do you have. And if the only tool you have is Excel, what can they do with that? Some of the basic feature engineering, you can still you could do this inside of Excel. You know, so you can see here I'm looking for specific words.

If I go back into our Excel spreadsheet, I'll just add a column here. That was a much bigger column than I expected it to be. And then I could create, for example, let's go to the top. Um let's call this marketing. I can do uh equals count if in this cell.

And it's basically going to give me a count of the number of times marketing appears in that cell. Right. And I so I could repeat this column by column for all those different topics and engineer the features in here. And then I could do uh using Excel's built-in correlation function, I could do an sort of accurate uh version of that the multiple regression model in Excel. Uh I don't know if I if you can do multivariate as easily as you can in R or not, but you could at least identify you know, column by column, okay.

Yeah, there's there's at least something here. So you could do 60% of this in Excel. I think it is way easier to use either R or platform like Watson Studio if you uh or SPSS or Alter X, any of the the bigger, you know, more expensive BI tools. If you've got the access to those, even like the free public versions of them, uh, you could certainly do uh that type of analysis. Like Watson Studio, you get 50 hours free a month.

Um so you don't have to buy anything from IBM in order to be able to try it out on a data set like this. Which I think is that's really good news for people who are short on budget and resources, but I think that it's important, you know, Chris, as you've pointed out, you've run six different uh tests, and the data set isn't necessarily answering the question. Um, you know, so it sounds like you still need to do all of that prep up front before you bring it to the machine learning models because you could be wasting a lot of those free hours that you have on data that's not going to answer the question. Yep, exactly. That's why you should sign up for multiple accounts.

Well, let's not talk about gaming the system. Let's stay focused on this. Yes. So doing requirements gathering, doing that initial analysis, doing feature uh engineering, trying to come up with uh you know as many options as possible on the first run through is important, right? The the more that you can do that, the easier it's going to be to actually run those models because you still have to do all the tuning and stuff in the model itself.

You want to make sure the data going into the model is as good as it's going to get. Um, you know, for example, uh you could look for the syntax around like, you know, is this a Twitter chat I was participating in? Yes or no, and then code that as well. Maybe Twitter chats work or they don't. But it's important to be able to dig into those questions up front as much as you can and create as much of that engineered data as possible.

It's not easy, but it is something that is an important skill. Well, and you know, to your point, if you're being asked what we should be doing with our social media this year, and the company is looking to hand over a lot of money to invest in social media, you want to make sure that you can answer that question with some data behind it because come the end of the year when nothing's changed and nothing's happened, you know, you're gonna be SOL. Exactly. So you have to do the analysis up front. Well done, brother.

Uh on that note. So coming up next week, we're gonna be doing the more advanced feature engineering if uh appropriate, and then taking a look at what it looks like to um take out some of the features. So we want to talk about cleaning and taking out stuff that doesn't matter because uh we didn't cover that in today's episode, and there's a lot of junk in here too that needs to come out. So we'll that's that'll be in next week's episode. Yep, that'll be part three, and then part four we get to the meat of it.

The so what? We've done all this work. So what? What do we do with it? What do we do with all this data?

Did we answer the question? How do we put it all together? So stay tuned for those, you know, the following two Thursdays, 1 p.m. Eastern, same bat time, same bat channel. We'll talk to you next time.

Thanks for watching today. Be sure to subscribe to our show wherever you're watching it. For more resources and to learn more, check out the Trust Insights Podcast at TrustInsights.ai slash TI podcast and a weekly email newsletter at TrustInsights.ai slash newsletter. Got questions about what you saw in today's episode? Join our free analytics for marketers Slack group at TrustInsights.ai slash analytics for marketers.

See you next time.