So What? How do I prep my data for analysis part 4

Well, hey everyone, happy Thursday. Welcome to So What, The Marketing Insights Analytics and Insights Live Show from Trust Insights. That's our weekly Thursday, 1 p.m. Eastern show, um, joined by Chris and John. This week, we are wrapping up the data prep for analysis series that we've been doing for the past three weeks.

If you missed the previous episodes, you can find them on our YouTube channel, trust insights.ai/slash YouTube. This week we're wrapping it up with the so what of the so what? Now that we've done the analysis, now that we've figured now that we've like figured out what's in the box, what the heck do you do with it? And that's the piece that we just want to make sure that we're covering. So what, Chris?

Well, before we talk about so what, let's review where we've been because it's it has been a bit of a journey, and for folks who are tuning in for the first time, just get caught up quickly. We started out three weeks ago looking in attribution analysis on a website to see what social media channels were working for driving conversions on my personal blog. And we set out saying, okay, we looked at the the sources, we took out the obvious ones, right? You know, email search. And of the channels that were left, Twitter was the the highest ranking channel in terms of hey, this this appears to have generated uh at least something.

Um so that got us asking, okay, well, what could we learn about Twitter? What could we figure out what works on Twitter so that we could do more of the things that work and less of the things that don't work? And so that was our goal, our strategy was okay, what could we take from Twitter data and turn it into something maybe useful? And we went through this process of getting the data out of Twitter, cleaning it up, um looking at for anomalies, trying to figure out if we had the data we needed to answer the question. Then we prepared it, uh, we did a lot of feature engineering on it, and we modeled it.

And what happened at the end of last week's episode was a big great pile of nothing. Um we ended up with an analysis uh using uh IBM Watson Studio that showed there isn't a there, right? There's there's no data available that it was a strong enough conclusion said, yes, this is the thing that is the answer for sure. So we're now left with uh kind of a quandary, which is what do we do? Right?

The so what doesn't exist because there isn't uh viable enough data uh in what we've done. So there's a couple of different choices that we could take here. One we could ask ourselves, did we have the right goal and strategy to begin with? If it, you know, was that even the right question to be asking? What do you think, Katie?

If you've got the uh an attribution analysis that says, you know, maybe see what's going on with Twitter. Um, what do you think? Well, I think you're exactly right, Chris. The first thing I would do if my analysis came up empty is I would go back to my requirements and see if that's where um you know I got it wrong. So, you know, to your point, am I asking the right questions?

Do I care about the right things? And so, Chris, I believe you were trying to figure out, you know, what leads to Twitter engagement or what leads to link clicks, and so some a form of engagement. Um, you know, is that the thing you should care about the most? And do you need to step backward to see where Twitter fits into your customer journey overall? Is it even the right channel to be looking at?

And I think that that's such an important step, because if you didn't do the business requirements to begin with, you have nothing to go back to to say, are we answering the question being asked? Yep. Um, and so it turns out that uh I did some analysis on the side, and it turns out that yes, um, link clicks are probably the best way to judge Twitter's direct impact on marketing because uh certainly we want to make sure that we're we're getting traffic that converts. Um but there is something to, and we've seen this with a couple of our clients where just the impressions do help reinforce brand a little bit. So in this situation, the goal we could possibly change the goal and strategy, and I think that's would be worth exploring maybe in a subsequent episode, okay.

Well, like how would we change our analytical approach? The other thing is if you're pretty sure that you've got uh the right strategy and the right goals, and the data is not good enough. If you've got if you've if you've communicated it well to your stakeholders, you can say, okay, of the analysis analyses that we've done, which one is the least bad, right? Um, what is the least bad and then what do we do with that? Um so I'll show you a quick example here.

Let's go ahead and minimize this and this and bring over our test results. So these are three different analyses that show the uh error rate, the root mean squared error, and the R squared uh error rate. And for those who are not familiar with statistics, uh R squared is essentially uh the how well does this explain what happened. Uh and RMSC is a relative number saying there's a lot of you know how how well does it fit to the regression that you've done. In these are three different models that we've run, and we see two models had closer performance, not great, still not really explaining the model well, but lower error rates.

And so in this situation, as long as we are able to I guess spend some time communicating to stakeholders properly, this is not the answer. The data doesn't support the answer. There's not something we can immediately leap to from here, but it gives us some things to test, it gives us future hypotheses to construct for testing purposes, then you can still make use of this data, even though it's not great. So in this analysis, uh, we have low statistical strength. It says that what causes link clicks, impressions, likes, retweets, detail expands, user profile clicks, proper nouns.

Uh I added some new features last week on all different parts of speech to see maybe it's how I'm saying things instead of what I'm saying. Um URL count, handle count. And what we've got here, some things are directly under my control, right? I can't control uh how many times somebody likes something. I can't control how many times somebody retweets something beyond asking people to retweet my stuff.

Um I can sort of influence detail expands by including like a photo or a video in my tweets, um, because it requires you the detail expand. And I absolutely can control the number of things like proper nouns that I use in my tweets or the number of conjunctions or adjectives. So at this point, we could say, well, let's construct construct some hypotheses. Um, based on what I've just listed out, uh, what would you what do you guys think for uh a reasonable hypothesis to test? I mean, so you covered a lot of ground right there.

So that's I think it's sort of the oh, my mind is a little bit blown by what you've done. So, in terms of a reasonable, reasonable hypothesis, you know, you've talked about you know parts of speech and you know, the way in which you're talking about it. And so I think that starting there, if that seems to be giving you more insight into what makes someone click on something. So, you know, for example, if you say, hey, click on this link, super direct, you know, then does that make someone click on the link versus so I did this really cool thing and let me tell a story, and at no point do you ever say the call to action, click on the link. Does someone click on the link?

So that might be a place to start is with the actual the way in which you're saying it, the call to action itself. Do you say, you know, click here to learn more, click on this link, or you know, do you leave that out and people just naturally take that stuff because they're interested in the story that you're telling? Okay. Well, the other side is data set too, right? I mean, because we this is our house data, our stuff.

Would we want to take another completely different data set and see, you know, is some has somebody else cracked the the code? Does someone else get different results? Or do we see that this is something, you know, everybody's getting similar results across the whole universe? That's a really good point because one of the things that we could check is using certain software like uh social media monitoring software, I could try to pull out the number of tweets of people who've mentioned me and do the do a side-by-side comparison. Maybe other people talking about me uh is more influential than me talking about myself.

Um now there is there's another thing that's really important in here, and that's that there's some missing things. One of the things that we we're not able to get, uh and I actually tried doing it early, and uh I've run into some interesting data problems, is that we never validated just how often am I sharing URLs to my own website? Because if I never share them, then of course I'm not going to get any link clicks, and therefore the analysis is actually kind of pointless because you're trying to measure something that doesn't exist. On the other hand, if all I share are links to my website, then it's obviously uh that would be a more cogent analysis. In this case, I don't share enough of my website.

Like I share, you know, maybe one or two links a day because everything else I'm sharing is either uh curated content or stuff for the Trust Insights website. And so already the analysis is that's part of the reason why it's so thin, is that I'm not promoting myself enough. I need to be more of an ecomaniac. Oh goody. When we do hypothesis testing, one of the things that people screw up the most by far is multiple condition hypotheses, which is a terrible idea, multiple variable things.

So trying to do too many things at once. In this case, um, an easy thing that would be under my control that uh you know has a real six percent relevance to uh the outcome that we care about is the use of proper nouns. So I could set up a basic hypothesis if, because hypotheses are always if then statements that are approvably true or false, if I increase my usage of proper nouns by 20% in my tweets, then I should see a 20% increase in the number of URL clicks. That's does it seem like a reasonable hypothesis? It does.

Um, you know, it'll be interesting if you do that to see how you start talking about yourself in the third person. Um, and so I can see why that would be, you know, like it makes sense as a hypothesis, but then in execution, you know, you have to think about what that looks like because if you're tweeting from your account about you and the third person, well, people think you've lost your mind. I people already have pretty sure they have lost their mind. But the proper noun doesn't have to be me, it can be any named entity. So one of the things that this then opens up for questioning is am I sharing the right kind of content, right?

Um, should I be creating content that does name check, you know, or name drop uh things more often? That would make logical sense. So, like for example, right now I'm in the middle of a working on a blog post on how Google works, right? Um, and so I've got this gigantic huge long blog post that's talking about like how Google collects information. I should be thinking about even in my content here, do I need to how should I name check Google in here?

Because Google's a proper noun um and you know should I be increasing the number of these in the headline because the headline automatically gets fed into our Twitter posting software to increase the uh the content so even something as simple as you know which a part of speech in an analysis like this could lead to sort of a ripple effect in our content strategy well you know and it's interesting because when you say proper name my first thought is are you you know calling out your friends to get them to then click on the link or are you using their handles to tag them so if you're using that example of Google is it enough to just mention Google in the tweet itself or do you also need to uh include their handle and those are two different tests. They are in fact line 10 is the number of handles in in a tweet and that has less relevance so at least according to this this statistically shaky analysis um compared to the just the use of the proper nouns in general um the other thing that really surprised me was that things like um replying to people uh stuff didn't really seem to have an impact you know so you you would think uh one of the things people love to talk about with social media marketing is it's all about engagement it's all about conversation you know return on uh what is it return on engagement return on conversation return on you know basically anything that's not a the actual ROI formula and in this case for my data the way I use Twitter with the caveat that I'm you know, everybody's unique and gonna do things differently, those conversations don't seem to play a role. The other thing that I thought was really strange and is an avenue for investigation is I went through it, we went through uh two weeks ago and we tagged our topics, right? You know, the tweets about marketing and data and social analytics and stuff like that. No impact and stuff.

So one of the questions we have to ask, and this is going right back to our exploratory data analysis is are we even talking about the right things on social media? Right? If it's not generating results, there could be a number of things really wrong here. Well, and for me, it brings up another question of so your audience size and who makes up that audience. And so, are you followed by you know a bunch of random people who don't care about your personal website, your personal brand, or are you followed by the right audience?

And does the size of the audience matter if they're highly engaged? I think one of the things that we've talked about is as long as you have the right people following your stuff that engage with it, then you know, having a million followers doesn't really matter if you have a hundred of the right people. And so, where does that fit into this analysis? It doesn't. And that's one of the challenges uh with this particular data set is that this measures the what happened, right?

And so we have those three what's what happened, so what now what? Um, we're at the so what phase, and things like audience composition are not something that we've we've been you could build into this model because this model is purely looking at the what happened of social media engagement, the who it's going to a very different question. Um, in terms of how you would measure that, uh, one of the things you probably want to look at would be um collecting all of your social media mentions and conversations and things like that from your your monitoring software, and then going through and classifying, categorizing, like, yes, this person, you know, based on their bio is somebody I would want to be engaging with. This person is not. So uh there are tools that you can use that will take uh a set of Twitter handles and append the biographies, the people's bios of that.

And you could then very simply go through and tag anybody who's got you know marketing in their bio, like, yeah, that person's somebody I want my audience. It says, like, you know, you know, Camstar, like, yeah, nope, uh not not interested in that in that kind of person. Um based on that, you could work out an audience composition, but that goes back to our original question, which is do we have the right goal and strategy? And part of that is you know, knowing what's not going to be in the data and will never be in this particular data set. I would argue with you a little bit on that.

I think that the goal and strategy isn't the thing that changes, it's your data requirements that changes in order to answer the question. And so, you know, if the export from Twitter straight out of the box isn't going to answer the question about audience, for example, then what other data can you pull that would answer that question? So the goal and the strategy doesn't change, the uh requirements for your data changes. Um, and so I do think it's interesting that because now we're talking about a different kind of analysis. You're talking about your, you know, for lack of a better term, the influencer map of who's the most talked about, and then seeing if uh engaging with those people increases your engagement.

Yep. And that's where you then you start to get into some really, really interesting questions about who is in that audience because uh and what kinds of people they are. Uh, not you know, good or bad people. Um, just you know, for example, when I'm we we operate a you know a bunch of Twitter scripts, and one of the things that we look at from things like marketing Twitter, which is a popular hashtag, is what is the likelihood based on performance that an account is automated? Let's go ahead and sort through this here.

And so there are some accounts that you know we've been able to figure out based on their behavior and performance. Yeah, this account is pretty clearly just a bot, and all it does is is you know hit copy paste in an automated fashion all the time. By the way, if anybody who is watching this video who this is who's Twitter handle, I'm sorry, but your account looks like it's a bot. Um well, you do actually have an account in there online for digital bot retweet. So that is a literal bot.

At least we know the software's working. Um but then you would combine this with things like the biographies that people are providing uh and be able to say, like, yeah, this is definitely something where you know this account is not probably going to add a whole lot of value uh to our audience, right? If it's a bot, it doesn't matter whether it engages or not, because they ain't never gonna buy anything. Or for you know, to play devil's advocate, if you do if you are followed like by an account like DigitalBot Retweet, is the act of that bot retweeting everything that you tweet, then reaching their audience and therefore the likelihood that people might engage with it. Mm-hmm.

Yep. Um when you look in what you get out of Twitter's API, you actually get a decent amount of stuff that you could make those decisions on. So when we look at some of these bios, um, some of these people might be okay, like e-commerce executive, that person might be okay. Uh head of SEO at this company might be okay. Uh person here with uh lyrics from the Bob Dylan's watchtower, maybe not as relevant.

We don't know. But uh this would be the you know a separate analysis on the on who is in the audience of the people who are talking to you or talking about you. And so where do you factor in that influencer analysis of the nearness, like the you know, the those clusters? Um and I know that I've sort of gone a little bit abstract because we haven't talked about that kind of analysis on the show before. So can you just sort of give like a quick rundown of what that looks like?

Um it's called, yeah, it's a network graph. So you're you're building a network graph of who is the most talked about and who does the most talking in in any given environment. Um and then that answers the question who should we be talking to? Um, because when you're if you're trying to get you know an objective of engagement, like likes, retweets, stuff like that. Uh if we went back to this model here, uh, retweets, you know, 10% of our uh uh of what does drive URL clicks.

So um if we were to go out and reach out to some of those influencers, we might be able to say, hey, could you retweet my thing? In fact, um, you know, there are our Twitter pods, groups of people who all agree to we retweet each other. Um technically it's a violation of Twitter's terms of service, but uh it is one of those things that you could do so the next step would be okay well then who would we want to talk about so let's go ahead um see if we can do this without completely blowing up uh my computer and our live stream um and try to construct one of these these network maps we're gonna use uh an open source tool called geffy uh it is uh you can find it at geffy.org g e p h i and we're gonna do let's what do you think case we do uh twitter marketing twitter chats or marketing twitter itself i think the marketing twitter itself i mean it's an interesting uh group of folks it's actually gotten fairly large so if you don't if you don't know what we're referring to there's a hashtag on twitter hashtag marketing twitter that is highly engaged and is working together to create this uh network of people within twitter that are all marketers that you know engage with each other's stuff that help each other find jobs uh promote each other's material so I think that that would be a really interesting place to start okay so we're gonna vacuum in here uh how much Twitter data do we have from marketing Twitter we have 14 megabytes of marketing twitter stuff which is 122000 tweets and not insignificant not insignificant in network graphing, um, what we're looking for is the connections between nodes. So if we were tweeting the three of us, and uh I tweeted something, and then you replied to me, and then John replied to me. In a model like this, we could construct lines like Chris talks to Katie, Katie talks to Chris, Chris talks to John, John talks to Chris.

But if you and John never tweeted at each other, uh there would be no connection between you two. So that's kind of how a network graph works. Now we're going to explode this at a much, much larger level to try and figure out just how dense is this network and who is the most talked about. Now, one of the challenges with influencers and influencer uh marketing is that there's a whole lot of people talking, right? We all know people who just love to talk to the air and you know, and just wax rhapsody about things, but nobody's listening.

Um, so one of the challenges that we run into is can we prune that noise out? Um, the people who talk just to the air and get to the people who are having actual conversations. So if we scroll down here to where this uh particular category of relevancy ends, you'll see there's some people they're going out and tagging and tweeting at 50 people at a time, right? Uh, but nobody talks to them, right? That's just so they're they're all junk.

So we're just gonna get rid of all those people uh because they're not helpful, and uh and we'll do a couple of runs of this model to see okay, can we get to a a density that is actual conversations or actual engagements? Because again, we're we're looking for those engagements. Uh we want to grow our retweets. Let's go ahead and do this one or two more times. I would also like to point out that you have inadvertently created fictional drama between John and I, because in your in your story, John doesn't tweet at me at all, and now we are going to fictionally have to fight about this because why does he tweet to you but not to me, John?

This is I was reading somewhere saying that like this never really exists. Like this this is something they actually use for fraud detection because that would never happen. Like if I'm always talking to Chris and you're always talking to Chris, we will talk to each other. That's just like that's the way humans work. I think that that's really interesting too, because you're absolutely right.

Like you can see when something looks completely fishy because it's an unlikely scenario of things to happen. But we're gonna fight John. We can fight. Well, as we're looking at the ant diagram here, it's it's obvious we're gonna have to do some soft shoe to let the let the math run. So go ahead.

No, go ahead. No, please. I was still clicking things. Well, so I was gonna ask, so you know, um, I know like I do a lot of automated posting through Twitter. So I use the um content curation service that we have at Trust Insights, but in the past couple of months, I've tried to tweet a little bit more like not live tweet, but tweet like in the moment, something that isn't pre-scheduled to see if I can get more engagement, more followers on my account.

And I know, John, you're you're not a heavy Twitter user at all. So when you're sort of like seeing all this analysis and conversation, what does it make you think about your own social media strategy? Does it change anything for you? Well, that there's it's funny, we're in this time where there's such dramatic sea change with what's going on with this. And I think the a huge thing that gets missed with all of this is that both Twitter and Facebook have made this transition from what was originally thought to be social networks and conversation networks, and they've really just become publishing platforms.

You know, they've just become platforms that are almost broadcast. I mean, the fact that you can respond to a tweet, you know, when you look at the top 5,000 Twitter accounts, if you respond to them, you're not going to hear back. You know, the Kardashians aren't going to talk to you about your opinion on their shoes. They just have no interest in that. So it's a real shift.

And yeah, I I still treat a lot of these social channels as social. You know, I'm putting stuff out there, and you know, I if I get five replies or five likes, that's a huge win because I am actually talking with those people and getting, you know, feedback from them. And I mean, even think about some of the nodes on here that we're looking at, right? I mean, some of them are fully automated. So yeah, they look like they have a lot of impact on the map, but they're just retweeting.

And so that creates another user case too of people that follow these retweet tweet bots. You know, is that a thing? Are there people that follow those feeds because that's how they learn about new stuff? Um, and yeah, I always love how watching these things go from just a field of stars to Chris dials in to find that perfect focus, whereas suddenly now you can you can you can see who the winners and losers are. So a little bit of background uh for people who don't follow marketing Twitter, aren't familiar with what we're talking about.

So the largest note in there is that Christina G. And so the whole marketing Twitter movement was actually started by Christina. Um, and you can see it's her pinned tweet uh from December is if you have less than a thousand followers and work in marketing in some capacity, introduce yourself to marketing Twitter. And that tweet I continue to see every single day people engaging with, retweeting it, responding to it, um, introducing themselves, becoming part of that marketing Twitter hashtag to the point where there's a marketing Twitter hashtag on Peloton. They've started subgroups of people who work out together, they've done meetups, they've done well virtual meetups, I should say.

They're actually now talking about a marketing Twitter um event in different cities. And so, like, it's blown up to this big thing in just a matter of what, like 10 weeks. Yep. So we've constructed our map and we can see uh, you know, as John was saying, there's some definite winners and losers. Uh well, I should say more prominent notes, people who are talked about more in the marketing Twitter conversation.

And it makes total sense. The founder there join Clubhouse, the audio platform talked about. So whether or not we like it, it is being talked about within this context. Um, in fact, if you look at the spreadsheet version of this of that lovely chart, uh you can see Join Clubhouse is number three on the list of of accounts that people are talking about. And so we want to uh to do consider you know taking a look at that, but also having conversations with people who are in that space.

So what's interesting, because I do see this in my feed, the majority of people talking about about join clubhouse are complaining about the number of invites that they have to give away. And then apparently at one point people were trying to sell invites to Clubhouse. So there was a lot of conversation about that. And I think I saw um Jennifer Yanolo yesterday, for example. She was uh complaining that she got rid of all her invites and they gave her more to give away.

And it became this like sort of sarcastic thread of like, enough, I can't give them away. That's excellent. So when it comes to then sort of this idea of influencer marketing, um, we want to use technologies like this to identify who are the people we need to be talking to. Now, here's kind of a fun power tip with this. Christina, um, if she was somebody like, say, a Kardashian, and no offense, Christina, I'm sure you're well on your way.

Um, but because we can identify people who are in her neighborhood, not physically but conversationally, if you can't get a hold of Christina, if she's just too unobtainable, you can take that group of people that she uh associates with most closely and see who's next in line, right? This here is um Michelle Garrett, right? Um so if we can't get Christina's attention, maybe we can get Michelle's attention, maybe she's uh a more accessible person, and so now we're starting to dig into how these network graphs actually come to life. This is the so what of a network graph. It's cool to look at, right?

We everybody loves these visualizations. Um, you know, we do them for conferences, but the the meat and potatoes is the so what who do we want to talk to within Christina's sphere, this modularity number that could get her attention. And then that goes back to the original goal and strategy of how do you, Chris, get more clicks on the links in your Twitter posts. Well, it looks like perhaps just the data from Twitter wasn't enough. So we took it a step farther with that influencer marketing to see who could you be engaging with in order to get your stuff shown even more.

Exactly right. So this um sort of wraps up this overall process and how we've how we've used it, what's gone wrong, which I think I would actually argue that having the analysis fail on us was probably one of the best things that we could have demonstrated, albeit inadvertently and certainly not intentionally uh set up to fail, because it shows the incompleteness of the data that we were working with to solve this marketing problem. And now we're at a place where okay, there might be some additional solutions and at the very least um some ideas. And we used the marketing data that we had available to us combined with uh the ideas of you know what can what else could we extend it with to try and solve this challenge and figure out what was missing, figure out what we didn't know, uh, and give us an experimentation plan. And so the takeaways here do this process, obviously.

The we've covered the last four weeks, have some ideas about what could go wrong, and when you have something go wrong, obviously try and fix it. And if it turns out, like in our example, there wasn't a fix in the data itself, then start to ask, okay, well, what are the things that we could test, knowing that the underlying statistical model still isn't great, but we could test it and then build hypotheses and testable experiments on top of that. So I guess the big question, Chris, is what are you partially going to do about your Twitter account? Absolutely nothing different because I don't care enough about it to fix anything. Email marketing is 100% more effective for me.

No, it's actually, you know, 10x more effective for me based on the numbers. And so all of my uh efforts and focus and time go into making my email uh newsletter as as valuable as it can be because that's where the money is. Sounds right. Um, you know, so if you want to learn more about you know this process, definitely check us out on our Slack group, um, trustinsights.ai slash analytics for marketers. Hit up Chris on Twitter, he may or may not respond to you.

I'm a bot. Chris is a bot. Well, you know, it's interesting. You do have uh a retweet bot set up. So anytime your handle is tagged in something, it automatically retweets it.

Yeah, my account. I'm having the real conversations, unless you're trying to get to Katie because I don't talk. We don't talk at all. That's going in your performance review, sir. All right.

On next week's show, uh, I believe uh Katie and John will be talking about podcast advertising. I will be overdoing a LinkedIn live show, so I won't be around. But uh if you want to learn how one of the longest running podcasts in marketing, marketing over coffee, 15 years running, uh has managed to stay uh successful and profitable uh with uh paid sponsors every quarter for 15 years. Uh you're not gonna want to miss next week's show. Uh we'll talk to you soon.

Talk uh take care and stay safe. Thanks for watching today. Be sure to subscribe to our show wherever you're watching it. For more resources and to learn more, check out the Trust Insights podcast at TrustInsights.ai slash TI Podcast and our weekly email newsletter at Trustinsights.ai slash newsletter. Got questions about what you saw in today's episode?

Join our free Analytics for Marketers Slack group at TrustInsights.ai slash analytics for marketers. See you next time.