Phil asks, “How do you determine a large enough sample size for things like our survey? I always thought 10% sample would be enough, but you seemed to think that’s not true?” It depends on the size of the overall population. The smaller the population, the larger the sample you need. It also depends on…
Summary In today's episode, I break down how to determine the right sample size for surveys and why a flat 10% rule rarely works in practice. Here's what this means for you. You'll stop wasting money on surveys that produce unreliable results and start making decisions backed by statistically sound data. You'll also learn these…
Jessica asks, “How would you differentiate hypothesis formation and searching for relevant variables WITHOUT “data snooping”?” Data snooping, or more commonly known as curve fitting or data dredging, is when you build a hypothesis to fit the data. The way to avoid this is by using evidence not included in the dataset you used to…
Summary In today's episode, I show you how simple spreadsheet statistics like mean and median can reveal actionable insights hiding in your marketing data. Here's what this means for you. You can use these two numbers side by side to judge whether your campaigns are actually working without any fancy analytics tools or advanced degrees.…
Courtney asks, “How many web sessions (or users) would you need to send to a website for the results to be significant?” At first, I thought this question was about A/B testing, which has a whole range of statistics for it. But Courtney is actually asking about how many visitors you need for web analytics…
Summary In today's episode, I break down how much web traffic you actually need for statistically meaningful web analytics, separating the question from A/B testing and getting to what really matters for marketers. Here's what this means for you. You'll discover that tracking improvement in your own metrics month over month delivers far more value…
Maggie asks, “Would you use CHAID analysis to discover relationships between multiple categorical predictor variables? For example, using GA dimensions (traffic channel, device type, country, landing page, etc.) and group them according to their likelihood to convert. What else can be used for a similar analysis?” Maggie knows her stuff when it comes to statistics.…
Summary In today's episode, I explore CHAID analysis as a classical statistics method for uncovering relationships between multiple categorical variables, compare it to decision trees, and explain when modern machine learning approaches like stacked ensembles outperform it. Here's what this means for you. You gain a clear framework for choosing the right technique based on…
