Almost Timely Archives - Page 3 of 19 - Christopher S. Penn

Almost Timely News, August 25, 2024: 🗞️ How to Make Generative AI Sound More Like You

Almost Timely News: 🗞️ How to Make Generative AI Sound More Like You (2024-08-25) :: View in Browser

👉 Enjoy my latest on-demand free webinar, Generative AI for Hospitality!

Content Authenticity Statement

80% of this week’s newsletter was generated by me, the human. You’ll see lots of AI outputs in the opening section, plus an AI summary. Learn why this kind of disclosure is a good idea and might be required for anyone doing business in any capacity with the EU in the near future.

Watch This Newsletter On YouTube 📺

Almost Timely News: 🗞️ How to Make Generative AI Sound More Like You (2024-08-25)

Watch this video on YouTube.

Click here for the video 📺 version of this newsletter on YouTube »

Click here for an MP3 audio 🎧 only version »

What’s On My Mind: How to Make Generative AI Sound More Like You

This week, let’s tackle one of the most common complaints about content generation using generative AI:

“It doesn’t sound like me.”

Why doesn’t AI sound like you, and what can you do about it?

TL;DR

Don’t feel like reading this week’s issue? Here’s the AI-generated summary, summarized with Google Gemini and complete with very AI-sounding content.

This issue of the newsletter explores the mechanics of how AI writes, emphasizing the influence of training data and the prevalence of common, and often bland, language. It delves into the complexities of writing style, highlighting the numerous components that contribute to an individual’s unique voice. The newsletter then provides a step-by-step guide on how to prompt AI to emulate your style, involving detailed analysis, scoring rubrics, and iterative refinement.

Now, if you’re ready to dig in, grab a beverage, your laptop or computer, and a comfortable chair.

Part 1: How AI Writes

Let’s go back to the basics to review how generative AI systems – language models in particular, like the ones that power ChatGPT and Claude – work. These models are built using massive amounts of text data. As I mentioned in last week’s newsletter, the average model is trained on 7-10 petabytes of text data. That’s a bookshelf full of text-only books that stretches around the equator twice.

Every word is broken down into small word fragments (tokens), and then each token’s position and frequency is measured against every other token, word, phrase, sentence, paragraph, and document. What you end up with after this extremely time-intensive process is a giant database of statistics.

When we prompt AI to generate text with naive, short prompts, it naturally selects the highest probability tokens and assembles them into words. Here’s an example from a LinkedIn post I wrote the other day.

LinkedIn image

You can see that in this naive example – the prompt was, “continue this LinkedIn post” – it’s choosing the most likely word pieces to assemble.

Here’s the problem: by definition, the most probable words are also the most common. And the most common words are… well, boring.

Take some of the hallmark words of unguided AI-generated text like “delve”, “absolutely!”, and “embark”:

Delve chart

Why are these so frequent in the text AI generates?

Well, because they’re frequent in the text AI was trained on. Think about it for a moment: what’s one of the largest sources of publicly available, free text data? Marketing content. We’ve been generated marketing content at a blistering pace for 25 years, ever since the dawn of SEO and search engines. Company websites are filthy with terms like “embark” and other corporate mumbling. And all that has filtered into the content AI generates.

Here’s an additional gotcha about this kind of language. Remember that AI generates content based on what it’s already seen in any given conversation. If it can tell just from our prompts or a couple of paragraphs of sample information that we’re working on marketing content, the probabilities it chooses will be from… marketing content it’s already seen.

Cue the tsunami of “delves”.

Delve occurs in the style of text we’re writing; if you give generative AI a different kind of starting document or prompt, it’s going to come up with different high-probability words. Give it some purple prose, and you’ll see an avalanche of words and phrases that commonly occur in fan fiction (one of the largest sources of text data from sites like Archive Of Our Own).

Here’s an example. This is with a very short prompt:

Gemini WLW fiction snippet

Not a delve in sight. Why? Just from the prompt alone, the language model understands that this isn’t marketing content. Delve isn’t going to show up because in the context of this language, it’s not common.

That’s a key point we’ll come back to. Language models choose words based on the context of the words that have been in the conversation so far.

If we want less boring writing, we need to instruct the language model to generate less common words.

Part 2: Writing Style Basics

What is writing style? How do we know what does and doesn’t sound like us? This is a more complicated question than it first seems, because writing style isn’t a thing. Writing style is an umbrella concept that rolls up dozens of sub-concepts.

For example, diction – word choice – is a huge component of writing style, and one that sets up apart from other writers (including AI). Every individual person’s diction is different because we’ve all been exposed to different language, different words. The words you remember are tied to your emotional experiences in life; even if they’re broadly common words, the order in which you remember them is unique to you.

Don’t believe me? Can you remember the words someone you cared about said that made you fall in love, or broke your heart? Probably. You might even remember the tone of voice they were said in. But that sequence of words is seared into your brain and when you write, it’s part of how you select language. If you’ve ever tried your hand at writing fiction and really gotten into it, you’ve probably noticed that little bits of language that you’ve said or had said to you crept in.

Diction is just one of more than two dozen pieces of writing style, like pacing, tone, clarity, conciseness, sentence length, paragraph length, etc. – all things that identify you individually, like a fingerprint.

This is why it’s so hard for the average user to prompt AI well to write like them. If you can’t articulate the fine details of your own writing style, there’s no way you’re going to be able to tell AI what to do.

Part 3: Prompting AI to Write Like You, Step by Step

So if we want AI to write like us, how do we do that? We have to create a writing environment that helps AI understand what it’s supposed to be doing. To accomplish this, we need samples of our writing – and more is better. Ideally, we have samples of our writing in the context we want to use AI for. How we write an email is different than how we write a book, or how we write a LinkedIn post, or how we caption an image on Instagram.

More is better. A lot more. How much more? AI is going to struggle capturing your style from just a handful of blog posts, in the same way that a ghostwriter would struggle to capture it. If you had 20? That’s better. A year’s worth? Even better. When I build prompts for writing style, a year’s worth of content is good, two years is better.

That’s the bare ingredients. Second, we need a language model that can hold that much information in memory. A year’s worth of content might be a couple hundred thousand words, depending on how prolific you are. A model like GPT-4-Omni, the model that powers ChatGPT, can only remember about 90,000 words or so. If you have more than that, you need to move to a larger model. The model I use for this sort of work is Google’s Gemini 1.5, which can remember about 1.5 million words in a conversation. Use the model that best fits the amount of data you have to work with.

With an appropriate model and a corpus of our own data, we’re ready to get started.

Step 1. Ask the Model What It Knows

We’re going to follow the Trust Insights PARE AI Framework for this process, and the first step is to prime the mode. We ask the model what it knows about writing style components, because every model is different. Here’s an example prompt:

Today, we’re going to analyze the writing style of an author, (your name here). What are the major components of a writing style (such as diction, intertextuality, etc.) for an author in the (genre)?

Here’s what it would look like for me:

Today, we’re going to analyze the writing style of an author, Christopher Penn. What are the major components of a writing style (such as diction, intertextuality, etc.) for an author in the nonfiction business book genre?

What we’re doing is invoking the model’s knowledge of writing style from a specific, technical perspective. This is one of the reasons many prompts fail; you need at least a little domain expertise to know how to word your prompts with the specifics a model needs to know. We’ll want to augment it with the other questions in the PARE framework, like:

What are common mistakes made when analyzing writing style components?
What aspects of writing style components haven’t we talked about?

Once the model has returned all the major knowledge it has about writing style, ask it to outline all the best practices for analyzing writing style in an outline. This helps reinforce the things we want to focus on.

Step 2: Build an Analysis

Now that we’ve got an outline of what writing style even is, we’re ready to move onto the next part, which is analyzing our writing. Here’s an example prompt:

Using the best practices for analyzing a writing style’s components as outlined above, analyze the writing style of (your name here) based on these uploaded documents. Provide a detailed, comprehensive analysis in outline format.

Then upload the text files of your writing. Remember, more is better and more task-specific is better.

Let the language model produce its detailed analysis of your writing style based on the components it analyzed. We’ve now got some very nice details about our specific writing style, broken out by component. This is critical; remember that writing style is an umbrella term. By getting far more granular with each of the components, we’ll be able to give the model better directions.

We’ll also want to have a page or two of examples in a separate document, perhaps a signature piece of content you’ve written. We’ll come back to this part later. Having a piece of signature content is important because it shows the frequency of our writing style components, something that is almost never captured correctly in an analysis of our writing style.

Step 3: Build a Scorecard

As we covered on this past week’s Trust Insights livestream, one of the ways to force a model to do a better job is to force it to self-evaluate. The best way to do this is to have it analyze and score its own work, but to do that, we need to give it a framework for doing so, a scorecard. The magic word here that invokes the right form of analysis is rubric. A scoring rubric reliably helps a model understand what we’re doing.

Here’s an example prompt.

Using the analysis of (your name here)’s writing style components, build a scoring rubric to assess how well a piece of content aligns to the writing style of (your name here). The scoring rubric should use variable integers (0-3, 0-5, 0-7) to score each writing style component, and the scoring rubric should total 100 points.

The model should return a scorecard that we will use to evaluate whether a piece of writing aligns to our writing style or not.

We should now have four documents: a list of writing style best practices, an analysis of our individual writing style, component by component, a page of signature content, and a scorecard to measure content to see how well it aligns with our writing style.

Step 4: Generate Content

We’re now ready to put these pieces into action. Let’s give the model some instructions and a starting topic to build some content in our writing style. This point is where you could start a new chat; all the work we’ve done up to this point is preparatory work that you only have to do every so often, as long as you saved the documents generated in the previous steps.

Here’s an example prompt. Note that we are still using the Trust Insights RACE AI Framework to guide the model.

You’re an expert in (roles). Your task is to (action). Some background information: (context). Using the writing style of (your name here) as provided, write the blog post about (topic) in the writing style of (your name here). Write 800 words in three sections: what the concept is, why the concept is important, and how to implement the concept. Use the writing style of (your name here) strictly; reference the (your name here signature content) to understand how the writing style should read.

Here’s what it would look like if I did this:

You’re an expert in marketing analytics and marketing attribution. You know Google Analytics 4, Google Tag Manager, data layers, attribution analysis, multi-touch attribution, uplift modeling, and marketing mix modeling. Your task is to write a blog post about the importance of uplift modeling to measure the effects of offline marketing activities. Background information: uplift modeling is a useful way to determine the impact of an activity outside a clickstream and digital analytics, by declaring which time periods in a measurement set have the activity question. Using techniques such as propensity score matching where the control and test units are individual days, we can measure the impact of a tactic while discounting data that’s “business as usual”. Using the writing style of Christopher Penn as provided, write the blog post about uplift modeling in the writing style of Christopher Penn. Write 800 words in three sections: what the concept is, why the concept is important, and how to implement the concept. Use the writing style of Christopher Penn strictly; reference the Christopher Penn Signature Content to understand how the writing style should read.

You’ll add the Signature Content and the Writing Style Analysis documents along with the prompt.

What does the model produce? A really nice blog post:

Gemini First Draft

Now, we’re not done. We’ve only done the generation. We need to know how well the model did.

Step 5: Score the Content

Like an intern or a student, we need to give feedback to an AI model about how well it did on a task. The most precise way to do this is the same as we do with schoolwork: we give it a grade. The nice thing about AI that isn’t true of schoolchildren is that AI is capable of grading its own work. We’ll use a prompt like this:

Using the (your name here) Writing Style Scoring Rubric, analyze and score the blog post you wrote according to the rubric. Return your analysis and a score of your blog post.

The model will return its outline of what it did well, what it did less well, and how it scored:

Model score

Not bad, right? We’re not done. This is only the first draft.

Step 6: Refine the Content

Now that we have a grade, we want to give our AI model a chance to improve its grade. Here’s an example prompt:

Using the score and analysis of your blog post and the (your name here) Writing Style Scoring Rubric, create a refinement of the blog post that improves on the score.

Let the model do its work to create a revision. Once it does so, ask it to score its revision:

Using the (your name here) Writing Style Scoring Rubric, analyze and score your revision.

What you should get is a score that helps you understand what the model changed and improved to achieve a higher score:

Gemini Score Analysis

And let’s take a look at the final output. How well does this match me?

Gemini Final Post

Not bad, right?

Part 4: Scaling Your Writing Style

You’ll notice that this process is incredibly thorough. It requires a lot of effort the first time around, though you can see after step 4 that it gets more efficient with subsequent uses. However, there’s still a lot of manual work.

What if you wanted to do this at scale? That’s a topic for a different newsletter, but the short version is that tools like Google Gemini have APIs. The APIs can take things like prompts, documents, etc. And programmatically run and repeat the process from Step 4 onwards as many times as you want – with the caveat that each use of the API costs you money above and beyond what you already pay for generative AI.

But if you had a need to drastically scale you, the process is something that can be automated and still do a very good job.

While that might not be something you want to do with your personal writing, you could see how useful that would be for deploying with a corporate style guide, for example. Suppose you’ve had a series of bloggers over the years at your company and all your content is a wild mashup of styles. With code, an API, a credit card with a decently large limit, and the pieces we’ve generated here, you could have a language model rewrite every piece of content to conform to your current style guide.

Shameless plug: if you want help doing that, that’s something that Trust Insights can do for you.

Part 5: Wrapping Up

I’m impressed that you made it all the way to the end. Congratulations. And I’m even more impressed that you tried each of the steps in the process and saw how they work. Perhaps you even have the necessary pieces to do this on your own now, and if so, I’m glad it was helpful.

The reason AI can’t easily mimic writing style without the extensive set of processes we walked through today is because writing style is such a complex beast. There are tangible aspects an AI model needs to know to create. There are intangible aspects that a model needs to know to mimic. And like all writers, the first draft is rarely the best draft. By forcing the model to self-evaluate and then improve, we force it to become a better writer.

Is it an exact match of how you write? No. It never will be. So much of your writing, as we talked about with diction, is rooted in your emotional knowledge, something the machines don’t have access to. But will it generate a piece that’s really good? If you follow the process, the answer is yes.

How Was This Issue?

Rate this week’s newsletter issue with a single click. Your feedback over time helps me figure out what content to create for you.

Share With a Friend or Colleague

If you enjoy this newsletter and want to share it with a friend/colleague, please do. Send this URL to your friend/colleague:

https://www.christopherspenn.com/newsletter

For enrolled subscribers on Substack, there are referral rewards if you refer 100, 200, or 300 other readers. Visit the Leaderboard here.

Advertisement: Bring Me In To Speak At Your Event

Elevate your next conference or corporate retreat with a customized keynote on the practical applications of AI. I deliver fresh insights tailored to your audience’s industry and challenges, equipping your attendees with actionable resources and real-world knowledge to navigate the evolving AI landscape.

Christopher S. Penn Speaking Reel – Marketing AI Keynote Speaker

Watch this video on YouTube.

👉 If this sounds good to you, click/tap here to grab 15 minutes with the team to talk over your event’s specific needs.

If you’d like to see more, here are:

Christopher S. Penn Speaking Reel – Marketing AI Keynote Speaker

Watch this video on YouTube.

speaker preview reel (YouTube)

Almost Timely News, August 18, 2024: 🗞️ You Can’t Reasonably Measure AI Brand Strength

Almost Timely News: 🗞️ You Can’t Reasonably Measure AI Brand Strength (2024-08-18) :: View in Browser

👉 Register for my upcoming free webinar, Generative AI for Hospitality!

Content Authenticity Statement

90% of this week’s newsletter was generated by me, the human. You’ll see some AI outputs in the opening section. Learn why this kind of disclosure is a good idea and might be required for anyone doing business in any capacity with the EU in the near future.

Watch This Newsletter On YouTube 📺

Almost Timely News: 🗞️ You Can't Reasonably Measure AI Brand Strength (2024-08-18)

Watch this video on YouTube.

Click here for the video 📺 version of this newsletter on YouTube »

Click here for an MP3 audio 🎧 only version »

What’s On My Mind: You Can’t Reasonably Measure AI Brand Strength

Be very cautious of AI search results tools.

Not that it’s a bad idea to think about how your brand is positioned within language models like GPT-4o (the model that powers ChatGPT today), Claude, or Gemini, but because it’s a nearly impossible task to make that determination.

If You Don’t Feel Like Getting Into The Weeds

Today’s newsletter is going to be very technical so here’s the TL;DR:

Using “AI optimization” tools to determine your “AI SEO” is a bit of a fool’s errand because AI models do not work the way search engines do. If you want to influence AI models, the best thing you can do is publish very high volumes of very high quality content – in other words, traditional SEO. The main difference is that instead of links as the gold standard of measurement, it’s appearances around the public web.

That’s it. Go home. See you next week.

…now, if you’d like to stick around for the detailed explanation, grab your libation of choice, find a comfortable position, and let’s dig into the weeds.

Part 1: How AI Models Work

To learn why trying to measure our findability in AI models is a bit of a fool’s errand, we first need to know how these things work. As a reminder, a model is the engine of today’s generative AI tools. Services like ChatGPT use models like GPT-4omni under the hood.

When these models are built by deep-pocketed technology companies, they’re trained on massive amounts of text data, 7-10 petabytes of text, mostly extracted from the public web. To give you a rough sense of how much text that is, all of William Shakespeare’s plays fit inside roughly 5.4 megabytes. 7 petabytes is 1.3 billion copies of Shakespeare’s works. If each volume is about 3 inches tall, 7 petabytes is a stack of Shakespeare’s books that circles the equator twice.

That’s how much text you need to train today’s models. It’s also why only a few companies can make state of the art foundation models – it costs millions of dollars in time and hardware just to digest that amount of text.

The first part of the process is breaking down words into tokens, little three and four letter fragments of words.

Then, every token is turned into a number.

Then, every token’s position is compared to every other token’s position in a massive statistical database. That’s what is under the hood in every large language model: a big pile of statistics. The way I recommend people think about this conceptually is to think about word clouds. In any given piece of text, what’s the word cloud for EVERY WORD in that text? That’s essentially what’s going on, conceptually. (Mathematically, that’s not true but no one wants to do linear algebra in a newsletter)

When we ask a model something, using a prompt, the model breaks down our question into tokens, then goes into its library and fetches all the word clouds it can find that match those tokens.

For example, suppose you looked at all the word clouds on the Internet around the term Starbucks. You would see words like coffee and tea. You would also see thousands of other words. That’s what is inside a large language model. When we give a model a prompt, what comes back are the most related words (actually tokens, but words is fine for this conceptually) that the model maker’s software has seen all over the Internet.

Starbucks word cloud

This is why attempting to measure a brand’s presence in a language model is a bit of a fool’s errand. It’s not like regular SEO, where you can see how web pages relate to each other through links. Instead, every part of every word is related to every part of every other word, phrase, sentence, paragraph, and document.

Part 2: How People Behave

The technical obstacles to measurement are tough enough to deal with. However, there’s an additional wrinkle. In old school SEO, we pay attention to things like words. You’ll still see things like keyword lists floating around, and keyword planning tools. Those are fine when you’re measuring links and checking the text of words in links as a way to determine what a page is authoritative about.

All that goes out the window in generative AI, because that’s not how people talk to AI. People don’t type in things like:

“Trust Insights consulting” or “management consulting firm”

They ask real questions and have real conversations: “Give me a list of recommended management consulting firms that specialize in healthcare in the Boston area,” and “Who is Trust Insights and what do they do?” And “What’s a good recipe for guacamole for people who are allergic to citrus?”

You might think, “Oh, then we can just ask common questions and measure how often we come up in responses, right?”

Except, going back to part 1, every word gets tokenized when we prompt a model. The more words, the more tokens, and the more specific and clear the response… and the harder it is to predict what people themselves are going to type into an AI model, and the more likely it is the model is going to return different results.

Here’s an example. Suppose I type in:

“Who is Trust Insights and what do they do?”

I’ll get a response from Google’s Gemini model that looks like this:

Gemini Results

Now, suppose I type:

“What does Trust Insights do and who are they?”

Before we look at the results, it should be clear that these are basically the same question. We should get basically the same answer, right?

Nope.

More Gemini Results

These are wildly different answers. Both are factually correct, but you could not easily predict what the answers would be. If you’re trying to measure even whether or not your brand is in an AI model, you can see how complicated this could get.

Why does this happen? Remember that the way these models work, they take into account the order in which words appear when they’re first trained.

The order of words matters. In search engines, in traditional search, word order matters very little because it’s a much different language technology. In language models, word order and even seemingly unimportant words can trigger different token activations and lead to very different results.

Imagine how problematic this becomes when an informational query like “Recommend some top consulting firms in the Boston area” and “Recommend some Boston area top consulting firms” yields totally different results. Good luck measuring your brand presence on every possible way of wording informational queries.

Part 3: A Cornucopia of Models

Okay, so the conversational nature of generative AI means it’s very, very difficult to predict what an AI model will even say, much less measure our brand strength in it. Let me add an even bigger wrinkle.

There are many, many more models in generative AI than there are in traditional search. In traditional search, you mainly have to worry about ranking well in Google. In generative AI, you have a much, much wider field. Here are just a few:

Google’s Gemini powers Gemini (the app) as well as Google’s AI Answers and every other place they try to stuff Gemini.
GPT-4omni powers ChatGPT and the many, many companies that have built their infrastructure on OpenAI’s platform, like Hubspot’s ChatSpot.
Anthropic’s Claude models power a variety of writing apps as well as their own web app.
Meta’s Llama family of models power Meta AI and the AI search in all of Meta’s apps like Facebook, Instagram, WhatsApp, and Messenger.
Mistral’s family of models power apps and services around the world but especially in the EU.

And there are many, many, many more. You can see and test the top 89 (yes, 89) in LMsys’ Chatbot Arena:

Chatbot Arena example

Any company or tool saying they can measure your brand’s presence in AI had better be able to query at least these major models, not to mention the tens of thousands of smaller models that are in production, like IBM’s Granite in WatsonX.

In fact, IBM WatsonX is a great example of how challenging even answering the question of AI presence can be. People building enterprise apps on that platform can choose from over a dozen different models. If you want to influence large enterprises using the WatsonX system, you’d need to influence each of those models.

If any company or piece of software is telling they can measure your brand’s presence in an AI model, and it can return that result in less than a day, they’re lying. There’s no kind way to put it. They’re lying. You would need to query dozens, if not hundreds of different ways of asking a model questions about your brand, tens of thousands of different ways to ask questions about your industry or niche, and ask those questions of at least half a dozen, if not more models. Chatbot Arena, one of the top sites for testing and measuring models, has 89 top models to test. Even fully automated, you’re not getting that answer in under a day, and certainly not for free. The compute costs alone would be in the hundreds of dollars just for one model.

Part 4: Ground Truth

Most generative AI services that offer consumer interfaces have started creating grounding in their software; that is, the software gets answers from the model and then essentially fact checks them with website data, typically from search engines. Meta’s AI model does this. ChatGPT does this. Google’s Gemini does this.

Which means… while people may not necessarily use search engines the way they used to or as much as they used to, optimizing your content for search engines is still vitally important. Showing up in search engines means that when an AI model is unsure about its answer, if the model has the ability to call tools (like web searches), it’s going to ask the nearest search engine.

Those sites and pages that do well in legacy, traditional search will be the points of reference for AI models that are checking their own work, which is a polite way of telling you not to fire your SEO team just yet.

Part 5: So What Do We Do?

Let’s get to the So What? as Katie Robbert likes to say. Okay, we effectively cannot measure our brand presence in an AI model because the architecture of the model simply makes that infeasible.

But what we can do, what we know how to do, is create content on the Internet that’s available for models to learn from. Almost every AI company making models uses some or all of the Common Crawl archive of public web pages as the base data for its model training. Common Crawl is a regularly updated massive archive of text, 6.2 petabytes as of the last crawl, and is something you can – if you have the disk space – inspect yourself to see what’s inside. You can download portions of it and see what part of your website has been scraped. You can see what news outlets, what blogs, what podcasts, etc. are inside their archive – and by extension, inside the training data for language models.

We can and should be creating high quality, high volume content everywhere we can. Make YouTube videos at scale – we know for sure companies like NVIDIA have scraped millions of hours of YouTube video for their model training. I am almost 100% confident OpenAI has scraped the same from YouTube, because in their voice transcription model, Whisper, it recognizes and tags my voice in audio

We should be creating content in all the places we know scrapers go, and AI models go to obtain training data. Github. StackExchange. Reddit. YouTube. Local news sites. Substack. Various social networks.

And finally, but probably most important, we should be building brand like crazy. You should be building your personal brand by being in as many places as you feasibly can be. You should be publishing guest content wherever you can. You should be on every podcast that will have you. You should be joining livestreams and being on any YouTube channel you can be.

We want models to train on our names and our topics from as much text as possible, which means we need to create that text in as many places on the Internet as we can. However, instead of chasing inbound links as we used to in SEO, now we’re just chasing placements, period. Believe it or not, this is where public relations firms and teams could make a huge difference for their companies. The best firms and teams excel at getting media placements, so their strategy just has to pivot from “we want top tier media” to “we’ll take any media we can get”.

You Can’t Reasonably Measure AI Brand Strength

The bottom line is this: yes, you can measure your brand’s placement in AI, but it’s a time-intensive, compute-intensive, very expensive process to do so if you do it well. (it’s free and easy if you want inaccurate, incorrect answers)

However, you can absolutely measure the impact of AI on your business using the gold standard of measurement: asking people how they learned about you. There is no substitute for this, there never has been, and when done properly, will always be a timeless, valuable source of marketing attribution.

If you’d still like to try measuring brand strength in AI and feel like lighting a lot of money on fire, my company Trust Insights will try to talk you out of that but will still do it if you absolutely insist..

How Was This Issue?

Rate this week’s newsletter issue with a single click. Your feedback over time helps me figure out what content to create for you.

Share With a Friend or Colleague

If you enjoy this newsletter and want to share it with a friend/colleague, please do. Send this URL to your friend/colleague:

https://www.christopherspenn.com/newsletter

For enrolled subscribers on Substack, there are referral rewards if you refer 100, 200, or 300 other readers. Visit the Leaderboard here.

Advertisement: Bring Me In To Speak At Your Event

Elevate your next conference or corporate retreat with a customized keynote on the practical applications of AI. I deliver fresh insights tailored to your audience’s industry and challenges, equipping your attendees with actionable resources and real-world knowledge to navigate the evolving AI landscape.

Christopher S. Penn Speaking Reel – Marketing AI Keynote Speaker

Watch this video on YouTube.

👉 If this sounds good to you, click/tap here to grab 15 minutes with the team to talk over your event’s specific needs.

If you’d like to see more, here are:

Christopher S. Penn Speaking Reel – Marketing AI Keynote Speaker

Watch this video on YouTube.

speaker preview reel (YouTube)

Almost Timely News: 🗞️ How To Use Generative AI To Understand Good Content (2024-08-11)

Almost Timely News: 🗞️ How To Use Generative AI To Understand Good Content (2024-08-11) :: View in Browser

👉 Register for my upcoming free webinar, Generative AI for Hospitality!

Content Authenticity Statement

90% of this week’s newsletter was generated by me, the human. You’ll see some AI outputs in the opening section, and if you’re watching or listening, I used Eleven Labs to synthesize the voice of a former Supreme Court Justice who died in 1985. Learn why this kind of disclosure is a good idea and might be required for anyone doing business in any capacity with the EU in the near future.

Watch This Newsletter On YouTube 📺

Almost Timely News: 🗞️ How To Use Generative AI To Understand Good Content (2024-08-11)

Watch this video on YouTube.

Click here for the video 📺 version of this newsletter on YouTube »

Click here for an MP3 audio 🎧 only version »

What’s On My Mind: How To Use Generative AI To Understand Good Content

What constitutes good content? And can generative AI help us make it?

I’ve been thinking a lot about good content lately, about compelling content. What makes one thing worth paying attention to, and not another?

For example, you can’t go for more than two thumb swipes on any social network without someone complaining that reach is in the toilet, that engagement is down, that people aren’t paying attention. And yet, when you hit up the channels and accounts of the top tier content creators, or when you listen to system admins like Adam Mosseri of Instagram, they all say the same thing: there’s no economic incentive for content channels to throttle popular content. It makes no business sense to show people less of what they love. Taking away great content from people would reduce ad exposures, and that’s contrary to their business model.

That makes sense.

So why are people complaining about reduced reach? A good number of folks who are complaining are… just not creating content people want.

This is a familiar challenge for marketers. Most marketers know we’re creating stuff people don’t love. We’re creating – on a good day – mediocre content, not good content, not great content.

So how would we change this around? How do we build good content?

Part 1: What is Good Content, Anyway?

First we have to know what good content is. Quick, what is it? What are the common characteristics of good content, and how can we use generative AI to help?

A lot of people crap on generative AI for making bland, boring, generic content, and there’s validity to that criticism. Generative AI is very much like the mythical genie of fairy tales and legends. You get exactly what you ask of it. If you ask bland, boring questions – like, “What makes good content?” – you get bland, boring answers. It’s the nature of AI itself to return probabilities based on the questions we ask it. The more average the question, the more average the answer.

But that doesn’t answer the question of what good content is, does it? It just sort of shifts blame. So how do you define good content?

Here’s how a couple of folks talk about it. My friend Ann Handley:

“There are three keys to creating good, quality, engaging content… And the three keys are bigger, braver, and bolder, right? Bigger stories. Braver marketing or braver marketers as well. And bolder writing and bolder tone of voice.”

Or this quip from Supreme Court Justice Potter Stewart in 1964 in Jacobellis v. Ohio:

“I shall not today attempt further to define the kinds of material I understand to be embraced within that shorthand description; and perhaps I could never succeed in intelligibly doing so. But I know it when I see it…”

While that’s Justice Stewart’s definition of obscenity, you could make the case that’s the definition of good content as well. Good content is content we like, content we enjoy, content we’re compelled to consume, whether it’s celebrities suffering eating increasingly hot wings or investigative journalism answering obscure questions like the hardest nightclub to get into in the world.

And therein lies one of the greatest challenges of good content. It varies from person to person.

Part 2: Good Content is Good Storytelling

Okay, so if good content is an individual choice, how are we supposed to create it at scale? There are some fundamental truths to good content. One of those truths is that good content is often a good story.

Invariably, good content tells a story, because we’re adapted well to consume stories. Millennia before the printing press, we were sitting around campfires telling stories. What to eat, what not to eat, why Bob isn’t around the campfire because he got mauled by a bear… we’ve learned through millennia to capture the information we need to share in a story.

Here’s one of the challenges of storytelling. Besides the infinite number of ways there are to tell a story, our audience has specific kinds of stories that resonate with them best. If we want to be effective in our storytelling, in our content creation efforts, we need to align our content with what our audience finds most compelling.

How do you do that? This is an area generative AI can’t help you with, but a real audience can. With your existing audience, ask them a simple question: what shows do you enjoy, especially long form content like podcasts?

This is a critical question because the marketplace has already produced good content. The audience has already decided for themselves what is good and what isn’t, especially for content people consume outside of work. We don’t need to define what good content is – and in aggregate, we can’t anyway. But we can absolutely get a sense from our audience of the content they already appreciate.

Once you have some answers from your audience, we can put generative AI to work in a way that we really can’t. Here’s why.

Stewart’s definition encapsulates a really important concept. Language itself is a form of code, a way to take very deep concepts and shorten them. Think about how much data is encoded in the word apple or daughter or sushi. There’s two challenges with language like this. First, the language is so compact that we have to fill in a lot of the blanks ourselves.

What kind of apple? Fuji? Granny Smith? The computer company? There’s a lot of concepts packed into that one word.

Second, almost every word is relative to our experiences. I love sushi. You may hate sushi. And there are so many kinds of sushi that my association of the word may bring up a very specific kind of sushi as a mental concept, and that is guaranteed to be different in someone else’s head.

For example, I might have a certain mental and emotional association with even a simple word like mother. Some people have strong positive associations with it. Others have strong negative associations. When we use that term, we may be invoking wildly different associations in people’s heads.

Because of this, we have a very hard time putting into words why we like something. We know when we like it. We know when we don’t like it. But we generally struggle to explain it in a complete and comprehensive way, partly because of how much language we need to encapsulate it, and partly because we have our own emotional associations that can color our perceptions.

That’s what Justice Stewart was talking about 60 years ago in his ruling. We know it when we see it, not because we don’t have the words to describe it, but because the sheer amount of language we’d need to surround the concept would be greater than an army of ants surrounding a slice of watermelon.

Part 3: Generative AI Understands Good Content If We Tell It To

Generative AI models don’t have these problems. They have a nearly infinite vocabulary, and because they’re not sentient, self aware, or emotionally intelligent, they don’t have the loaded associations that you or I might have as individuals. That same averageness that causes them to generate bland content with poor prompts becomes an advantage when we need to find words to describe something.

Let’s put this to practical use. Suppose we learn from our audience that they really love shows like Freakonomics or Search Engine. That alone is helpful information, information that could go in something like an ideal customer profile. But what if we could understand the meta-structure of these shows, to see how they’re formatted and structured to be compelling?

We absolutely can. Today’s generative AI models like GPT-4-omni, Claude Sonnet 3.5, and Google Gemini can all accept things like audio and video files as inputs. Instead of trying to describe good content ourselves, what if we could have a generative model consume some of our audience’s favorite shows and help us understand the practices these specific shows use to create good content?

In other words, we take the final dish and we reverse engineer the recipe using generative AI. If we can understand the recipe, then we can take the recipe’s concepts and apply it to our own content, converting it from good to better.

Let’s step through the basic process. The video edition of this newsletter shows each step in action, should you want to see it. First, using the Trust Insights PARE AI Framework, I prompt the model to spit out everything it thinks it knows about great storytelling. This step helps us diagnose whether the model even knows what it’s talking about or not. It also serves to build an insanely rich prompt for future questions because every subsequent prompt in a conversation is composed of the entire conversation to that point.

PARE Prompt

Once we have a solid base of knowledge for what good storytelling is, by adjusting the model’s output if necessary, we need to show, not tell the model what good storytelling is. We need, as Justice Stewart said, to help the model know it when it sees it. Once we’ve loaded a few episodes, we ask the model to analyze the structure of the shows. How do they do what they do? What are the strategies, tactics, and ways they implement their practices?

Upload audio

We next ask the model to build us a scoring rubric, a way for us to analyze our own content against the best practices that the model observed from what we know to be good content. This scoring rubric accomplishes two things. First, it helps the model be more granular about how it assesses and analyzes content. Second, it helps us, the humans, better understand the criteria for content we create.

Build a scoring rubric

Finally, once we’ve built the informational infrastructure for understanding compelling storytelling in a very specific format, we can take that as a template and have generative AI examine our existing content and reformat it, apply these best practices to it. It’s important to note here that we want to apply the structure and the concepts of our best practices content, not replicate the content itself.

Depending on the level of control you want to have over the final output, you could direct generative AI to rewrite your content to adhere to the general best practices, or simply produce an outline that you can then manually reorder your content with.

Conceptually, imagine you had a country song, and you knew what made a great rock song. You’d apply the principles of rock to your song, but it would still be your song. This is critical – if you want this technique to work well but you don’t want to lose your own voice and unique style, you have to bring your own first draft to the party. Don’t have AI build it – you build the country song, then have it make the rock song from it.

And if you want to turn the dial to 11, you could also incorporate your ideal customer profile in at this stage as well, to have the model critique not only your storytelling skills but also how well aligned the content itself is to what your audience cares about.

Part 4: Let AI Help You Make Good Content

So, let’s wrap up. There is no such thing as good content in aggregate, any more than there’s such as thing as the average person. There is content your specific audience appreciates.

There are thus no best practices to create good content for the average audience member, but patterns we can learn from other content our audience already appreciates.

We use generative AI to identify these patterns, distill them down into concepts we apply to our own content, to better make it fit what our audience appreciates – but we do so with an eye towards maintaining our voice and originality, just putting the content in a more appealing format.

And when we do so, when we apply the structures and practices our audiences appreciate to our existing content, we’ll make something they specifically like better.

The road to good content has been paved by the marketplace as a whole. We just need the sense and the tools to see what’s been done and travel down that road with our own content.

And if you’d like help doing this sort of content analysis work with AI, shameless plug, my company Trust Insights does that (and much more).

How Was This Issue?

Rate this week’s newsletter issue with a single click. Your feedback over time helps me figure out what content to create for you.

Share With a Friend or Colleague

If you enjoy this newsletter and want to share it with a friend/colleague, please do. Send this URL to your friend/colleague:

https://www.christopherspenn.com/newsletter

For enrolled subscribers on Substack, there are referral rewards if you refer 100, 200, or 300 other readers. Visit the Leaderboard here.

Advertisement: Bring Me In To Speak At Your Event

Elevate your next conference or corporate retreat with a customized keynote on the practical applications of AI. I deliver fresh insights tailored to your audience’s industry and challenges, equipping your attendees with actionable resources and real-world knowledge to navigate the evolving AI landscape.

Christopher S. Penn Speaking Reel – Marketing AI Keynote Speaker

Watch this video on YouTube.

👉 If this sounds good to you, click/tap here to grab 15 minutes with the team to talk over your event’s specific needs.

If you’d like to see more, here are:

Christopher S. Penn Speaking Reel – Marketing AI Keynote Speaker

Watch this video on YouTube.

speaker preview reel (YouTube)

Almost Timely News, August 4, 2024: 🗞️ The ROI of AI

Almost Timely News: 🗞️ The ROI of AI (2024-08-04) :: View in Browser

👉 Register for my upcoming free webinar, Generative AI for Hospitality!

Content Authenticity Statement

100% of this week’s newsletter was generated by me, the human. I wish I could have written the opening piece with AI, but I needed to think it through myself thoroughly. Learn why this kind of disclosure is a good idea and might be required for anyone doing business in any capacity with the EU in the near future.

Watch This Newsletter On YouTube 📺

Almost Timely News: 🗞️ The ROI of AI (2024-08-04)

Watch this video on YouTube.

Click here for the video 📺 version of this newsletter on YouTube »

Click here for an MP3 audio 🎧 only version »

What’s On My Mind: What’s the ROI of AI?

“Remember that Time is Money.” – Benjamin Franklin, Advice to a Young Tradesman, 1748

Let’s talk about the ROI of AI today; while the focus will predominantly be on generative AI, it can be classical AI too. A lot of folks are asking the very sensible question, what is the ROI of AI? How would you go about determining it? So let’s dig in.

Part I: ROI

What’s the ROI of [insert thing here] is an age old question. Those folks with my color hair probably remember questions like “What’s the ROI of a website?” to skeptics in the 1990s. “What’s the ROI of social media?” in the 2000s, “What’s the ROI of mobile?” And so on. Sometimes the questions were asked in good faith, trying to figure out how much to invest in a platform. Other times, for more conservative folks, it was asked in bad faith to shut down any kind of initiative.

The amusing thing is, most people get ROI wrong. ROI – return on investment – is a financial calculation. It is expressed either in dollars or percentages, and there is no substituting anything in the equation. (There are a lot of people who love things like “return on [insert buzzword here]” that are nonsensical in this context)

Here’s the equation:

( earned – spent ) / spent = return on investment

That’s it. That’s ROI. To calculate the ROI of anything, you have to know what you earned, and you have to know what you spent.

If you don’t have those numbers, you cannot calculate ROI. Period, end of story.

A fair number of folks use the term ROI when they really mean value. That’s incorrect. The value of AI is different than the ROI of AI; things can have value that you might not be able to express in monetary terms.

For example, suppose you’re running a campaign to get someone elected to office. There’s a clear, measurable binary outcome – either the person is elected or they are not. But that result is not financial in nature (unless the person is corrupt and then you can measure bribes later, I suppose), and so while there is value, there is not ROI.

Here’s something else that’s rarely discussed with ROI. Even when you have the numbers, it’s not always an appropriate measurement. ROI is best suited for low resource environments, for when you have very limited resources and you need to make every dollar count. You use it to judge whether or not something is worth investing in.

However, not every scenario or situation is a low resource environment. There are plenty of companies who are in growth mode, who want to acquire audience or customers regardless of cost, or with cost as a secondary consideration. They need market share more than anything else. In those situations, ROI might unnecessarily constrain your strategy, and it would be an inappropriate measure of success.

Part 2: Calculating ROI

It’s at this point where I wish I had written this with AI, but I did not. So we press on. Let’s talk through calculating ROI. It seems pretty simple – (earned – spent)/spent fits on a sticky note. The challenge is, what does that mean?

Earned is the gross revenue earned by a particular thing, be it a strategy, tactic, or technology.

Spent is what you spent to achieve that earning.

Sounds simple, right? Except it really, really isn’t.

Earned

Marketers in particular have struggled to determine earned for decades, through attribution analysis. What does each marketing activity result in? Forget spent – can you even determine whether what you did earned any money or not? This gets into things like “what’s the value of brand?” And individual channel analysis. What’s the value of email marketing, of social media, of SEO, etc.?

There was a brief period of time, in the mid 2010s, when digital attribution was something the average non-technical marketer could do with a reasonable level of confidence. Tools like Google Analytics and a total disregard for individual privacy allowed marketers to get a sense of what was working. However, those days are long past; today, with things like privacy blocking in many operating systems, dark social (places like group chats, Slack, Discord, etc. That offer zero analytics to marketers) and AI-powered guessing in software like Google Analytics make attribution substantially more difficult overall and impossible for the non-technical marketer. You can still measure and calculate attribution, but it requires substantially more investment and technical skill.

So earned is kind of a mess. Let’s move onto spent. That’s easy, right?

Spent

Spent is another area where things usually go off the rails. Why? Because there’s two kinds of spent, hard money and soft money.

Hard money is literal dollars or kroner or euros spent, given to some other entity. When you buy ads, you are spending hard money with a vendor.

Soft money is the opportunity cost of doing something. We pay people to work for us, and every hour someone spends doing one task is an hour they can’t spend on another task. Thus, we have to account for that in the spent portion.

Let’s say someone is in charge of writing social posts, and their hourly rate works out to be 30 USD per hour. If the task takes them an hour, then we effectively spent 30 USD on that task – and that goes into our spent calculation.

This soft money cost can dramatically change our ROI computations. Let’s say you spent 5000 USD on Facebook Ads, and you earned 7500 USD in gross sales. So far, our ROI is:

(7500 – 5000) / 5000 = 50% ROI

That’s terrific. For every dollar you spent, you earned it back plus 50 cents. Good job, right?

But… it took:

A meeting of the marketing team to decide what ads to run, which was 1 hour with 5 people
The creative team to design the ad assets, which was a designer that took 4 hours of design time plus two rounds of revisions for a total of 8 hours
The marketing team had to write the ads, which took 2 hours and 2 people, or 4 hours total
Someone to actually put the ads into Meta Ads, which took 1 hour
Someone to monitor the ads and adjust them, which was 2 hours a week for the 8 weeks the campaign was in flight

Add up those hours spent at a blended rate of 50 USD per hour, and your spent was 5000 USD PLUS the 34 hours of labor, or an additional 1700 USD.

Now your ROI calculation is:

(7500 – 6700) / 6700 = 11.9% ROI

That’s still positive, meaning if you spent a dollar, you earned back the dollar plus 12 cents, but it’s a far cry from 50% ROI.

This is the challenge of truly measuring ROI, taking into account what you earned and what you spent. You can obviously do it at a very gross level on a company’s basic finances – the company’s overall gross revenue minus expenses, but once you start trying to break things down, it gets messy in a hurry.

Part 3: You Probably Can’t Calculate The ROI of AI

Here’s the zinger: calculating the ROI of AI is contingent on already knowing the ROI of what you’re currently doing.

If you don’t know the ROI of what you’re doing today, you cannot calculate the ROI of AI’s impact on it.

That’s the bench test for when a stakeholder asks about the ROI of AI. Ask to see the spreadsheet or ledger which shows your current ROI calculations for any given strategy, tactic, or execution. If that doesn’t exist, then you won’t be able to calculate the ROI of AI because you have no idea where you’re starting from.

But let’s say you’ve vanquished the mythical beast of ROI computation. You have an operating understanding of ROI for something like the Facebook Ads example we showed above. You’ve figured out what the earnings are for a particular strategy, tactic, or execution and you’ve calculated the hard and soft money.

Now you can finally start figuring out what the ROI of AI is. Here’s how. First, we have to determine where AI has fit into the picture, and what the benefits are. Fundamentally, we care about three things in business: save time, save money, make money.

When we save time, we fundamentally reduce the soft money cost in any process. If a task takes 10 hours and we’re paying a rate 50 USD per hour and we can reduce that task to 5 hours, we save 250 USD in soft money through that time savings.

When we save money, we’re just spending less. That one’s pretty obvious.

When we make money, either we’re innovating and creating new lines of income, or we’re optimizing and we’re getting more income out of the lines of business we already have, the ways we already make money.

So how might this apply to the very tactical process of running Facebook Ads?

Remember that 1700 USD chunk of soft money that reduced our ROI from 50% to 12%? If we can reduce the amount of time spent, we reduce the soft money cost, and our overall ad campaign ROI goes up.

If we generate ad copy or ad design using AI that performs better in testing and deployment than our current ads, then that performance brings in more revenue by getting people to convert or more better.

If we improve our targeting, our audience understanding, etc. to the point where we can reduce our ad costs, then we reduce the hard money cost. You would think this would be baked into ad platform software, but it isn’t; ad platforms inherently have a bias towards getting us to spend as much money as possible with them, not get the greatest possible results. (This is a common criticism of many big AdTech companies – the “smart” bidding algorithms are smart and do a great job – for the ad platform.)

Once we identify where we can put AI to use in one or more of these three dimensions, we can measure the change. Let’s look at our Facebook Ads example again.

A meeting of the marketing team to decide what ads to run, which was 1 hour with 5 people. This part probably won’t change. People love their meetings.
We use AI to generate most of the ad creative, which cuts down from 8 hours to 4 hours (because revisions and review still takes time)
AI writes all the ad copy. With a very skilled prompt and/or appropriate infrastructure setup, this will take 5-10 minutes.
Someone to actually put the ads into Meta Ads, which took 1 hour. This won’t change much unless someone automates this via the API.
Someone to monitor the ads and adjust them, which was 2 hours a week for the 8 weeks the campaign was in flight. This also won’t change much.

We go from 34 hours of labor to 26 hours of labor, which is a savings of 8 hours. At our 50 USD hour blended rate, our cost drops from 1700 USD to 1300 USD, a savings of 400 USD. Obviously, if we can trim back more, like dramatically shorten that initial meeting or change how we do ad monitoring, we can find additional savings.

Let’s also say that our AI ads perform better. Instead of earning 5000 USD, we earn 6000 USD on our campaign because AI helped write more compelling ads.

If we add up the savings in soft money plus the increase in income, we have a total of 1400 USD that AI changed for the better. That’s the ‘earnings’ of AI. Now, we need to determine what we spent. If we’re using foundation models and skillful prompting, we’re probably spending 50 USD a month per user on the tools. If we’re using APIs and programmatic AI, the costs might be higher based on how many API calls we’re consuming. (your Dev and IT teams would be able to help with that)

Let’s say for sake of example that we have a ChatGPT Teams subscription and four seats. That’s 100 USD per month. Let’s say we also consume 100 USD of API calls from a script we have that can mass generate Facebook ads. That’s 200 USD per month in costs. If we plug that into our ROI formula, our ROI of AI, in this example, is:

( 1400 – 200 ) / 200 = 600%

Thus, our ROI of AI here is 600% ROI. For every dollar we spent, we got it back plus 6 more.

Soft Money ROI Only

Suppose the ads didn’t earn more. If we just went with the time savings and our ads didn’t perform any better, we would have:

( 400 – 200 ) / 200 = 100%

The time savings ROI would be 100% – for every dollar we spent, we got it back plus another dollar.

This is how most organizations will probably find the ROI of AI. And because it lacks the income side, the earned side, and focuses just on the savings side, this is the “worst case scenario” ROI for AI – and it’s still positive.

Here’s a clear example I gave from a talk recently about a time savings-based ROI. I wrote a prompt to help a PR team build pitch ideas based on a piece of content it was given. For those who don’t know, I worked at a PR firm for 5 years prior to Trust Insights. I sat in on many, many of these meetings. For one set of pitch ideas – not the pitching itself, just the ideation – the cost that was billed to a client was 12,000 USD for 66 hours of a team’s time, at an agency blended bill rate of 182 USD per hour.

That same process I replicated in Google’s Gemini for a client’s PR team. The client was very, very happy with the outcome and thought the ideas were quite effective, so we’re at parity in terms of quality of outcome. The same process that took a team of humans 66 hours in my agency days took Gemini 1 minute, 52 seconds. At a bill rate of 182 USD per hour, that was 5.66 USD.

Slide from my AI keynote

What’s the ROI of AI? For a client in this case, it’s the cost difference as the earnings minus the expense of AI. Even if I used the highest bill rate I charge, 1000 USD per hour, the cost would be 33 USD to run the software. Let’s be generous and say it took me 3 hours to build and test the software (it did not) for a cost of 3000 USD. What’s the ROI of AI in this case?

‘Earned’ is what the client didn’t spend on the old way, 12,012 USD
Spent is what the client spent in this instance, 3033 USD

( 12012 – 3033 ) / 3033 = 296% ROI

For every dollar the client spent, they ‘earned’ (saved) that dollar plus 2.96 additional – on this run of the project. They can then take that project and reuse it, over and over again, to generate idea after idea. What does the ROI look like on the second run, when it’s just the cost of running it, even at my bill rate?

( 12012 – 33 ) / 33 = 36,300% ROI

It’s a ludicrous 36,300% ROI. For every dollar they spend on subsequent uses of the project, they get that dollar back (in savings) plus 36,300 more.

Part 4: What To Do With the ROI of AI

If you’re able to calculate the ROI of AI for different strategies, tactics, and executions, then you’ll be able to build business cases for why you need to expand or enhance your offerings. But what do you do with that number, other than brag to your CFO?

One of the most important use cases for a stellar ROI of AI number is to re-look at all your business processes. There are some strategies, tactics, and executions which, when human-led, have had negative ROI. They cost more than they earned, and it was judged that it wasn’t worth doing.

Once you become skillful at applying AI to business processes, you can dust off those old use cases. If a strategy, tactic, or execution was only marginally negative in ROI, applying AI to it might be enough to flip the balance to a positive ROI. In the previous example, 296% ROI was great for the first use of AI to replicate a more manual process. 36,300% ROI is just bonkers and could make almost any unprofitable process profitable if the process itself is a good candidate for AI.

Here’s an example. Many, many, MANY companies that do ROI computation correctly for organic social media find that organic social media generally has negative ROI. You do it because you have to, for brand reputation management, retention, community building, and crisis aversion, not because it’s intensely profitable. Very, very few companies can demonstrate strong profitability with organic social media marketing alone.

Why? The soft money cost of organic social media is very high. Now, if you were to decompose your organic social media processes and apply AI to them wherever you could – especially on content creation – you might be able to change that balance. If you could reduce your soft money expenditures on social media content creation by 2x or more, then you might find that your organic social ROI could start heading towards positive ROI territory. What was previously not a viable channel in terms of ROI could be again.

AI has the strongest impact today on soft money expenditures – saving time. Like Ben Franklin said, time is money. If you want to demonstrate the value of AI, that’s where you’ll find the easiest benefit, and unlike the tangled web that is attribution modeling and proving the impact of marketing methods, you can demonstrate the ROI of AI with cost savings alone. The example above where we ignored the increase in ad revenue and just showed cost savings in time is the easiest way to get to an ROI of AI.

Wrapping Up

Here’s the unpleasant reality: very few companies will be able to show the ROI for AI because they can’t show the ROI of anything they currently do. The best case scenario for them is showing the impact of AI on cost savings.

For those companies that have their house in order, they can branch out into showing saving hard money or making more money through things like innovation, improved product market fit, better competitive analysis, etc. Like social media, mobile, and the Internet itself, AI has a transformative effect on business that is a direct correlation of how well the business itself is run. to paraphrase the first Captain America movie, it’s an amplifier. It makes the good into great and the bad into worse.

My partner and CEO Katie Robbert frequently says that new technology can’t solve old problems, which is 100% true. If you’re not able to calculate the ROI of anything else, you won’t be able to calculate the ROI of AI either.

If you can show the ROI of what you’re currently doing, then you’ll be able to show the ROI when you apply AI to those processes as well – and the numbers will probably astonish you.

If someone has asked you recently about the ROI of AI, please share this issue of the newsletter with them.

And if you’d like help doing this sort of investigation into the ROI of your own use of AI, shameless plug, my company Trust Insights does that (and much more).

How Was This Issue?

Rate this week’s newsletter issue with a single click. Your feedback over time helps me figure out what content to create for you.

Share With a Friend or Colleague

If you enjoy this newsletter and want to share it with a friend/colleague, please do. Send this URL to your friend/colleague:

https://www.christopherspenn.com/newsletter

For enrolled subscribers on Substack, there are referral rewards if you refer 100, 200, or 300 other readers. Visit the Leaderboard here.

Advertisement: Bring Me In To Speak At Your Event

Elevate your next conference or corporate retreat with a customized keynote on the practical applications of AI. I deliver fresh insights tailored to your audience’s industry and challenges, equipping your attendees with actionable resources and real-world knowledge to navigate the evolving AI landscape.

Christopher Penn AI Keynote Speaker Reel 2024

Watch this video on YouTube.

👉 If this sounds good to you, click/tap here to grab 15 minutes with the team to talk over your event’s specific needs.

If you’d like to see more, here are:

ICYMI: In Case You Missed it

Besides the recently updated Generative AI for Marketers course I’m relentlessly flogging, this week Katie and I talked through identifying generative AI use cases.

Skill Up With Classes

These are just a few of the classes I have available over at the Trust Insights website that you can take.

Premium

Free

Advertisement: Generative AI Workshops & Courses

Imagine a world where your marketing strategies are supercharged by the most cutting-edge technology available – Generative AI. Generative AI has the potential to save you incredible amounts of time and money, and you have the opportunity to be at the forefront. Get up to speed on using generative AI in your business in a thoughtful way with Trust Insights’ new offering, Generative AI for Marketers, which comes in two flavors, workshops and a course.

Workshops: Offer the Generative AI for Marketers half and full day workshops at your company. These hands-on sessions are packed with exercises, resources and practical tips that you can implement immediately.

👉 Click/tap here to book a workshop

Course: We’ve turned our most popular full-day workshop into a self-paced course. The Generative AI for Marketers online course is now available and just updated as of April 12! Use discount code ALMOSTTIMELY for $50 off the course tuition.

👉 Click/tap here to pre-register for the course

If you work at a company or organization that wants to do bulk licensing, let me know!

Get Back to Work

Folks who post jobs in the free Analytics for Marketers Slack community may have those jobs shared here, too. If you’re looking for work, check out these recent open positions, and check out the Slack group for the comprehensive list.

Advertisement: Free Generative AI Cheat Sheets

Grab the Trust Insights cheat sheet bundle with the RACE Prompt Engineering framework, the PARE prompt refinement framework, and the TRIPS AI task identification framework AND worksheet, all in one convenient bundle, the generative AI power pack!

Download the bundle now for free!

How to Stay in Touch

Let’s make sure we’re connected in the places it suits you best. Here’s where you can find different content:

My blog – daily videos, blog posts, and podcast episodes
My YouTube channel – daily videos, conference talks, and all things video
My company, Trust Insights – marketing analytics help
My podcast, Marketing over Coffee – weekly episodes of what’s worth noting in marketing
My second podcast, In-Ear Insights – the Trust Insights weekly podcast focused on data and analytics
On Threads – random personal stuff and chaos
On LinkedIn – daily videos and news
On Instagram – personal photos and travels
My free Slack discussion forum, Analytics for Marketers – open conversations about marketing and analytics

Listen to my theme song as a new single:

Advertisement: Ukraine 🇺🇦 Humanitarian Fund

The war to free Ukraine continues. If you’d like to support humanitarian efforts in Ukraine, the Ukrainian government has set up a special portal, United24, to help make contributing easy. The effort to free Ukraine from Russia’s illegal invasion needs your ongoing support.

👉 Donate today to the Ukraine Humanitarian Relief Fund »

Events I’ll Be At

Here are the public events where I’m speaking and attending. Say hi if you’re at an event also:

MAICON, Cleveland, September 2024
Lab Products Association, Philadelphia, September 2024
MarketingProfs B2B Forum, Boston, November 2024
Social Media Marketing World, San Diego, April 2025

There are also private events that aren’t open to the public.

If you’re an event organizer, let me help your event shine. Visit my speaking page for more details.

Can’t be at an event? Stop by my private Slack group instead, Analytics for Marketers.

Required Disclosures

Events with links have purchased sponsorships in this newsletter and as a result, I receive direct financial compensation for promoting them.

Advertisements in this newsletter have paid to be promoted, and as a result, I receive direct financial compensation for promoting them.

My company, Trust Insights, maintains business partnerships with companies including, but not limited to, IBM, Cisco Systems, Amazon, Talkwalker, MarketingProfs, MarketMuse, Agorapulse, Hubspot, Informa, Demandbase, The Marketing AI Institute, and others. While links shared from partners are not explicit endorsements, nor do they directly financially benefit Trust Insights, a commercial relationship exists for which Trust Insights may receive indirect financial benefit, and thus I may receive indirect financial benefit from them as well.

Thank You

Thanks for subscribing and reading this far. I appreciate it. As always, thank you for your support, your attention, and your kindness.

See you next week,

Christopher S. Penn

Almost Timely News, July 28, 2024: 🗞️ Generative AI for Advanced Citizenship

Almost Timely News: 🗞️ Generative AI for Advanced Citizenship (2024-07-28) :: View in Browser

👉 New! Take my new webinar on demand, Generative AI for Email Marketing!

Content Authenticity Statement

95% of this week’s newsletter was generated by me, the human. You’ll see AI outputs from Google Gemini and Google NotebookLM in the opening section and in the live demo in the video. Learn why this kind of disclosure is a good idea and might be required for anyone doing business in any capacity with the EU in the near future.

Watch This Newsletter On YouTube 📺

Almost Timely News: 🗞️ Generative AI for Advanced Citizenship (2024-07-28)

Watch this video on YouTube.

Click here for the video 📺 version of this newsletter on YouTube »

Click here for an MP3 audio 🎧 only version »

What’s On My Mind: Generative AI for Advanced Citizenship

In this week’s issue, let’s talk about a very different use case for generative AI, one that people overlook, one that people don’t realize is possible, and one that is absolutely vital: advanced citizenship.

Caveat lector: this issue touches on politics (mostly local politics) but if you want absolutely nothing to do with the topic, I respect that. Skip this issue and I’ll see you next week.

What Is Advanced Citizenship?

First, what do I mean by advanced citizenship? Many of us do our required or encouraged duties as citizens, from voting to jury duty to participating in the politics of our preferred points of view. But very few of us participate in the operation and activation of the governments we support.

It’s like attending the grand opening of a park, and then we never set foot in it again for years.

For example, suppose a new law is proposed or is put on the books. How often do you check out new, pending legislation to see what it says? I rarely do, because law isn’t a strength of mine and not something I specialize in.

Those same proposed and actual laws impact us every day, in every part of our lives. They set our tax rates, they tell us what we may and may not do, they govern our health, our wealth, even what we can or cannot do with our own bodies.

So why wouldn’t we pay more attention to them? Because they’re really, really onerous to read. They’re dry. They’re technical. They’re boring.

And yet… they have such an outsized effect on us. This is true regardless of where you are; laws exist in every nation (even if they’re enforced differently).

What if it didn’t have to be that way? What if we could know the impact of laws, the impact of proposals before they happen? What if we could take action before proposed legislation becomes law, and (at least for those folks who live in places with democratically elected governments) possibly even participate in the making of the law.

How Generative AI Helps

Generative AI – large language models (LLMs) in particular – are good at creating stuff, but they’re better at processing the data you already have. One of the bedrock truths of language models is that the more data you bring to the party, the less they’re inclined to make things up because they don’t have to do any of the work to bring words out of long-term memory (possibly incorrectly).

Think of a large language model like a library. You go to the librarian and ask for a book, but you’re not as precise as you could be. The librarian does their best and brings back what they think is what you asked for – only because you weren’t clear, they didn’t get it quite right.

Now, suppose you went to the library and asked the librarian for a book just like the one you brought with you. The librarian instantly reads the entire book you just brought, and finds the book that’s the closest match in the library. You’ll get a much closer result when the librarian doesn’t really need to ask you detailed questions to find what you’re looking for. You just give the book over and the librarian does the work.

Conceptually, that’s what happens when you provide data to an LLM. It reads through everything you provide it and tends to use that information first before relying on its own long-term memory (which may be less current and less reliable). The more data you provide, the less it has to look up.

So how does this advance our citizenship skills? If we provide pending or actual legislation to an LLM, we can ask pointed questions of it, questions that we might not necessarily be able to quickly find answers to ourselves. More important, today’s large foundational models are VERY fluent at reading and interpreting law.

What sorts of questions might we want to ask? Well, anything that concerns you.

For example, you might write a long prompt that describes who you are, what interests you have, what kind of business or work you do, a sort of biography. Then you could ask an LLM to read through some legislation and ask how, if at all the proposed or actual legislation impacts you, asking it to return the results with exact citations and verbatims so you can check that it’s accurately recalling information.

One of my favorite questions to ask about is unintended consequences. Given a piece of legislation, I’ll ask a model to infer what some second or third order effects could be, what the unintended consequences of a piece of legislation could be.

Another question I’ll ask frequently is how a law could be misused. I’ll ask the model to assume that an unethical government official is in office. How could that government official use the law to their own benefit, or to do harm, or to further marginalize already vulnerable populations. You’d be surprised at some of the answers.

Finally, and this is where generative AI really shines, I’ll ask it for recommendations. It’s not enough to complain about something – to effect change, we have to do something. We have to take action.

For example, in the city where I live, I loaded all our city’s bylaws and ordinances into Google’s Gemini 1.5 Pro model. I prefer Gemini 1.5 for this kind of task because it can ingest up to 1.5 million words at a time, and many government documents are quite hefty. It identified 12 city ordinances that are so vague and poorly worded, an unethical mayor or government official could weaponize against any part of the population they deemed unacceptable.

One of the regulations is:

“No person shall own or keep in the City any dog which, by barking, biting, howling or in any other manner disturbs the quiet of any person for a prolonged period such that a reasonable person would find such behavior disruptive to one’s quiet and peaceful enjoyment.”

On the surface, that seems reasonable – except that it’s really vague. What’s a prolonged period? How do you disturb the quiet of a person? What does that mean?

It’s poorly written laws like that which would allow an unethical mayor to selectively enforce the law. They could direct police to only answer noise complaints in certain neighborhoods. They could direct police to to enforce the law only against dog owners of a certain ethnicity. And because the law is so vague, a person in a majority could say, “well, my neighbor’s dog was disturbing my quiet” and the complaint would have to be taken at face value because it’s aligned with the poorly written law.

Gemini suggested the following:

“No person shall own or keep in the City any dog which, by barking, biting, howling or in any other manner disturbs the quiet of any person for a prolonged period. By quiet, the dog shall be making noise in excess of 65dB measured at the property line. By prolonged period, the disturbance must last for 30 continuous minutes or more during the hours of 9 AM – 9 PM, and 15 continuous minutes or more during the hours of 9 PM – 9 AM.”

The level of specificity in the revision means it’s much more difficult for anyone to weaponize a law against a neighbor they didn’t like, but especially for a government to weaponize it against a specific population.

Now, what do we do with it? In my case, I lobby City Hall and my district’s City Councilor to have that law amended. That’s how we upgrade our laws to be more inclusive, less vague, and more helpful.

Here’s another use case that’s relevant right now in the USA where I live. There’s a big election cycle coming up, and various parties and groups have released their platform statements, some of which are hundreds of pages long. I could load those into an LLM along with a voice memo of the things I care about and see which candidates or parties are most closely aligned to my beliefs.

I could ask questions of parties and candidates that I don’t know and quickly get a sense of how I might want to vote. For example, we know who the big names are in an election. There isn’t much left to know at the highest offices in the land. But what about something like the 6th Middlesex County District State Representative? The holder of that office votes on laws that directly affect me locally, and yet I might not know what their record is, what they voted for or against, or what they’re working on.

And realistically, that person I don’t know at the local level has far more impact on my life than the big names on the big stages. Shouldn’t I know what they’re doing? With the power of generative AI, I could.

What To Do Next

Here’s what you should do next, what your key takeaways are. First, familiarize yourself with good research AI tools. I like Google’s Gemini; if you want something that’s really strict, Google’s free NotebookLM is exceptionally rigid. It won’t return results if you didn’t provide the data, and is ideally suited for high-risk applications like law, finance, and medicine.

Second, get hold of the necessary data. Start with the laws and legislation in your city or province, and start loading those laws into the AI tool of your choice. Write up your concerns and who you are so that the models can interpret the law appropriately, digesting it down to the parts that are relevant to you.

Third, build your prompts to ask questions you care about – how this impacts you, how might someone misuse it intentionally, what the unintended consequences are. Watch the video version of this to see this in action. Ask those questions, have a real, in-depth conversation, and understand what it is that your government is working on.

Fourth, take action. Analysis is fine, but as we’ve said at Trust Insights for years and years, analysis without action is distraction. Do something with the knowledge you gain. Suggest legislative changes. Share your findings. Support those people running for office or in office that are aligned with your views and the changes you want to see made. Maybe even run for office yourself, now that you have the power of AI at your side.

The power to understand, monitor, and challenge your government to improve the quality of its work is yours more than ever now with the power of AI. Politicians can’t hide gimmicks and gotchas inside obscure laws, not if you are empowered as a citizen to find them and call them out using AI. The power is in your hands – use it accordingly.

How Was This Issue?

Rate this week’s newsletter issue with a single click. Your feedback over time helps me figure out what content to create for you.

Here’s The Unsubscribe

It took me a while to find a convenient way to link it up, but here’s how to get to the unsubscribe.

If you don’t see anything, here’s the text link to copy and paste:

https://almosttimely.substack.com/action/disable_email

Share With a Friend or Colleague

If you enjoy this newsletter and want to share it with a friend/colleague, please do. Send this URL to your friend/colleague:

https://www.christopherspenn.com/newsletter

For enrolled subscribers on Substack, there are referral rewards if you refer 100, 200, or 300 other readers. Visit the Leaderboard here.

Advertisement: Bring Me In To Speak At Your Event

Elevate your next conference or corporate retreat with a customized keynote on the practical applications of AI. I deliver fresh insights tailored to your audience’s industry and challenges, equipping your attendees with actionable resources and real-world knowledge to navigate the evolving AI landscape.

👉 If this sounds good to you, click/tap here to grab 15 minutes with the team to talk over your event’s specific needs.

If you’d like to see more, here are:

ICYMI: In Case You Missed it

Besides the recently updated Generative AI for Marketers course I’m relentlessly flogging, this week I did a five part series on AGI.

Skill Up With Classes

These are just a few of the classes I have available over at the Trust Insights website that you can take.

Premium

Free

Advertisement: Generative AI Workshops & Courses

Imagine a world where your marketing strategies are supercharged by the most cutting-edge technology available – Generative AI. Generative AI has the potential to save you incredible amounts of time and money, and you have the opportunity to be at the forefront. Get up to speed on using generative AI in your business in a thoughtful way with Trust Insights’ new offering, Generative AI for Marketers, which comes in two flavors, workshops and a course.

Workshops: Offer the Generative AI for Marketers half and full day workshops at your company. These hands-on sessions are packed with exercises, resources and practical tips that you can implement immediately.

👉 Click/tap here to book a workshop

Course: We’ve turned our most popular full-day workshop into a self-paced course. The Generative AI for Marketers online course is now available and just updated as of April 12! Use discount code ALMOSTTIMELY for $50 off the course tuition.

👉 Click/tap here to pre-register for the course

If you work at a company or organization that wants to do bulk licensing, let me know!

Get Back to Work

Folks who post jobs in the free Analytics for Marketers Slack community may have those jobs shared here, too. If you’re looking for work, check out these recent open positions, and check out the Slack group for the comprehensive list.

Advertisement: Free Generative AI Cheat Sheets

Grab the Trust Insights cheat sheet bundle with the RACE Prompt Engineering framework, the PARE prompt refinement framework, and the TRIPS AI task identification framework AND worksheet, all in one convenient bundle, the generative AI power pack!

Download the bundle now for free!

How to Stay in Touch

Let’s make sure we’re connected in the places it suits you best. Here’s where you can find different content:

My blog – daily videos, blog posts, and podcast episodes
My YouTube channel – daily videos, conference talks, and all things video
My company, Trust Insights – marketing analytics help
My podcast, Marketing over Coffee – weekly episodes of what’s worth noting in marketing
My second podcast, In-Ear Insights – the Trust Insights weekly podcast focused on data and analytics
On Threads – random personal stuff and chaos
On LinkedIn – daily videos and news
On Instagram – personal photos and travels
My free Slack discussion forum, Analytics for Marketers – open conversations about marketing and analytics

Listen to my theme song as a new single:

Advertisement: Ukraine 🇺🇦 Humanitarian Fund

The war to free Ukraine continues. If you’d like to support humanitarian efforts in Ukraine, the Ukrainian government has set up a special portal, United24, to help make contributing easy. The effort to free Ukraine from Russia’s illegal invasion needs your ongoing support.

👉 Donate today to the Ukraine Humanitarian Relief Fund »

Events I’ll Be At

Here are the public events where I’m speaking and attending. Say hi if you’re at an event also:

MAICON, Cleveland, September 2024
MarketingProfs B2B Forum, Boston, November 2024
Social Media Marketing World, San Diego, April 2025

There are also private events that aren’t open to the public.

If you’re an event organizer, let me help your event shine. Visit my speaking page for more details.

Can’t be at an event? Stop by my private Slack group instead, Analytics for Marketers.

Required Disclosures

Events with links have purchased sponsorships in this newsletter and as a result, I receive direct financial compensation for promoting them.

Advertisements in this newsletter have paid to be promoted, and as a result, I receive direct financial compensation for promoting them.

My company, Trust Insights, maintains business partnerships with companies including, but not limited to, IBM, Cisco Systems, Amazon, Talkwalker, MarketingProfs, MarketMuse, Agorapulse, Hubspot, Informa, Demandbase, The Marketing AI Institute, and others. While links shared from partners are not explicit endorsements, nor do they directly financially benefit Trust Insights, a commercial relationship exists for which Trust Insights may receive indirect financial benefit, and thus I may receive indirect financial benefit from them as well.

Thank You

Thanks for subscribing and reading this far. I appreciate it. As always, thank you for your support, your attention, and your kindness.

See you next week,

Christopher S. Penn

Almost Timely News: 🗞️ My AI-Enabled Content Marketing Process (2024-07-21)

Almost Timely News: 🗞️ My AI-Enabled Content Marketing Process (2024-07-21) :: View in Browser

👉 New! Take my new webinar on demand, Generative AI for Email Marketing!

Content Authenticity Statement

100% of this week’s newsletter was generated by me, the human, as evidenced by all the screwups in the video version. Learn why this kind of disclosure is a good idea and might be required for anyone doing business in any capacity with the EU in the near future.

Watch This Newsletter On YouTube 📺

Almost Timely News: 🗞️ My AI-Enabled Content Marketing Process (2024-07-21)

Watch this video on YouTube.

Click here for the video 📺 version of this newsletter on YouTube »

Click here for an MP3 audio 🎧 only version »

What’s On My Mind: My AI-Enabled Content Marketing Process

It’s been a little while since I’ve last looked at my overall content marketing process, but over on LinkedIn, Peter and a couple of other folks were asking about it.

My Content Marketing Strategy

First, the overall strategy is one I picked up from Jay Baer years ago. Jay said, “Having the recipe doesn’t make you a chef,” which is an amazing way to encapsulate how I think about creating and sharing content.

Strategically, I think of my content as a cooking show, more or less. I create content and show off recipes around AI, data science, and analytics with the understanding that it’s not the whole picture, nor is it the process in full. Just as a cooking show doesn’t last for 6 hours while a roast is in the oven, nor does my content show the boring, long-wait parts of the process.

And like a cooking show, the strategic intent is to entertain, educate, and inspire, with the understanding that a significant portion of the audience has absolutely no interest in cooking the dish themselves, but would gladly show up at a restaurant to have it cooked for them.

In general, there are three kinds of ways your content helps others, just as there are three broad categories of consuming food:

Done for you
Done with you
Done by you

Done for you is the restaurant. You show up, you pay a premium to have someone else make the dish for you, but you have almost no effort on your part other than eating and paying the bill. When you hire Trust Insights, a good amount of the time clients are looking for us to do the cooking. They want to eat at our restaurant. I enjoy that kind of work immensely.

Done with you is the meal kit. You order from a delivery service, and they provide a meal kit that lets you do some of the cooking, but the majority of the boring stuff is already done, like all the slicing and chopping. You still pay a premium, but it’s not necessarily as high a premium as going out to eat. When you hire me to lead a workshop or as a keynote speaker, that’s more or less what you’re buying.

Done by you is you doing the cooking. You get the recipe, you buy the groceries, you cook the dish in your kitchen. You pay for the raw materials in money, and then you pay in time and effort to make the dish, as well as pay in terms of skill development. Anyone who’s watched Nailed It! knows that skills are a substantial differentiator. When you buy my books, subscribe to this newsletter, or read my content around the Internet, you’re accepting the responsibility of doing the cooking. It’s the lowest cost in financial terms.

And that’s a key point to the entire spectrum. The overall cost is more or less the same – it’s a question of how much of the cost is money versus time, effort, and skill. No matter what approach you prefer, you pay in some fashion.

My Content Marketing Process

So that’s the strategy. Now, what about the recipe, the implementation? I have a flow for my personal content that goes roughly like this:

Seed
Post
Video
Audio
Article
Newsletter

Let’s step through each of these pieces. First, the seed – the idea that gives rise to content. Where do I get ideas? Ideas come from a few different places. I spend a lot of time on LinkedIn and Threads, reading through what people have to say. Often, I’ll comment on posts that interest me, and those comments can become full posts of their own. I talked about this recently. Other times, I’ll see something that annoys me so much, I need to write a contrarian piece without denigrating the original author.

Another source of seeds is from webinars and events. I’ve given dozens of these talks over the years, and collected the questions from the Q&A portion. Now, you might think that I’d run out of questions after a while, and if I stuck with the original questions that would be true. But I take the questions, feed them to generative AI and ask the language model to infer what additional questions the audience might ask based on the questions they’ve already asked, and an audience profile. You’ll never, ever run out of questions that way.

I was at a CMO virtual event recently and was asked about 40 questions. Because it was a private event, I knew the audience by name, so I took their LinkedIn profiles, put them all into a language model, and said, “Okay, here are the questions they already asked, give me 20 new questions based on the questions they asked.” I got a great list of questions.

And then I extended this with Gemini. I prompted, “Okay, now write 20 questions that one of these folks would ask privately, a trusted advisor, a confidant—not in public, not in a forum, not in a place where they would feel embarrassed asking questions.” Like, “Gosh, I wonder if I can get it to do this.” You can imagine where that line of thinking is going. But the questions it came up with were fantastic.

These were questions like, “What if our AI-powered campaign goes rogue and we can’t control it? If everyone starts using AI to game the system, won’t marketing just become a noise war nobody can win? What happens to human connection, to authenticity and marketing, if everything’s driven by algorithms? Is it wrong to use AI to create hyper-personalized ads that play on people’s insecurities? It’s super effective, but is it wrong?”

These are the questions that a CMO might ask, maybe privately or maybe not out loud at all. With this technique, you never run out of questions this way. As long as you’ve got existing questions to get a sense of what’s on their minds, and you’ve got your audience profile, you have an infinite amount of targeted, focused, intelligent questions that are very likely the questions people have in their minds.

And of course, the third source of seeds is just whatever’s bouncing around in my head at any given time. I use voice notes, the reminders app on my phone and Apple Watch, and pretty much any other memory capture mechanism I can so that when I think of something, I can write it down. Often, as I did with last week’s newsletter, it’s just a half-baked mishmash of words, but occasionally it gels together. (at the moment the last two items on my ideas list are “iPhone Peltier cooler case” and “what is responsible AI anyway”)

Once I have a seed, I write a post for LinkedIn. LinkedIn is my first destination to post these idea seeds because of the length limits. I want to see if the idea has any legs at all, and so I’ll draft posts for LinkedIn. I also get the benefit of seeing how well people engage with an idea, whether it deserves to be made into something bigger or just left out there as is.

For those ideas that get traction and feedback (which I mentally define as any comments that aren’t AI spam and at least 1,000 impressions in the platform), I then bundle those up on Sunday afternoons and shoot them as videos, ideally a total of five. I typically record natively on my Mac or if I need to show something, in Streamyard. I do these in bulk, sitting down for the 25-45 minutes it takes to generate the videos as one big block.

I then take that block and put it into Adobe Premiere, do color adjustment and audio engineering on the block as a whole, then cut it up into pieces. After that, each piece gets rendered as video, then converted to MP3 audio. The audio I then feed to whisper.cpp, the local transcription model I use. That becomes rough transcripts I stick into Google Gemini to clean up and also make YouTube captions and tags. After that, each video gets posted to YouTube and I put up an accompanying blog post, plus the MP3s as a podcast.

If I’m feeling motivated and caffeinated, that process takes about 90 minutes on a Sunday afternoon/early evening. I schedule the posts and that’s that. When a local, unlimited version of software like Opus Clips comes out that uses local AI models, I’ll probably start making shorts and Reels from the videos but right now I’m not inclined to invest the time or money to do that.

After all that, if there’s a seed that really stuck out and has outperformed, I’ll make that into a newsletter topic. If not, I’ll just write the newsletter normally, which is usually on a Saturday afternoon or evening. For whatever reason, I prefer to write my newsletter on my iPad and then do all the work to produce it at my desk, but I enjoy the iPad writing experience a lot.

We do a more advanced version of this process for the Trust Insights podcast and livestream that involves much more stuff like those Reels and social media sharing, but I don’t feel like investing the time in it for my personal stuff, nor do I feel like paying someone to do it. As AI evolves, I’m sure I’ll be able to have a model and some Python code do that for me eventually, but not now.

That’s the implementation of my content marketing strategy. What’s notable about it is idea reuse. I follow the reverse of my friend Todd Defren’s content atomization strategy; back in 2008, Todd said the ideal strategy was to produce a piece of huge pillar content and divide it up into as many pieces as possible to distribute it everywhere. That’s a sound strategy that still works today, but my strategy goes in reverse. Like the T-1000 Terminator, I have lots of little bits of content all over the place that flow together and aggregate, getting larger and more complex, from Threads replies and LinkedIn comments ultimately into long form pieces.

My upcoming book, The Intelligence Revolution, will be the culmination of that process as I’m using generative AI to glue together two years worth of LinkedIn posts, blog posts, YouTube videos, and newsletters. It’s not plagiarism if you’re only stealing from yourself! (well, technically it is still plagiarism but there are no legal consequences)

This process is also a guideline. It’s not rigid; I can change it at any time. Since (for my personal stuff) I have only a team of 1 human (me) plus AI, I can make alterations whenever I want. Sometimes an idea will skip the line. Other times I want to publish an idea on an owned property first so that I can lay claim to it as soon as possible (you’ll see this next week in the Trust Insights newsletter). But in general, this is the recipe I try to stick to, mainly so I don’t have to make distinct content for every possible channel.

So that’s my workflow for now. As tools and technologies evolve, it will change; the release of Gemini 1.5 earlier this year changed it substantially from what it used to be, and I anticipate future changes will alter it even more. But what’s important to note is that even though I leverage AI a ton in the process of varying types, the ideas and the outcomes are still human-led.

Not because AI can’t do those parts (it can, quite capably) but because I enjoy it. I enjoy the process of making content, and because this newsletter and my associated properties have always been a bit of a hobby, I see no reason to turn over anything to the machines that I enjoy. I absolutely turn over the parts I don’t enjoy (like transcript cleanup) as quickly as I can, but for something that’s yours, you should never feel compelled to automate away the fun parts.

That’s the behind-the-scenes on my content marketing cooking show. How does it compare to your processes? What, if anything, would you do differently or better if you were in my shoes? What, if anything, would you borrow from my recipe to incorporate into your own content marketing cooking?

How Was This Issue?

Rate this week’s newsletter issue with a single click. Your feedback over time helps me figure out what content to create for you.

Share With a Friend or Colleague

If you enjoy this newsletter and want to share it with a friend/colleague, please do. Send this URL to your friend/colleague:

https://www.christopherspenn.com/newsletter

For enrolled subscribers on Substack, there are referral rewards if you refer 100, 200, or 300 other readers. Visit the Leaderboard here.

Advertisement: Bring Me In To Speak At Your Event

Elevate your next conference or corporate retreat with a customized keynote on the practical applications of AI. I deliver fresh insights tailored to your audience’s industry and challenges, equipping your attendees with actionable resources and real-world knowledge to navigate the evolving AI landscape.

👉 If this sounds good to you, click/tap here to grab 15 minutes with the team to talk over your event’s specific needs.

If you’d like to see more, here are:

ICYMI: In Case You Missed it

Besides the recently updated Generative AI for Marketers course I’m relentlessly flogging, this week we walked through advanced prompting techniques in generative AI on the livestream.

Skill Up With Classes

These are just a few of the classes I have available over at the Trust Insights website that you can take.

Premium

Free

Advertisement: Generative AI Workshops & Courses

Imagine a world where your marketing strategies are supercharged by the most cutting-edge technology available – Generative AI. Generative AI has the potential to save you incredible amounts of time and money, and you have the opportunity to be at the forefront. Get up to speed on using generative AI in your business in a thoughtful way with Trust Insights’ new offering, Generative AI for Marketers, which comes in two flavors, workshops and a course.

Workshops: Offer the Generative AI for Marketers half and full day workshops at your company. These hands-on sessions are packed with exercises, resources and practical tips that you can implement immediately.

👉 Click/tap here to book a workshop

Course: We’ve turned our most popular full-day workshop into a self-paced course. The Generative AI for Marketers online course is now available and just updated as of April 12! Use discount code ALMOSTTIMELY for $50 off the course tuition.

👉 Click/tap here to pre-register for the course

If you work at a company or organization that wants to do bulk licensing, let me know!

Get Back to Work

Folks who post jobs in the free Analytics for Marketers Slack community may have those jobs shared here, too. If you’re looking for work, check out these recent open positions, and check out the Slack group for the comprehensive list.

Advertisement: Free Generative AI Cheat Sheets

Grab the Trust Insights cheat sheet bundle with the RACE Prompt Engineering framework, the PARE prompt refinement framework, and the TRIPS AI task identification framework AND worksheet, all in one convenient bundle, the generative AI power pack!

Download the bundle now for free!

How to Stay in Touch

Let’s make sure we’re connected in the places it suits you best. Here’s where you can find different content:

My blog – daily videos, blog posts, and podcast episodes
My YouTube channel – daily videos, conference talks, and all things video
My company, Trust Insights – marketing analytics help
My podcast, Marketing over Coffee – weekly episodes of what’s worth noting in marketing
My second podcast, In-Ear Insights – the Trust Insights weekly podcast focused on data and analytics
On Threads – random personal stuff and chaos
On LinkedIn – daily videos and news
On Instagram – personal photos and travels
My free Slack discussion forum, Analytics for Marketers – open conversations about marketing and analytics

Listen to my theme song as a new single:

Advertisement: Ukraine 🇺🇦 Humanitarian Fund

The war to free Ukraine continues. If you’d like to support humanitarian efforts in Ukraine, the Ukrainian government has set up a special portal, United24, to help make contributing easy. The effort to free Ukraine from Russia’s illegal invasion needs your ongoing support.

👉 Donate today to the Ukraine Humanitarian Relief Fund »

Events I’ll Be At

Here are the public events where I’m speaking and attending. Say hi if you’re at an event also:

MAICON, Cleveland, September 2024
MarketingProfs B2B Forum, Boston, November 2024
Social Media Marketing World, San Diego, April 2025

There are also private events that aren’t open to the public.

If you’re an event organizer, let me help your event shine. Visit my speaking page for more details.

Can’t be at an event? Stop by my private Slack group instead, Analytics for Marketers.

Required Disclosures

Events with links have purchased sponsorships in this newsletter and as a result, I receive direct financial compensation for promoting them.

Advertisements in this newsletter have paid to be promoted, and as a result, I receive direct financial compensation for promoting them.

My company, Trust Insights, maintains business partnerships with companies including, but not limited to, IBM, Cisco Systems, Amazon, Talkwalker, MarketingProfs, MarketMuse, Agorapulse, Hubspot, Informa, Demandbase, The Marketing AI Institute, and others. While links shared from partners are not explicit endorsements, nor do they directly financially benefit Trust Insights, a commercial relationship exists for which Trust Insights may receive indirect financial benefit, and thus I may receive indirect financial benefit from them as well.

Thank You

Thanks for subscribing and reading this far. I appreciate it. As always, thank you for your support, your attention, and your kindness.

See you next week,

Christopher S. Penn

Category: Almost Timely

Almost Timely News: 🗞️ AI Agents 101 (2024-09-22)

The Big Plug

Content Authenticity Statement

Watch This Newsletter On YouTube 📺

What’s On My Mind: AI Agents 101

Part 1: What Is an AI Agent?

Part 2: Why Do We Care About AI Agents?

Part 3: What Does the Agent Ecosystem Look Like?

Part 4: How Do You Build an AI Agent?

Wrapping Up

How Was This Issue?

Share With a Friend or Colleague

Advertisement: Bring Me In To Speak At Your Event

ICYMI: In Case You Missed it

Skill Up With Classes

Premium

Free

Advertisement: Generative AI Workshops & Courses

Get Back to Work

Advertisement: Free Generative AI Cheat Sheets

How to Stay in Touch

Advertisement: Ukraine 🇺🇦 Humanitarian Fund

Events I’ll Be At

Required Disclosures

Thank You

Almost Timely News: 🗞️ Next Generation AI Models (2024-09-15)

Content Authenticity Statement

Watch This Newsletter On YouTube 📺

What’s On My Mind: Next Generation AI Models

Part 1: What is a Next Generation AI Model?

Part 2: How Will We Use Next Generation Capabilities?

Part 3: What are the Second Order Effects of Next Generation Models?

How Was This Issue?

Share With a Friend or Colleague

Advertisement: Bring Me In To Speak At Your Event

ICYMI: In Case You Missed it

Skill Up With Classes

Premium

Free

Advertisement: Generative AI Workshops & Courses

Get Back to Work

Advertisement: Free Generative AI Cheat Sheets

How to Stay in Touch

Advertisement: Ukraine 🇺🇦 Humanitarian Fund

Events I’ll Be At

Required Disclosures

Thank You

Almost Timely News: 🗞️ The Basics of How To Launch a Podcast From Scratch (2024-09-08)

Content Authenticity Statement

Watch This Newsletter On YouTube 📺

What’s On My Mind: The Basics of How To Launch a Podcast From Scratch

Part 1. Podcast Strategy

Part 2. Podcast Tactics

Create

Distribute

Activate

Measure

Part 3: Outcomes

How Was This Issue?

Share With a Friend or Colleague

Advertisement: Bring Me In To Speak At Your Event

ICYMI: In Case You Missed it

Skill Up With Classes

Premium

Free

Advertisement: Generative AI Workshops & Courses

Get Back to Work

Advertisement: Free Generative AI Cheat Sheets

How to Stay in Touch

Advertisement: Ukraine 🇺🇦 Humanitarian Fund

Events I’ll Be At

Required Disclosures

Thank You

Almost Timely News: 🗞️ Generative AI’s Most Useful Healthcare Use Case (2024-09-01)

Content Authenticity Statement

Watch This Newsletter On YouTube 📺

What’s On My Mind: Generative AI’s Most Useful Healthcare Use Case

How Was This Issue?

Share With a Friend or Colleague