Christopher S. Penn – Marketing AI Keynote Speaker

Almost Timely News, March 17, 2024: A Large Language Model Power Technique

—

by

Christopher S Penn
in , , , , ,

Almost Timely News: A Large Language Model Power Technique (2024-03-17) :: View in Browser

Almost Timely News

Check out these two new talks, free for you to enjoy:

Content Authenticity Statement

90% of this week’s newsletter was generated by me, the human. You will see many outputs of generative AI in the opening section. Learn why this kind of disclosure is a good idea and might be required for anyone doing business in any capacity with the EU in the near future.

Watch This Newsletter On YouTube 📺

Almost Timely News: A Large Language Model Power Technique (2024-03-17)
Watch this video on YouTube.

Click here for the video 📺 version of this newsletter on YouTube »

Click here for an MP3 audio 🎧 only version »

What’s On My Mind: A Large Language Model Power Technique

This week, I’m going to share with you one power technique that will dramatically increase your productivity and accuracy when using large language models, like the ones that power ChatGPT and similar software. But before I delve into the technique itself, we need to go over some very basic foundational aspects of transformer-based models.

Large language models have two conceptual types of memory. This explanation is mathematically incorrect, so know that from the beginning, if you go read the relevant academic papers and you do the calculus yourself, this is not how the models mechanically work under the hood. But for now, this is a good conceptual model to remember.

Like you and me, language models have two kinds of memory, long-term and short-term. Long-term memory, also known by the fancy technical term latent space, is the statistical library of everything the model was trained on. To be clear, the original documents it was trained from are NOT in its long-term memory. Instead, a model’s long-term memory is essentially a bunch of associations. It’s seen “I pledge allegiance to the” so many times that it can predict with near-mathematical certainty that the next word in the sentence should be flag, at least in North American English.

In general, the long-term memory of a language model is very large and diverse. It’s seen everything from the most credentialed academic papers to your drunk uncle’s Reddit shitposts, and it has learned language patterns from it (this is why all models have biases). The process of building that long-term memory is so intensive and expensive that companies rarely do it. When they do, there’s usually a big announcement, like when GPT-4, Claude 3, or Gemini 1.5 were announced, to much fanfare. This is why models often warn that they only have knowledge up to a certain date.

The short-term memory of a language model starts empty. It’s the memory used during a conversation, so when you first open up a browser window to Gemini or Claude or ChatGPT, there’s literally nothing there. The short-term memory is governed by some internal mechanics, but all models have a defined short-term memory; after you reach the limit of that memory, it starts forgetting the earliest parts of your conversation. How short is short-term memory? For some models, like the open-source LLaMa 2 model, the memory is as little as 2,200 words. For other models, like Claude 3 Opus and the upcoming Google Gemini 1.5, it’s as large as 700,000 words. Big or small, eventually models do run out of memory and delete the earliest information in their memories.

What makes transformers-based models – which is basically almost all language models today – different from their predecessors is a technical concept called attention. Without getting into details, the short conceptual summary is that attention means a model takes into account everything in its short-term memory to help predict what it’s going to say next. Everything in our prompt and subsequent conversations is used by the model for what it generates next.

I often call language models the smartest, most forgetful interns you’ve ever worked with. Their knowledge in their long-term memories is substantial, more than any human could know. Their short-term memory means they have to constantly be reminded of what they know, even within the space of a single conversation.

So, with that somewhat technical background, let’s talk about this prompting power technique. It goes like this:

What do you know about this topic?

Kind of anti-climactic, isn’t it? Let’s see what it looks like in practice so we can see the value of it. I’ll start with the basic RACE framework prompt (grab the free, no form or data required PDF here):

You are an expert virologist and epidemiologist. You know viruses, viral replication, RNA, mRNA, protein structures, spike proteins, COVID-19, SARS-CoV-2, COVID variants, and immune escape. Your first task is to explain the importance of position 13,468 on Open Reading Frame 1ab of the SARS-CoV-2 virus and its significance for immunity.

That prompt fulfills the basic RACE framework. Let’s see what a model like ChatGPT says:

ChatGPT output

And for good measure, let’s see what Google Gemini says:

Google Gemini output

What’s important to note here is that both models have failed to mention the most salient fact about ORF1ab and position 13,468: it’s the mutation of the current COVID variant.

Now, let’s add our power technique:

You are an expert virologist and epidemiologist. You know viruses, viral replication, RNA, mRNA, protein structures, spike proteins, COVID-19, SARS-CoV-2, COVID variants, and immune escape. Your first task is to explain the importance of position 13,468 on Open Reading Frame 1ab of the SARS-CoV-2 virus and its significance for immunity. What do you know about this topic?

Let’s see what ChatGPT has to say:

ChatGPT output

and Gemini:

Google Gemini output

In both cases, the query that ends with asking it what it knows delivers more detailed results. That alone is helpful. However, more important is the continued omission of the basic fact that the JN.1 variant is why this mutation is so significant.

What does that tell us? Right out of the gate, it tells us that the long-term memory of these models does NOT have any information about the JN.1 variant, which in turn means that if we want to use generative AI for this topic, we will need to supply that information ourselves. This, by the way, is one of the reasons why subject matter expertise and human review still matter a great deal. To the untrained eye, these responses might look fine, but for anyone in virology, these are deeply deficient responses.

Second, if we were to proceed, then the short-term memory already has quite a bit more information in it. Remember that these transformers-based models take into account everything within the short-term memory, so by asking a model what it knows, we’ve effectively multiplied the length of the starting prompt by several times.

Let’s take something that requires a bit less current knowledge to further illustrate the point. Suppose you are a real estate agent and you want to write up a description of a property for sale. Let’s say these are the basics of the property:

  • 2 bedrooms, 1 full bath
  • House built in 1920
  • House in terrible condition but livable
  • 6 rooms total
  • 4770 square foot lot
  • $4,064 annual taxes
  • Seller asking $440,000
  • House has 1,122 square feet total with a shingle roof, block foundation, bungalow style
  • House has city water and sewer
  • Not in a particularly nice part of the city
  • Houses selling fast in the city, only 10 single family homes available in the entire city of 70,000 people, very hot market

Now, if you were to use a language model with the RACE framework to prompt it to write a property description, it might go like this:

You are a top-selling real estate agent. You know real estate, realtors, house sales, single family homes, residential real estate, buyer’s agent, seller’s agent. Your first task, as the seller’s agent for a property, is to write a compelling description of this house to attract potential buyers. Here are the details: (paste in the list above) Write a compelling housing description.

What kind of description will we get?

Let’s see what ChatGPT has to say:

ChatGPT output

and Gemini:

Google Gemini output

These descriptions aren’t bad, but the models don’t have much to work with in the prompt, so what they come up with is a bit bland, generic, and dry.

Now, let’s start the conversation a little differently, using our power technique.

You are a top-selling real estate agent. You know real estate, realtors, house sales, single family homes, residential real estate, buyer’s agent, seller’s agent. Your first task, as the seller’s agent for a property, is to write a compelling description of this house to attract potential buyers. What do you know about the best practices for writing compelling real estate property descriptions to attract the attention of buyers?

Each model will tell us what it knows about the topic and some general best practices. What it’s doing as well is filling in what’s basically a very long prompt filled with the general best practices.

Let’s see what ChatGPT has to say:

ChatGPT output

and Gemini:

Google Gemini output

Now, with these pre-loaded expert prompts, let’s take a look at the revised listings.

Let’s see what ChatGPT has to say:

ChatGPT output

and Gemini:

Google Gemini output

Qualitatively, these responses are not just a little better, they’re a lot better. There’s still definitely some tweaking to do to make them better listings, but you can see just how including the basic best practices – pulled from the model’s long-term memory – into the conversation made for a better result.

Naturally, if your firm has a specific style of writing, or you have a database of listings that sold really fast, you’d want to include that context as well to create exactly the kind of output you want.

Asking a model what it knows about a topic is a fast way to tell whether or not the model is even capable of generating correct outputs, and pre-loading the conversation with rich information that substantially improves what it generates. Add this into your workflow to make your work with generative AI and large language models much better.

How Was This Issue?

Rate this week’s newsletter issue with a single click. Your feedback over time helps me figure out what content to create for you.

Share With a Friend or Colleague

If you enjoy this newsletter and want to share it with a friend/colleague, please do. Send this URL to your friend/colleague:

https://www.christopherspenn.com/newsletter

For enrolled subscribers on Substack, there are referral rewards if you refer 100, 200, or 300 other readers. Visit the Leaderboard here.

ICYMI: In Case You Missed it

Besides the new Generative AI for Marketers course I’m relentlessly flogging, I did a four-part series this week as a reaction video to an OLD talk I did on podcast marketing back in 2007.

Skill Up With Classes

These are just a few of the classes I have available over at the Trust Insights website that you can take.

Premium

Free

Advertisement: Generative AI Workshops & Courses

Imagine a world where your marketing strategies are supercharged by the most cutting-edge technology available – Generative AI. Generative AI has the potential to save you incredible amounts of time and money, and you have the opportunity to be at the forefront. Get up to speed on using generative AI in your business in a thoughtful way with Trust Insights’ new offering, Generative AI for Marketers, which comes in two flavors, workshops and a course.

Workshops: Offer the Generative AI for Marketers half and full day workshops at your company. These hands-on sessions are packed with exercises, resources and practical tips that you can implement immediately.

👉 Click/tap here to book a workshop

Course: We’ve turned our most popular full-day workshop into a self-paced course. The Generative AI for Marketers online course is now available and just updated this week! Use discount code ALMOSTTIMELY for $50 off the course tuition.

👉 Click/tap here to pre-register for the course

If you work at a company or organization that wants to do bulk licensing, let me know!

Get Back to Work

Folks who post jobs in the free Analytics for Marketers Slack community may have those jobs shared here, too. If you’re looking for work, check out these recent open positions, and check out the Slack group for the comprehensive list.

What I’m Reading: Your Stuff

Let’s look at the most interesting content from around the web on topics you care about, some of which you might have even written.

Social Media Marketing

Media and Content

SEO, Google, and Paid Media

Advertisement: Business Cameos

If you’re familiar with the Cameo system – where people hire well-known folks for short video clips – then you’ll totally get Thinkers One. Created by my friend Mitch Joel, Thinkers One lets you connect with the biggest thinkers for short videos on topics you care about. I’ve got a whole slew of Thinkers One Cameo-style topics for video clips you can use at internal company meetings, events, or even just for yourself. Want me to tell your boss that you need to be paying attention to generative AI right now?

📺 Pop on by my Thinkers One page today and grab a video now.

Tools, Machine Learning, and AI

Analytics, Stats, and Data Science

All Things IBM

Dealer’s Choice : Random Stuff

How to Stay in Touch

Let’s make sure we’re connected in the places it suits you best. Here’s where you can find different content:

Advertisement: Ukraine 🇺🇦 Humanitarian Fund

The war to free Ukraine continues. If you’d like to support humanitarian efforts in Ukraine, the Ukrainian government has set up a special portal, United24, to help make contributing easy. The effort to free Ukraine from Russia’s illegal invasion needs your ongoing support.

👉 Donate today to the Ukraine Humanitarian Relief Fund »

Events I’ll Be At

Here’s where I’m speaking and attending. Say hi if you’re at an event also:

  • Society for Marketing Professional Services, Boston, April 2024
  • Lab Products Association, Nashville, April 2024
  • Society for Marketing Professional Services, Los Angeles, May 2024
  • Australian Food and Grocery Council, Melbourne, May 2024
  • MAICON, Cleveland, September 2024
  • MarketingProfs B2B Forum, Boston, October 2024

Events marked with a physical location may become virtual if conditions and safety warrant it.

If you’re an event organizer, let me help your event shine. Visit my speaking page for more details.

Can’t be at an event? Stop by my private Slack group instead, Analytics for Marketers.

Required Disclosures

Events with links have purchased sponsorships in this newsletter and as a result, I receive direct financial compensation for promoting them.

Advertisements in this newsletter have paid to be promoted, and as a result, I receive direct financial compensation for promoting them.

My company, Trust Insights, maintains business partnerships with companies including, but not limited to, IBM, Cisco Systems, Amazon, Talkwalker, MarketingProfs, MarketMuse, Agorapulse, Hubspot, Informa, Demandbase, The Marketing AI Institute, and others. While links shared from partners are not explicit endorsements, nor do they directly financially benefit Trust Insights, a commercial relationship exists for which Trust Insights may receive indirect financial benefit, and thus I may receive indirect financial benefit from them as well.

Thank You

Thanks for subscribing and reading this far. I appreciate it. As always, thank you for your support, your attention, and your kindness.

See you next week,

Christopher S. Penn

You might also enjoy:

Want to read more like this from Christopher Penn? Get updates here:

subscribe to my newsletter here

AI for Marketers Book
Take my Generative AI for Marketers course!

Analytics for Marketers Discussion Group
Join my Analytics for Marketers Slack Group!

Comments

Leave a Reply

Your email address will not be published. Required fields are marked *

Pin It on Pinterest

Shares
Share This