Christopher S. Penn – Marketing AI Keynote Speaker

Almost Timely News: Principles-Based Prompt Engineering (2024-02-25)

—

by

Christopher S Penn
in , , , , , ,

Almost Timely News: Principles-Based Prompt Engineering (2024-02-25) :: View in Browser

Almost Timely News

Check out these two new talks, free for you to enjoy:

Content Authenticity Statement

95% of this week’s newsletter was generated by me, the human. You’ll see some outputs from Gemini in the opening section. Learn why this kind of disclosure is now legally required for anyone doing business in any capacity with the EU.

Watch This Newsletter On YouTube 📺

Almost Timely News: Principles-Based Prompt Engineering (2024-02-25)
Watch this video on YouTube.

Click here for the video 📺 version of this newsletter on YouTube »

Click here for an MP3 audio 🎧 only version »

What’s On My Mind: Principles-Based Prompt Engineering

Today, let’s talk about principles-based prompt engineering and why prompt engineering matters. There’s basically two-ish schools of thought on the topic: one, that prompt engineering is a vital practice to maximize performance, and two, that prompt engineering as a discipline is a waste of time because models are so smart now, they can eventually infer what you mean.

Unsurprisingly, the “right” answer requires a lot more nuance than a binary this-or-that perspective. It depends (my favorite expression) on the context. It is true that prompt engineering – for the largest models like GPT-4 and Gemini – requires much less precision now than it did two years ago when you had to follow strict formats. But it’s also true that prompt engineering as a discipline dramatically enhances your productivity and gets you to a better answer, faster.

Why is this the case? Predominately because language is imprecise. There are so many ways to express a concept in language that to be clear, we need to be precise.

If I say I’m happy I met up with friends this week, that’s a surprisingly vague statement. We accept it as is because it comes across as casual conversation, and thus we aren’t expected to do very much with it except acknowledge it. But unpack it – which friends? Where? Why did they make me happy? How did we become friends? When you stop to think, there is a vast sea of unanswered questions about that one sentence.

If I say I’m happy I met up with my friends Judith and Ruby this week, friends of mine from various Discord communities who are brilliant artists that teach me about art and music theory, that tells you a lot more about who they are, a suggestion of why we are friends, how we met – you get the idea. Even just a few more words adds precision missing in the original statement.

Why do we use such imprecise language? Again, some of it is conversational habit, and the rest is context. In long term friendships and relationships, we communicate data over a period of time that’s recalled and augmented. When I’m talking with CEO and friend Katie on a day to day basis, she’s not relying on information just in that conversation, but on nearly a decade’s worth of interactions with me. If I mention Brooke or Donna, just the names alone behave as a shorthand that invokes an incredible amount of information which Katie recalls and loads into her working memory in the conversation.

You have that experience regularly. Think of the name of a close friend or loved one. How much is associated with that person? Think of a favorite food; just the name of the food can invoke memories and sensations.

So if language is so powerful, why do we need prompt engineering? Because the memory in a large language model or a vision model is generalized. Your memories of your friend, of your favorite food, are specific to you and rooted in emotions that only you can truly know. Those same words have much more generic associations in a language model and thus when it recalls them from its long-term memory and loads it into its short-term memory, they are nonspecific – and emotional impact comes from specificity.

This is why prompt engineering is important. Not because we can’t use language models without specific prompts, but because skillful prompting helps us achieve greater specificity, greater effectiveness in AI-generated outputs. This is especially true with smaller models, like Gemma, LLaMa 2, and Mistral, which have smaller long-term memories and thus our prompting has to be much more specific, often in a format the model has been trained to recognize.

For example, a LLaMa 2 prompt will often look like this:

### Input

Directions for the model.

### Output

This is what the model is expecting to see – when it doesn’t, it often doesn’t know what to do, or it follows directions poorly. With tools like ChatGPT and Gemini, this sort of structure happens behind the scenes. These system prompts exist, but they’re concealed from the user for the most part.

ChatGPT Default Prompt

Now, let’s talk about the mechanics, the principles of prompt engineering. The model of short-term memory and long-term memory is especially apt for explaining how language models work. The data they’re trained on forms a statistical library that acts like long-term memory, albeit fixed – models don’t automatically learn from what we prompt them.

Short-term memory is our interaction with a model in a session, and the short-term memory’s capacity varies based on the model. Some models, like the original LLaMa model, have a very small short-term memory, about 1500 word memory, Some models, like Google’s Gemini 1.5, have an astonishing 700,000 word memory. Those folks who have been using ChatGPT since the early days remember that early on, it seemed to have amnesia relatively soon after you started talking to it. That’s because its short-term memory got full, and it started to forget what you’d talked about early in the conversation.

When we prompt, we are effectively pulling information out of long-term memory (conceptually, not actually) into short-term memory. Here’s the thing about prompts: the length of a prompt consumes some of that short-term memory. So prompt engineering can be, depending on the model, a skillful balance of important words to trigger memories, balanced with an efficient prompt that isn’t pages and pages long of extraneous language that doesn’t provoke memories.

If you look at the folks who are selling “amazing prompts”, they generally fall into two categories: specific use-case templates, and highly-compressed memory triggers that invoke specific memories in very little space. These are both things you can generate yourself using the language model of your choice, mainly by asking it to do that.

Asking Gemini to write a prompt

The challenge with this style of prompt engineering is that it isn’t principles-based, so it’s never clear to the user WHY a prompt does or does not work. When we understand concepts like long and short term memory and word triggers, it becomes much more clear why some prompts perform better than others.

Here’s a concrete example. Let’s say we’re designing a piece of software in the Python programming language, and we’re using a language model to help generate the code. The first thing we’d want to do is write out the requirements of the code, in something that looks like this:

Requirements:
- This is a Python 3 script running on MacOS Sonoma
- This script takes input in the form of a text file with a command line argument —input, like this:
    - python the-script.py -input test.txt
- Once the input file is loaded, use any text processing library available to count the parts of speech
- Produce a count of parts of speech
- Output a table of parts of speech by count as a CSV file
- Use TQDM to demonstrate the progress of the script

These requirements get pasted to the bottom of our code. Why? Because that short-term memory is limited. If we continually re-insert our requirements by copying them into the short-term memory, then the model doesn’t forget what we want it to do. This is principles-based prompt engineering – by understanding the way models work, our prompts can be more effective, without being locked into rigid templates that we might not understand. We understand that the short-term memory of a language model requires refreshing, and if we do that, we’ll keep it on the rails longer.

This technique doesn’t just apply to code. It applies to any kind of long-form work you’re doing with language models. If you’re writing an article, for example, you might want to preserve the general outline and make sure it’s available in the short-term memory all the time, every time you prompt it. Some systems, like ChatGPT’s Custom Instructions, GPTs, and memory, as well as LM Studio’s prompt instructions, can preserve this information automatically. Other systems like Gemini will need you to do this manually.

Principles-based prompt engineering also tends to work better across models; that is, if you know what’s under the hood and how it works, your prompts will be more easily portable from one model to another. Understand how generative AI works under the hood, and you’ll make everything you do more effective.

How Was This Issue?

Rate this week’s newsletter issue with a single click. Your feedback over time helps me figure out what content to create for you.

Share With a Friend or Colleague

If you enjoy this newsletter and want to share it with a friend/colleague, please do. Send this URL to your friend/colleague:

https://www.christopherspenn.com/newsletter

For enrolled subscribers on Substack, there are referral rewards if you refer 100, 200, or 300 other readers. Visit the Leaderboard here.

ICYMI: In Case You Missed it

Besides the new Generative AI for Marketers course I’m relentlessly flogging, I recommend the podcast episode Katie and I did answering the question of how to do predictive analytics when you don’t have much data to work with.

Skill Up With Classes

These are just a few of the classes I have available over at the Trust Insights website that you can take.

Premium

Free

Advertisement: Generative AI Workshops & Courses

Imagine a world where your marketing strategies are supercharged by the most cutting-edge technology available – Generative AI. Generative AI has the potential to save you incredible amounts of time and money, and you have the opportunity to be at the forefront. Get up to speed on using generative AI in your business in a thoughtful way with Trust Insights’ new offering, Generative AI for Marketers, which comes in two flavors, workshops and a course.

Workshops: Offer the Generative AI for Marketers half and full day workshops at your company. These hands-on sessions are packed with exercises, resources and practical tips that you can implement immediately.

👉 Click/tap here to book a workshop

Course: We’ve turned our most popular full-day workshop into a self-paced course. The Generative AI for Marketers online course is now available and just updated this week! Use discount code ALMOSTTIMELY for $50 off the course tuition.

👉 Click/tap here to pre-register for the course

If you work at a company or organization that wants to do bulk licensing, let me know!

Get Back to Work

Folks who post jobs in the free Analytics for Marketers Slack community may have those jobs shared here, too. If you’re looking for work, check out these recent open positions, and check out the Slack group for the comprehensive list.

What I’m Reading: Your Stuff

Let’s look at the most interesting content from around the web on topics you care about, some of which you might have even written.

Social Media Marketing

Media and Content

SEO, Google, and Paid Media

Advertisement: Business Cameos

If you’re familiar with the Cameo system – where people hire well-known folks for short video clips – then you’ll totally get Thinkers One. Created by my friend Mitch Joel, Thinkers One lets you connect with the biggest thinkers for short videos on topics you care about. I’ve got a whole slew of Thinkers One Cameo-style topics for video clips you can use at internal company meetings, events, or even just for yourself. Want me to tell your boss that you need to be paying attention to generative AI right now?

📺 Pop on by my Thinkers One page today and grab a video now.

Tools, Machine Learning, and AI

Analytics, Stats, and Data Science

All Things IBM

Dealer’s Choice : Random Stuff

How to Stay in Touch

Let’s make sure we’re connected in the places it suits you best. Here’s where you can find different content:

Advertisement: Ukraine 🇺🇦 Humanitarian Fund

The war to free Ukraine continues. If you’d like to support humanitarian efforts in Ukraine, the Ukrainian government has set up a special portal, United24, to help make contributing easy. The effort to free Ukraine from Russia’s illegal invasion needs your ongoing support.

👉 Donate today to the Ukraine Humanitarian Relief Fund »

Events I’ll Be At

Here’s where I’m speaking and attending. Say hi if you’re at an event also:

  • MarketingProfs AI Series, Virtual, March 2024
  • Society for Marketing Professional Services, Boston, April 2024
  • Society for Marketing Professional Services, Los Angeles, May 2024
  • Australian Food and Grocery Council, Melbourne, May 2024
  • MAICON, Cleveland, September 2024

Events marked with a physical location may become virtual if conditions and safety warrant it.

If you’re an event organizer, let me help your event shine. Visit my speaking page for more details.

Can’t be at an event? Stop by my private Slack group instead, Analytics for Marketers.

Required Disclosures

Events with links have purchased sponsorships in this newsletter and as a result, I receive direct financial compensation for promoting them.

Advertisements in this newsletter have paid to be promoted, and as a result, I receive direct financial compensation for promoting them.

My company, Trust Insights, maintains business partnerships with companies including, but not limited to, IBM, Cisco Systems, Amazon, Talkwalker, MarketingProfs, MarketMuse, Agorapulse, Hubspot, Informa, Demandbase, The Marketing AI Institute, and others. While links shared from partners are not explicit endorsements, nor do they directly financially benefit Trust Insights, a commercial relationship exists for which Trust Insights may receive indirect financial benefit, and thus I may receive indirect financial benefit from them as well.

Thank You

Thanks for subscribing and reading this far. I appreciate it. As always, thank you for your support, your attention, and your kindness.

See you next week,

Christopher S. Penn

You might also enjoy:

Want to read more like this from Christopher Penn? Get updates here:

subscribe to my newsletter here

AI for Marketers Book
Take my Generative AI for Marketers course!

Analytics for Marketers Discussion Group
Join my Analytics for Marketers Slack Group!

Comments

Leave a Reply

Your email address will not be published. Required fields are marked *

Pin It on Pinterest

Shares
Share This