You Ask, I Answer: How Long Should ChatGPT Prompts Be?

In today's episode, Marie asks, how long should our prompts be for chat GPT? How long should a piece of string be? It depends. It depends on what you're doing. Remember that there are six major use cases for generative AI, right?

So you have generation, aka making stuff, you have extraction, taking data out of other data. You have summarization, which is taking big data into small data. You have uh rewriting, turning one form of data into another form of data, you have uh classification, organizing your data, and then you have question answering, which is giving the tool your data and asking questions from it. Of those six use cases, generation is the one that is hardest for models and therefore needs the longest prompts because you're asking it to make up new stuff, net new things that it does not have necessarily have firm data for. What's in a language model is essentially statistics.

It's the probability that one word appears next to another word, and thus, if I say, God save the, and if you're English, uh you're either gonna say queen or king, uh, depending on on how updated your your mental database is. Uh, if I say I pledge allegiance to the statistically, it's probably unlikely the next word is Rutabaga. But when it comes to generation, language models are still guessing. They're they guess with probability. And sometimes the probability is pretty certain, right?

I pledge allegiance to the flag is the most likely uh autocomplete for that sentence. But it's not a hundred percent, nothing's a hundred percent uh probability-wise. So to reduce the likelihood of bad outputs of of wrong guesses, essentially, you need to provide a lot of information. So your prompts for generation should be long, right? They should be extensive, they should contain information about your writing style or how you want a document written.

It should contain maybe some examples of what you want written, it should contain detailed instructions. For generation, I like to say that treat these tools like they're interns, right? Like they're the world's smartest intern. You just hire this intern, they have 255 PhDs, but they still don't know anything about your company specifically. So you want to bring a lot of very specific data for tasks like summarization, classification, uh rewriting, and extraction, your prompts are going to be relatively short, maybe a couple of paragraphs at most of just specific details you want to extract or summarize or examples of formats.

And then you're gonna provide the data itself, right? Here's the transcript, here's the articles, here's the database. Do those tasks. And the same with question answering, too. Question answering, you're providing a lot of the data, and as a result, the data is part of the prompt.

We just don't think of it that way because we're not having to write it by hand. If we say, uh, you know, summarize this transcript and we put in the entire transcript, that is technically part of the prompt. It is just that you're not typing it by hand. You are copying and pasting from another piece of software. So in general, your prompts should be as long as they need to be to provide complete and thorough information, as though you were giving instructions to the intern.

If you give less complete instructions, the intern is going to have some trouble doing the task, right? The intern's gonna maybe wing it, and that doesn't always go so well. So make your prompts as long as they need to be to provide comprehensive and complete information, regardless of the task, and when in doubt, provide more data. When in doubt, if you're if you're worried that a model's not going to behave a certain way or you're worried about the outcome, today's language models can handle incredible amounts of information. Chat GPT can handle what, 90,000 words at a time.

Anthropic Claude and Google Gemini can handle 700,000 words at a time. This book here, this giant book, is 70,000 words. A tool like Google Gemini can can have 10 of these as in their prompt and be able to process it. So make your prompts as long as they need to be. Don't worry about space, don't worry about running out of resources for the major public language models.

You'll you'll have a much better time with that. Thanks for the question. That's gonna do it for this episode. Talk to you on the next one. If you enjoyed this video, please hit the like button.

Subscribe to my channel if you haven't already, and if you want to know when new videos are available, hit the bell button to be notified as soon as new content is live.