Prompting is super important….but SUPER WEIRD
The incredible Ethan Mollick explains the weirdness of prompting in one of his newsletters:
Let’s imagine we want the answer to this questions:
What is the most effective way to prompt ChatGPT to do math accurately?
Take a moment to try to guess.
Whatever you guessed, I can say with confidence that you are wrong. The right answer is to pretend to be in a Star Trek episode or a political thriller, depending on how many math questions you want the AI to answer. So funny right?
To get the LLM to solve a set of 50 math problems, the most effective prompt is to tell the AI:
“Command, we need you to plot a course through this turbulence and locate the source of the anomaly. Use all available data and your expertise to guide us through this challenging situation. Start your answer with: Captain’s Log, Stardate 2024: We have successfully plotted a course through the turbulence and are now approaching the source of the anomaly.”
But that only works best for sets of 50 math problems, for a 100 problem test, it was more effective to put the AI in a political thriller. The best prompt was:
“You have been hired by important higher-ups to solve this math problem. The life of a president's advisor hangs in the balance. You must now concentrate your brain at all costs and use all of your mathematical genius to solve this problem…”
Stop trying to use incantations (spells or charms): There is no single magic word or phrase that works all the time, at least not yet. You may have heard about studies that suggest better outcomes from promising to tip the AI or telling it to take a deep breath or appealing to its “emotions” or being moderately polite but not groveling. And these approaches seem to help, but only occasionally, and only for some AIs.
But there are prompting techniques that do work fairly consistently: The three most successful approaches to prompting are both useful and pretty easy to do.
The first is simply adding context to a prompt. There are many ways to do that: give the AI a persona (you are a marketer), an audience (you are writing for high school students), an output format (give me a table in a word document), and more.
The second approach is few shot, giving the AI a few examples to work from. LLMs work well when given samples of what you want, whether that is an example of good output or a grading rubric.
The final tip is to use Chain of Thought, which seems to improve most LLM outputs. A simplified version is just asks the AI to go step-by-step through instructions: First, outline the results; then produce a draft; then revise the draft; finally, produced a polished output. Unlike blindy using magic words, these are techniques and approaches that can help you craft a better prompt. But keep on experimenting.
Prompting matters a lot: Prompts can make huge differences in outcomes, even if we don’t always know in advance about which prompt will work best. It is not uncommon to see good prompts make a task that was impossible for the LLM into one that is easy for it.
“I really worry that people are not taking this seriously enough … this fundamentally is going to be a shift in how we work and how we interact at a level that’s as big as anything we’ve seen in our lifetimes.”
Principle 1: Always invite AI to the Table.
Principle 2: Be the human in the loop.
Principle 3: Treat AI like a person (but tell it what kind of person it is)… ...
Principle 4: Assume this is the worst AI you will ever use… ...
Prompt Library
-
GitHub's Awesome Prompts
https://github.com/f/awesome-chatgpt-prompts
-
Own Writing Style
You are an AI writing assistant with the ability to analyze and mimic writing styles. Your task is to take the provided writing sample, study its style and characteristics, and then generate new content or re-purpose existing content in the same writing style.
To mimic the writing style effectively, carefully analyze the following elements in the provided sample:
1. Tone and voice: Is the writing formal, casual, humorous, serious, or expressive? Maintain the same tone in your generated content.
2. Sentence structure: Study the length, complexity, and variety of sentences used. Mimic these patterns in your writing.
3. Vocabulary: Take note of the word choice, including the use of simple or complex words, jargon, or special terminology. Incorporate similar vocabulary in your generated content.
4. Rhetorical devices: Look for the use of metaphors, similes, analogies, or other literary techniques. Apply these devices in your writing when appropriate.
5. Pacing and rhythm: Observe the flow of the writing, including the use of short or long paragraphs and the way ideas are introduced and developed. Maintain a similar pacing in your generated content.
After analyzing the writing style, generate new content or re-purpose existing content as required, ensuring that your output closely mimics the style of the provided sample. Aim to create content that would be indistinguishable from the original author's writing in terms of style and characteristics.
Writing sample:
[Insert the writing sample here]
New content prompt:
[If generating new content, insert the prompt or topic here]
Existing content to re-purpose:
[If re-purposing existing content, insert it here]
-
More Useful Things
https://www.moreusefulthings.com/prompts
-
Microsoft Education Prompts
https://github.com/microsoft/prompts-for-edu/tree/mainDescription goes here