I’ve Stopped Writing Prompts—DSPy Does It Better

This piece is more technical than we usually publish—because you’ll be learning alongside our builders via a video tutorial linked below. DSPy is a prompt optimization framework for improving prompt quality and reliability that Every columnist Michael Taylor taught to Spiral general manager Danny Aziz, Cora general manager Kieran Klaassen, and Cora engineer Nityesh Agarwal. Watch the walkthrough and follow their steps to improve your own prompts.—Kate Lee

Was this newsletter forwarded to you? Sign up to get it in your inbox.

Having worked with AI since OpenAI released the GPT-3 beta in 2020, I’ve used language models for everything from automating bank fraud detection to generating and testing thousands of variations of advertisements. For the past year-plus, I’ve written a monthly column for Every on the differences between what works for AIs and humans. I even wrote a book about prompt engineering, the delicate art of tweaking strings of words to get a model to do exactly what I want.

Lately, though, I’ve stopped writing prompts myself. Instead, I use DSPy, an automated prompt-optimization tool that is still relatively obscure, but powerful enough that it could soon do away with prompt engineers like myself. (Shopify CEO Tobi Lutke recently called DSPy “severely underhyped.”)

When DSPy started to gain traction, people breathlessly exclaimed that "prompt engineering is dead." Rather than take it personally, I learned how to use it, and started sending my clients better prompts optimized by DSPy. I can still beat DSPy if I try hard enough, but for anyone with the time and my five years of prompt engineering experience, you’re better off relying on DSPy.

Make your team AI‑native

Scattered tools slow teams down. Every Teams gives your whole organization full access to Every and our AI apps—Sparkle to organize files, Spiral to write well, Cora to manage email, and Monologue for smart dictation—plus our daily newsletter, subscriber‑only livestreams, Discord, and course discounts. One subscription to keep your company at the AI frontier. Trusted by 200+ AI-native companies—including The Browser Company, Portola, and Stainless.

Create your team

Want to sponsor Every? Click here.

Here’s how you do it: You define the inputs you want to give to the AI (the information it needs to do the task) and define a reward function, or evaluation metric (a way to measure how well it did the task). It’s like checking to see if the answer your prompt gave matches the right answer to a question. So DSPy’s optimizers can tell if they’re doing a good job, as they automatically optimize the prompt instructions for me (given the same inputs, trying strategies for getting better outputs). It works across all the major language models, and I can swap the models in and out like Legos, without worrying about the specifics of how OpenAI works as compared to Google or Anthropic.

Say you wanted to extract the right information from thousands of differently formatted invoices rather than entering them manually. You could write a prompt, run it on a few invoices, then check what fields it got wrong, and keep manually adding rules to the prompt until it gets everything right. DSPy writes the instructions automatically, which, depending on the size of your task, could save you days worth of work.

If DSPy is so great, why isn’t it more popular? Its dense documentation is a natural filter. DSPy was created by professors at Stanford, and it can be hard for technical people to explain their inventions to the rest of us. The engineers on the Every team had heard how powerful DSPy was at prompt optimization, checked it out, and found it hard to understand, so they bookmarked it to come back to one day.

That day came when I was in Brooklyn recently and did an impromptu training session with the team. Once you suffer through the steep learning curve, you can get big improvements in your prompt performance, so I was eager to make sure the team weren’t missing out on a powerful tool.

As I told the team, there are three reasons I'm all in:

1. It's incredibly flexible. Just one line of code is enough to swap out the model you’re working with for another one. Want to see how your app performs across OpenAI, Anthropic, or Google’s LLMs? Done. Want to change your program to use chain of thought (think first before answering)? That’s also one line of code.
2. It handles the complex stuff. Building agents used to mean writing hundreds of lines of plumbing code for tool calls, retry logic, function formatting and parsing, and so on. DSPy's ReAct agent (an out-of-the-box template for creating an agent that plans and uses tools) allows you to just define the functions you want to carry out in Python, throw them in a list, and it handles everything.
3. It makes you think better. The framework forces you to define what you're trying to do. What inputs does the LLM need to do its job? What does a “good” output look like? It sounds basic, but most people skip this part and wonder why their AI apps get unreliable results.

The video that follows is a technical lecture, and I jump straight in the deep end with code examples—but if you or someone on your team wants to benefit from the powerful DSPy optimizers or learn more about context engineering, I hope you check it out and use the code as a starting point to optimize your own prompts.

To read more essays like this, subscribe to Every, and follow us on X at @every and on LinkedIn.

We also build AI tools for readers like you. Write brilliantly with Spiral. Organize files automatically with Sparkle. Deliver yourself from email with Cora. Dictate effortlessly with Monologue.

We also do AI training, adoption, and innovation for companies. Work with us to bring AI into your organization.

Get paid for sharing Every with your friends. Join our referral program.