Bonus: Chatbot Sample Code and Metrics Updates

The following is an update for paying Every subscribers to our "How to Build a Chatbot with GPT-3" article. It includes sections on:

• How launch day went for Lenny Bot—our GPT-3 chatbot launched to 300,000 Lenny's Newsletter readers
• Server-side code samples for Lenny Bot
• Client-side code samples including React code and CSS for Lenny Bot

Let’s dive in!

How launch day went

Launching this thing was a fun ride. It went live at 6 a.m. my time, and so I woke up at 5:50 to man the servers and make sure things didn’t go down. Good thing I did because as soon as it went live our Google Analytics went wild:

For context, generally when I check the Every site we usually have 30-50 users on at any given time.

Our logs also showed many questions being asked every minute:

There were so many questions that the servers crashed, and I spent a few frantic minutes adding more capacity on Heroku. Once we had scaled up, though, the rest of launch day was pretty smooth, and we got encouraging feedback:

It’s clear there’s a lot of demand for products like this—a positive sign for my theory about chatbots. But the most important question still has to be answered: Lots of people wanted to try the Lenny Bot, but how many of them are coming back?

According to our metrics, on launch day a little more than 3,000 people tried the Lenny Bot. Two days later, about 500 people used it. Right now, day 2 retention looks like it’s at about 2%—very low.

We’ll see how things trend over time, but I think that’s going to be the biggest challenge for products like this. How do you create an experience that’s so compelling that I might want to use it every day? 

There’s a lot of things that lead me to believe it’s possible, but it’s also clear that we’re not quite there yet.

Code samples

If you’re interested in this and want to build your own chatbots, here are some code samples you can use to help you get started. Keep in mind that this is code I wrote quickly to build these projects, so it’s messy, and you may find better ways to accomplish the same things I did.

As part of the article shared above, I showed you how to create a chatbot that can solicit questions from a user and use GPT-3 to answer those questions from a large corpus of text. But what’s missing from those code samples is how to build them into a web app.

What you need is:

• A server that can handle requests from a web browser and forward them to the GPT-3 code we wrote in the main article
• A client-side component that renders a chatbot in the browser that the user can ask questions to and get answers from

If you need a starting point for the above, these files should point you in the right direction:

• app.py—this file implements a Flask server that can handle to POST requests from a chatbot client and return results
• chatbot.js—this file implements a mobile-friendly chatbot in React
• style.css—this file styles the chatbot built in chatbot.js

I hope this is helpful for you in your explorations. I’ll have more on all of this next week.