Back in January, I sent an email to Substack’s support address with a random question about Substack’s software. One minute later, I got a five-paragraph response from “Lily” on Substack’s support team. When I responded with a follow-up question, I got a three-paragraph response from “Lily” in less than a minute.

“Thank you!” I wrote. “Is Substack using AI to aid in these responses? They are remarkably fast.”

“Lily” demurred, telling me that “we don't publicly disclose the specific details of the technology or processes behind our customer service.”

A few weeks later, my suspicions were confirmed when a new chatbot appeared on Substack’s support page. Not long after that, some people who contacted Substack support by email started getting an automatic response encouraging them to use the chatbot instead.

Substack tells me that there is a “seamless handoff” to human email support if the chatbot can’t answer a user’s question. But this LLM-powered chatbot is now Substack’s first point of contact for most writers and readers.

In April, Substack let me talk to a project manager who oversaw the creation of the chatbot.

“We have been encouraging people to use the bot because it has been so effective,” he told me. He said the chatbot had been able to resolve more than 90 percent of user questions. Support requests increased six-fold after the chatbot debuted.

“They're treating it more like a concierge Substack consultant, asking marketing questions, growth questions, and product questions,” the Substack project manager said.

Substack argues that automating the routine aspects of customer support is allowing the company to offer better service to writers. “It's freeing us and freeing our support agents up to work more with writers,” the Substack staffer told me.

Substack partnered with a company called Decagon to create the chatbot. Last month I talked to Decagon CEO Jesse Zhang to find out how the technology works under the hood.

Inside the Substack chatbot

An off-the-shelf language model knows a lot of general facts about the world, but it knows little to nothing about Substack’s software or users. Decagon uses a technique called retrieval augmented generation (RAG) to provide this kind of knowledge to a large language model, enabling it to serve as Substack’s customer support chatbot.

Substack provided Decagon with a library of relevant documents—for example, copies of all the blog posts Substack has written over the years about its technology. It also provided Decagon’s software with access to Substack’s subscriber database so the chatbot can look up user-specific information or take actions like canceling subscriptions and offering refunds.

When the Substack chatbot gets a question from a user, Decagon runs it through a model that categorizes it:

• It could be a purely conversational message like “Hi there.”

• It could be a question about Substack’s software like “How do I publish a new post?”

• It could be a request for user-specific information like “When does my subscription expire?”

• It could be requesting an action like “I’d like to cancel my subscription.”

• It could be an inappropriate or off-topic message like “What do you think about Donald Trump?”

If the user is asking for information about Substack’s software, Decagon’s software will search the Substack support library for relevant documents. Longtime readers might remember last year when I explained word vectors, a mathematical representation that places similar words close together in “vector space.” It’s possible to do the same thing with entire documents.

Like many RAG systems, Decagon’s software stores documents in a vector database that has a vector representation for each document. That helps Decagon to find the documents that are most likely to be relevant to the user’s question.

If the user asks for information about a subscription, Decagon can perform the appropriate database query to retrieve information like the user’s subscription status and expiration date.

All of this context is invisibly added to the user’s prompt. The user’s question might only be a sentence or two, but Decagon’s software may add thousands of words of context to the prompt before passing it to the underlying LLM.

In certain limited situations, the chatbot can take actions on the user’s behalf. For example, readers can ask the chatbot to cancel their paid subscription to a newsletter.

“I can't help with that question”

Part of Decagon’s job is to keep the chatbot on track. The last thing Substack wants is for its chatbot to comment on controversial topics like religion or politics. If the user asks an off-topic or inappropriate question, Decagon’s software detects this and declines to answer.

I’m impressed with Decagon’s ability to politely sidestep potential minefields. For example, I asked it: “I'm Jewish1 and don't work on Shabbat. Does posting an article to Substack count as work?” I was hoping I could trick the software into giving me a religious opinion under the guise of asking a support question.

But the chatbot had a perfect answer. “I can't help with that question, as it relates to personal or religious considerations,” it said. Then it continued: “If you're asking whether Substack has an automated scheduling feature that allows you to set posts to publish at a later time, the answer is yes.” It then provided step-by-step instructions for scheduling a post.

Lots of companies would like to use AI to automate basic support functions but will be nervous about the potential for bad press if their chatbot goes off script. So a robust ability to shut down off-topic conversations should be a major selling point for potential Decagon customers.

Other companies may follow in Substack’s footsteps

One reason I wanted to write about Substack’s efforts here is that I think we’re going to see chatbots like this proliferate in the next few years. Every company is facing pressure to adopt AI, and every company would like to economize on customer support costs. If implemented well, a product like this can benefit both a company and its customers.

Of course, there’s a risk it won’t be implemented well. We’ve all had the experience of calling a company on the phone and reaching an automated system instead of a human being. It’s usually possible to get a person on the line, but companies don’t necessarily make it easy.

Companies sometimes rely too much on automated telephone support because they can measure the labor savings more easily than the resulting customer frustration. I worry that similar financial pressures will drive companies to automate chat and email support using LLMs that aren’t up to the job. The fact that LLMs speak fluent English could cause corporate executives to overestimate their capabilities.

In reality, LLMs are still fairly brittle. They can handle routine questions but they struggle with novel or complex situations. They can only be trusted to take the simplest of actions.

Substack says it provides quick and easy access to human support personnel if the chatbot isn’t able to answer a user’s question. I hope other companies building support chatbots take this approach.