Welcome to my apparently-now-quarterly update.
The big news in my life is that—for the first time in four years—I have a new product making money!
Recurring money, no less!
The surprising MRR growth of my latest product, which I haven’t even started marketing.
The product is Scriv.ai—an AI-powered chatbot that can answer questions
about your product, project, company, or anything else.
In this post I’ll talk about what Scriv is, why and how I built it, and the optimistic and pessimistic
outlooks for the project moving forwards.
Building a chatbot to solve my own problems
Scriv—like many ideas that turn into products—started with trying to solve my own problem.
For me, the problem was support.
My main product, SaaS Pegasus, is a
SaaS boilerplate—a codebase for building SaaS applications.
Pegasus is a big piece of software, and people, of course, have questions about how to use it.
Over the years, I’ve built up some pretty comprehensive documentation to help answer them.
One of the perks of using Pegasus is access to a private Slack community where me and other Pegasus users hang out and talk shop.
Sometimes questions come up that can be answered directly from Pegasus’s documentation.
And so I thought—wouldn’t it be cool if there was a chatbot that could just answer these questions?
I had the vague sense the latest advances in AI had made this type of thing pretty easy,
so I decided to give it a shot.
And it was super easy!
At it’s core, it’s basically five lines of code
in a library called langchain.
Here’s the result in Slack:
An example Slack chat with my bot, which provides answers based on my own product docs.
The key is that the bot is providing a very specific answer based on the contents of my own documentation.
It’s not ChatGPT, it’s PegasusGPT!
This was cool! And a fun start. But it was also pretty limited.
When asked about something in the docs the bot would produce a great answer,
but if it couldn’t find any information, it was useless.
I needed to make the bot smarter.
And then something occurred to me.
There was a lot of useful information in the Slack conversations.
Many times someone would ask a good question and someone else would provide a great answer,
but it never made it into any “official” documentation channel.
Could I train the bot on the data in the Slack community?
This ended up being a bit more complicated, but possible.
First I had to get the data out of Slack.
Then figure out the right way to organize and index it (basically, by thread).
And finally, iterate until the bot started providing sensical answers.
And after a few iterations it worked pretty well!
The bot could now search both the documentation and Slack,
and answered questions based on anything it found in either of those places.
At this point I started getting excited by a larger vision.
The product wasn’t a bot trained specifically on your documentation, or your Slack data.
The product was a bot trained on whatever you wanted to train it on.
Anything you could get text out of was something that could be used by the bot.
With some semblance of a product vision in place, it was time to shift to the validation stage…
Are custom chatbots useful?
As is always good when building a new product—I was Scriv’s first target user.
But once the bot existed, there was another important question—would anyone else use it?
The answer turned out to be a pretty emphatic “yes”!
Here’s a recent screenshot of the support channel from the Pegasus Slack community:
People have been mentioning the bot on every question!
Sometimes the bot provides a perfect answer, like in this case:
Scriv nailing an answer.
Sometimes it says something useful, but that needs more context:
Scriv doing alright.
And yes, occasionally it says something nonsensical or even flat out wrong and needs to be corrected:
Scriv completely lying.
Still—even though the bot had some problems—it was clearly useful enough to get traction among the Pegasus community.
This was a pretty exciting bit of validation.
Are custom chatbots generalizable?
Having become somewhat convinced that Scriv was useful—at least for the Pegasus community—I then sought out the answer a harder question:
was it useful for other things?
To answer this question I had to find myself some early adopters—people who could try using Scriv for their own projects.
I hit up my personal networks and managed to come up with a list of potential users that included:
- Other Pegasus users
- My former company
- My mom (really!)
One Pegasus user wanted to build a chatbot on top of a database of drugs help pharmacists better recommend treatments.
My former company wanted a chatbot that to questions about its product.
And my mom wanted to index hundreds of scientific papers to help her with her research.
Everyone got a bespoke onboarding, where I helped them get their content loaded into Scriv.
And in all cases the product worked!
No, not perfectly. Sometimes not even all that well.
But it was clear that the seed of the idea was both generalizable and useful.
The devil would be in the execution…
Are people willing to pay for custom chatbots?
The final and most important form of validation is money.
It’s all fine and good when people say they’ll pay for something, but the real test is when you actually
ask them to pull out their credit cards.
Fortunately, this question basically answered itself.
Somehow, miraculously, while I was running these tests, and before I had done any marketing at all for Scriv,
two completely random people found it and paid for it!
One person came from YesRamen—a site for getting feedback on early-stage products I had listed it on.
Another came from my YouTube channel, maybe?
I don’t actually even know how they found it.
But, given that I wasn’t even trying to get users at this stage, it seemed like a good sign.
The bull case for Scriv
I recently got back from a six-week trip to Europe, visiting family and friends.
Whenever anyone asked me what I was up to and I mentioned Scriv, they immediately got excited.
“Oh, I would totally use that for [whatever thing]!” they’d say. One of these people actually turned into my third customer.
This is basically the bull case for Scriv.
A chatbot that knows all about your data and gives you smart answers to domain-specific questions is a panacea.
Everyone on earth probably has something they could use that for, whether it’s work, a hobby, or even something from their personal life.
And when it works it really is like magic. Scriv is to your knowledge what ChatGPT is to Google.
Rather than search, read, search again, read more, and finally find what you were looking for, you just get the right answer immediately.
There are other, perhaps, more specific reasons to be bullish on Scriv:
- It’s still somewhat early in the chatbot wave, and lots of people are starting to learn about tools like this for the first
time—with no clear market leader.
- There’s a high switching cost once someone is onboarded.
Getting all your content into the system can be a hassle, and if Scriv works, it’s unlikely people will go through the effort to switch to something else.
- The tight integration with Slack is something few other products have, and makes Scriv uniquely useful to anyone using that platform.
Getting it into people’s Slack workflows also increases the switching costs.
Pretty compelling, right? Retirement, here I come!
Well—not so fast. Let’s instead consider things from the other side…
The bear case for Scriv
I wrote above “when it works it really is like magic”. The key part of that sentence is when it works.
Chatbots are finicky beasts, and chatbots that work on custom data are doubly so.
If the answer isn’t in the data, the bot will fail, or—even worse—hallucinate a totally wrong answer out of thin air.
And even if the answer is in the data, it’s not guaranteed that the bot will find it correctly.
The underlying technology for the lookup-to-answer process involves mostly black-box AI magic, and,
while there are things you can do to improve and fine-tune it, a lot of the performance is outside your control.
So while the theory of Scriv is amazing, the reality of Scriv is a lot closer to earth.
It’s useful, but it’s more like an intern than an oracle.
It’ll give you some answers, but you’re still gonna want to double-check everything it says.
The other—perhaps larger—bear case is competition.
To my friends and family in Europe—aka normal people—Scriv is novel magic.
To the many of you reading this who are more tapped into the burgeoning AI scene,
Scriv is one of perhaps fifty other products you’ve seen in the last few months that do similar things.
This tweet—liked by over two thousand people—hits a little too close to home.
On top of competition is commoditization.
Is there really any moat for something that is essentially just plumbing and UX around someone else’s API?
Plus, isn’t every major app just going to have AI baked into it very soon?
It’s in thinking of these questions that my quit monkey continues to shout at me.
And how I feel about the future of Scriv depends on what hour of the day you ask me.
Sometimes I’m super excited and sometimes I feel hopeless.
This is, I suppose, how things are supposed to feel in the arena—so for that at least, I’m happy.
If Scriv flames out, at least it won’t be because I was too scared to try.
What’s next for Scriv?
Well, now that I’ve gotten beta-testers, real customers, and even my mom’s approval,
I guess it’s time to officially launch this thing.
So, uh, I guess that’s what this is?
If you think you might be interested in trying out Scriv for your company, project, or community, give it a go and let me know how it goes.
If you want help with getting your data in, or have any questions or feedback, I’d love to chat.
They say if you aren’t embarrassed by your MVP then you’ve launched too late.
I’m definitely still embarrassed, so—win?
If you don’t have a use case for Scriv, but are able to help me spread the word
that’s great too.
Onwards.
Notes