And So We Drive
A Digression from Cars: Startups, and the One Question You Should Ask to Assess Product Quality
Over the last ten years, I’ve been a decision-maker in about a dozen startups. In about half, I’ve been a founder or investor; in the other half, an early employee or consultant. Most have been software of one kind or another, ranging from videogames to SaaS platforms. (Though one of them made shirts.)
I spent last weekend using and testing product for one of these, and am spending this week doing the same for another. Looking back at all the companies I’ve worked with, I have a theory: the best approximation for the quality of a product is how much time the company’s own team spends using its product.
This metric isn’t something an incoming investor, prospective employee, or potential customer has ever asked me, but it should be.
More than one factor is at play here, and of course, the reason to use your own product varies by type of product. It’s very different for a team that makes videogames to use its product than it is for a team that makes a SaaS platform, particularly if that platform targets business users that are unlike the startup itself.
But that’s precisely why I like this as a metric: there are many different reasons to use your own product, and yet it all ends up in the same place. As a single metric, it cuts across just about every kind of startup.
I see three main reasons for the correlation:
2) Product/Market Fit
I’m sure there’s an exception to this rule somewhere, but I’ve never seen a product succeed that the founding team doesn’t use as much as it can — and in my experience, as soon as you see the opposite trend start to happen (the team beginning to use the product less), it’s a huge red flag.
To get a bit deeper into each, let’s start with Embarrassment. When a company knows its product isn’t quite right, the team is embarrassed (sometimes unconsciously), and doesn’t want to use the product as much.
For example, back when I made videogames, it was always the good ideas that got played the most and tested the most. When we had something that wasn’t as good, just playing it made us cringe. No one wanted to test, and no one played for fun. The difference between good content, with potential, and bad content, without, wasn’t just in how many bugs there were. Great content with a lot of bugs was still something everyone wanted to test, and bad content, even if it was bug-free, was always something you wanted to rush through.
The amount we played with something was a better indicator of quality than our actual feedback on the feature we were playing.¹
Next, Product/Market Fit. When the team doesn’t have reason to use their own product, either the market fit probably isn’t right, or the group isn’t diverse enough to include its intended users. Either way, this bodes poorly.
The details here will vary for different types of startups. If you make enterprise software for a different industry, it can be hard to use that software every day. But here, the metric is whether the team uses the software as much as they can. I once worked with a startup whose SaaS product was conceived to address a pain point the founder had had for years, and yet, when they got to release candidate stage, the founder still was not using the product. If it wasn’t the right fit for the founder, how could it ever be for a customer who supposedly had the same need?
Finally, the Details
Using a product to solve the same problem it should solve for your customers is the only way to get the details right. Hopefully, the team will get the broad strokes right in its design meetings, strategy sessions, and focus groups. Then it builds it, attaches all the buttons, and makes sure everything works. And hopefully, it does. But it’s by actually using the product, for its actual intended purpose, that the team makes it feel right to use, and gets it to the point that, when others use it, they can do so in a state of flow. That’s how a product gets sticky.
There are many reasons startups don’t get off the ground, even with the best products in the world. If a startup is using its own product, it still may not succeed. But in my experience, a team using its own product is absolutely necessary, even if not quite sufficient, for success.²Permalink
- Alas, that was 10 years ago, and I didn't learn this lesson until years later.
- The shirt company, incidentally, is one that didn't work for other reasons -- but for what it's worth, I have our inventory, and they're still the only dress shirts I own.