This startup pays 58,000 strangers to break your favorite apps

In the summer of 2012, Rainforest was struggling.

Rainforest CEO Fred Stevens-Smith and cofounder Russell Smith had been accepted into Y Combinator, the prestigious startup mentorship program.

But its core product - a service to track how much money a company was spending on the Amazon Web Services cloud - just wasn't coming together into something people actually wanted or needed. Friends and colleagues alike were skeptical of how this would shake out.

"Everyone was like, 'this is stupid, stop,'" Stevens-Smith recalls.

Frustrated, Stevens-Smith sent a blast e-mail out to Rainforest's beta users, the CEOs of friendly companies, and his fellow Y Combinator startups, asking a simple question: "What problem do you have that you'd pay $1,000 a month for?"

The replies were almost unanimous: They needed help with "quality assurance," also called QA, where human testers rigorously test every pixel of a software product to make sure it's ready for their customers. Stevens-Smith realized that they were finally onto something and signed up 9 customers from that e-mail alone.

Fast forward four years, and now, Rainforest employs 58,000 testers - strangers from around the world employed on a case-by-case basis who can test any app's new features or design and deliver feedback in half an hour or less. And it's gone beyond $1,000 a month, with its basic plan starting at $10,000/month.

"Our business sounds like one of those classic 'this doesn't sound like a good idea' things," says Stevens-Smith.

Still, it's a model that has plenty of believers: Rainforest has raised $16 million in investment capital, including a $12 million infusion from Bessemer Venture Partners in February of 2016. Hot startups like Zenefits and Soylent are among Rainforest's customers. And it's on track to be profitable by the end of this year, says Stevens-Smith.

Testers on demand

"We think of ourselves as the objective feedback platform," says Stevens-Smith.

When you sign up for Rainforest, you're connected with a company representative who helps you break down exactly what kinds of things you need to test in your app, such as "sign up for a new account," and "try to find the help button." You can also specify which devices and browsers test in, so you can see if a button on your service works on the Chrome browser but breaks on an iPhone.

Then, the magic happens. Whenever a Rainforest customer needs to run one of those tests, they just push a button. Rainforest's technology goes out and automatically posts the test on gig sites Amazon Mechanical Turk and CrowdFlower, which let people complete short, simple tasks for money.

The tests are designed to be super-focused and very short. Rainforest's software monitors the tester's behavior and it's delivered back to the customer as soon as Rainforest has gathered enough useful input. It's all done in an hour, and then the developer can go back to tweaking it as necessary.

With 58,000 Rainforest testers all over the world who have taken on these assignments, the testing gets done fast. Half of those testers are in India and Eastern Europe, Stevens-Smith says.

Fees for the testers are paid out automatically once they complete the test. Fred Stevens-Smith says the wages are generated algorithmically based on demand, and range from $1.50 to $2 an hour up to $7 or $8 an hour, depending on how many Rainforest tasks they've previously completed and the quality of their work.

"The hard thing we've done is automate the management," Stevens-Smith says.

Artificial lab rats

Products like Stripe and Amazon Web Services changed the face of technology, Stevens-Smith says, by making it incredibly easy to take payments or set up a server without the need for you to set up that infrastructure yourself.

In the same way, Rainforest wants to eliminate the need for companies to have their own QA team, which can both be costly and limiting. If you're a small startup, you probably don't have the budget to hire a dedicated tester. If you're a bigger company, you're still going to run into limits with how much the QA department can test at once.

The big idea is to make it faster, easier, and cheaper for companies to test concepts so they can push them into the real versions of their software. It's not quite as easy as setting up an Amazon Web Services account, Stevens-Smith says, but it's getting more automated all the time.

In fact, he says, Rainforest is working to train artificial intelligence to do this kind of testing. Right now, it's still the early phases, with Rainforest's AI slowly learning how real humans use apps. But within the next 18 to 24 months, Stevens-Smith says that he's hoping to get Rainforest's testers to be 80% robots, 20% humans.

"That's kind of the next step," Stevens-Smith says.