The Facebook engineer who taught its data how to dance is solving a new complicated problem
Cassandra would go on to take the world by storm, with everybody from Apple to Wikimedia using it to power their data-heavy web services.
Today, the man who invented Cassandra at Facebook, Avinash Lakshman, reveals his new startup: A storage company called Hedvig, which has raised $12.5 million in search of a repeat performance as it works to help companies hold the huge amounts of data they generate.
Before Facebook, Lakshman worked at Amazon Web Services, where he co-invented the Amazon DynamoDB data-crunching product.
He says he had a fine career ahead of him at Facebook. But working on Cassandra gave him the glimmer of some new ideas he knew he had to pursue.
"At Facebook, everything was going good for me, but I knew what it was I wanted to do," Lakshman says.
The basic problem Cassandra solved is that when you have a lot of data sitting on a lot of servers, as Facebook does, you end up with a house of cards. A single server going down can collapse the whole stack.
Cassandra is software that lets you store data distributed across many systems, such that just one individual server going down (as they inevitably do) has only the barest of impacts on the whole application.
That's great, as far as it goes, and it's why it has so many fans. But the actual hard drives in those servers just aren't as flexible or as fault-tolerant. It's a problem that's especially prevalent when companies like, yes, Facebook have to add tons of hard drives all the time just to hold the ever-increasing amounts of stuff we generate and store.
"Modern applications are changing the game about how data is generated," Lakshman says.
Right now, if you have a data center with lots of servers with lots of data, adding hard drives is a pain in the behind. Because of these complex systems, it's not as simple as plug-and-play. It takes a lot of planning and delicate architecturing, like playing Jenga with a stranger's pictures of cats.
This is why companies traditionally only update their storage every three to five years. Which is great, except when the you're doubling the amount of data you need every three months.
Which brings us to the specific problem Hedvig aims to solve.
Hedvig sells what you call "software-defined storage," where it gives a system for adding storage to a system without any kind of headache. Install their software on top of your storage, and it applies the same kind of fault-tolerant thinking behind Cassandra to the storage itself. Hardware from any manufacturer, not just the big guys, will immediately get added to the pool of available storage.
It's something that sounds like it should have been simple all along, but it's actually pretty hard. Lakshman actually got certified in some of those old-school storage platforms before founding Hedvig just to understand the issues.
"In order to understand the future, you have to understand the past," Lakshman says.
"Hedvig," incidentally, was chosen as the name for the company because it has nothing at all to do with what it does - something Lakshman apparently was told is important before founding the company. It stands for "Hyperscale Elastic Distributed Virtual Integral Granular," all qualities Lakshman says the product was designed around.
The funding round was raised by Atlantic Bridge Capital, with participation from True Ventures and Redpoint Ventures, at an undisclosed valuation.