How We Scaled Pinterest From Zero Users To A $2 Billion Valuation
Pinterest is a social network with boards of people's favorite things.
You can follow pins and boards from people you know.
Here, you can see the pins from everyone you follow.
So users have boards and relationships.
Here's Pinterest's page view count at the beginning.
In March 2010, the team wasn't working with much from an infrastructure point of view.
The product and architecture evolved.
Pinterest started doubling page views every month and a half, but everything was breaking.
So they ended up with five major technologies just for the data alone.
How We Scaled Pinterest From Zero Users To A $2 Billion Valuation
So they started dropping off technologies and did a massive restructuring of the architecture.
Here's what they changed the architecture to.
Pinterest's web traffic continued to increase.
Pinterest started to put more resources into its architecture to handle its growth.
Pinterest uses Amazon EC2/S3 for a few reasons. The main one: you can have new instances ready in a matter of seconds.
But there is limited choice.
How We Scaled Pinterest From Zero Users To A $2 Billion Valuation
Open source database MySQL has proven to be solid choice for Pinterest. It's incredibly mature and you can hire for it, as lots of engineers know MySQL.
Memcache is also incredibly mature, and it never crashes.
Redis isn't very mature, but it's simple.
Pinterest realized that during its rapid growth, it needed to spread the data evenly to handle the load. So they defined a spectrum of options between clustering and sharding.
With clustering, everything is automatic.
But sharding is a completely manual data placement process. It's used to separate databases into smaller, faster, and more manageable data pieces called shards.
How We Scaled Pinterest From Zero Users To A $2 Billion Valuation
How We Scaled Pinterest From Zero Users To A $2 Billion Valuation
How We Scaled Pinterest From Zero Users To A $2 Billion Valuation
If there's a massive bug, it will impact every single node. A SPOF brought down Pinterest four times.
How We Scaled Pinterest From Zero Users To A $2 Billion Valuation
How We Scaled Pinterest From Zero Users To A $2 Billion Valuation
With sharding, everything is manual. And that's a good thing.
If your project has a few terabytes of data, you should shard as soon as possible. When Pinterest's Pin table reached one billion rows, the indexes ran out of memory. That's when the company decided to shard.
So Pinterest froze some of its features to start the transition from clustering to sharding.
The less data you move, the more stable your architecture will be.
Since they wanted to shard on MySQL, they projected growth for next five years.
Pinterest initially put their databases on 8 physical servers.
For high availability, Pinterest ran MySQL in multi master replication mode.
With an increased load on a database, Pinterest replicated a server to handle some of the data nodes.
Since Pinterest is on AWS and MySQL queries took about 3 milliseconds, they decided to build the location into the ID.
If Pinterest has 50 IDs, for example, they split them up and run them in parallel. This is what Pinterest's lookup/rendering structure looks like.
How We Scaled Pinterest From Zero Users To A $2 Billion Valuation
All of Pinterest's data falls into two categories: objects or mappings.
How Pinterest brings up a user profile. Most of the calls are served from the cache (Memcache or Redis)
Pinterest built a huge scripting farm to move 500 million Pins and 1.6 billion follower rows. Scripting is what happens when you need to move from the old, unsharded system to the sharded one.
How We Scaled Pinterest From Zero Users To A $2 Billion Valuation
How We Scaled Pinterest From Zero Users To A $2 Billion Valuation
How We Scaled Pinterest From Zero Users To A $2 Billion Valuation
How We Scaled Pinterest From Zero Users To A $2 Billion Valuation
Need to give your brain a rest?
