Ben Margot/AP Photo
Facebook CEO and cofounder Mark Zuckerberg
- PyTorch, an artificial intelligence project started by Facebook engineers, has become the second fastest-growing open source project in the world, according to GitHub - and the fastest-growing AI project overall.
- Within Facebook, PyTorch is used for text translations, accessibility features for the blind, and even for fighting hate speech.
- PyTorch is now used at other companies like Microsoft, Toyota, Tesla, Uber, and Genentech.
- It's been used for drug discovery, identifying cancer cells, making self-driving cars safer, building video games, powering apps, and more.
- PyTorch is especially popular in the research community and used at top engineering schools like Stanford, Berkeley, and CalTech.
- Click here for more BI Prime stories.
Many of the features that we take for granted on Facebook - language translation in Messenger, for example - were made possible with a powerful artificial intelligence project called PyTorch.
PyTorch, which was first built by Facebook engineers, helps power many of the social network's services at scale. At Facebook, PyTorch is part of essentially every AI feature. PyTorch trains the translation systems, powering 6 billion translations a day. It's used for making social recommendations and making suggestions in Messenger. It can provide category suggestions for listings in Facebook Marketplace. It's used for detecting hate speech and images that violate Facebook's policies.
PyTorch even helps Facebook provide accessibility features for the blind or visually impaired. On Facebook and Instagram, users can tap an image, and the app will describe the image to them, using phrases automatically generated by a PyTorch-powered system.
"What PyTorch allows us to do is experiment very quickly," Srinivas Narayanan, head of Facebook AI Applied Research, told Business Insider. "It's showing incredible promise. What we are seeing, using these new modeling techniques, we are able to take some of the problems and experiment and deploy them into production in a very short time."
Since its release in 2016, PyTorch has spread at incredible speed. That's largely because it's available as open source, meaning that it's free for anyone to download, modify, or use as they please. PyTorch is just one of Facebook's many open source projects, which also include popular projects like React and Move. But PyTorch is the fastest-growing.
In fact, PyTorch has become the second-fastest growing open source project in the world, according to GitHub, the ubiqitious Microsoft-owned code-hosting site. This also makes it the fastest growing AI project, overall. And according to RISELab, research papers that mentioned PyTorch grew a massive 194% year-over-year, from January to June 2018 compared to January to June 2019.
The rise of PyTorch reflects the increased demand for AI technology. Over the last year, postings for AI jobs rose 29.1%, according to a report on job site Indeed.
PyTorch is certainly popular in real, production software - Microsoft, Toyota, Uber, Tesla, and Genentech are among the companies using PyTorch to power some of their AI efforts. However, it's found a special appeal among researchers and academics, with educators at UC Berkeley, Stanford, CalTech, and even online platform Udacity using it as their preferred platform for teaching AI concepts.
How it all began
Soumith Chintala, a Facebook AI software engineer and co-creator of PyTorch, says about every five to six years, he sees the trends in AI research trend.
For example, the early 2010s were about deep learning - a method that the Google-created TensorFlow, another very popular project, uses. Starting around 2015 or so, he noticed that the focus had shifted to neural networks, or computing systems that are inspired by,and work similarly to, biological neural networks in the brain.
"Something we started noticing is people started trying crazier and crazier neural networks," Chintala said. "They came up with ideas that were harder and harder to do."
Chintala says that the team at Facebook felt that AI research was changing course. So, these engineers decided to build a project that makes use of this idea, mostly targeted at researchers.
"There was a need for something like PyTorch right when it got released," Chintala said. "We built the right product at the right time. If we built the same product a couple of years early and a couple of years later, it would not have the same kind of success and growth."
Chintala recalls that when he first started working in AI, there were few tools to help him out. When he started at Facebook, he used Torch, another open source AI project, but found it limiting.
"If you wanted to build AI ideas, do research, or try neural networks, it was a struggle not because you couldn't express your idea but because there's no tooling to express your ideas in a reasonable way," Chintala said.
Eventually, he saw a need for a new version of Torch, but based on Python - one of the most popular programming languages in the world, and especially lauded for how friendly it is for novice programmers. This is important because today, Python is frequently used by data scientists, researchers, and developers for AI programming and training datasets. Hence, PyTorch was born.
Now, PyTorch is lauded for how easy it makes it for developers to experiment with other AI research ideas like natural language processing, the field of computer
Within Facebook, researchers are experimenting with PyTorch to fight hate speech through so-called classifiers, the way that AI systems sort and categorize data.
"There's a lot of text content on Facebook," Joe Spisak, Facebook AI Product Manager for PyTorch, told Business Insider. "We want to build classifiers that help us understand the intent. We have classifiers that identify if certain posts are hate speech."
How other companies are using PyTorch
While Google's TensorFlow is often used in production, researchers and data scientists tend to gravitate towards PyTorch.
"If you're a researcher, if you come from a research background, PyTorch is better," Ali Ghodsi, cofounder and CEO of Databricks, told Business Insider. "It's more flexible. You can write custom code more easily. In the case of PyTorch, Facebook has evangelized it a lot and they pushed for it a lot. Researchers want something that's more flexible than TensorFlow."
Still, PyTorch is finding practical use. Genentech uses PyTorch for drug discovery and development projects. For example, it uses PyTorch to sift through millions of chemical structures to develop drugs, and it helps builds models about how patients will react to treatment. It's even being used in the development of cancer vaccines.
And at Uber, research teams use PyTorch to investigate problems, like how long it takes for a rider to make a second trip after their first trip on the app. It's worth noting, however, that Uber uses both PyTorch and TensorFlow in conjunction to power its AI software.
"PyTorch was open sourced pretty recently and we started using that pretty much since then," Alex Sergeev, staff software engineer at Uber told Business Insider. "We work with the community and also contribute back...For us, we are actually very excited about competition that these frameworks have with each other. This actually drives a lot of
PyTorch in education
Chintala says that PyTorch is the most-cited AI framework at academic conferences, and it's been used for identifying breast cancer, making self-driving cars safer, building video games, and similar.
And in late 2018, Facebook released a PyTorch training course on Udacity, citing the increased need for AI skills.
"The predominant message that CEOs hear today is anything repetitive, the machines can learn in deep learning. It has people sitting in the office doing repetitive work," Sebastian Thrun, Udacity co-founder and president, told Business Insider. "They will have absolutely no difficulty finding jobs. It's a really hot field."
Roland Gavrilescu, engineering student at University College London, started his AI career with this course, for which he received a scholarship. In that course, he learned about neural networks, computer vision, and using open source AI tools like PyTorch.
This year, he landed a summer internship working in robotics.
"I have always been interested in getting started with AI and computer vision," Gavrilescu told Business Insider. "The announcement of the scholarship arrived just at the right moment for me. I thought this would be the easiest and most enjoyable way of breaking into the field. I thought gaining these skills would allow me to prepare the best I can."
A 'growing community'
PyTorch wouldn't be possible without its community of users and contributors - the major driving force behind its blockbuster growth. As an open source project, every line of code added to PyTorch since it was released by Facebook into the wild has come from a contributor in the community.
Over time, this has allowed PyTorch to train larger and larger datasets, allowing users to build AI applications at a bigger scale. And recently, PyTorch launched features that make it easier for users to reproduce research results and see how other people are using PyTorch. The engineers behind PyTorch only expect this project to continue growing
"We are reaping the benefits of faster research prototypes," Chintala said. "We can be more productive. We can solve problems that are hard to solve."
Spisak says that Facebook dedicates its resources to building smaller features for PyTorch, while pulling in ideas from the community. He added that the team is planning to add new features, such as dictionary support to help recognize even more words.
"We have a growing community that is on the cutting edge of things," Spisak said. "Just focusing on the user and users' needs kind of has this network effect of driving the community forward. A lot of the thought leaders in the space love PyTorch, and they use it."
