Ex-Apple engineers created an app to track basketball shots using AI, and it's already getting attention from Mark Cuban and the NBA
- Apple recently highlighted an app that uses an iPhone and machine learning to understand whether a basketball shot went in the hoop or not.
- The company, Nex Team, was founded by former Apple employees and is based in San Jose, California.
- The app, HomeCourt, is backed by a range of basketball insiders, including Mark Cuban, Sam Hinkie, Jeremy Lin, and Steve Nash.
- The app can determine metrics like how high you jump or what the arc of the ball is using artificial intelligence and the iPhone's camera.
For HomeCourt co-founder David Lee, the idea for his startup came because he couldn't hit a shot.
He was making web apps for Apple when one night he was playing in a recreational basketball game, and his wife and two daughters dropped by to watch him play.
"I couldn't make a single shot in front of them," Lee told Business Insider. So, he wondered, "can I make my phone understand my game?"
At first, he just wanted to write software that would automatically cut the footage into a highlight reel so that his family could see that he could actually make shots. But less than two years later, he was standing on stage with NBA legend Steve Nash at Apple's iPhone XS launch showing off his app.
It's a rapid rise for HomeCourt, a product out of a young San Jose startup called Nex Team founded by Apple, Facebook, and Google veterans that uses machine learning to understand what's happening on a basketball court.
On Tuesday, the startup launched the feature that it revealed on stage at Apple, called Shot Science - the app can now track a number of different metrics that affect how accurate a shooter is, including how long it takes to release the ball, how high you jump, and the angle the ball is released at, using nothing but an iPhone and its camera.
It can also track how many shots you take during a session, and the percentage of them you make and where they were taken from.
"The problem we want to solve is how to use the phone to track shots, and we incrementally moved more and more to AI, because it gives us more precise results," Lee said.
The company has raised $4 million in seed funding, and counts a lot of basketball players and pros as investors, including Dallas Mavericks owner Mark Cuban, former 76ers general manager Sam Hinkie, NBA point guard Jeremy Lin, and Nash.
'You're not making enough shots!'
HomeCourt's advanced learning algorithms analyze lots of footage of basketball courts and sessions to understand where, say, the three-point line is, or if a shot went in or not.
It has a headstart over competitors. It has already recorded over three million shots since launching the beta version of the app in Feburary, according to a person familiar with the startup. That data helps it improve its machine learning models.
But in the early days of the startup, the company had to collect data manually - often by filming themselves taking shots or collecting video from local teams. In fact, finding a nearby basketball court is a priority for the young startup's next office. "My team was complaining, 'you're not making enough shots!'" Lee said, joking that he was a better rebounder than shooter.
"We're the data," Homecourt cofounder Alex Wu, who leads marketing for the startup, said. "But the more people use our app, we see hoops from around the world, even backyard hoops, and different types of shots," which improve the company's image recognition models.
For example, when HomeCourt tested its app with Stanford, it had to improve its artificial intelligence models, because the college players were too good.
"When we started testing with Stanford - they were too good, they were too accurate, their balls were going in smooth and clean through the net, so our AI wouldn't recognize that they had shot," Wu said.
The app takes advantage of Apple's latest technology, called CoreML, that enables artificial intelligence models to run more efficiently and using less battery life on an iPhone's specialized processor - one of the reasons why Apple decided to highlight the young company at its biggest launch of the year.
"iPhone XS is the first iPhone that has the performance that we can actually ship" the real-time shot tracking feature, Lee said. "On the last generation, the delay is a couple seconds. The specific improvement in the hardware itself is dramatic."
"And it operates at one-tenth of the battery life too, so it's a performance improvement, not just processing wise, but also usability wise," Wu said.
Finding the next Jeremy Nash
"I would spend hours shooting by myself, just hoping I was refining the techniques to take me to the next level," said Nash at the feature's launch.
The way that the NBA is looking at HomeCourt and similar analytics is that they can streamline a system that's typically been paper and pencil based and labor intensive, according to a person with knowledge of how NBA front offices work. If teams can automate that data collection and even look at metrics that weren't tracked before, like launch angle, then pro players can improve the quality of their practice.
So far, elite teams including the Boston Celtics, Philadelphia 76ers, and Duke University have tested the HomeCourt app.
"I wish I had this when I was younger. Just imagine what happens when people have access to this technology when they're 13," Lee said.
But HomeCourt isn't just for players who want to go pro. The app also has a social component, where players can compete in shooting contests with other players around the world, even if they're in a completely different country.
For example, currently, the top challengers on HomeCourt's leaderboards are in Europe, an area that's traditionally not a basketball hotspot.
And going forward, the young startup hopes it can continue to refine its models so it can capture and understand entire basketball games, not just practice.
"We're working on solving 1-on-1 first, then onto 2-on-2, 3-on-3," Wu said.
"We're trying to capture more of your basketball life and be helpful in more places. So you can imagine us going into games," Lee said.