Elon Musk says a Tesla on Autopilot is 10 times less likely to crash than the average car, but experts say that stat is misleading
- Tesla says a car on Autopilot is almost 10 times less likely to crash than the average vehicle.
- But the carmaker's statistics lack context and have some fundamental flaws, two experts said.
- Autopilot is mainly used on the highway and the average car in the US is 12 years old.
As Elon Musk and local authorities spar over whether Tesla's Autopilot software was to blame in a fiery Model S crash that killed two people on April 17, one thing is clear: Regardless of the technology's involvement in this particular wreck, questions remain about the safety of the system.
Musk has repeatedly touted Autopilot - which automates some driving tasks but doesn't grant cars full autonomy - as a revolutionary safety feature that can slash the likelihood of getting in a crash by a factor of 10. But the data Tesla uses to back up that claim is incomplete at best.
According to the electric-car maker's safety report from the first three months of 2021 - which Musk tweeted about hours before the crash - the company logged one crash for every 4.19 million miles driven on Autopilot. Measured against the National Highway Traffic Safety Administration's US average of an incident roughly every 484,000 miles, Tesla's self-reported figure appears mighty impressive.
But Tesla's statistics don't show the full picture, experts say.
"They can give you this little snippet of data with absolutely zero context, and it is effectively meaningless. It doesn't really tell us anything useful," Sam Abuelsamid, a principal analyst at Guidehouse Insights who specializes in electric vehicles and automated driving, told Insider.
One major flaw in Tesla's comparison is that Autopilot, which keeps a car centered in its lane and maintains a constant distance to other cars, is primarily used on the highway. Drivers can technically switch it on at any speed and on any road, but as with any cruise-control feature, it's most useful during long uninterrupted stretches of highway driving.
So comparing miles driven on Autopilot to national statistics that include a huge variety of driving environments - from packed city streets to unmarked rural roads - gives Tesla a major edge. Highways generally see fewer crashes on the whole than urban environments, and only 17% of fatal crashes occurred on interstates and freeways in 2019, according to the Insurance Institute for Highway Safety.
Tesla's data also overlooks vast differences in vehicle age, which has a major bearing on safety. The average car on the road today is 11.9 years old. One in four of those vehicles more than 16 years old, according to data from IHS Markit. The majority of Teslas were sold in the last few years, and the oldest ones equipped with the hardware to run Autopilot were built in 2014.
That makes Tesla's comparison fundamentally flawed, Abuelsamid says. Not only do older vehicles lack Autopilot-like advanced driver-assistance systems (ADAS) altogether, but they're also more likely to get in a wreck due to things like worn-out brakes, less-responsive handling, and aging tires.
"Telling me that [a Tesla] is better than a car that has no ADAS features doesn't really tell me anything useful," he said. "I would expect the same thing to be true of every modern car that has adaptive cruise control and lane-centering."
In addition to drawing some questionable comparisons, there are other ways Tesla may be skewing the data in its favor.
For instance, Tesla isn't fully transparent about cases where Autopilot disengaged before crashing, Abuelsamid says. In its methodology, the company says it counts any incidents that occurred within five seconds of Autopilot disengaging. But Abuelsamid says it's not uncommon for a driver not to notice when Autopilot shuts off, since the alert is not very intrusive.
Data that potentially misrepresents Autopilot as safer than it is could embolden customers to misuse the system or pay insufficient attention to the road when it's engaged, Abuelsamid said.
For Missy Cummings, an engineering professor at Duke University who studies autonomous vehicles, a more pressing issue is that Tesla markets Autopilot - and its more advanced, $10,000 Full Self-Driving feature - in ways that exaggerate their capabilities.
"I think we should present legitimate data in a legitimate way, so I do think there is some element of false advertising there. But what really makes people misunderstand the car's capabilities is calling it Full Self-Driving," she told Insider. "Your average person who hears statistics - it doesn't even register with them."
Neither Autopilot nor Full Self-Driving allows cars to drive themselves and both require undivided driver attention, despite their monikers. But that didn't stop Musk from taking his hands off the wheel during a now-notorious "60 Minutes" segment from 2018, Cummings notes. For years, Musk has touted that truly autonomous Teslas are just around the corner, despite the current limitations of the company's driver-assistance tech.
Moreover, Cummings says she wouldn't trust any data that hasn't been reviewed by an independent organization - whether it's from Tesla, General Motors, or any other automaker.
Musk said Monday that records reviewed by the company so far show that Autopilot wasn't engaged at the time of the Texas crash. He offered few other details. To be sure, Autopilot is considered by many to be quite safe when used in the right conditions under full supervision.
However, despite years of scrutiny surrounding Autopilot's risks - NHTSA has opened investigations into at least 28 recent Tesla wrecks - the electric-car maker hasn't provided airtight statistics about the system's safety record to assuage consumer-safety advocates.
Abuelsamid says the issues with Tesla's quarterly data all boil down to one old adage: "Tell me what side of the argument you're on, and I'll give you the statistics to prove that you're right."