REUTERS/Mike Blake
Investors put hundreds of millions in Theranos despite its lack of published
- A new study suggests that many of the highest-valued healthcare startups - from buzzy health insurance companies like Oscar to biotechs with record-breaking IPOs like Moderna - are failing to publish quality scientific research.
- That could put us on the brink of the next Theranos, say the authors of the study, who hail from Stanford and New York University.
- While it isn't perfect, peer-reviewed research is a critical source of vetting for any startup whose product could put people's lives at risk.
Few tech companies have crashed and burned in recent years like Theranos, a blood-testing startup that claimed it would revolutionize healthcare by doing away with "big bad needles."
But money moves fast in Silicon Valley, where the company began - and sometimes it moves too quickly for the ideas behind it to keep pace.
In the case of Theranos, launched over a decade ago by Stanford drop-out Elizabeth Holmes, investors put hundreds of millions in a concept with little-to-no published science to support it. When it was gradually revealed that the advanced technology required for such an idea did not yet exist, the company - and Holmes, who'd amassed a net worth of $4.5 billion after the company was valued at $9 billion - toppled.
Yet we may be on the cusp of seeing the next Theranos, according to a team of Stanford and New York University researchers that includes John P.A. Ioannidis, an early Theranos whistle-blower and a researcher with a reputation for well-regarded critiques. In a new study published Monday in the European Journal of Clinical Investigation, Ioannidis and his coauthors reveal that dozens of other healthcare startups valued at more than $1 billion (also known as unicorns) have published little or no research in peer-reviewed literature.
Peer review involves subjecting your work to outside experts in the same field. Whether it's a health startup claiming its platform will save patients money or a biotech company claiming its new therapy can cure cancer or Alzheimer's, those assertions can and should be measured and quantified.
That's especially true when a startup's technology puts people's lives at risk, as is often the case in healthcare.
"To the extent that health care startups are responsible for pushing the frontiers of medicine, patients' lives hang in the balance," Ioannidis and coauthors wrote in a Stat opinion piece published the day after their study came out.
"Hope and hype cannot supplant data," they added.
'Possibly brilliant ideas, aggressive corporate announcements, and mass media hype'
Skye Gould/Tech Insider
Had Theranos been expected to publish in the scientific literature, some of the problems could have been avoided, according to Ioannidis and his team.
Theranos' claims were bold and expansive: easy blood tests, no needles required. The startup's allegedly revolutionary tech would transform the blood testing model and allow patients to spot critical diseases like cancer early enough to treat and even possibly cure them. On those claims, the startup - positioned as a dynamic and innovative alternative to slow-moving pharmaceutical behemoths - raised about $700 million.
But startups, just like their large competitors, must validate their claims, Ioannidis and his colleagues argue in their paper.
And the number of highly-valued health startups with little-to-no published science is booming, they write.
Indeed, the field is hot and growing. Last year, healthcare startups raised $20.3 billion from venture investors, according to a recent PwC and CB Insights report. That's the highest amount in history, according to data back to 2002.
For Ioannidis and his team to come to their conclusions, they identified 48 unicorn healthcare and biotech startups (29 of which had gone public or been acquired) using data from CB Insights that was current as of November 2017. Then they examined what those startups had published in the peer-reviewed literature up to that point in time.
Of all them, half published no highly-cited papers; a quarter published nothing at all. Eighteen companies published six papers or fewer.
Instead of peer-reviewed research, these highly-valued startups rely on something Ioannidis previously described in connection with Theranos: "stealth research," or internal studies free from expert scrutiny and at risk of hype. Stealth research manifests in "a confusing mix of possibly brilliant ideas, aggressive corporate announcements, and mass media hype," he wrote.
Biotech startups who've published very little
Several high-profile biotech startups working on everything from cancer therapies to HIV prevention have published very little research.
While "it would be tempting to dismiss the Theranos case as just one rotten apple," Ioannidis and his coauthors write in Stat, "we worry that stealth research may reflect a more systemic state of affairs."
In their new analysis, they highlight several high-profile biotech companies working on everything from cancer therapies to HIV prevention.
Many of them published very little, and what they did publish was rarely considered "high-impact," or meaningful enough to get picked up by other researchers, according to the authors.
One of the companies is Intarcia Therapeutics, a Gates Foundation-backed startup working on implantable devices for diabetes management and HIV prevention. Founded more than two decades ago, the startup has published only six peer-reviewed papers in the peer-reviewed literature, Ioannidis and his team found. Of those papers, none were high-impact. Yet the company has raised $1.4 billion and was last valued at $4 billion.
Seth Goldenberg, Intarcia's chief design officer, told Business Insider that although the company has not recently published any peer-reviewed papers, it has eight manuscripts under current development and has published a total of 20 peer-reviewed papers since its founding in 1997.
"Intarcia believes in, complies with, and fully supports the importance of peer-reviewed scientific reviews and publications as well as the high regulatory standards any biotech company must adhere to, public or private," Goldenberg wrote in an emailed statement.
Moderna Therapeutics, a startup founded in 2010 that aims to develop RNA-based therapies for cancer and infectious diseases, broke records last month with its sky-high initial public offering. Although the company's stock has declined since its IPO, it is currently valued at roughly $5 billion.
Moderna has only published one high-impact study, however, according to the analysis. Still, the authors found that the company had published 16 other peer-reviewed studies - just none that were highly cited by other researchers. A Moderna representative declined to comment on the study and pointed out that Moderna lists 21 published studies on the "publications" portion of its website.
Read more: Moderna just priced the biggest IPO in biotech history, valuing the startup at $7.5 billion
Buzzy unicorn healthcare startups with no high-quality research
Shutterstock
Healthcare startups claim to save patients money and time, but there's a lack of research to backup those claims.
In addition to biotech startups, Ioannidis and his coauthors included unicorn healthcare startups in their analysis. While these companies don't necessarily have potentially revolutionary therapies, they do claim to bring innovation into the healthcare space by doing things like saving patients money and time. Yet these assertions should also be borne out by peer-review, the authors suggest.
One of those startups is Oscar Health, the buzzy health insurance startup that recently scored $375 million from Google's parent company, Alphabet. Before the Alphabet announcement, the company was last reported to be valued at $3.2 billion.
Oscar's website claims it saves patients money and time, but the company hasn't published any peer-reviewed studies backing up those claims.
Read more: Oscar Health just raised $375 million from Alphabet
Cofounded in 2012 by Josh Kushner, whose brother Jared is a senior adviser to President Donald Trump, the company launched on the Affordable Care Act's insurance exchanges. In contrast to Oscar, established health insurers like Kaiser and Cigna have published peer-reviewed articles to evaluate some of the claims they make and to inform the work of their clinicians.
Oscar leadership says they vet their claims with internal research and may publish in the peer-reviewed literature in the future.
"Oscar is steeped in data science and transparency is a core value," Dennis Weaver, Oscar's chief clinical officer, said in a statement emailed to Business Insider.
"We are constantly evaluating and testing hypotheses in service of our vision to lower the cost of care and improve outcomes. As we scale, pursuing scientific validation and higher levels of evidence is fundamental to who we are and will continue to be at Oscar."
Outcome Health, a startup that aimed to outfit doctor's offices with educational materials and pharmaceutical ads and was once valued at $5.5 billion, also failed to publish a single peer-reviewed paper, according to Ioannidis. After facing a recent lawsuit and undergoing an internal investigation, the company is now struggling to survive.
All of these companies should use the peer-review process to vet and support their claims, Ioannidis and his coauthors write in Stat.
"Startups are key purveyors of innovation: Holding them to a minimum standard of evaluation is essential," they write.
