First, it’s easier to pile on when the media narrative turns. Kudos to media outlets (1A) and scientists (2, 3A, B, C) who questioned back when the uniform media narrative around Theranos was cloyingly fawning and credulous.
- Several scientists were already publicly dubious about Theranos’ disproportionate and baffling secrecy, lack of peer-reviewed publications and utterly non-scientific but heavily political Board of Directors.
- In an eerily prescient article published in on Feb 17, 2015 (2), one of the most important among such high-profile scientific critics, wrote, ‘Information about Theranos, a privately held biotechnology company that has developed novel approaches for laboratory diagnostic testing, has appeared in The Wall Street Journal, Business Insider, San Francisco Business Times, Fortune, Forbes, Medscape, and Silicon Valley Business Journal—but not in the peer-reviewed biomedical literature. As of January 5, 2015, a search in PubMed using Theranos as a search term identified affiliations for only 2 unrelated articles coauthored by Theranos Inc employees, although these 2 reports do not offer insights about their company…stealth research creates total ambiguity about what evidence can be trusted in a mix of possibly brilliant ideas, aggressive corporate announcements, and mass media hype…how can the validity of the claims made be assessed, if the evidence is not within reach of other scientists to evaluate and scrutinize?…unless stealth research adopts more scientific transparency, investors, physicians, patients, and healthy people will not be able to judge whether some proposed innovation is worth $9 billion, $900 billion, or just $9—let alone if the innovation will improve the health and well-being of individuals‘.
Second, Theranos’ technology and science. 12 years is a long time in science, especially for a company funded to the tune of millions of dollars to not have already done the following,
- Tightly IP (Intellectual Property)-protect the so-called novel Theranos technology (Edison) through patents.
- Publish data generated by Edison in peer-reviewed scientific journals.
- After all, there are only four simple checkerboard questions w.r.t. Theranos’ science and technology. All they needed to do was take samples at the same time from the same volunteers. Pick an analyte that’s part of a normal blood screen. Generate and show Edison test data for that analyte from:
- Traditional arm vein bleed (venipuncture) and traditional readout (Quest/LabCorp): Positive control.
- Finger prick and Edison readout (Theranos): the experiment.
- Venipuncture and Edison readout (Theranos): Accuracy and precision of Edison readout alone.
- Finger prick and traditional readout (Quest/LabCorp): Does finger prick change traditional test result?
- Do these tests on one analyte on hundreds of people at one site and repeat at many sites. Publish the data.
- For Theranos’ technology to pass the smell test, 2 should =1. 3 and 4 are critical controls.
- Not difficult to do, especially with the funding they got and given their Walgreen Wellness Centers in Arizona and California.
- What’s the hold up? No convincing answer from the company. Retreating defensively into the realms of paranoia and ad-hominem attacks on entrenched incumbents doesn’t engender credibility. Debate is and should be about their science and data. After all, this is about health, arguably the most important life issue.
- More troubling? The Oct 16, 2015 John Carreyrou WSJ (Wall Street Journal) article (12) alleges that Theranos’ Edison technology didn’t perform on par in proficiency testing. To quote (12), ‘Whether labs buy their testing instruments or develop them internally, all are required to prove to the federal Centers for Medicare and Medicaid Services that they can produce accurate results. The process is known as proficiency testing and is administered by accredited organizations that send samples to labs several times a year. Labs must test those samples and report back the results, which aren’t disclosed to the public. If a lab’s results are close to the average of those in a peer group, the lab receives a passing grade. In early 2014, Theranos split some of the proficiency-testing samples it got into two pieces, according to internal emails reviewed by the Journal. One was tested with Edison machines and the other with instruments from other companies. The two types of equipment gave different results when testing for vitamin D, two thyroid hormones and prostate cancer. The gap suggested to some employees that the Edison results were off, according to the internal emails and people familiar with the findings. Senior lab employees showed both sets of results to Sunny Balwani, Theranos’s president and chief operating officer. In an email, one employee said he had read “through the regulations more finely” and asked which results should be reported back to the test administrators and government. Mr. Balwani replied the next day, copying in Ms. Holmes. “I am extremely irritated and frustrated by folks with no legal background taking legal positions and interpretations on these matters,” he wrote. “This must stop.” He added that the “samples should have never run on Edisons to begin with.” Former employees say Mr. Balwani ordered lab personnel to stop using Edison machines on any of the proficiency-testing samples and report only the results from instruments bought from other companies… The former employees say they did what they were told but were concerned that the instructions violated federal rules, which state that a lab must handle “proficiency testing samples…in the same manner as it tests patient specimens” and by “using the laboratory’s routine methods.”‘
- Accuracy is among the two most important aspects of medical assays, the other being precision. The two aren’t synonymous. Accuracy = how well the technology measures industry consensus reference standards. Precision = how much test results vary for same sample tested in repeat tests. Even if a technology is very precise, if it isn’t accurate, it could be precise about exactly the wrong thing (9). That’s why accuracy in proficiency testing is a clinical lab gold standard and pretty much everything else about a blood testing technology is irrelevant without it.
Third, Silicon Valley has been excellent at using computing power to provide consumers products of convenience. Buy a smart phone or download an app. Change mind or too many glitches? Just return or exchange or delete. Biotech is not IT (infotech). Accuracy and precision of a health test result can literally mean the difference between life and death. Obviously self-evident, only the suicidal or suicidally foolhardy would gamble with their health.
- Theranos exemplifies a dangerous mission creep of the excessive, bombastic Silicon Valley hype infiltrating octopus-like into realms such as health where it has no business to be in and in fact is outright dangerous for it to be in.
- Matters even more with a privately held company with potential political backing to have laws written or re-written, tilting the balance in its favor at the expense of people’s health.
Fourth, maybe most important, the dangers of wellness testing or Direct-to-Consumer (DTC) health care products.
- As writes (2), ‘overdiagnosis, false-positive findings, or the potential for escalation of iatrogenic disease secondary to misplaced and perhaps overly zealous diagnostic and screening efforts‘.
- Eleftherios P. Diamandis, professor of clinical pathology at the University of Toronto, raised similar concerns in the May, 2015 issue of Clinical Chemistry and Laboratory Medicine (3A), ‘In this paper, I analyze the Theranos technology and their promises, and contrast this information with the currently used technologies, to show that most of the company’s claims are exaggerated. While it remains to be seen if this technology will revolutionize diagnostics, in this Opinion Paper, I also draw attention of associated issues, such as self-testing and self-interpretation of results, over-testing, over-diagnosis and over-treatment, along with their associated harms‘.
- Other than those that detect infections, with few exceptions, most routine lab tests are by themselves not indicative of disease. Most important exception is blood sugar for diabetes and patients’ already self-test for that.
- False positives are obviously a huge problem in diagnostic tests, especially for the asymptomatic. Is it occult disease or test precision issue? Is it reasonable to expect a lay person to understand the difference? Isn’t this why doctors spend a decade or more in intensive training and practice?
- Dangers of false positives, over-testing and over-diagnosis aside, there is also a much more dangerous conflict of interest at play here. Who clearly benefits from changing the status quo of doctors interpreting test results? Neither patients nor doctors but only service providers benefit from DTC diagnostic blood tests. But how?
- As Wired (13 B) points out, passed in the wake of another WSJ report of serious flaws in lab reports on pap smears, the 1988 (CLIA) regulations were supposed to strengthen patient safety. Every new test is rated on a complexity range. At one end, ‘highest complexity‘, i.e., only physician or nurse can administer. At the other end, ‘waived‘, i.e., self-administered, e.g., insulin and pregnancy. To quote Wired (13B), ‘But diagnostics companies eventually realized that CLIA, designed to close a loophole, had actually opened one. “Originally these were academic medical centers coming up with new way of testing,” says Master. “Someone at some point realized it was great business model.” Theranos apparently recognized that opportunity, too. Its strategy is all about patient access, so it wants waived status for its tests and in fact received it for its herpes simplex I test. Make tests easy and accessible to patients, even at severely discounted prices, and any company stands to make a lot of money—and potentially revolutionize healthcare‘.
- This is a huge red flag and undeniable conflict of interest. Bodes ill for a society when a privately held entity schemes its way to a healthcare outcome that the public should reach after vigorous public debate and consensus.
For further reading, the information sources I used in composing this answer:
- Business Insider
A) Kevin Loria, April 25, 2015.
B) Jillian D’onfro, Oct 20, 2015.
- Ioannidis, John PA. “Stealth research: is biomedical innovation happening outside the peer-reviewed literature?.” JAMA 313.7 (2015): 663-664.
- A) Diamandis, Eleftherios P. “Theranos phenomenon: promises and fallacies.” Clinical Chemistry and Laboratory Medicine (CCLM) (2015).
B) Li, Michelle, and Eleftherios P. Diamandis. “Theranos phenomenon− part 2.” Clinical Chemistry and Laboratory Medicine (CCLM) (2015).
C) Plebani, Mario. “Evaluating and using innovative technologies: a lesson from Theranos?” Clinical Chemistry and Laboratory Medicine (CCLM) (2015).
- Monday Note, , former Apple exec, Oct 18, 2015. His first person accounts of his and his wife’s all-over-the map Theranos blood test results make for engrossing reading. Also very revealing about Theranos’ tone-deaf corporate culture.
- Bloomberg Business rolling coverage post-WSJ
Oct 15, 2015, Melissa Mittelman.
Oct 15, 2015, Caroline Chen.
Oct 19, 2015, Caroline Chen.
- Fortune magazine, Roger Parloff, Oct 15, 2015.
- Fox Business, Steve Tobak, Oct 19, 2015.
- The Tech Insider, Kevin Loria, Oct 16, 2015.
- Bobby Gladd at the KHIT blog for the most comprehensive summation of all the recent media hoopla about Theranos. Holmes is no Sherlock. Edison is no Watson. Theranos, continued.
- Matthew Holt at the HealthCare blog, Oct 16, 2015. Similar implication to 4 of tone-deaf corporate culture in this first person account.
- The Los Angeles Times, Michael Hitlzik, Oct 16, 2015.
- The Wall Street Journal, John Carreyrou, Oct 16, 2015.
- Wired Magazine
A) Oct 15, 2015, Issie Lapowsky.
B) Oct 20, 2015, Nick Stockton.