Who’s right?

Not that long ago, I was one of the very many people that was extremely excited by the potential of a company called Theranos. Theranos has developed a product that can run hundreds of blood tests from a single drop of blood, attained from a finger stick. Rather than having to maintain huge systems for running a full series of blood tests, these large systems could be replaced by a desktop device that could be operated by almost anyone.

This really is the direction that everything is going. Computer systems that once upon a time took up rooms of space, have been replaced by our cell phones which are also a million to a billion times more capable than the computers of old. Basically anything that is designed in a lab as a huge lumbering device, eventually makes it to market in a far more compact version.

Another classic example is augmented reality. Microsoft’s major hardware and software technical meeting (Build 2016) recently demonstrated the latest version of their augmented reality system, called Hololens. This is the year that this wireless [a key point] device is going to hit the market. I would be very surprised if it is not a major hit from day one. Potentially, the cost will initially limit the number of customers. But the cost will come down and capabilities will increase, and this will become literally a household item, just like gaming machines, 32 inch TVs and iPods.

I want to just quickly add that augmented reality is a transformative technology. It will find the place in pretty much every aspect of our lives. While at the moment, it is a helmet with a visor, eventually this will be shrunk down to small devices that seamlessly clip on to a pair of glasses. And for those people who don’t wear glasses, or who wear contact lenses, there will be solutions for them as well. The day will come when the idea of not having a constant inflow of accessory information will make a person feel as if they have forgotten their portable phone at home. I’ve spoken about this in the past and I will not belabor this point now.

In terms of the company Theranos, rather than being a multibillion-dollar breakout, it is suffering from a serious question of reliability. Of late, a study was published in the Journal of Clinical Investigation. To quote from an article from Forbes:

Researchers at the Icahn School of Medicine at Mt. Sinai have done their own analysis of Theranos’ tests, which replace needle jabs with finger pricks. They have found them wanting.

The Mt. Sinai researchers analyzed results from 60 patients using Theranos and tests from larger rivals Laboratory Corporation of America and Quest Diagnostics. The study found that Theranos consistently gave cholesterol readouts that were five to ten points too low. It was 12.5 times more likely to say it could not get results from a sample. It was 60% more likely to say a test value was too low or too high.

First of all, there are a number of statistics included in this quote. One could argue that 60 patients is not enough, and that the relative values of 12.5 times and 60% do not reflect clinically significant differences [which is part of the claim of Theranos].

There is definitely something to be said for the retorts by Theranos. When doing a comparative study, you need something that is considered a reference point or the gold standard. The other two labs that tested the blood results sent to them by the Mount Sinai researchers, also had significant disparities between them. Generally speaking each lab is calibrated independently and the range of normal can vary. It should not vary by very much because that would effectively invalidate any values that are presented in the literature. But a cholesterol level of 160, might be considered on the border of high in one lab, but still comfortably within the normal range in another. If this  raises the question in your mind as to how we know that anything is absolutely correct, you are raising an excellent point.

As somewhat of a segue, I could ask how you know that a meter is really a meter long. What do you measure a meter stick against, in order to know that it is really a meter long. How do we know how long a second is? It’s not based on someone standing somewhere with a stopwatch. As noted in the ultimate source of all celestial knowledge, Wikipedia, “It is quantitatively defined in terms of a certain number of periods – about 9 billion – of a certain frequency of radiation from the caesium atom: a so-called atomic clock.” I like the fact  that as part of this definition, the author used the term  “about”. So much for accuracy.

The point is that there are many things that are a part of our daily life that get measured, without us having any absolute knowledge of what the right answer is. In many cases, we decide what the right answer is and then build devices, structures and our science, all on the basis of these assumed absolutes. The key is not so much to find the absolute truth. In clinical work, what I care about is that the patient presently has  a series of blood tests that are considered within the normal range. And how do I know that the normal range quoted from the lab is valid?

Well, once upon a time, somebody did a big study where they, for example, drew blood from lots and lots of people and measured all types of components using equipment  that they had designed.  On the basis of the values that came out of this study, a probability curve was drawn. That’s just a fancy way of saying that based on this population, they figured out what the  range of normal was.

Admittedly, you have to assume that everyone in this clinical study was healthy. So if you accidentally included a bunch of people with undiagnosed diabetes and kidney disease, that could definitely skew the results. And more so, you have to be sure that the equipment that people will use in the future is equivalent to the equipment that was used during the study. If another device yields a result that is twice the value from this “normal” population, it doesn’t actually mean that the device is wrong. What it means is that this new device is not “calibrated” to the standards set in the original study. If all the results from the new device are twice what is considered normal, it’s possible that the developers would modify the internal software to simply divide everything by two. And then magic – they have a new device to sell to the public.

So Theranos is challenging this study performed by Mount Sinai researchers on a whole number of points. The problem is that Theranos has to start making money at some point, or else its valuation will collapse, along with the entire business. At the moment, I think it is fair to say that there is significant suspicion as to the reliability of the device. Trust is like a mirror – once broken, it can never be perfectly restored [unless you are a major glass craftsman, but please don’t confuse me with facts].

Whether it is Theranos or another company, at some point, it will be possible to do the entire range of blood tests that are done in major labs, via a compact unit that sits in a doctor’s office or in a person’s home. When the day comes that 3-D imaging displays your entire body  to a remote doctor, and all blood tests can be done at home, and vital signs like blood pressure and pulse and weight can be measured also at home, we will enter a new phase of telemedicine.

I recently saw an advertisement for an ultrasound device that is comprised of a handheld unit and your portable phone. A doctor definitely doesn’t have to be the one to do the test, in a case like this. In theory, a minimally trained patient, or definitely a nurse or formal ultrasound technician could come to the patient’s house, do the study, and have the information uploaded to the patient’s online file. At this point, if there are any anomalies, they will most likely require a more formal evaluation in the doctor’s office or possibly directly in a hospital.

I will leave you with a final quote from the Forbes article on this topic.

“In God we trust,” the mid-century statistician William Edward Deming quipped. “All others bring data.”

Thanks for listening.

About the Author
Dr. Nahum Kovalski received his bachelor's of science in computer science and his medical degree in Canada. He came to Israel in 1991 and married his wife of 22 years in 1992. He has 3 amazing children and has lived in Jerusalem since making Aliyah. Dr. Kovalski was with TEREM Emergency Medical Services for 21 years until June of 2014, and is now a private consultant on medicine and technology.