Home Testing

Feeling a little lost about home testing for Covid-19? If you’re not, either you are an expert or you aren’t paying attention. I say this because there is no possible way to interpret the results of this kind of medical test without knowing three numbers, all of which are at best hard to find. Once you do find them, the implications are underwhelming, to say the least, as I’ll explain below.

This isn’t some pedantic technicality–tests truly don’t mean jack if you don’t know those numbers. The best you can do without them is to go by some simple rule of thumb like positive=quarantine. That’s generally fine for positive results because if you get a false positive, all it means is a few unnecessary days alone watching Netflix. Unfortunately, not knowing what a negative result means gets people killed.

It’s bizarre that two years into the pandemic, this is so little understood either by the public or by the people who present the news. The public health people certainly understand, because it’s basic medical statistics, but mostly continue to speak to the public as if this were not a thing. At the risk of seeming cynical, testing is a very positive sounding thing to talk about at a time when the public health authorities have conspicuously failed to cover themselves with glory. It took almost two years to make rapid testing readily available. After all that, it’s only natural to want it seen in the best light.

There are a few numbers involved, but it doesn’t much math to get the gist of it. Percentages and multiplication, basically. If you are at risk, or you’ve got anyone in your orbit who is especially at risk, such as your aging Granny, it’s definitely worth reading this.

If the TLA “PPV” means anything to you, feel free to skip to the section The Numbers for At-Home Tests near the bottom–you’ll probably know everything between here and there.

Test Results Don’t Mean What You Probably Think

All statistics students and most doctors will have run into these ideas at some point, but I suspect that the average family doctor would need a quick refresher if they ever had to apply it, because the situations where you need it mostly come up after the case has already been booted up to a specialist. The average GP probably remembers this about as well as you remember trigonometry.

There Is No Such Thing As Test Accuracy

You see everywhere that such and such a test is some percent accurate, but in fact, there is literally no such thing as “test accuracy.” If someone tells you a test is X% accurate, either they don’t know what they are talking about or they do, and they are afraid that the full story would just confuse you.

The mathematical fact is that any meaningful statement about the accuracy of a medical test needs at least two numbers, and usually three. So universally disregarded is this truth that it takes some effort to even find out what the three numbers are for Covid tests. They’re out there, but you have to know what to look for and what it means.

I encourage the reader to sit still for the explanation–there’s not much math in it–It’s more about words. The first two numbers have such exact and non-intuitive meanings that you’d think they were dreamed up by a lawyer.

Sensitivity is the first number. A sensitivity score of X means that when you test 1000 people who you know for a fact have the disease, on average, X percent will test positive. Say X = 99% (which would be very good.) If you test 1000 people who have a disease, 990 will test positive and 1%, i.e 10 of them will falsely test negative.

That sounds great, but there’s a catch. Any number of people who don’t have it can also test positive and it has no effect on the sensitivity number. These are called false positives.

The second number is specificity. A specificity of Y means that if we test 1000 people who definitely do not have the disease, Y percent will in fact test negative. So if it’s highly specific, almost everyone who doesn’t have it will test negative, but it doesn’t say anything about how many who do have it will also test negative.

You can see that high specificity tends to squelch false positives because if they don’t have it, they test negative.. And the opposite is true too–high sensitivity tends to squelch false negatives, because if they do have it, they test positive.

A test that’s strong in both is great–high sensitivity catches false negatives and high specificity catches false positives, leaving only mostly the true negatives and true positives.

Unfortunately, the two numbers can be very different for the same test and neither is usually 99%. For instance, imagine a test that is very sensitive, i.e., it catches almost every case, but not very specific, i.e., it can also have lots of false positives. If you get a negative result for a test like that, great! Why? Because the high sensitivity rules out most false negatives leaving only the true negatives. The reverse is the case for a positive result. The test isn’t very specific, so it’s bad at ruling out false positives. Therefore, a positive result doesn’t tell you much, except that that the disease hasn’t been ruled out.

Some tests are the other way around. Not very sensitive but highly specific. So if you get a positive result for a test like that, it has a high chance of being accurate, because the high specificity squelches most of the false positives, leaving the true ones. So if you did get a positive, it’s most likely true. On the other hand, the negative results are not to be relied upon because low sensitivity does a bad job of canceling out false negatives. If such a test says you have it, you probably do, but if it says you don’t, you can’t rely on it.

To sum up,

  • High sensitivity and high specificity gives good results both ways.
  • High sensitivity and low specificity means gives reliable negatives but positive result doesn’t tell you much.
  • Low sensitivity and high specificity gives reliable positives, but unreliable negatives.

Simple, right?

The Pesky Third Number

Actually, it’s not that simple, because I glossed over the third number which is the density of actual cases in the population of concern. This is the giant gotcha in interpreting test results.

Say Dr Bob has a super good test for a hideously awful, incurable disease. The test is 99% sensitive, i.e. and 99% specific. As good as it gets, basically. Fortunately, the disease it tests for is extremely rare; only one in a million Americans have it. (That makes it a little more common than leprosy, aka Hansen’s disease.)

Now say the over-zealous Doctor Bob decides to just randomly give the test to Alice, and she happens to test positive. Alice is screwed, right? She got a positive test with a 1% false positive rate so I guess it’s time for her to subscribe to the Hemlock Society YouTube Chanel.

Certainly not! The key here is that Dr. Bob just randomly picked Alice without any reason to think she has it. The test result has 99.99% probability of being a false positive.

Let’s see how it works. If you gave the test to all 325 million Americans, you would get 3.25 million false positives, i.e. 1%. But it actually affects only one person in a million, so only be about 325 people in the US actually have it. Of them, you would expect about 321 would test positive because the sensitivity is 99%. In the absence of any reason to think she Alice has it, the probability that the result is valid is the ratio of the number of true positives to the number of all positives, both true and false.

This is a really tiny number. 325/(3,250,000 + 321) = 0.0001, which is one in ten thousand. So even with a positive result from an excellent test, Alice is less likely to have the disease than she is to die in a car accident this year, a risk so small she rarely even thinks about it.

This calculation is the essence of what statisticians call “Positive Predictive Value” or PPV.

The Numbers for At-Home Tests

So what does that say about home tests? There are five at-home tests for Covid on the market. One is strikingly worse than the other four, with a sensitivity of 34.1 and a specificity of 88.1. The rest have sensitivities of between 44% and 54% and they all have specificity of either 100% or very close. (Which is phenomenal.)

Let’s toss the outlier and round the others four to 50% and 100%.

As we saw above, the meaning can depend heavily upon that third number, which is the percentage of cases in the population. In the example, the expected number of false positives completely swamped the number of true positives, giving it a PPV of almost 0.

However, in this case, the specificity is almost perfect, which means there are zero false positives. The PPV is the number of true positives divided by the total number of all positives, true and false. With zero false positives, you get a PPV of 1.0 (which is freakishly good.) This means that the probability that a positive result means you have Covid is 100%.

That’s nice, but who cares? Even with a less reliable result, you would still going to have to quarantine as it it were, so what does that perfection really get you?

What we really care about are the false negatives. A negative result, true or false, is what gets you on the plane or gets you a seat at the table with all of your elderly, obese, or immune compromised relatives. So what does it say about negative tests?

Just glancing at the numbers tells the story. All the test promises is that if you’re positive, then you definitely have it. The test caught only half the people who have it–the rest all got false negatives. Low sensitivity means low protection from false negatives.

We know that half the cases went uncaught, so if you got a negative result, all it means is that the probability that you have Covid is about half of what you would have estimated the probability was if you didn’t take a test at all. Not a very impressive result, to say the least.

Are The Test Kits Going To Protect Granny?

If you get a positive result, great, it’s nice to know. Wait, I mean sorry–you definitely have Covid. It’s only great if you’re into statistics and statistically speaking, you probably are not. But random people taking the test will overwhelmingly get negative results, primarily because most people don’t have it, but also because about half of those who do have it test negative anyway.

I don’t have any idea what percentage of random people in my area who have no special reason to suspect that they have Covid, have it. I doubt anyone knows. So how is a negative result useful to me, if all is says is that the probability that I have Covid is 50% less than damn-if-I-know?

I’m not even sure if making them generally available is a good idea. The problem is, while a negative result makes it 50% less likely that you have Covid, nobody really knows how much more likely the false sense of security makes people to do things that spread Covid.

The bottom line is, a negative at-home test result is only somewhat more reliable than crossing your fingers. Do it if you want, but it’s no substitute for having everyone who comes in contact with people who are at risk being vaccinated and boosted to the max.

Advertisement

He Said Six Weeks

Photo by Thomas Chan https://unsplash.com/@c5m2h3

A couple of days ago (it’s September 17th 2020) Dr. Robert Redfield, the director of The Center for Disease Control, testified to Congress that universal mask wearing in the US would bring Covid-19 under control in the US in six weeks. He’s has said this before but this time he said it under oath to Congress. Once again, didn’t make a ripple.

Dr. Redfield isn’t your drunk uncle Bob—the CDC is the deep duck in the epidemiology puddle and Redfield is their top guy. They have a budget twice as large as the NIAID (Dr. Fauci’s organization) and collectively know more about controlling infectious diseases than any other organization in the world.

His testimony barely made the papers. Control in six weeks with just masks that you can get for a buck a pop. Not masks plus economy-crippling isolation. Not masks plus vaccine. Not even masks plus elaborate social distancing. Just masks. Anything else you do is gravy. Redfield has made the same statements on camera before and it seems to have had no impact whatsoever. I’m at a loss to explain the lack of reaction. It’s a giant get-out-of-jail-free card for the whole country and the economy. It could save 250,000 more lives in the US this Winter for pocket change and make hundreds of millions of people less poor, bored, and anxious. Yet nobody is interested.

It’s not some pipe dream. His calculation is based on definitive research from a recent study on the efficacy of masks and backed up by practical experience around the world. The calculation is trivial, immediately obvious if you read the research. Moreover, the research would have to be wildly wrong to substantially change Dr. Redfield’s conclusion. Any plausible error would mean only that it wouldn’t be six-weeks, but eight, or twelve. The principle would hold up.

Continue reading “He Said Six Weeks”

99% Disappointing

When I first heard there was an antibody (serum) test I thought wow, this is fantastic!  If you are certified to have already had it, then you know that it’s safe for you to be around others and others can be confident that they are safe around you. It could be like a license to go to work.virus_antibody_illustration

Then I thought about it.  Actually, the test is probably useless for you, personally. (It has other uses, like making policy, but that’s not what we’re talking about here.) The problem has nothing to do with not knowing whether Covid-19 guarantees future immunity. You don’t need to go there in order to show that it’s useless for the average person.

This isn’t an Internet crackpot thing—it’s real math you can verify yourself.  It’s a disappointment but the reasons are interesting and the principle applies to all tests that yield a positive/negative result. The smaller the proportion of people in the population that have the condition in question, the more this principle applies.

I’m just going to explain one small aspect of this. One of the main places this applies is in diagnosing illnesses and that water gets very deep. Still, it’s interesting to poke around in it and it might help you understand what your doctor is doing someday.

Continue reading “99% Disappointing”

You Might Be Thinking About It All Wrong

Geoffrey Chaucer, who lived through the black death, wrote the line “Ech man for himself, ther is non other” in The Knight’s Tale and it quickly entered English permanently as “Every man for himself and the devil take the hindmost.”
plague

 

Every day I see despairing posts and re-posts of articles and blogs claiming that the pandemic in America is a lost cause. One post that is currently getting massive attention asserts that the epidemic in the USA is now in a runaway state that can no longer be brought under control. Another simply assumes that this is true, and concludes that Covid-19 will eventually infect everyone in America, killing 1% (3.25 million people) and crippling or otherwise disabling many tens of millions of us in gruesome ways.

None of it is true. The ubiquity of graphs like the one below make this feeling understandable on an emotional level but the despair it engenders is completely inconsistent with the facts. The appropriate emotions in response to the graph below are (a) fury and (b) hope.

By way of making the case for hope, I’d like to lay on you one of the most remarkable and under-publicized bits of research I’ve come across but first we need to look at some basics.

Continue reading “You Might Be Thinking About It All Wrong”

Am I Missing Something?

It’s May 16th, 2020, and there’s an undeniable feeling of optimism in the US about Covid-19. A vibe that we’ve got this thing on the run. All over the country businesses are opening up and we’re getting ready for Summer.

I feel like I must be missing something.  We all see the same data but to me it looks anything but reassuring. The current numbers look like a failure and a setup for a calamity in the fall.

The conclusion I’ll outline below could be wrong—I hope it’s wrongbut when you look at the larger context it’s such an obvious inference that even if it is wrong, it seems like it should be the default conclusion that the uninformed jump to, the one that people in the know contemptuously debunk, as they do when someone says that “it’s no worse than the flu.”  So right or wrong, either way, something seems off.

To see why it looks so bad to me, consider a couple of points first.

In What Sense Is Covid-19 Under Control?

First of all, the widespread conviction that the epidemic is winding down in the US is itself a mystery.

dailycase

Continue reading “Am I Missing Something?”

We Ain’t Seen Nothing Yet

tarpit

This sobering article in The Economist last year outlined the consequences to expect from a Brexit-without-a-deal. Most of it still applies, but to me, a non-economist, the diversity and magnitude of malign consequences suggests that Brexit could be a more interesting experiment than anyone thinks.

It’s not that we’ve lacked for economic turmoil since the age of inter-networking for business and the general public took off in the late 1980’s, but the problems have been fairly conventional in economic terms. Recessions, bubbles, the CMO meltdown, and so on; none of it has been greatly different from the trouble we’d gotten into for many decades previous.

Brexit brings up the possibility of a truly modern meltdown—an economic calamity that as yet has no name.

Continue reading “We Ain’t Seen Nothing Yet”

Christianity and Ducks

Atheists invariably haul out the religious wars of Europe to make the case that religion is pernicious. It is indisputable that from The Age of Faith to the Reformation/Counter Reformation numerous bloody wars and slaughters were committed in the name if the Christian God and there were countless more if you include the wars fought in His second best known name, Allah. It’s not an obviously wacky point.8bdfe51795

But hold on there—religious war is monstrous just as all war is monstrous, and it is possible that religion is monstrous as well, but the proposition that wars being fought over religion proves that religion is monstrous is a classic example of what philosophers call “an association fallacy of the red herring type.” The herring, i.e., the thing dragged in that is not logically connected is religion. Helen’s face launched a thousand ships only in a poetic sense; Paris and Agamemnon are to blame for the Trojan war, not Helen, love, or beauty.

Such a fundamentally flawed attack shouldn’t require any defense, but logic isn’t a central concern in religion; Christians consistently fall for this argument and end up defending Christianity from the accusation with words to the effect that “sure, there were wars, but those people weren’t real Christians” or “they weren’t acting consistently with Christian principles.

This is a terrible argument but not because it’s inherently fallacious. It’s weak because it invites the accuser to apply what is known as the duck test, AKA Occam’s Razor. You’d sneer if I defended, say, Nazism, using the same logic. Try it out: “Nazism didn’t really underly the horrors of the Holocaust; the problem was that bad people coopted a good idea. Let’s let bygones be bygones and give it another chance.” No, the killers were Nazis and the actions were consistent with the principles of Nazism as advocated by the founders, so you can go out on a limb and say Nazism caused the Holocaust. It is logically possible that a small animal that looks like a duck, swims like a duck, quacks like a duck, and is often seen in the company of ducks, is not a duck, but that’s not where a wise ornithologist will take the argument.

Continue reading “Christianity and Ducks”

Bitchin’

Against a cultural background of morbid touchiness about references to gender, race, etc., the word bitch is a lens into what’s going on beneath the fig leaf of political correctness.

Calling a particular person a bitch in private is bad manners but if you disregard all the ostentatious political posturing, the visceral reaction most people have to bitch used in that way isn’t much different from their reaction to gender-neutral epithets such asshole, or to male-specific insults such as prick, or dick. Bitch, used in this way, is an insult to a particular person, and after all, offending is the goal. Bitch is not nearly as shocking to the ears as the c-word-that-dare-not-be-spoken (here in the US, anyway—in the UK cunt is practically a term of endearment when applied to a man.)

Gender-specific insults are tricky but you have to be willfully obtuse to deny that much obnoxious behavior has gender. I’m not talking about the gender of the obnoxious person, but the behavior itself. Being a prick is definitely a yang personality trait and being a bitch is yin, regardless of the sex of the insulted or insulting party. Asshole, on the other hand, is neuter, despite the fact that many more men than women are assholes. Assholes are like happy families but the epithet pig can connote at least three distinct kinds of obnoxiousness, one specific to each gender plus one neuter. I like the gender-specificity of all of these words and usages and doubly applaud their use when the target is not gender-consistent with the epithet. The comprehensibility of cross-gender insults is a sign of progress in the relationship of the sexes.

Continue reading “Bitchin’”

Don’t Be So Cynical!

diogenes
Diogenes the Cynic searched the world for an honest man.

I had an epiphany about what’s up with people who claim to “believe” things that are manifestly not so. I’m not talking about things people believe that are arguably wrong about or about matters of faith like belief in God or in karma. After all, most of us are wrong about most things most of the time.

The things that I’m talking about are things that you’d think it would be manifestly impossible to believe in good faith. Like saying you “believe” that the Earth is flat, or that two flatly contradictory lines of Scripture are both literally true.  Plausibility is subjective but how does one argue with a person who denies the rules of logic?

Continue reading “Don’t Be So Cynical!”

Yep, I Definitely Believe Her.

I wasn’t a reflexive Kavanaugh hater when this started. His politics are deplorable but they are pretty much of a piece with those of anyone else who was in the running. His evasive responses to virtually all questions were dismaying but they were not much worse than has become customary in Senate hearings.

kavanaugh-ford

The spectacle only strayed into the bizarre with Kavanaugh’s response to Christine Blasey Ford’s testimony. Who carries on like that at a job interview, attacking and insulting the interviewers, raging, crying, acting hurt, citing massive conspiracies to get revenge for work he did for Ken Starr twenty-some years ago?  You can lose touch with how weird that performance was.

Continue reading “Yep, I Definitely Believe Her.”