Feeling a little lost about home testing for Covid-19? If you’re not, either you are an expert or you aren’t paying attention. I say this because there is no possible way to interpret the results of this kind of medical test without knowing three numbers, all of which are at best hard to find. Once you do find them, the implications are underwhelming, to say the least, as I’ll explain below.

This isn’t some pedantic technicality–tests truly don’t mean jack if you don’t know those numbers. The best you can do without them is to go by some simple rule of thumb like positive=quarantine. That’s generally fine for positive results because if you get a false positive, all it means is a few unnecessary days alone watching Netflix. Unfortunately, not knowing what a negative result means gets people killed.

It’s bizarre that two years into the pandemic, this is so little understood either by the public or by the people who present the news. The public health people certainly understand, because it’s basic medical statistics, but mostly continue to speak to the public as if this were not a thing. At the risk of seeming cynical, testing is a very positive sounding thing to talk about at a time when the public health authorities have conspicuously failed to cover themselves with glory. It took almost two years to make rapid testing readily available. After all that, it’s only natural to want it seen in the best light.

There are a few numbers involved, but it doesn’t much math to get the gist of it. Percentages and multiplication, basically. If you are at risk, or you’ve got anyone in your orbit who is especially at risk, such as your aging Granny, it’s definitely worth reading this.

If the TLA “PPV” means anything to you, feel free to skip to the section The Numbers for At-Home Tests near the bottom–you’ll probably know everything between here and there.

Test Results Don’t Mean What You Probably Think

All statistics students and most doctors will have run into these ideas at some point, but I suspect that the average family doctor would need a quick refresher if they ever had to apply it, because the situations where you need it mostly come up after the case has already been booted up to a specialist. The average GP probably remembers this about as well as you remember trigonometry.

There Is No Such Thing As Test Accuracy

You see everywhere that such and such a test is some percent accurate, but in fact, there is literally no such thing as “test accuracy.” If someone tells you a test is X% accurate, either they don’t know what they are talking about or they do, and they are afraid that the full story would just confuse you.

The mathematical fact is that any meaningful statement about the accuracy of a medical test needs at least two numbers, and usually three. So universally disregarded is this truth that it takes some effort to even find out what the three numbers are for Covid tests. They’re out there, but you have to know what to look for and what it means.

I encourage the reader to sit still for the explanation–there’s not much math in it–It’s more about words. The first two numbers have such exact and non-intuitive meanings that you’d think they were dreamed up by a lawyer.

Sensitivity is the first number. A sensitivity score of X means that when you test 1000 people who you know for a fact have the disease, on average, X percent will test positive. Say X = 99% (which would be very good.) If you test 1000 people who have a disease, 990 will test positive and 1%, i.e 10 of them will falsely test negative.

That sounds great, but there’s a catch. Any number of people who don’t have it can also test positive and it has no effect on the sensitivity number. These are called false positives.

The second number is specificity. A specificity of Y means that if we test 1000 people who definitely do not have the disease, Y percent will in fact test negative. So if it’s highly specific, almost everyone who doesn’t have it will test negative, but it doesn’t say anything about how many who do have it will also test negative.

You can see that high specificity tends to squelch false positives because if they don’t have it, they test negative.. And the opposite is true too–high sensitivity tends to squelch false negatives, because if they do have it, they test positive.

A test that’s strong in both is great–high sensitivity catches false negatives and high specificity catches false positives, leaving only mostly the true negatives and true positives.

Unfortunately, the two numbers can be very different for the same test and neither is usually 99%. For instance, imagine a test that is very sensitive, i.e., it catches almost every case, but not very specific, i.e., it can also have lots of false positives. If you get a negative result for a test like that, great! Why? Because the high sensitivity rules out most false negatives leaving only the true negatives. The reverse is the case for a positive result. The test isn’t very specific, so it’s bad at ruling out false positives. Therefore, a positive result doesn’t tell you much, except that that the disease hasn’t been ruled out.

Some tests are the other way around. Not very sensitive but highly specific. So if you get a positive result for a test like that, it has a high chance of being accurate, because the high specificity squelches most of the false positives, leaving the true ones. So if you did get a positive, it’s most likely true. On the other hand, the negative results are not to be relied upon because low sensitivity does a bad job of canceling out false negatives. If such a test says you have it, you probably do, but if it says you don’t, you can’t rely on it.

To sum up,

• High sensitivity and high specificity gives good results both ways.
• High sensitivity and low specificity means gives reliable negatives but positive result doesn’t tell you much.
• Low sensitivity and high specificity gives reliable positives, but unreliable negatives.

Simple, right?

The Pesky Third Number

Actually, it’s not that simple, because I glossed over the third number which is the density of actual cases in the population of concern. This is the giant gotcha in interpreting test results.

Say Dr Bob has a super good test for a hideously awful, incurable disease. The test is 99% sensitive, i.e. and 99% specific. As good as it gets, basically. Fortunately, the disease it tests for is extremely rare; only one in a million Americans have it. (That makes it a little more common than leprosy, aka Hansen’s disease.)

Now say the over-zealous Doctor Bob decides to just randomly give the test to Alice, and she happens to test positive. Alice is screwed, right? She got a positive test with a 1% false positive rate so I guess it’s time for her to subscribe to the Hemlock Society YouTube Chanel.

Certainly not! The key here is that Dr. Bob just randomly picked Alice without any reason to think she has it. The test result has 99.99% probability of being a false positive.

Let’s see how it works. If you gave the test to all 325 million Americans, you would get 3.25 million false positives, i.e. 1%. But it actually affects only one person in a million, so only be about 325 people in the US actually have it. Of them, you would expect about 321 would test positive because the sensitivity is 99%. In the absence of any reason to think she Alice has it, the probability that the result is valid is the ratio of the number of true positives to the number of all positives, both true and false.

This is a really tiny number. 325/(3,250,000 + 321) = 0.0001, which is one in ten thousand. So even with a positive result from an excellent test, Alice is less likely to have the disease than she is to die in a car accident this year, a risk so small she rarely even thinks about it.

This calculation is the essence of what statisticians call “Positive Predictive Value” or PPV.

The Numbers for At-Home Tests

So what does that say about home tests? There are five at-home tests for Covid on the market. One is strikingly worse than the other four, with a sensitivity of 34.1 and a specificity of 88.1. The rest have sensitivities of between 44% and 54% and they all have specificity of either 100% or very close. (Which is phenomenal.)

Let’s toss the outlier and round the others four to 50% and 100%.

As we saw above, the meaning can depend heavily upon that third number, which is the percentage of cases in the population. In the example, the expected number of false positives completely swamped the number of true positives, giving it a PPV of almost 0.

However, in this case, the specificity is almost perfect, which means there are zero false positives. The PPV is the number of true positives divided by the total number of all positives, true and false. With zero false positives, you get a PPV of 1.0 (which is freakishly good.) This means that the probability that a positive result means you have Covid is 100%.

That’s nice, but who cares? Even with a less reliable result, you would still going to have to quarantine as it it were, so what does that perfection really get you?

What we really care about are the false negatives. A negative result, true or false, is what gets you on the plane or gets you a seat at the table with all of your elderly, obese, or immune compromised relatives. So what does it say about negative tests?

Just glancing at the numbers tells the story. All the test promises is that if you’re positive, then you definitely have it. The test caught only half the people who have it–the rest all got false negatives. Low sensitivity means low protection from false negatives.

We know that half the cases went uncaught, so if you got a negative result, all it means is that the probability that you have Covid is about half of what you would have estimated the probability was if you didn’t take a test at all. Not a very impressive result, to say the least.

Are The Test Kits Going To Protect Granny?

If you get a positive result, great, it’s nice to know. Wait, I mean sorry–you definitely have Covid. It’s only great if you’re into statistics and statistically speaking, you probably are not. But random people taking the test will overwhelmingly get negative results, primarily because most people don’t have it, but also because about half of those who do have it test negative anyway.

I don’t have any idea what percentage of random people in my area who have no special reason to suspect that they have Covid, have it. I doubt anyone knows. So how is a negative result useful to me, if all is says is that the probability that I have Covid is 50% less than damn-if-I-know?

I’m not even sure if making them generally available is a good idea. The problem is, while a negative result makes it 50% less likely that you have Covid, nobody really knows how much more likely the false sense of security makes people to do things that spread Covid.

The bottom line is, a negative at-home test result is only somewhat more reliable than crossing your fingers. Do it if you want, but it’s no substitute for having everyone who comes in contact with people who are at risk being vaccinated and boosted to the max.

It’s May 16th, 2020, and there’s an undeniable feeling of optimism in the US about Covid-19. A vibe that we’ve got this thing on the run. All over the country businesses are opening up and we’re getting ready for Summer.

I feel like I must be missing something.  We all see the same data but to me it looks anything but reassuring. The current numbers look like a failure and a setup for a calamity in the fall.

The conclusion I’ll outline below could be wrong—I hope it’s wrong—but when you look at the larger context it’s such an obvious inference that even if it is wrong, it seems like it should be the default conclusion that the uninformed jump to, the one that people in the know contemptuously debunk, as they do when someone says that “it’s no worse than the flu.”  So right or wrong, either way, something seems off.

To see why it looks so bad to me, consider a couple of points first.

In What Sense Is Covid-19 Under Control?

First of all, the widespread conviction that the epidemic is winding down in the US is itself a mystery.

Continue reading “Am I Missing Something?” →

This sobering article in The Economist last year outlined the consequences to expect from a Brexit-without-a-deal. Most of it still applies, but to me, a non-economist, the diversity and magnitude of malign consequences suggests that Brexit could be a more interesting experiment than anyone thinks.

It’s not that we’ve lacked for economic turmoil since the age of inter-networking for business and the general public took off in the late 1980’s, but the problems have been fairly conventional in economic terms. Recessions, bubbles, the CMO meltdown, and so on; none of it has been greatly different from the trouble we’d gotten into for many decades previous.

Brexit brings up the possibility of a truly modern meltdown—an economic calamity that as yet has no name.

Continue reading “We Ain’t Seen Nothing Yet” →

Atheists invariably haul out the religious wars of Europe to make the case that religion is pernicious. It is indisputable that from The Age of Faith to the Reformation/Counter Reformation numerous bloody wars and slaughters were committed in the name if the Christian God and there were countless more if you include the wars fought in His second best known name, Allah. It’s not an obviously wacky point.

But hold on there—religious war is monstrous just as all war is monstrous, and it is possible that religion is monstrous as well, but the proposition that wars being fought over religion proves that religion is monstrous is a classic example of what philosophers call “an association fallacy of the red herring type.” The herring, i.e., the thing dragged in that is not logically connected is religion. Helen’s face launched a thousand ships only in a poetic sense; Paris and Agamemnon are to blame for the Trojan war, not Helen, love, or beauty.

Such a fundamentally flawed attack shouldn’t require any defense, but logic isn’t a central concern in religion; Christians consistently fall for this argument and end up defending Christianity from the accusation with words to the effect that “sure, there were wars, but those people weren’t real Christians” or “they weren’t acting consistently with Christian principles.“

This is a terrible argument but not because it’s inherently fallacious. It’s weak because it invites the accuser to apply what is known as the duck test, AKA Occam’s Razor. You’d sneer if I defended, say, Nazism, using the same logic. Try it out: “Nazism didn’t really underly the horrors of the Holocaust; the problem was that bad people coopted a good idea. Let’s let bygones be bygones and give it another chance.” No, the killers were Nazis and the actions were consistent with the principles of Nazism as advocated by the founders, so you can go out on a limb and say Nazism caused the Holocaust. It is logically possible that a small animal that looks like a duck, swims like a duck, quacks like a duck, and is often seen in the company of ducks, is not a duck, but that’s not where a wise ornithologist will take the argument.

Continue reading “Christianity and Ducks” →

Against a cultural background of morbid touchiness about references to gender, race, etc., the word bitch is a lens into what’s going on beneath the fig leaf of political correctness.

Calling a particular person a bitch in private is bad manners but if you disregard all the ostentatious political posturing, the visceral reaction most people have to bitch used in that way isn’t much different from their reaction to gender-neutral epithets such asshole, or to male-specific insults such as prick, or dick. Bitch, used in this way, is an insult to a particular person, and after all, offending is the goal. Bitch is not nearly as shocking to the ears as the c-word-that-dare-not-be-spoken (here in the US, anyway—in the UK cunt is practically a term of endearment when applied to a man.)

Gender-specific insults are tricky but you have to be willfully obtuse to deny that much obnoxious behavior has gender. I’m not talking about the gender of the obnoxious person, but the behavior itself. Being a prick is definitely a yang personality trait and being a bitch is yin, regardless of the sex of the insulted or insulting party. Asshole, on the other hand, is neuter, despite the fact that many more men than women are assholes. Assholes are like happy families but the epithet pig can connote at least three distinct kinds of obnoxiousness, one specific to each gender plus one neuter. I like the gender-specificity of all of these words and usages and doubly applaud their use when the target is not gender-consistent with the epithet. The comprehensibility of cross-gender insults is a sign of progress in the relationship of the sexes.

Continue reading “Bitchin’” →

I had an epiphany about what’s up with people who claim to “believe” things that are manifestly not so. I’m not talking about things people believe that are arguably wrong about or about matters of faith like belief in God or in karma. After all, most of us are wrong about most things most of the time.

The things that I’m talking about are things that you’d think it would be manifestly impossible to believe in good faith. Like saying you “believe” that the Earth is flat, or that two flatly contradictory lines of Scripture are both literally true.  Plausibility is subjective but how does one argue with a person who denies the rules of logic?

Continue reading “Don’t Be So Cynical!” →