With the "Unstatistic of the Month," we—along with the newsletter's author, Berlin psychologist Gerd Gigerenzer, Dortmund statistician Walter Krämer, and RWI Vice President Thomas K. Bauer—question both the recently published figures and their interpretations every month. All "Unstatistics" can be found on the internet at www.unstatistik.de and under the Twitter account @unstatistik. Today's newsletter builds on the Unstatistic from December 2020.
Some issues of the Unstatistic in recent months have differed from the familiar character. Instead of just pointing out where statistics have been misused, we have regularly made suggestions on how to do better. The December Unstatistic, like much else in the Corona year 2020, is characterized by ambivalence and points to a formally successful example of statistical communication that, at the same time, conveys a questionable substantive message.
RKI Infographic Compares Rapid Test Variants Clearly
On December 10, the Robert Koch Institute (RKI) published an infographic to help understand the results of antigen rapid tests for SARS-CoV-2. It compares two different scenarios: targeted testing of symptomatic individuals and mass testing—i.e., random testing of individuals who neither have symptoms nor contacts with infected persons.
The results are presented in the form of trees with natural frequencies: Based on 10,000 individuals, among whom 1,000 (in targeted tests) or 5 (in mass tests) are actually infected, it illustrates how many people are falsely identified as infected or not infected by the antigen tests. In percentages, these values represent prevalences of 10% and 0.5%, respectively.
Source: RKI Infographic
Sensitivity indicates the percentage of actually infected individuals a test correctly identifies. Specificity indicates the probability of correctly diagnosing a truly healthy person as such.
By simply counting and relating the respective individuals in the four groups (correctly positive or negative; falsely positive or negative), readers can calculate two important probabilities using basic arithmetic:
- What is my risk of being infected even though the test is negative?
- What is my risk of being infected if the test is positive?
Mass Rapid Tests Yield Significantly More False Positives
The scenarios presented make it immediately clear that while mass rapid tests can reliably detect infected individuals, the risk of being infected despite a negative test is only 0.01%. With targeted testing, it increases somewhat—to 2.2%. This small absolute increase in the risk of misdiagnosing infected individuals by 2.19 percentage points brings a significant reduction in the risk of misdiagnosing non-infected individuals. In mass testing, the risk that a positive COVID rapid test is false, meaning no infection is actually present, is a whole 98%. Almost all, namely 98 out of 100 individuals diagnosed as infected by a mass rapid test, are healthy! Targeted testing significantly reduces this risk. It drops in this case by 79.6 percentage points to 18.4%. At least four out of five positively tested individuals are actually infected in this scenario.
The Unspoken Message of the Scenario Comparison
However, the unspoken message of the scenario comparison is hard to miss: one should only get tested if there is a concrete suspicion or typical symptoms appear. This is remarkable because the Federal Ministry of Health recommended increasing the use of rapid tests in schools, daycare centers, and nursing homes the day after the infographic was released.
But is the situation really that clear?
An incidence of only five actual infections among 10,000 tested individuals is hardly likely given the current incidence rates. Assuming a dark figure of about factor five, which RKI director Lothar Wieler communicated in November, a prevalence of 1%, or 100 infected individuals among 10,000 mass tested, is much more realistic.
This changed assumption hardly affects the risk of being infected when the test is negative. In this scenario, it lies at 0.2%, an absolute 2.0 percentage points below the risk of a negative misdiagnosis of infected individuals. Conversely, however, the impacts are significant. The risk that a positive COVID rapid test is false is now only 71.2%. Among 100 diagnosed as infected, then, 29 are actually infected, 71 are not.
Relative Risks are Often Misunderstood
In previous Unstatistics, we have regularly pointed out how misleading the presentation of relative frequencies or relative risks can be, most recently in our November Unstatistic on the efficacy of the Corona vaccine. Even many medical professionals cannot correctly interpret test results when they receive information about sensitivity, specificity, and prevalence in the form of relative frequencies or probabilities, as this study shows.
This is because calculating with conditional probabilities is very abstract. Natural frequencies not only make it easier for doctors to derive the correct probabilities of an illness from test results. They help everyone who faces the question of whether to get tested as a precaution without specific suspicions to make a rational decision