The COVID-19 pandemic has demonstrated the importance of research regarding the accuracy of diagnostic tests, and the importance of understanding, applying, and teaching the related subject matter. An often-recommended mnemonic for interpreting biostatistics associated with diagnostic modalities is SpPIn-SnNOut, which stands for Specific/Positive rule In (SpPIn) and Sensitive/Negative rule Out (SnNOut). In other words, a positive result from a test with high specificity (Sp) can be trusted. A negative result from a test with high sensitivity (Sn) can be trusted. The mnemonic is based on the Sn and Sp characteristics of any diagnostic test when compared with a gold standard test or other data source that is known to be accurate. The following contingency table and formulas represent how such tests are performed.
|
Gold Standard Positive |
Gold Standard Negative |
Totals |
|
|
New Test Positive |
True Positives |
False Positives |
New Test Positives |
|
New Test Negative |
False Negatives |
True Negatives |
New Test Negatives |
|
Totals |
Gold Standard Positives |
Gold Standard Negatives |
Total Subjects |
The logic behind SpPIn-SnNOut is that, mathematically, a test with high Sp will have a low number of false positives and a test with high Sn will have a low number of false negatives. To demonstrate, let us consider a test with 99% Sn and 99% Sp. In this scenario, we imagine a new anti-body test compared with a gold standard test for COVID-19 anti-bodies in a study that includes 100,000 samples from adults all over the United States, representative of the country’s demographic profile.
Based on currently reported cases and the most recent U.S. population data, the prevalence of COVID-19 is 0.541%. In a study including 100,000 subjects, the following contingency table is developed, showing 99% Sn and 99% Sp of the new antibody test.
|
Known Positive |
Known Negative |
Total |
|
|
Anti-body Test Positive |
536 |
995 |
1,530 |
|
Anti-body Test Negative |
5 |
98,464 |
98,470 |
|
Total |
541 |
99,459 |
100,000 |
By the SpPIn rule, a positive result is associated with a low false-positive rate. In this example, 995 individuals receive a false positive test out of 99,459 individuals who have not had the infection according to the gold standard test result, for a false positive rate of 1.0%. However, there are two other statistics that are important to consider, positive and negative predictive values (PPV and NPV). Without taking these statistics into account we could miss important information.
The calculations for PPV and NPV produce 35.0% and 99.99% respectively. This indicates that there is only a 35% probability that those tested with the positive anti-body test results are truly anti-body positive.3 The reason for the strong discrepancy between the PPV and the SpPIn rule is the low prevalence of the condition. As such, only the SnNOut half of the mnemonic is reliable. A negative antibody test result is highly likely to be accurate due to having the dual characteristics of high NPV (99.9%) and high Sn (99.0%).
The Centers for Disease Control and Prevention (CDC) raised this issue in their most recent guidance on the subject of anti-body testing. The CDC stated that tests with high Sp are valuable for conditions in populations with prevalence at or above 5%. If the present data are accurate, prevalence is about one-tenth that amount in the United States today, though there are important regional differences and clusters. If COVID-19 reached 5% prevalence, the anti-body test would need to have Sp of 99.9% to reach at least 98% PPV. However, hopefully, there will not be such an increase in prevalence. Currently, a stand-alone testing procedure with an anti-body test that has 99.99% Sp and the present level of prevalence will attain 98% NPV. Based on this situation, the CDC recommends additional testing strategies to overcome the low NPV that is likely to be associated with any anti-body test, unless tests achieve 99.99% Sp.
The takeaway from these observations is that it is essential those who teach health professional students about interpreting diagnostic test results, also teach them about the limitations of SpPIn-SnNOut. If students who complete our courses remember only the SpPIn-SnNOut mnemonic, we might be failing their patients. Patients who engage in behaviors that increase their exposure due to a false sense of security from a positive antibody test result (given by a test that has low PPV) can suffer serious consequences. The better our students understand these statistics, the more likely they will provide helpful information to their patients to avoid this hazard. This is a moment in our history when what we teach can significantly impact the well-being of the population.