Real-World Performance of COVID-19 Rapid Antigen Tests

COVID-19 Rapid Antigen Test

Source: https://commons.wikimedia.org/wiki/File:Covid19_Antigen_Rapid_Test_Diagnostic_(50688135107).jpg

SARS-CoV-2 rapid antigen-based tests have been touted as having the potential to significantly reduce the spread of COVID-19, or even end the pandemic, if the tests are used regularly by a high percentage of the population. Modeling studies have predicted that frequent utilization of rapid antigen-based tests could significantly reduce the spread of COVID-19, but this has not been demonstrated in experimental or observational studies. 

Relative Sensitivities & Specimen Type Matter

When used at home by the general public, rapid-COVID-19 antigen tests perform at the very bottom of the COVID-19 testing cascade. Similar to a cascading waterfall, there are 

Nasal swabs are less uncomfortable for patients than nasopharyngeal (NP) collection, so this has been almost exclusively chosen as the specimen type for rapid antigen-based tests and represents another significant drop in the cascade, demonstrating a 

Studies of rapid antigen-based testing on nasal swabs compared to nasal swab PCR may claim >85% relative sensitivity, but in reality, the antigen test will not detect even close to 85% of true cases. Here, the fact that gold standard PCR on NP swabs does not achieve 100% clinical sensitivity and the expected reduction in sensitivity due to compromising on nasal sampling instead of NP are effectively hidden.

If these drops in sensitivity were observed all at once, either due to technology, specimen type or performance environment, the problem would be more obvious and difficult to accept, but the reductions have been incremental and multi-factorial. Thus, many remain unaware of the true impact and misunderstand how this relative sensitivity will play out in real life.

Population Prevalence Impacts Testing Accuracy

Similarly, the term accuracy may be misunderstood when discussing COVID-19 tests. “Accuracy” is a measurement of the percentage of tests that are correct (whether positive or negative), but a test may have high accuracy in some settings, even as high as 99%, and still not be a good test. For example, if testing in a population with low prevalence for COVID-19, let’s say 1% truly positive, a test that simply returns exclusively negative results would be 99% accurate, and 100% useless—unless, of course, you really want to get into a venue or on a plane that requires a negative test.

Real-World Performance Differs from Modeling Studies

Many countries have spent considerable money subsidizing, or providing for free, rapid-antigen based SARS-CoV-2 tests to try to help limit the spread of the virus. If this strategy is a game-changer, these countries should have fewer COVID-19 cases, hospitalizations and deaths—reported cases may be partially inflated by increased community testing, but hospitalizations and deaths should be reduced.

Two countries that have been lauded in media reports for having widely available and free or low-cost testing available to the general public are the U.K. and Germany. In the U.K., which 

The U.S. seems to have fairly limited utilization of rapid antigen tests, relative to these 2 countries, although the 

Japan, until very recently, appears to have not been using rapid antigen tests to any significant degree in community settings. Until September 2021 pharmacies in Japan were prohibited from selling “medical-grade” antigen-based COVID-19 test kits, and now that they are allowed to do so, testing instructions must be delivered by the pharmacist, and the purchaser must sign a document that a negative result does not release them from taking other preventative measures.

Summary

Although there isn’t a great way to separate the impact of rapid antigen-based testing on the pandemic landscape of these 4 countries, given the other interventions and differences between them, on the surface, it doesn’t seem that rapid antigen-based COVID-19 tests have made a significant impact on community spread in the European countries where they have been most heavily utilized (see the table below for per capita COVID-19 case and death numbers since the beginning of the pandemic with information from the 
Per capita COVID-19 case and death numbers (accessed Dec. 2, 2021).

Source: https://www.nytimes.com/interactive/2021/world/covid-cases.html

Rapid antigen testing for COVID-19 among symptomatic patients, when PCR or other molecular based testing is not readily available, can certainly be useful, and has been recommended for this purpose by various public health entities, including the World Health Organization (WHO). However, in their October 2021 update to recommendations on the use of antigen-based COVID-19 tests, the WHO stated that, “Given the significant costs involved, the lack of evidence on impact and cost-effectiveness of such approaches and the concern that this cost-intensive approach risks diverting resources from higher priority testing indications, mass community-based testing of asymptomatic individuals is not currently recommended."

It is also important to consider the cost and feasibility of large-scale, at-home implementation, as well as the many confirmatory tests that would be required for the majority of positive results at the low population prevalence that has prevailed for most of the pandemic. In the absence of evidence that mass-testing of asymptomatic people for COVID-19 has been beneficial, we cannot know whether the resources poured into such measures will return the public health value being pursued.

SARS-CoV-2 rapid antigen-based tests have been touted as having the potential to significantly reduce the spread of COVID-19, or even end the pandemic, if the tests are used regularly by a high percentage of the population. Modeling studies have predicted that frequent utilization of rapid antigen-based tests could significantly reduce the spread of COVID-19, but this has not been demonstrated in experimental or observational studies. Concerns about the performance characteristics of rapid antigen-based SARS-CoV-2 tests include low sensitivity, which is sometimes accepted, since patients who are the most infectious are more likely to test positive, and frequent testing can partially compensate for low sensitivity. Additional concerns beyond performance characteristics include questions about how people use (or refuse to use) them and whether or not results are reliably reported to public health departments.When used at home by the general public, rapid-COVID-19 antigen tests perform at the very bottom of the COVID-19 testing cascade. Similar to a cascading waterfall, there are several drops in sensitivity along the way from true cases (100% “clinical sensitivity”) to the real-world performance of rapid antigen-based SARS-CoV-2 tests , which demonstrate about 50% or lower clinical sensitivity. Even the most sensitive testing method (PCR) on the best specimen type (nasopharyngeal swabs) does not achieve 100% clinical sensitivity, but is closer to 80% sensitive , even during the first week of symptoms. PCR testing, therefore, represents the first drop in the testing cascade. Comparison of rapid testing and PCR performance in real-world settings, as opposed to small, controlled studies, predicts that rapid-antigen based SARS-CoV-2 tests are around 50% as sensitive as PCR Nasal swabs are less uncomfortable for patients than nasopharyngeal (NP) collection, so this has been almost exclusively chosen as the specimen type for rapid antigen-based tests and represents another significant drop in the cascade, demonstrating a 15% reduction in sensitivity , just for convenience.Studies of rapid antigen-based testing on nasal swabs compared to nasal swab PCR may claim >85% relative sensitivity, but in reality, the antigen test will not detect even close to 85% of true cases. Here, the fact that gold standard PCR on NP swabs does not achieve 100% clinical sensitivity and the expected reduction in sensitivity due to compromising on nasal sampling instead of NP are effectively hidden.If these drops in sensitivity were observed all at once, either due to technology, specimen type or performance environment, the problem would be more obvious and difficult to accept, but the reductions have been incremental and multi-factorial. Thus, many remain unaware of the true impact and misunderstand how this relative sensitivity will play out in real life.Similarly, the term accuracy may be misunderstood when discussing COVID-19 tests. “Accuracy” is a measurement of the percentage of tests that are correct (whether positive or negative), but a test may have high accuracy in some settings, even as high as 99%, and still not be a good test. For example, if testing in a population with low prevalence for COVID-19, let’s say 1% truly positive, a test that simply returns exclusively negative results would be 99% accurate, and 100% useless—unless, of course, you really want to get into a venue or on a plane that requires a negative test.Many countries have spent considerable money subsidizing, or providing for free, rapid-antigen based SARS-CoV-2 tests to try to help limit the spread of the virus. If this strategy is a game-changer, these countries should have fewer COVID-19 cases, hospitalizations and deaths—reported cases may be partially inflated by increased community testing, but hospitalizations and deaths should be reduced.Two countries that have been lauded in media reports for having widely available and free or low-cost testing available to the general public are the U.K. and Germany. In the U.K., which planned to spend up to $130 billion U.S. dollars on their rapid COVID-19 testing program, rapid tests are readily available at testing centers and citizens may request up to 7 free rapid antigen tests per day for home delivery. In Germany rapid antigen tests have been widely available and free or heavily subsidized. For example, tests can be found for less than $1.00 at many grocery stores. It is hard to get good data for the number of “at-home” antigen tests being utilized, since these records are not generally included in reported test results, unless the results are positive and lead to official confirmatory testing (typically via PCR). However, neither the U.K. nor Germany are doing particularly well limiting community spread this Fall (upswings in cases predated the rise of Omicron), making it difficult to see where the tens to hundreds of billions of dollars spent on rapid antigen tests have delivered significant value.The U.S. seems to have fairly limited utilization of rapid antigen tests, relative to these 2 countries, although the U.S. government has earmarked up to $15 billion for them in various legislation packages . Still, the amount spent on fixed numbers of rapid antigen tests in the U.S doesn’t translate directly to the number of tests that have been used. For instance, Missouri is out 1 million government funded tests, acquired for school testing programs, but only about 3% have reportedly been used.Japan, until very recently, appears to have not been using rapid antigen tests to any significant degree in community settings. Until September 2021 pharmacies in Japan were prohibited from selling “medical-grade” antigen-based COVID-19 test kits, and now that they are allowed to do so, testing instructions must be delivered by the pharmacist, and the purchaser must sign a document that a negative result does not release them from taking other preventative measures.Although there isn’t a great way to separate the impact of rapid antigen-based testing on the pandemic landscape of these 4 countries, given the other interventions and differences between them, on the surface, it doesn’t seem that rapid antigen-based COVID-19 tests have made a significant impact on community spread in the European countries where they have been most heavily utilized (see the table below for per capita COVID-19 case and death numbers since the beginning of the pandemic with information from the NY Times accessed Dec. 2, 2021). Conversely, Japan has the highest vaccination rate of the 4, and dramatically lower COVID-19 numbers, but it does not appear that rapid antigen tests have had any substantive role in the country’s response to the pandemic.Rapid antigen testing for COVID-19 among symptomatic patients, when PCR or other molecular based testing is not readily available, can certainly be useful, and has been recommended for this purpose by various public health entities, including the World Health Organization (WHO). However, in their October 2021 update to recommendations on the use of antigen-based COVID-19 tests, the WHO stated that, “Given the significant costs involved, the lack of evidence on impact and cost-effectiveness of such approaches and the concern that this cost-intensive approach risks diverting resources from higher priority testing indications, mass community-based testing of asymptomatic individuals is not currently recommended."It is also important to consider the cost and feasibility of large-scale, at-home implementation, as well as the many confirmatory tests that would be required for the majority of positive results at the low population prevalence that has prevailed for most of the pandemic. In the absence of evidence that mass-testing of asymptomatic people for COVID-19 has been beneficial, we cannot know whether the resources poured into such measures will return the public health value being pursued.