Diagnostic Accuracy of Rapid Antigen Test Kits for Detecting SARS-CoV-2: A Systematic Review and Meta-Analysis of 17,171 Suspected COVID-19 Patients

Early diagnosis is still as crucial as the initial stage of the COVID-19 pandemic. As RT-PCR sometimes is not feasible in developing nations or rural areas, health professionals may use a rapid antigen test (RAT) to lessen the load of diagnosis. However, the efficacy of RAT is yet to be investigated thoroughly. Hence, we tried to evaluate the overall performance of RAT in SARS-CoV-2 diagnosis. Based on our PROSPERO registered protocol (CRD42021231432), we searched online databases (i.e., PubMed, Google Scholar, Scopus, and Web of Science) and analysed overall pooled specificity and sensitivity of RAT along with study quality, publication bias, heterogeneity and more. The overall pooled specificity and sensitivity of RAT were detected as 99.4% (95% CI: 99.1–99.8; I2 = 90%) and 68.4% (95% CI: 60.8–75.9; I2 = 98%), respectively. In subgroup analyses, nasopharyngeal specimens and symptomatic patient’s samples were more sensitive in RAT, while cycle threshold (Ct) values were found to have an inverse relationship with sensitivity. In the European and American populations, RAT showed better performance. Although the sensitivity of RAT is yet to be improved, it could still be an alternative in places with poor laboratory set up. Nevertheless, the negative samples of RAT can be re-tested using RT-PCR to reduce false negative results.


Introduction
Severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2) is responsible for the coronavirus disease 2019 (COVID- 19), characterised mainly by fever, cough, sore throat, fatigue, joint and muscle pain and loss of smell and taste [1][2][3][4][5]. It was first identified in Hubei province in Wuhan, China, in December 2019 and until now, it continues to be a significant health issue worldwide. Because there is no specific therapy or medication, early and accurate diagnosis is critical in preventing the spread of the disease [6,7]. In vitro diagnostics have long been recognised as a valuable tool for outbreak control and patient care [8]. The current gold standard diagnostic test for laboratory diagnosis of SARS-CoV-2 is nucleic acid amplification tests (NAAT); reverse transcription polymerase chain reaction (RT-PCR) to be specific [9]. The advantage this has over other tests is its high sensitivity due to the nucleic acid amplification step while also having a high specificity due to the specific primers used. This is consistent with its purpose at referral centres, which is to advise clinical care of patients. However, since it can take several hours to obtain results from RT-PCR, it may be inappropriate to use during an emergency. It also requires costly technology and competent workers, both of which may not be accessible in remote health clinics, particularly in underdeveloped countries.
Rapid antigen test (RAT) kits have emerged as an important alternative tool to aid in the clinical diagnosis of COVID-19. RAT is based on immunochromatography, which employs antibodies spotted onto nitrocellulose membranes that interact with specific antigens from the patient sample. The antigen-antibody interaction can be visualised manually or by using an immunofluorescence machine reader. For the diagnosis of COVID-19, the target analyte is often the virus' nucleocapsid protein. A similar strategy has been used for the rapid diagnosis of HIV, malaria, influenza and other diseases [9]. It is a reasonably inexpensive and simple test with quick results that may be used for point-of-care testing [10].
The advantage of RAT is that the test is more accessible to patients. It also enables appropriate infection control measures to be instituted earlier which is important in a pandemic [10,11]. However, a negative RAT result cannot rule out COVID-19 infection [12]. Patients with typical clinical presentation or close contacts of COVID-19 cases should undergo repeat testing [12,13].
For diagnosis of SARS-CoV-2, the World Health Organisation has recommended that a RAT kit needs to meet a minimum performance requirement of at least 80% sensitivity and 97% specificity compared with a NAAT reference assay to be used [14]. Studies regarding the diagnostic accuracy of RAT have produced a wide range of sensitivity. The varied results may be due to the various study designs, manufacturer of the RAT kits, patient selection, types of specimens and the phase of illness at the point of sample collection. While research and development of RAT to detect SARS-CoV-2 continue, this systematic review and meta-analysis aims to provide an update on the diagnostic accuracy in terms of estimating the specificity and sensitivity of RAT kits in patients with suspected COVID-19.

Study Protocol and Guideline
Based on the current literature, this systematic review and meta-analysis on the diagnostic accuracy of the available RAT kits for COVID-19 diagnosis was undertaken according to the PRISMA guideline [15]. The protocol of this study was registered in the PROSPERO database (CRD42021231432).

Eligibility Criteria
As the objective was to investigate the pooled sensitivity and specificity of the available RAT kits detecting SARS-CoV-2, we included studies in which RAT kits were used to identify SARS-CoV-2 to confirm COVID-19. Original studies without restricting study design or language were included. Review articles, opinions, case reports, news, press releases, blogs and data from websites were not considered eligible.

Search Strategies
Based on the eligibility criteria, we searched online databases such as PubMed, Google Scholar, Scopus, and Web of Science to identify studies of our interest published between 1 January 2020 and 13 January 2021. The following keywords were used to search different databases: COVID-19, SARS-CoV-2, coronavirus, nCoV, antigen, detection, diagnostic, diagnosis, test, testing, assay, assays and combined with appropriate Boolean operators (Table S1). Additionally, the reference lists of the included studies were also reviewed to identify any potentially eligible studies. EndNote X8 software (Clarivate Analytics, Philadelphia, PA, USA) was used to identify and exclude duplicate studies.

Study Selection
Three authors (S.S.K., N.H.H.N.H. and R.H.S.) independently screened the original pool of papers, then assessed their eligibility to be included in this meta-analysis using title, abstract, and full-text evaluation. Disagreements about whether a study should be included or excluded were discussed with the other authors (M.A.I. and Z.Z.D.) and resolved with acceptable consensus.

Data Extraction
Three authors (S.S.K., N.H.H.N.H. and R.H.S.) independently undertook the data extraction, and two authors (M.A.I. and Z.Z.D.) validated it. Initially, the data of the total number of positive and negative specimens confirmed by a reference standard followed by the number of positive and negative results of those same specimens were evaluated through RAT kits and they were extracted from each of the eligible studies. The major characteristics of the studies including the Study ID (last name of the first author and year of publication), location, total number of subjects, percentage of female subjects, the mean or median age of the participants, type of participants, specimen types, the test method of RAT kit used, percentages of positive samples detected by the reference standard (RT-PCR), ranges of Ct values of the reference standard and the manufacturer of the RAT kits were documented.

Quality Assessment
Two authors (S.S.K. and M.A.I.) independently assessed the quality of the included studies following the diagnostic test accuracy quality assessment tool of the Joanna Briggs Institute (JBI). To resolve the discrepancies, all authors took part in the discussion and resolved with consensus. If the total score was ≤49, 50-69, or ≥70%, the studies were classed as low-quality (high-risk of bias), moderate-quality (moderate-risk of bias), or highquality (low risk of bias) [16]. To detect publication bias, funnel plots were constructed, and the Egger's test was performed.

Data Analyses
A random-effects model with 95% confidence intervals (CIs) was used to analyse the overall pooled sensitivity and specificity of the RAT kit to detect COVID-19. I 2 statistics were used to analyse heterogeneity among the included studies (I 2 > 75% indicating substantial heterogeneity), followed by Cochran's Q test to determine the significance of the heterogeneity (p < 0.05 was considered statistically significant). Furthermore, a Galbraith plot was generated to determine the outlier studies.

Subgroup and Sensitivity Analyses
Subgroup analyses were undertaken to investigate sensitivity and specificity based on symptomatic and asymptomatic patients, days of symptom onset, types of specimen, Ct values, countries, continents and manufacturers of RAT. To investigate the robustness of results and the possible source of heterogeneity, sensitivity analyses were performed through strategies such as excluding small studies (<100), excluding low-or moderatequality studies, using a fixed-effects model, and excluding outlier studies. The analyses and plots were constructed by using the metaprop codes in the meta (version 4.15-1) and metafor (version 2.4-0) packages of R (version 3.6.3) in RStudio (RStudio, Inc., Boston, MA, USA) (version 1.3.1093).

Study Selection
Primarily, based on the search strategies, a total of 1201 published articles were identified from the online databases. During the initial screening process, 872 articles including review articles (n = 31), case reports (n = 7), articles that included non-human subjects (n = 18), editorials, letters and comment (n = 65) and duplicate studies (n = 751) were excluded. From the remaining 329 studies, 300 studies did not comply with the objective of this meta-analysis (irrelevant to the objective of the meta-analysis, repetitive studies, protocols only or missing data of interest); hence regarded as ineligible. The remaining 29 studies fulfilled the eligibility criteria and were finally included in this meta-analysis ( Figure 1

Characteristics of the Included Studies
A total of 17,171 COVID-19 suspects, including RT-PCR-positive and negative participants who were further tested in RAT, were reported in this meta-analysis. Different types of specimen (i.e., nasopharyngeal swab, saliva, nasal swab, sputum, throat swab or endotracheal aspirates) were taken from suspected symptomatic or asymptomatic participants. The RT-PCR-positive participants had a wide range of Ct values. Although a total of 15 different RAT kits were assessed from 13 different manufacturers, the RAT test methods were based on either immunochromatographic (ICG) assay or fluorescence immunoassay (FIA). The studies were carried out in several countries encompassing five continents. Table 1 shows the detailed features of the studies we included.

Quality Assessment and Publication Bias
The quality of each of the included studies was extensively examined using the JBI diagnostic accuracy checklist where the highest and lowest quality scores of the included studies were recorded as 88.8% (five studies) and 55.5% (two studies), respectively. Overall, there were no low-quality studies, 26.6% of high-quality, and 72.4% of moderate-quality studies (Table S2). There was no indication of substantial publication bias in the funnel plots evaluating the specificity and sensitivity of RAT kits to confirm SARS-CoV-2 ( Figure 2). J. Clin. Med. 2021, 10, x FOR PEER REVIEW 9 of 18

Quality Assessment and Publication Bias
The quality of each of the included studies was extensively examined using the JBI diagnostic accuracy checklist where the highest and lowest quality scores of the included studies were recorded as 88.8% (five studies) and 55.5% (two studies), respectively. Overall, there were no low-quality studies, 26.6% of high-quality, and 72.4% of moderate-quality studies (Table S2). There was no indication of substantial publication bias in the funnel plots evaluating the specificity and sensitivity of RAT kits to confirm SARS-CoV-2 ( Figure  2).

Meta-Analysis
The overall pooled specificity and sensitivity of RAT were 99.4% (95% CI: 99.1-99.8; I 2 = 90%) and 68.4% (95% CI: 60.8-75.9; I 2 = 98%), respectively (Figure 3). Four outlier studies [20,33,38,44] were identified using the Galbraith plot ( Figure 4). Except for the pooled specificity of subgroups based on specimen types (i.e., nasopharyngeal swab: 71.0% (95% CI: 14.1-100.1) and saliva: 80.7% (95% CI: 41.8-100.0)), the specificity did not vary in most of the subgroups and ranged between 99 and 100% (Table 2 and Figure S1). On the other hand, in subgroup analyses, the sensitivity for each subgroup was lower than the specificity. Each of the subgroups estimating the sensitivity had high levels of heterogeneity except for the subgroups defined by the onset of symptoms (<5 and >5 days; I 2 = 0%) and Ct values (Ct values ≤20 and 36-40; I 2 = 0%). When compared with asymptomatic patients (54.5%), the sensitivity of RAT kits was higher in symptomatic patients (78.5%). The sensitivity of the nasopharyngeal swab was higher (70.1%) than that of saliva (50.4%) and throat swab or saliva (38.4%). The CT indicates the number of cycles needed for the fluorescence signals to reach the threshold. Ct value for a particular gene (i.e., N, ORF, E, S or M) is inversely correlated with viral load in a targeted specimen [45,46]. Surprisingly, the sensitivity of RAT kits was shown to have an inverse relationship with the Ct values, which also corresponded with the sensitivity of subgroups based on the symptom onset days ( Table 2 and Figure S1). The sensitivity of RAT kits dropped to 15

Meta-Analysis
The overall pooled specificity and sensitivity of RAT were 99.4% (95% CI: 99.1-99.8; I 2 = 90%) and 68.4% (95% CI: 60.8-75.9; I 2 = 98%), respectively (Figure 3). Four outlier studies [20,33,38,44] were identified using the Galbraith plot ( Figure 4). Except for the pooled specificity of subgroups based on specimen types (i.e., nasopharyngeal swab: 71.0% (95% CI: 14.1-100.1) and saliva: 80.7% (95% CI: 41.8-100.0)), the specificity did not vary in most of the subgroups and ranged between 99 and 100% (Table 2 and Figure S1). On the other hand, in subgroup analyses, the sensitivity for each subgroup was lower than the specificity. Each of the subgroups estimating the sensitivity had high levels of heterogeneity except for the subgroups defined by the onset of symptoms (<5 and >5 days; I 2 = 0%) and Ct values (Ct values ≤20 and 36-40; I 2 = 0%). When compared with asymptomatic patients (54.5%), the sensitivity of RAT kits was higher in symptomatic patients (78.5%). The sensitivity of the nasopharyngeal swab was higher (70.1%) than that of saliva (50.4%) and throat swab or saliva (38.4%). The CT indicates the number of cycles needed for the fluorescence signals to reach the threshold. Ct value for a particular gene (i.e., N, ORF, E, S or M) is inversely correlated with viral load in a targeted specimen [45,46]. Surprisingly, the sensitivity of RAT kits was shown to have an inverse relationship with the Ct values, which also corresponded with the sensitivity of subgroups based on the symptom onset days ( Table 2 and Figure S1). The sensitivity of RAT kits dropped to 15

Sensitivity Analyses
To assess the range in outcomes, sensitivity analyses were performed removing small studies (n < 100), low-or moderate-quality studies, using a fixed-effects model instead of a random-effects model, and omitting outlier studies. Surprisingly, the specificity of RAT kits for each of the sensitivity analysis results remained very constant when compared to the total pooled specificity (99.4%), ranging from 99.4% to 99.9% (Table 3 and Figure S2). Using a fixed-effects model, the overall pooled sensitivity was enhanced to 79.9% as compared with the overall pooled sensitivity observed using the random-effects model (68.4%). However, when small (n < 100) studies, low-and moderate-quality studies, and outlier studies were excluded, the sensitivity of RAT kits was found to be 62.0%, 69.8%, and 71.2%, respectively, which was close to the overall pooled sensitivity (68.4%) ( Table 3 and Figure S2).

Discussion
RT-PCR has been recognised as the diagnostic gold standard for COVID-19 diagnosis to date; however, it has several drawbacks, including false-negative and false-positive findings [47,48]. The rate of false-negative results was estimated to be about 67% in the first 4-5 days following the onset of symptoms. [48]. On the other hand, the reasons behind the false-positive results were contamination during specimen collection, contamination of reagents, PCR amplicons, cross-contamination of the sample, and cross-reaction with other genetic materials or viruses [47]. Even in a cohort study, chest CT was found to be more sensitive (88%) as compared with the diagnostic result of RT-PCR (59%) [49]. Additionally, RT-PCR is costly and requires well-equipped laboratories [47,48,50]. More importantly, the release of RT-PCR results in many instances may take longer than expected due to high sample volumes and a lack of technical support, leading to delayed management of patients and control of outbreaks. As a result, an emphasis on developing RAT kits was necessary to close the diagnostic gaps.
So far, several RAT kits have been developed by various manufacturers from multiple countries and evaluated independently by researchers. We observed that most of the RATs were developed based on either ICG assay or FIA. Although both the assays are generally based on the antibody-antigen interaction technique, for the detection, a signal generator (i.e., gold colloid) is used in ICG whereas a fluorescent molecule is bound to the detection antibody in FIA [51][52][53]. In this meta-analysis, we looked at the overall sensitivity and specificity of the available reported RAT kits. We found that although the overall pooled specificity (99.4%) was acceptable, the overall pooled sensitivity (68.4%) was not.
Except in a few subgroups where there was a lot of heterogeneity across the studies, overall, the specificity did not differ substantially in the subgroup analysis. The study results, on the other hand, differed considerably when it came to assessing the sensitivity. Interestingly, the sensitivity of RAT kits for symptomatic and asymptomatic patients was 78.5% and 54.5%, respectively, indicating that RAT kits can identify both kinds of patient, albeit at a lower detection rate in the latter patient group. However, early detection is critical for RAT, as the sensitivity maintained at 82.0% when the symptom onset came after less than 5 days but dropped to 75.1% after five days. In the case of RT-PCR, a similar association between days of symptom onset and correct diagnosis had been reported [48]. This correlation was further justified when we analysed the correlation between Ct values and sensitivity. Previously, an analogous observation was investigated where a negative correlation of Ct values of RT-PCR and sensitivity in just one antigen kit for a specimen was identified [54]. When we checked the overall scenario with the studies we included, we found a similar inverse correlation of Ct values and sensitivity. According to the Centers for Disease Control and Prevention (CDC), the higher Ct value (>33) can be considered as a non-contagious stage [55] which can justify the usage of RAT kits. However, early diagnosis using RAT kits can be suggested according to these observations. Again, among different specimen types, the nasopharyngeal swab was identified to be the best for SARS-CoV-2 identification using RAT kits. A recent study also suggested that the nasopharyngeal swab is more sensitive (89%) as compared with saliva (72%) to detect SARS-CoV-2 which confirms our findings [56].
In the current study, when the RAT sensitivity to detect SARS-CoV-2 in people from various continents and nations was examined, it was demonstrated that the sensitivity of RAT in the population of Europe and America was higher as compared to that of Asia and Africa. Because most RAT kits are manufactured in European countries, China, and Korea, the sensitivity may have diminished in Asia and Africa owing to issues during RAT kit transportation. As reported earlier, a repetitive freeze-thaw process during transportation can change the behaviour of proteins used in RAT kits [57]. This was further justified when a study by Gupta et al. [26] from India exhibited a much higher sensitivity (81.8% (95% CI: 73.2-90.4)) and the RAT kit used in this study was in fact manufactured in India. Hence, based on our findings, it can be suggested that local manufacturers or inventors should be prioritised for the RAT kit development and utilisation.
Further investigations are required to determine the correlation of different sample conditions such as transportation or storage with the sensitivity of RAT. Again, recent reports about different genetic and structural mutations of SARS-CoV-2 enhance the possibility to hamper the sensitivity of diagnostic tools [58,59]. Hence, researchers need to focus on the improvement of sensitivity of RAT without compromising the specificity to detect both the original and mutated variants of SARS-CoV-2. Results of this meta-analysis support the recommendations of the Infectious Diseases Society of America Guidelines on the Diagnosis of COVID-19 that the sensitivity of RAT kits is highly correlated with viral load, symptoms, and the timing of the test in relation to the onset of symptoms [60].
This meta-analysis also shows that the sensitivity of RAT kits differs depending on the manufacturer and the country in which the kits are produced.

Conclusions
To summarise, the use of RAT kits is largely recommended for the early detection of patients suspected of having COVID-19, particularly in densely populated and isolated locations with limited resources and laboratory equipment. It may also be useful for prompt diagnosis in central city or rural areas. However, the negative RAT samples may need to be further analysed using molecular tests to confirm the results, particularly when the symptoms of COVID-19 are present.