Comparative performance of WANTAI ELISA for total immunoglobulin to receptor binding protein and an ELISA for IgG to spike protein in detecting SARS-CoV-2 antibodies in Kenyan populations

Highlights • Both assays discriminate SARS-CoV-2 seropositive and seronegative individuals.• Both assays correctly estimate seroprevalence in a wide range of seroprevalences.• WANTAI has slightly higher sensitivity than KWTRP.• KWTRP has slightly higher specificity than WANTAI.


Introduction
In December 2019, a severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2) emerged in Wuhan and rapidly spread around the globe causing a pandemic. SARS-CoV-2 is an enveloped positive sense RNA virus with the genome encoding structural and non-structural proteins in the 3 ′ and 5 ′ regions respectively [1]. The spike (S) and the nucleoprotein (N) structural proteins are targets of neutralizing antibodies and immunodominant proteins respectively making them attractive for designing serological diagnostic kits [1].
Although nucleic acid amplification tests (NAAT) remain reference standard for SARS-CoV-2 diagnosis, serological assays have become very useful tools for estimating viral transmission and seroprevalence especially in regions where there is limited NAAT coverage, like Kenya. Sero-surveys are also useful for quantifying cumulative incidence of infection and viral transmission in regions with significant asymptomatic infection. We developed an in-house (KWTRP) ELISA based on the whole trimeric spike protein and validated it extensively on local samples and international standards. The validation process included participation in a WHO-sponsored multi-laboratory study of SARS-CoV-2 antibody assays [2]. The assay results were consistent with majority of the test laboratories [2]. We have since used the KWTRP ELISA to estimate SARS-CoV-2 seroprevalence in several target groups [3][4][5].
In this study we aimed to do a performance comparison between KWTRP ELISA measuring SARS-CoV-2 IgG and WANTAI ELISA measuring total immunoglobulins (Table 1) using the same set of samples. WANTAI is a commercial microplate ELISA based on the receptor binding domain of the SARS-CoV-2 spike protein and widely used around the world [6] and recommended by WHO, which has supplied the kits to us and other countries enrolled in UNITY studies [7].

Materials and methods
Detailed characteristics of both assays are presented in Table 1.

WANTAI ELISA
The WANTAI ELISA was performed using the manufacturer's protocol. Briefly, 50 µl of positive and negative calibrator and 100 µl of serum or plasma samples were added to the plates and incubated at 37 • C for 30 min. After washing 5 times with diluted wash buffer, 100μl of HRP-Conjugate was added into each well except the Blank and incubated at 37 • C for 30 min. After further 5 washes, 50μl of Chromogen Solution A and 50μl of Chromogen Solution B were added and the plate incubated at 37 • C for 15 min avoiding light, then 50μl of Stop Solution added and the absorbance read at 450 nm. The ratio of absorbance (A) to cut-off (C.O.) (A/C.O. <1) was interpreted as having no SARS-CoV-2 antibodies (negative) and ≥1 having SARS-CoV-2 antibodies (positive).

KWTRP ELISA
The procedure has been reported extensively before [3]. Briefly, Nunc MaxiSorp™ flat-bottom 96-well plates (ThermoFisherScientific) were coated with 2 µg/ml of whole trimeric spike protein at 37 • C for 1 h, washed 3 times in wash buffer (0.1% Tween 20 in 1X phosphate buffered saline) and blocked with Blocker™ Casein (ThermoFisher-Scientific) for 1 h at room temperature. Then heat-inactivated serum or plasma samples were diluted 1:800 in Blocker™ Casein, added to the plates and incubated for 2 h at room temperature. After 3 further washes, 100 µl horseradish peroxidase-conjugated goat antihuman IgG antibody (Catalogue number 074-1002, KPL-SeraCare), diluted 1:10, 000 in wash buffer, was added to the plates and incubated for 1 h at room temperature. After 3 washes the plates were developed with o-phenylenediamine dihydrochloride (OPD) substrate (Sigma) for 10 min and read on an Infinite® 200 PRO microplate reader (TECAN) at 492 nm. The results were expressed as the ratio of test OD to the OD of the plate negative control; samples with OD ratios > 2 were considered positive for SARS-CoV-2 IgG and those with OD ratio ≤ 2, negative.
To confirm the reproducibility and specificity of the KWTRP ELISA before performing the comparison, we re-tested 467 pre-pandemic samples in the validation set reported in 2020 using the reported procedure [3].

Sample sets and ethical considerations
The characteristics of the test sample populations are summarized in Table 2. The gold-standard negative 'pre-pandemic' serum/plasma panels comprised: (1) sera from 327 adult blood donors collected in 2018 as part of research into the quality of transfused blood in coastal Kenya and (2) 189 sera samples from annual cross-sectional surveys for malaria surveillance in coastal Kenya collected between April-May 2018. The gold-standard positive 'pandemic' plasma panel comprised plasma from 149 COVID-19 patients sampled ≥7 days after their PCRpositive diagnosis. In addition, we tested a pandemic test panel consisting of serum/plasma from COVID-19 PCR testing (n = 676) and samples from early in the pandemic (COVID-19 wave 1, n = 176) when SARS-CoV-2 antibody seroprevalence was expected to be low and later when higher SARS-CoV-2 antibody seroprevalence was expected (COVID-19 wave 4, n = 176).
Commercial negative and positive control calibrators were supplied by the manufacturer for WANTAI. For the KWTRP ELISA the positive control was monoclonal antibody (CR3022) and a pool of pre-pandemic sera from 50 Kenyan adults was used as negative control.
This study was approved by the Scientific and Ethics Review Unit (SERU) of the Kenya Medical Research Institute (Protocol SSC 3426). Before the blood draw, donors gave individual consent the use of their samples for research. Ethical approval was obtained for collection, storage and further use for the sample sets used in the validation assays (SERU numbers: 1433, 3149, 3426).

Statistical analyses
Data analysis was conducted using GraphPad Prism v9, R v4.1.0 and Stata 15.0. Percentage agreement, specificity, sensitivity and prevalence were calculated for both assays using the same sample sets. Pairwise comparisons between the WANTAI and KWTRP were done using the McNemar's test. To explore whether differences between the two assays are intrinsic to the assay or simply a result of the selected cut-off for each assay, we plotted ROC curves for both assays using the positive and negative gold standard panels as test samples and calculated their area under the curves. Finally, we assessed the assays' reproducibility by examining the raw ODs and coefficient of variation (CV) for the negative and positive controls for all the test runs.

Results
In the intra-assay comparison, the KWTRP ELISA was highly reproducible with specificity of 0.99 (95% CI, 0.98-1.00) in both 2020 and 2021 ELISA. Among the 467 true negative samples both assays classified 3 (0.64%) samples as false positives and the 2021 ELISA classified an additional 1 (0.21%) as false positive (Fig. 1). There was positive correlation between the 2020 OD ratios and those of 2021 ELISA (Fig. 1). Because of minimal changes in the assay performance, WANTAI results were compared to KWTRP results already obtained in 2020.
In the inter-assay comparison, both assays showed very high specificity and sensitivity (Table 3). WANTAI specificity was slightly lower than KWTRP specificity using both negative sample sets, the 2018 prepandemic coastal Kenya blood donors and the 2018 malaria crosssectional survey samples. By contrast, WANTAI showed a slightly higher sensitivity than KWTRP using the 149-gold standard positive samples. Among the three sets of unknown samples, COVID-19 diagnostic testing patients, blood donors during COVID-19 wave 1 and blood donors during COVID-19 wave 4, the prevalence was 0.179, 0.023 and 0.830, respectively, for WANTAI and 0.152, 0.011 and 0.773, respectively, for KWTRP ( Table 3).
Results of a pairwise comparison showed there was very high overall agreement between the assays (Table 4). However, WANTAI resulted in more false positives than KWTRP in the pre-pandemic samples, the 2018 coastal Kenya blood donors and 2018 malaria cross-sectional survey samples. Both assays classified 4.7% of the positive samples as false negatives. Generally, WANTAI resulted in more positive results than KWTRP in the pandemic test panel reflecting its slightly higher sensitivity and slightly lower specificity compared to KWTRP. In a comparison of ROC curves for the WANTAI and KWTRP assays using all the gold standard positive and negative samples, the WANTAI curve was marginally closer to the optimal point at the top left of the plot with AUC of 0.9841 slightly higher than the KWTRP AUC of 0.9726.
Performance of both ELISAs, examined by coefficient of variation of negative and positive controls, was as expected with little inter-assay variation (Fig. 3). The WANTAI and KWTRP positive control CV were 15.7% and 2.64% while negative control CV were 23.3% and 8.75% respectively (Fig. 3).

Discussion
We compared the two ELISAs using the same sample sets. Overall, at the standard cut-offs, the KWTRP assay showed slightly higher specificity than the WANTAI assay which was in turn slightly more sensitive.    This was evident in the higher number of positive samples detected by WANTAI in all the sample sets including the positive gold standard panel and false positives in the pre-pandemic sample set. The false positives in the pre-pandemic sample set implies that at low prevalence, the WAN-TAI assay is likely to overestimate population seroprevalence, transmission and cumulative infection compared to the KWTRP assay and therefore less accurate. By contrast, at high prevalence, WANTAI assay is likely to detect more true positives than KWTRP and therefore more accurate. This trade-off between sensitivity and specificity illustrates that the differences in performance between the assays reported in Table 3 are partly attributable to differences in threshold selection along the ROC curves (Fig. 2). Nonetheless, the additional positives observed with WANTAI could also be contributed by other immunoglobulin classes such as IgA and IgM which are detected by WANTAI but not KWTRP. There were some samples in the pandemic test panel that were positive by KWTRP but negative by WANTAI, which was unexpected because of the slightly higher sensitivity by WANTAI. However, since WANTAI is based on the receptor binding domain while KWTRP on the whole spike protein, the latter contains additional epitopes that could drive responses not observed by the former [12]. The sensitivity of WANTAI observed in this study was comparable to its reported sensitivity, 95.3% compared to 94.4%, while specificity was slightly lower but still comparable in the two sample sets 97.9% and 98.4% compared to the reported 100% [8]. Comparable high sensitivities and specificities have been reported elsewhere for the WANTAI assay [6,8,13]. Although the sample numbers and sets (for WANTAI) were different from the ones used to define the reported sensitivities and specificities, finding comparable values inspire confidence in both assays, as they also showed high degrees of reproducibility (CV = <30%) illustrated by the raw OD values of their positive and negative controls.
Overall, both assays showed excellent characteristics to discriminate SARS-CoV-2 seropositive and seronegative individuals and estimate seroprevalence accurately across a wide range of seroprevalences.