Comparison of the Biofire FilmArray RP, Genmark eSensor RVP, Luminex xTAG RVPv1, and Luminex xTAG RVP Fast Multiplex Assays for Detection of Respiratory Viruses

Respiratory viral infections remain a leading cause of medical visits and can contribute significantly to morbidity and mortality. Most respiratory viruses present with similar symptoms, making a diagnosis difficult without laboratory testing. Accurate and timely identification of respiratory viruses benefits the patient, particularly when the pathogen is one for which therapy exists, such as influenza viruses (1). Additional benefits of respiratory viral identification include epidemiologic tracking of local outbreaks or epidemics, applying appropriate infection control measures to admitted patients (e.g., droplet and/or contact precautions and considerations in creating cohorts), and decreasing the use of unnecessary antimicrobial therapy, when appropriate (2, 3).

Current diagnostic techniques for the detection and identification of respiratory viruses are somewhat limited. Although rapid antigen testing offers quick results, the sensitivity and specificity of rapid antigen testing vary greatly (4, 5). The sensitivities of two rapid influenza virus tests for the detection of influenza virus A/H1N1/2009 averaged 21% (6), and respiratory syncytial virus (RSV) rapid antigen sensitivity as low as 59% has been reported (7). Direct fluorescent antibody testing and viral culture have a greater degree of sensitivity and have the advantage of detecting several viruses simultaneously; however, both are time consuming and demand significant experience to perform the testing and correctly interpret the results. To close the gap between sensitivity and time to result, amplification-based technologies have been developed. Several nucleic acid amplification tests are available as in vitro diagnostics for respiratory virus detection. Some assays only detect a single viral type, such as adenovirus (AdV) (Hologic Gen-Probe, San Diego, CA), parainfluenza viruses (PIV) (Hologic Gen-Probe), metapneumovirus (MPV) (Hologic Gen-Probe and Quidel, San Diego, CA), and influenza viruses (Cepheid, Sunnyvale, CA; IQuum, Marlborough, MA; and Qiagen, Gaithersburg, MD). Other assays offer limited multiplex testing, such as for influenza viruses and RSV (Focus Diagnostics, Cypress, CA; Nanosphere, Northbrook, IL; and Hologic Gen-Probe) and for RSV and metapneumovirus (Quidel). Currently, four molecular multiplex respiratory virus panels for the simultaneous detection of more than three respiratory viruses have been cleared by the U.S. FDA, namely, the FilmArray RP (respiratory panel) (BioFire Diagnostics, Salt Lake City, UT), the eSensor RVP (respiratory viral panel) (GenMark Dx, Carlsbad, CA), the Luminex xTAG RVPv1, and the Luminex xTAG RVP fast (Luminex Molecular Diagnostics, Austin, TX). Table 1 summarizes the details of these four FDA-cleared assays.

Studies comparing these molecular multiplex platforms to in-house molecular methods have found overall sensitivities and specificities, respectively, of 89.4% and 99.6% for the FilmArray RP (8), 95.4% and 99.7% for the eSensor RVP (9), 91.2% and 99.7% for the xTAG RVPv1 (10), and 78.8% and 99.6% for the xTAG RVP fast (11). Although there are additional published data on the performance of each of these four respiratory platforms, it is difficult to compare assay performance among the multiplex panels since the comparative method varies from study to study (12–17). To our knowledge, there has not been a comprehensive analysis of how these multiplex respiratory viral panels compare to each other. This study aims to compare the four FDA-cleared molecular multiplex respiratory viral panels with regard to sensitivity, specificity, and workflow parameters, such as hands-on time, time to result, and relative ease of use.

Without doubt, the commercial availability of multiplex molecular respiratory viral panels represents a significant advancement in the laboratory diagnosis of respiratory viral illnesses. Nonetheless, each system has its advantages and disadvantages, and each user should determine which system is appropriate for their specific laboratory and/or patient population. Since not all laboratories are able to do a side-by-side study of the four FDA-cleared molecular respiratory viral panels, we have attempted to demonstrate the relative performance of each assay, including a workflow assessment. Table 5 shows the four assays as they compare in terms of hands-on time, time to result, and number of steps (as an indicator of ease of use).

BioFire's FilmArray RP was the easiest to use and had the shortest time to result. For MPV, influenza virus A/H3, PIV1, PIV2, PIV3, and RSV B, its sensitivities were >92%, whereas the sensitivities for AdV, influenza virus A/H1N1/2009, influenza virus B, RSV A, and RhV/EV were lower. The assay's AdV sensitivity was significantly lower than those of the eSensor (P < 0.0001) and xTAG RVP fast (P < 0.05) but similar to that of the xTAG RVPv1 (P = 0.13); its influenza virus A/H1N1/2009 sensitivity was lower than those of the eSensor and xTAG RVPv1 (P < 0.05) but comparable to that of the xTAG RVP fast (P = 0.67); and its influenza virus B sensitivity was lower than that of the eSensor (P < 0.05) only. However, the FilmArray RP was significantly more sensitive than the xTAG RVP fast for influenza virus B (P < 0.05), and there were no statistically significant differences in sensitivity rates between the assays for RSV A or RhV/EV. The FilmArray RP was 100% specific. An extended FDA-cleared panel of 20 targets is now available that includes Chlamydophila pneumoniae, Mycoplasma pneumoniae, and Bordetella pertussis, making the FilmArray RP the panel with the most targets and the only one to include bacterial targets. We did not assess the performance of these additional targets in this study. As the only closed system in our study and the only system to provide only qualitative results, the FilmArray RP is amenable to testing outside a dedicated high-complexity molecular laboratory and is appropriately labeled by the FDA as being of moderate complexity. This is in contrast to the other three panels evaluated in this study that are FDA cleared as high-complexity tests and are not closed systems. On the downside, each FilmArray instrument can only test one sample at a time, limiting its capacity for high-throughput testing and requiring the need for more than one instrument in most laboratories.

The GenMark eSensor RVP was the most sensitive of the four assays, detecting 100% of the samples known to be positive for AdV, influenza virus A, influenza virus B, MPV, PIV1, PIV2, PIV3, RSV A, and RSV B. The eSensor RVP is the only assay to report adenovirus species (AdV C or AdV B/E), doing so with 100% accurate results compared to our sequencing results. However, the loss of enterovirus detection is a potential limitation, particularly for laboratories that routinely test for enteroviruses in respiratory specimens. The increased sensitivity of the eSensor RVP is countered by a lower specificity than the other three assays (P < 0.05). Whether the false positives represent increased analytical sensitivity, greater variability in assay performance, or the ease with which runs can be contaminated cannot be determined. Although the eSensor RVP does provide a quantitative value in nanoamps (nA), the package insert states that it is not intended to provide a quantitative value for the virus present. Furthermore, Pierce and Hodinka concluded that an accurate quantitative correlation between real-time PCR C_T values and nA signal strength could not be made (9). Similarly, our observed nA signal strengths for false positives were not statistically different from those obtained for true positives. The workflow for the eSensor RVP is a slight improvement over that of the predicate xTAG RVPv1, with two fewer steps and less hands-on time. However, the time to result is still over 7 h, and postamplification material is still manipulated, increasing the risk of laboratory contamination. The eSensor RVP is best suited for batched testing in a high-complexity laboratory with significant molecular experience.

xTAG RVPv1 was the first large multiplex panel cleared by the FDA. With multiple steps, it had the longest hands-on time and longest time to result. Although its sensitivities for AdV (74%) and RSV A (86%) were somewhat lower, there was no statistical difference between the xTAG RVPv1 and the FilmArray RP or xTAG RVP fast for AdV, and there were no statistical differences in RSV A sensitivities between any of the tests (P = 0.81). Similar to the eSensor RVP, the xTAG RVP assay provides quantitative results in median fluorescent intensity (MFI) units, but the test is only intended to provide qualitative results, and we have observed that these MFI units are not linear compared to the C_T values of real-time PCR (data not shown). However, specimens with high titers of PIV3 also caused a positive PIV1 signal; thus, the ratio of these signals must be assessed in determining PIV3 calls. The xTAG RVPv1 is best utilized as a batched test performed in an experienced molecular laboratory, as there are multiple steps where postamplification material is manipulated.

Although the xTAG RVP fast does decrease the time to result by 3 h relative to the xTAG RVPv1 and reduces the number of hands-on steps, it does not offer PIV detection, and there is a significant drop in overall sensitivity. The sensitivities were below 92% for all targets but MPV and RhV/EV, and importantly, the influenza virus B sensitivity was 45.5%. There were minimal specificity concerns with the xTAG RVP fast.

It is known that the detection of adenovirus presents a significant challenge to the molecular multiplex tests. While not all cause respiratory infection, there are presently 57 recognized human adenovirus serotypes. Based on the respective package inserts, the xTAG RVPv1 has decreased detection for species C (serotypes 1, 2, 5, and 6) and serotypes 7A (species B) and 41 (species F), while the FilmArray RP has decreased detection of serotypes 2 and 6 (species C). Thus, the lower sensitivities we observed for these two assays are not surprising, though the 57.1% sensitivity we observed for the FilmArray RP is significantly lower than what is stated in the package insert (88.9%). Although the FilmArray RP did miss five AdV serotype 2 specimens, it also missed three serotype 1 specimens, two serotype 5 specimens, and four specimens that were unable to be typed due to low viral quantity (Table 4). The observed sensitivity of the xTAG RVPv1 for adenovirus (74.3%) is very similar to the package insert data of 78.3%. The serotypes of AdV missed by the xTAG RVPv1 were 1, 3, 5, and untypeable (Table 4). The risk of offering a test with lower sensitivity for adenovirus will be dependent on the patient population served and whether additional testing for adenovirus is offered.

The xTAG RVPv1 and xTAG RVP fast will subtype influenza virus A-positive samples as seasonal H1 or seasonal H3. The eSensor RVP and FilmArray RP will also subtype H1N1/2009. Two specimens positive for the influenza virus A matrix gene only (subtype negative) by xTAG RVP and xTAG RVP fast were actually seasonal H3, as established by the other tests and confirmed by the Prodesse ProFAST+ assay. Four samples initially showed influenza virus A-positive/subtype-negative results by the eSensor RVP. All four were lower-level positives (nA of 44.7 to 59.0; for all positives, the mean nA was 152.6 [95% CI of 135.2, 167.1]). As recommended by the package insert, the samples were tested again, and all were negative. With the disappearance of seasonal H1 (which was oseltamivir resistant), there is not currently a clinical need to offer influenza virus A subtype information. However, subtype information is helpful for our epidemiology and infection control colleagues, and it may prove to be beneficial should differences in antiviral susceptibility develop.

Our study is partially limited by the inability to do a true prospective study. Unfortunately for the study, the respiratory season (2011-2012) during which the study was performed was not as robust as past seasons, including the noticeable absence of significant numbers of influenza virus-positive specimens. Therefore, the numbers per target included in the consecutive study are relatively low, making it difficult to ascertain the significance of differing performance among the assays in a true clinical setting. However, the same trends in sensitivity were observed in both the retrospective specimens and the archived, consecutive specimens (e.g., lower adenovirus and RSV sensitivities [data not shown]). Furthermore, the extended freezer time and freeze-thaw cycle for some of the specimens could affect assay performance. To minimize any differential effect on assay performance, we tested thawed specimens on all four platforms simultaneously and did not use a retrospective reference result, but it should be noted that such results may vary from the results for freshly obtained specimens that have not been frozen.

Another limitation is the use of a composite gold standard. Ideally, individual LDTs for each target would also be performed on every specimen to establish the reference result, but this was not possible in our laboratory due to the lack of LDTs for every target, as well as financial constraints. Thus, we used “positive by at least two platforms” as the reference result by which we calculated sensitivity and specificity. As such, an assay with greater sensitivity than all other assays tested would show lower specificity. There is also inherent bias associated with selecting archived specimens for comparative studies. The frozen specimens had primarily been tested with the xTAG RVPv1, although we did attempt to select as many specimens as possible that had been tested by LDTs. Thus, the retrospective sensitivity data may favor the xTAG RVPv1. Lastly, the eSensor RVP assay that was used was the research-use-only (RUO) version, as the assay had not yet been FDA cleared at the time of this study. We did not, however, analyze any RUO targets in the present work and note that the chemistry for target detection and the threshold for positivity (3 nA) are the same as for the in vitro diagnostics version. Even considering the study limitations, we assert that this head-to-head comparison of the four commercially available respiratory viral panels provides useful information regarding the relative performance of each assay.

Lastly, a significant consideration for clinical laboratories is the comparison of the relative costs of these assays. Although there is a paucity of data regarding the cost effectiveness of implementing molecular multiplex testing for respiratory viruses, Mahony et al. (23) demonstrated a Can$291 per inpatient cost saving by eliminating testing algorithms and using a molecular multiplex assay as the primary test. Although all three platforms require specialized instrumentation, none of the vendors require that the equipment be a capital purchase. In terms of reagents, the relative costs are FilmArray RP > eSensor RVP > xTAG RVPv1. This is also the order for the number of targets offered. However, note that the list is reversed when ranking hands-on time (Table 5). When all expenses are considered, the three competitive multiplex assays offer nearly the same cost per target. Therefore, when considering an FDA-cleared multiplex respiratory panel, it is important to focus on what is best for your patient population and your laboratory. Future generations of these tests and/or the development of new products promise to improve the ability to offer accurate, rapid, and cost-effective laboratory diagnoses for respiratory tract illnesses.

We acknowledge the kind generosity of BioFire Diagnostics, GenMark Dx, and Luminex Molecular Diagnostics for providing the reagents used in this study.

M.B.M. has received speaker honoraria from Abbott Molecular (Luminex distributor), BioFire Technologies, and GenMark Dx. M.B.M. is a scientific advisory board member for BioFire Technologies.