Automated or semiautomated systems have been widely used for species identification and susceptibility testing due to the increasing volumes of clinical specimens processed by clinical laboratories, perceived cost-effectiveness, and convenient interfaces with laboratory and hospital information systems. These systems are able to decrease the in-laboratory turnaround time compared to that required for standardized methods and supply physicians with susceptibility profiles to help them guide antimicrobial therapy (8, 21). Each system has inherent strengths as well as recognized limitations. Unfortunately, reporting errors by any test system can have serious implications for the clinical outcome for patients (16). Numerous studies have reported on the accuracies of various automated systems when several organism-antimicrobial combinations are tested. The most frequently reported errors have involved Pseudomonas aeruginosa and select members of the family Enterobacteriaceae, especially when these organisms are tested against β-lactam antimicrobial agents (1, 2, 4, 10-15, 19, 21, 22). Negative results of such studies have encouraged the updating of product software and the periodic issuance of manufacturer notices recommending alternative testing methods for certain organism-antimicrobial combinations.

Several mechanisms may lead to resistance to broad-spectrum β-lactams among gram-negative bacteria, including (i) the hyperproduction of AmpC β-lactamase or other broad-spectrum β-lactamases, such as various carbapenemases; (ii) decreased outer membrane permeability (porin deletion); and (iii) active efflux. Some of these mechanisms may preferentially affect particular β-lactam compounds, and some automated systems may have problems in correcting interpretive results (11, 13, 22). Variations in inoculum concentrations and incubation times may also affect the detection of resistance secondary to β-lactamase production. Thus, rapid automated susceptibility testing systems may have difficulty in detecting resistance to some β-lactam compounds for technical reasons, including the methodologies used by the test system and software calculations due to the underlying resistance mechanisms of the organism (1, 2, 7, 11).

Recent reports from an increasing number of hospitals of antibiogram inconsistencies between broad-spectrum cephalosporins and piperacillin-tazobactam when P. aeruginosa is tested drew our attention to possible technical problems with the prevailing automated test systems. In this study, we evaluated the antimicrobial susceptibility testing accuracies of three commonly used automated systems, the MicroScan WalkAway system (Dade Behring, Deerfield, IL) and the VITEK and VITEK 2 systems (bioMerieux, Hazelwood, MO), against a challenge collection of P. aeruginosa isolates and five antipseudomonal β-lactams: aztreonam, cefepime, ceftazidime, imipenem, and piperacillin-tazobactam.

One hundred recent clinical strains of P. aeruginosa from hospitals worldwide were evaluated. The isolates were selected according to cefepime and piperacillin-tazobactam MIC results to overrepresent strains that fall within ±2 log₂ dilutions of current Clinical and Laboratory Standards Institute (CLSI; formerly the National Committee for Laboratory Standards) susceptible and resistance breakpoints (Table ​(Table1)1) (5, 17, 18). This would allow the detection of systemic errors and facilitate the characterization of the errors in log₂ dilution steps.

The susceptibilities of all isolates were tested by three reference methods, by two other manual standardized methods, and with the three cited automated systems. The broth microdilution (frozen panels), agar dilution, and disk diffusion methods were used as reference benchmarks to establish consensus categorical and MIC results for each organism-antimicrobial combination (5, 17, 18). In addition, commercially prepared dry-form panels (TREK Diagnostics, Cleveland, OH) and E-test (AB BIODISK, Solna, Sweden) were used to further confirm the consensus MIC, when needed. The MicroScan WalkAway tests were performed at a commercial laboratory (Weland Clinic, Cedar Rapids, IA) by using a gram-negative MIC panel type 30 (B1017-308). The VITEK 2 tests were performed at a university medical center laboratory (University of Washington, Seattle, WA) with GN09 susceptibility cards, and the results were analyzed with advanced expert system software, version WSVT2-R03.01. The VITEK and all reference and standardized test methods were performed at JMI Laboratories (North Liberty, IA). VITEK GNS-122 susceptibility cards were used with the VITEK system, and the data were interpreted by using software version WSVTK-R09.01. All tests by the reference and standardized methods and automated system susceptibility testing were performed in compliance with current CLSI methods (M7-A6 and M7-A8) (17, 18) and/or as recommended by the manufacturers' package inserts with the products. Quality control was monitored by using the following organisms: Staphylococcus aureus ATCC 29213, Enterococcus faecalis ATCC 29212, P. aeruginosa ATCC 27853 and Escherichia coli ATCC 25922 and ATCC 35218 (5).

Consensus categorical results were initially obtained by comparing the results of the frozen broth microdilution and agar dilution methods (5, 17), which were the methods used to establish and validate each automated method. The agreement between both methods was considered the “consensus result.” If the results of these test methods did not agree, discords were resolved by the CLSI disk diffusion method (18). Categorical consensus results were obtainable for 100.0% of the strains by these three methods. The categorical results of each automated system were then compared to the consensus results. Categorical disagreements were classified as very major errors (false susceptibility), major errors (false resistance), and minor errors (involving the intermediate category interpretation).

The MIC consensus results were also calculated by the frozen broth microdilution and agar dilution methods. If the MIC results of these methods were different, the disk diffusion test result was consulted if a categorical shift was involved. As an example, if the ceftazidime results were 16 and 32 μg/ml and the disk diffusion provided a resistant result, the consensus MIC was established at 32 μg/ml. If a consensus MIC could not be achieved by these three methods, the dry-form broth microdilution and E-test results were sequentially compared to determine a log₂ quantitative value.

Table ​Table11 shows the distribution of the consensus MIC results for the P. aeruginosa strains. The proportions of MIC results within ±1 log₂ dilution of the breakpoints were 90% for aztreonam, 75% for cefepime, 62% for ceftazidime, 64% for imipenem, and 59% for piperacillin-tazobactam (Table ​(Table1).1). Based upon the consensus results for P. aeruginosa, the rates of susceptibility and resistance were as follows: 47 and 27%, respectively, for aztreonam; 59 and 14%, respectively, for cefepime; 44 and 43%, respectively, for ceftazidime; 71 and 19%, respectively, for imipenem; and 50 and 50%, respectively, for piperacillin-tazobactam (Table ​(Table22).

TABLE 2.

Evaluation of accuracies of three automated systems for susceptibility testing of 100 P. aeruginosa strains against β-lactam antimicrobial agents

Antimicrobial and category	% of isolates
	Consensusa	MicroScan WalkAway	VITEK 2	VITEK
Aztreonam
Susceptible	47	41	33	49
Intermediate	26	32	41	14
Resistant	27	27	26	37
Cefepime
Susceptible	59	39	57b	44
Intermediate	27	29	22	35
Resistant	14	32	21	21
Ceftazidime
Susceptible	44	43	44	42
Intermediate	13	16	12	20
Resistant	43	41	44	38
Imipenem
Susceptible	71	68	72b	69
Intermediate	10	9	14	1
Resistant	19	23	14	30
Piperacillin-tazobactam
Susceptible	50	68	78b	71
Resistant	50	32	22	29

Open in a separate window

^aConsensus result among the broth microdilution (frozen dry-form panels), agar dilution, and disk diffusion methods.

^bThe system did not provide results for one strain.

The highest numbers of intermethod discords were detected when piperacillin-tazobactam was tested, with all automated systems showing a clear tendency toward false-susceptible results (very major errors). Piperacillin-tazobactam consensus susceptibility rates increased from 50% to 68% (MicroScan WalkAway system), 71% (VITEK system), and 78% (VITEK 2 system). Thus, the rates of very major errors when this agent was tested were 27% for the VITEK 2 system, 21% for the VITEK system, and 19% for the MicroScan WalkAway system, while major errors were detected only with the MicroScan WalkAway system (1%) (Table ​(Table33).

TABLE 3.

Evaluation of accuracies of automated systems for susceptibility testing of 100 P. aeruginosa strains against β-lactam antimicrobial agents

Antimicrobial and error typea	Error rate (%)
	MicroScan WalkAway	VITEK 2	VITEK
Aztreonam
Very major	0	0	2
Major	0	1	2
Minor	28	31	28
Cefepime
Very major	0	0	0
Major	3	1	0
Minor	32	18	26
Ceftazidime
Very major	0	1	2
Major	0	1	0
Minor	13	9	11
Imipenem
Very major	0	1	0
Major	2	2	2
Minor	10	8	11
Piperacillin-tazobactam
Very major	19	27	21
Major	1	0	0

Open in a separate window

^aError rates were calculated based on the consensus result among the broth microdilution (frozen dry-form panels), agar dilution, and disk diffusion methods.

When cefepime was tested, the major error rates did not exceed 3% (the MicroScan WalkAway system results were at the upper end of acceptable performance) (Table ​(Table3).3). In contrast, however, the rates of minor errors were usually high for all combinations (8 to 32%), reflecting the high number of isolates whose breakpoints were within ±1 log₂ dilution of the categorical breakpoints (Tables ​(Tables11 and ​and3).3). When aztreonam was tested, a clear tendency toward less susceptible results was detected with the VITEK 2 system (a shift from susceptible to intermediate) and the VITEK system (a shift from intermediate to resistant) (Tables ​(Tables22 and ​and3).3). A similar trend was discovered with cefepime, especially when it was tested on the MicroScan WalkAway system. The cefepime susceptibility rates dropped from 59% (consensus) to only 39% (MicroScan WalkAway system) and 44% (VITEK system), while the cefepime resistance rates increased from 14% (consensus) to 32% (MicroScan WalkAway system) and 21% (VITEK 2 and VITEK systems). In addition, a slight tendency toward more resistant results (a shift from intermediate to resistant) was observed when imipenem was tested with the VITEK system.

P. aeruginosa has become the most common gram-negative bacterial species associated with serious nosocomial infections in many hospitals, particularly within intensive care units (9, 23). The accurate determination of P. aeruginosa antimicrobial susceptibility is critical for both the immediate management of infected patients and the timely introduction of appropriate infection control measures. The attributable mortality rate associated with P. aeruginosa infection has been reported to be greater than 20%, and the rapid application of appropriate antimicrobial treatment has been associated with improved clinical outcomes. Conversely, other studies have demonstrated that inappropriate initial antimicrobial therapy is associated with higher rates of patient morbidity and mortality as a result of infections caused by P. aeruginosa (16).

The automated systems evaluated in this study are market leaders in the United States and many other countries (6, 19). A recent survey of the College of American Pathologists with a P. aeruginosa isolate indicated that 5.9% of respondents using an automated test system reported a false-susceptible result for a highly resistant challenge organism (6). That survey also indicated that 29.5% of responses from laboratories that used the VITEK 2 system (46 of 156 results) were falsely susceptible to piperacillin-tazobactam (MICs ≤ 64 μg/ml). This was an unacceptable level of performance for those laboratories reporting the organism as falsely susceptible to piperacillin-tazobactam, given that the P. aeruginosa strain had a piperacillin-tazobactam MIC of ≥1,024 μg/ml by reference methods (17). Those results were confirmed by our findings with the VITEK 2 system and the β-lactamase inhibitor combination.

The results of this study dramatically demonstrated that the automated systems studied (the MicroScan WalkAway, VITEK 2, and VITEK systems) generally failed to accurately detect piperacillin-tazobactam resistance among clinically significant isolates of P. aeruginosa. All three systems showed high and unacceptable rates of very major errors (19 to 27%). It is important to emphasize that these results should be interpreted in light of the criteria used to select the strains (MICs close to the breakpoints or within the resistance range); this population was chosen to increase the sensitivity of detecting and quantifying significant categorical disagreements. It is also important that data from the SENTRY Antimicrobial Surveillance Program indicate that the majority of piperacillin-tazobactam-resistant strains (MICs ≥ 128 μg/ml) show piperacillin-tazobactam MIC results at or only 1 log₂ dilution higher (128 to 256 μg/ml) than the resistance breakpoint, therefore emphasizing the importance of those errors and the validity of the strains selected for use in this experiment (Table ​(Table1).1). Among 397 P. aeruginosa strains evaluated by the USA SENTRY Program (2003), 34 (8.6%) were resistant to piperacillin-tazobactam; and 28 (82%) of those strains showed piperacillin-tazobactam MICs of 128 or 256 μg/ml (H. S. Sader, D. J. Biedenbach, T. R. Fritsche, and R. N. Jones. Abstr. 105th Gen. Meet. Am. Soc. Microbiol., abstr. C-007, 2005).

Earlier studies of P. aeruginosa isolates from cystic fibrosis patients (mucoid and nonmucoid isolates) have established the accuracies of the CLSI reference methods (20) and the agar diffusion methods (disk diffusion and E-test) (3) but recognized that the VITEK and the MicroScan WalkAway systems were not recommended for use for the testing of this species (4). The trends toward false-intermediate and -resistant results by automated methods with cefepime and aztreonam have also become a chronic problem (1, 2, 11, 19), leading to inaccuracies in local antibiograms and the selection of alternative agents for individual patient therapy. Thus, the high rates of false-susceptible and false-resistant results provided by the automated systems may have a significant impact on formulary positioning of antipseudomonal agents and, ultimately, guidelines for the empirical treatment of P. aeruginosa infections in medical centers worldwide. Reevaluation of the β-lactam interpretative algorithms of these systems for tests with P. aeruginosa would be prudent to minimize or eliminate the biases detected and correct the various types of errors detected compared to the results of recognized reference methodologies (5, 17, 18).