1Clinical criteria in the diagnosis

Clinical Criteria in the Diagnosis of Ventilator- Associated Pneumonia* Richard G. Wunderink, MD, FCCP (CHEST 2000; 117:191S–194S) Abbreviations: CPIS 5 clinical pulmonary infection score; VAP 5 ventilator-associated pneumonia T his section discusses the accuracy of the traditionalclinical criteria of fever, leukocytosis, and purulent tracheal secretions, usually with abnormal radiographic signs, as initial possible indicators of ventilator-associated pneumonia (VAP). A major methodological problem in assessing the sen- sitivity or specificity of these clinical criteria is obtaining the correct denominator. Instead of providing the total number of patients at risk, many studies give only the number who met at least one criterion or the number who gave a subjective clinical impression of being at risk for VAP. Therefore, sensitivities may be lower and specifici- ties higher if data from these studies are applied to the entire population of ventilator-assisted patients. Conversely, the sensitivity derived from the entire population is inappropriately high when applied only to a preselected population, such as all febrile patients or all patients with purulent tracheal secretions. The appropri- ate calculation depends on the question being addressed. If the clinical criteria are used to select a population at high risk of VAP, population-based statistics should be used. If, however, the goal is to discriminate between patients with VAP and those with a mimicking condition, the criteria characteristics in the suspected VAP group should be used. Both issues are clinically relevant, and operating characteristics in each population are calculated when possible. Use of the sensitivity and specificity calcu- lated from these studies requires careful attention to the population being discussed. Most studies assume that all patients who had no clinical findings did not have pneumonia. No study men- tions that unsuspected pneumonia was documented in patients who did not meet study entry criteria. The fact that autopsy studies regularly show pneumonia in patients not treated with antibiotics suggests that a subjective clinical impression that pneumonia is not present may be inaccurate in some patients. Systematic overestimation of sensitivity may result. In one autopsy study, 9% of patients who were not given antibiotic therapy, and died, had VAP.84 Data on the original clinical suspicion were avail- able for eight studies (Tables 7–8).11,12,20,21,38–41 Performance Characteristics Patient Selection Six of eight studies prospectively identified patients for study entry. Two were retrospective studies. All but one of the studies included consecutive patients meeting case definitions.6 The indications for study entry in the eight studies were as follows: autopsy, in four studies; suspected VAP, in three studies; and clinical findings and results of broncho- scopies performed at intervals driven by an ARDS proto- col, in one study. The study of Sutherland et al12 corre- lated evidence of VAP in patients not overtly suspected of having VAP. In contrast, the primary end point of the study by Fagon et al11 was to examine the accuracy of the clinical diagnosis in patients suspected of having VAP. The only two studies using specific objective criteria (new or changing radiographic infiltrates and purulent tracheal secretions) rather than subjective criteria are the Fagon studies in 1988 and 1993.11,40 ARDS is somewhat overrepresented in the review studies. Three studies include only ARDS patients, and ARDS was present in at least 248 of the 567 patients (44%). The incidence of VAP in the studies varies from 15 to 74%. The incidence in studies of patients clinically sus- pected of VAP was consistently in the 30 to 40% range and was consistently higher in autopsy studies. Since variable incidence rates affect subsequent sensitivity and specificity calculations, we partially compensated by calculating like- lihood ratios. Methodology The only studies in which investigators were blinded to the reference standard were three of the four autopsy studies20,38,39 and the study of Sutherland et al,12 in which data were collected prospectively and the timing of bron- choscopy was not based on clinical suspicion. The quality of the assessments was only for the refer- ence standard. Fagon et al11 had at least three indepen- dent subjective evaluations of the probability of VAP. To further validate the findings, several studies used clinical manipulation, including stopping antibiotics or determin- ing alternate explanations if the findings from the refer- ence culture were negative. Reference Standard Four studies compared clinical findings exclusively to histology findings at autopsy. Confirmation of the accuracy of the test results for bronchoscopic specimens was also available at autopsy in another 33% of the remaining patients. A definite diagnosis was available in 386 of the 597 patients (65%), or in 367 of the 492 patients (75%) if the study of Sutherland et al12 is excluded. In addition to histology findings at autopsy, definite diagnoses were based on the results of cultures of blood or pleural fluid, and negative diagnoses were based on recovery without antibiotic therapy. *From the Department of Pulmonary and Critical Care Re- search, Methodist Hospitals of Memphis, Memphis, TN. CHEST / 117 / 4 / APRIL, 2000 SUPPLEMENT 191S Table 7—Clinical Indicators of VAP* Study/Year Selection Indication Description of Patients Investigators Blinded to Reference Standard Quality Assessment Andrews et al39/1981 Retrospective, consecutive Autopsy ARDS (100%) NA Multiple specimens, standardized autopsy criteria Bell et al38/1981 Retrospective, consecutive Autopsy ARDS (100%) Yes None Fagon et al11/1988 Prospective, consecutive Suspected VAP Med/surg ARDS (8%) No None Fagon et al6/1993 Prospective, selective† Suspected VAP Med/surg No Antibiotics stopped if culture results were negative; at least 3 independent evaluations Torres et al41/1994 Prospective, consecutive Autopsy Respiratory ICU No None Papazian et al20/1995 Prospective, consecutive Autopsy‡ Med/surg ARDS (34%) Yes Entire lung sectioned Sutherland et al12/1995 Prospective, consecutive Protocol Trauma/med ARDS (100%) Yes None Timsit et al21/1995 Prospective, consecutive Suspected VAP Med/surg ARDS (40%) No Alternative sources of fever and infiltrates identified *Med 5 medical; surg 5 surgical. See Table 5 for other abbreviations. †Thirteen of 97 patients excluded. ‡Thirty-eight cases among 156 patients who died. Table 8—Clinical Criteria for VAP* Study/Year Patients PN Patients Receiving Antibiotics, % Reference Standard Clinical Criteria Total Population Suspected VAP Sens Spec LR Sens Spec LR Andrew et al39/1981 24 14 92 Autopsy Overall clinical 64.3 (NC) 80.0 (NC) 3.22 NA NA NA Fever 100 (NC) 20.0 (NC) 1.25 NA NA NA Leukocytosis 100 (NC) 20.0 (NC) 1.25 NA NA NA Improve with antibiotics 18.2 (NC) 30.0 (NC) 0.26 NA NA NA Bell et al38/1983 47 35 NA Autopsy Overall clinical 54.3 (NC) 83.3 (NC) 3.25 NA NA NA Fagon et al11/1988 147 45 61 PSB RI, PS 100§\ (NC) 80.5 (NC) 5.12 100¶ (NC) CC CC Autopsy (68) RI, PS, and F or L 95.6 (NC) 82.0 (NC) 5.31 95.6 (NC) 7.8 (NC) 1.03 Definite† Dx (96) RI, PS, F, and L 48.9 (NC) 89.8 (NC) 4.79 48.9 (NC) 48.0 (NC) 0.94 Fagon et al6/1993 84 27 44 PSB Clinical suspicion NA NA NA 61.8 (NC) 83.8 (NC) 3.81 Autopsy (16) Unanimous CS NA NA NA 84.6 (NC) 91.7 (NC) 10.2 Definite Dx (51)‡ Best CS NA NA NA 70.4 (NC) 86.0 (NC) 5.03 Worst NA NA NA 58.3 (NC) 78.3 (NC) 2.69 Torres et al41/1994 30 18 100 Autopsy F CC (55) CC (58) 1.31 NA NA NA PS CC (83) CC (33) 1.24 NA NA NA PS 1 RI or L CC (70) CC (45) 1.27 NA NA NA Papazian et al20/1995 38 18 66 Autopsy CPIS NA NA NA CC (72.2) CC (85.0) 4.81 Sutherland et al12/1995 105 16 89 BAL Any (PS, F, L, ARI) 100 (NC) 3.4 (NC) 1.03 NA NA NA Autopsy (19) Any 2 of above 68.8 (NC) 18.0 (NC) 0.84 NA NA NA Any 3 of above 25.0 (NC) 65.2 (NC) 0.72 NA NA NA All 4 of above 6.3 (NC) 95.5 (NC) 1.40 NA NA NA Timsit et al21/1995 122 45 46 BAL, PSB RI and 1 of F, T, or PS 100§ (NC) 83.6 (NC) 6.10 100§ (NC) CC CC Autopsy (49) RI and 2 of F, L, or PS 84.4 (NC) 85.6 (NC) 5.86 84.4 (NC) 12.1 (NC) 0.96 Definite Dx (81)¶ All (RI, PS, F, and L) 48.9 (NC) 94.4 (NC) 8.73 48.9 (NC) 65.6 (NC) 1.42 *PN 5 patients with pneumonia; RI 5 radiographic infiltrate; ARI 5 asymmetric radiographic infiltrate; PS 5 purulent secretions; F 5 fever; L 5 leukocytosis; CS 5 clinical suspicion; CC 5 cannot calculate; Dx 5 diagnosis; sens 5 sensitivity; spec 5 specificity; LR 5 likelihood ratio. See Tables 5 and 6 for other abbreviations. Values in parentheses are No. of patients. †Definite 5 autopsy within 2 weeks or resolution without antibiotics in negative cases. Of 96 patients with a definite Dx, 24 were positive and 72 were negative. (One patient with Legionella VAP and two patients with influenza pneumonitis were considered as negatives.) ‡Of 51 patients with a definite Dx, 17 were positive and 34 were negative. §Assumes no patients with VAP were excluded. \Subsequent publication suggested 567 patients total and 420 patients who did not have RI and PS were used for calculations. ¶Of 81 patients with a definite Dx, 23 were positive and 58 were negative. 192S Evidence-Based Assessment of Diagnostic Tests for Ventilator-Associated Pneumonia Although the use of antibiotics is less important in this situation than in diagnosis based on culture results, 44 to 100% of patients were receiving antibiotics at the time of clinical evaluation. Partial antibiotic treatment may have altered some clinical findings, such as fever and leukocytosis. Results Accuracy Clinical suspicion consistently was associated with a high likelihood of VAP in all ventilator-assisted patients and in the subgroup with suspected VAP. Even the worst clinical impression was associated with twice the likelihood that the patient had VAP.11 Overall clinical impression also outperformed any objective criteria, except for the clinical pulmonary infection score (CPIS).20,30 However, a major component of the CPIS score is the result of tracheal secretion cultures. Unless such cultures are routinely performed in asymptomatic patients, this in- formation is available to calculate the CPIS score only if the clinician already has some degree of clinical suspi- cion of VAP. Combinations of the objective criteria of fever, leuko- cytosis, purulent secretions, and a radiographic infiltrate performed well in the population preselected for sus- pected VAP. When patients were not preselected, as in the studies of Andrews et al,39 Torres et al,41 and Sutherland et al,12 no individual sign or combination of signs increased the likelihood that the patient had VAP. While this finding may reflect the populations studied, which were composed predominantly of ARDS patients, the operating character- istics of these objective signs may be significantly worse if applied to an unselected group of patients. Requiring all three clinical findings and radiographic abnormalities increased the specificity but lowered the sensitivity to an unacceptable 48%.11,12,21 Requiring two rather than one clinical finding with abnormal radio- graphic findings did not improve accuracy. Objective criteria did not help to distinguish between patients with and without VAP if clinical suspicion was present. Diagnostic accuracy was not improved by the presence or absence of any sign, the degree of abnormality of any sign (data not shown), or the combinations of signs. Reproducibility Fever and leukocytosis are quantitative observations and would be expected to be highly reproducible. The presence and degree of purulence and the volume of tracheal secretions are subjective observations, and no study has documented their consistency. Recent data suggest that the presence of aspirated material near the endotracheal tubes in critically ill patients might increase the amount of secretions, without necessarily indicating pneumonia.88 The presence of aspirated material is subject to variables: patient positioning, gastric emptying, endo- tracheal tube cuff pressures, and other factors. Interobserver reliability was suggested only in the study of Fagon et al.40 They found no significant differences in the accuracy of the clinical suspicion of VAP between either individual physicians or physicians grouped by level of training. The likelihood ratios for clinical impression ranged from 2.5 to 5.0 between the worst and the best individual clinicians. Unanimous decisions occurred in only 58%, but increased the likelihood ratio to 10. Risk The risks of inaccurate clinical diagnosis are antibiotic treatment of patients without VAP and lack of treatment of those with VAP. A false diagnosis of pneumonia increases the possibility of missing an alternative cause of the clinical manifestations. Meduri et al83 demonstrated that multiple potential alternative sites of infection can be found in patients with negative findings from quantitative bron- choscopy cultures. Timsit et al89 and Papazian et al81 have found that the mortality rate of patients with negative findings on bronchoscopy is equal to that of patients who are suspected of having VAP who have positive results of cultures. Perhaps other potentially lethal infections are not accurately diagnosed, and, even if antibiotics are pre- scribed, the spectrum of coverage may be inappropriate for the actual infection. In addition, antibiotic therapy for noninfectious prob- lems may increase the risk of subsequent infection with drug-resistant organisms. VAP associated with these more lethal organisms may be an important cause of death. Using decision analysis and data from the literature, Sterling et al90 found that overall survival was greater if antibiotics were withheld in patients with VAP who were diagnosed by clinical criteria. One-way sensitivity analysis suggested that withholding antibiotic therapy is favored when the positive predictive value of clinical criteria is # 28%, the mortality rate of antibiotic-treated patients with VAP is $ 40%, the mortality rate of untreated patients with VAP is # 86%, or the risk of developing VAP is $ 22%. Two-way sensitivity analysis demonstrated that the model was most sensitive to the positive predictive value of the clinical diagnosis and the mortality rate of antibiotic-treated patients with VAP. While this decision analysis is only as accurate as the data used for the calculations, the results illustrate that empirical antibiotic therapy for clinical pneumonia is not without risk. A third aspect of the overdiagnosis of VAP using clinical criteria is that much of the epidemiologic data for empir- ical treatment are based on studies using clinical criteria. Requiring all three clinical criteria (fever, leukocytosis, and purulent tracheal secretions) in addition to abnormal radiograph findings may increase the probability that the patient has VAP but does so at the risk of missing 50% of cases. Whether using loose or restrictive clinical criteria for the diagnosis of VAP, the risk factors, prevention strategies, and treatment outcomes found may be incorrect. The alternative dilemma is that if antibiotic treatment is withheld in patients with true pneumonia, morbidity and mortality may be adversely affected. Little accurate data exist to determine the risk of withholding antibiotics, even temporarily, but the mortality rate in patients given inap- propriate antibiotic therapy is approximately 50%.44 These data are skewed for more lethal organisms, such as CHEST / 117 / 4 / APRIL, 2000 SUPPLEMENT 193S Pseudomonas spp, for which inadequate antibiotic therapy is more likely than for Haemophilus spp or Streptococcus spp.90 Conclusions Although biased by study entry criteria, the diagnostic sensitivity of a radiographic infiltrate and one clinical feature (fever, leukocytosis, or purulent tracheal secre- tions) is high for VAP, but the specificity is low. The only way to increase the specificity of clinical criteria is to require all four criteria, but this results in an unacceptably low sensitivity (, 50%). These findings suggest that the presence of abnormal clinical manifestations, combined with abnormal radio- graphic findings, can be used for the initial screening for VAP. However, the lack of specificity with this method suggests that additional procedures are needed, such as cultures of lower respiratory tract secretions (grade B recommendation). In the final analysis, clinical suspicion is as sensitive and specific as any fixed objective set of criteria, with the possible exception of composite CPIS score. Unfortu- nately, in most cases the complete score can be calculated only in retrospect, after VAP is suspected clinically. The operating characteristics of the clinical diagnosis of VAP suggest that the high sensitivity is appropriate for initial clinical suspicion and screening, but that the lack of specificity suggests the need for additional testing or information. 194S Evidence-Based Assessment of Diagnostic Tests for Ventilator-Associated Pneumonia