Analysis of the SHARPEN Score in the Prediction of In-Hospital Mortality of Patients With Infective Endocarditis Undergoing Cardiac Surgery

Central Illustration : Analysis of the SHARPEN Score in the Prediction of In-Hospital Mortality of Patients With Infective Endocarditis Undergoing Cardiac Surgery PPV, NPV, and accuracy of the analyzed scores, according to the cutoff point. Observed mortality was 29.5%, except for the RISK-E, which was 29.0%: five cases of pulmonary/tricuspid OE were excluded, as they are not included in the analysis of this score. Error bars denote the 95% CI. NPV: negative predictive value; PPV: positive predictive value; Pts: points. *Cutoff point for the high-risk category according to the original score study. †Cutoff point defined according to Youden’s J index Abstract Background The SHARPEN score was developed to predict in-hospital mortality in patients hospitalized for infective endocarditis (IE), undergoing or not undergoing cardiac surgery. A comparison with other available scores has not yet been carried out. Objective To evaluate the performance of the SHARPEN score in predicting in-hospital mortality in patients hospitalized for IE undergoing cardiac surgery and compare it with that of both nonspecific and IE-specific surgical scores. Methods Retrospective cohort study including all admissions of patients ≥18 years who underwent cardiac surgery due to active IE (modified Duke criteria) at a tertiary care university hospital between 2007 and 2016. The SHARPEN score was compared to the EuroSCORE, EuroSCORE II, STS-IE, PALSUSE, AEPEI, EndoSCORE and RISK-E scores. Differences P<0.05 were considered statistically significant. Results A total of 105 hospitalizations of 101 patients (mean age 57.4±14.6 years; 75.2% male) were included. The median SHARPEN score was 11 (9-13) points. The observed in-hospital mortality was 29.5%. There was no statistically significant difference in observed vs. estimated mortality (P = 0.147), with an area under the ROC curve of 0.66 (P = 0.008). In comparison with the other scores, no difference was observed in discriminative ability. The statistics of the SHARPEN score at a cutoff >10 points — positive predictive value (PPV): 38.1%, 95%CI:30.4-46.6; negative predictive value (NPV): 80.0%, 95%CI:69.8-87.4; and accuracy: 58.1%, 95%CI:48.1-67.6 — showed overlapping 95%CIs, indicating no significant difference between scores. Conclusions The SHARPEN score did not present parameters with a significant difference in relation to the other scores analyzed; despite the easy obtainment of its few variables, it has limited applicability in clinical practice, like other existing scores.


Introduction
Despite advances in its medical and surgical treatment, infective endocarditis (IE) remains associated with severe complications and high mortality.The presentation and course of IE are highly variable, depending on host factors (preexisting heart disease, prosthetic valve, implantable cardiac device), the causative organism, and the adequacy of treatment (antibiotics, surgery).The interaction of these factors results in an in-hospital mortality rate for patients with IE ranging from 15 to 30%. 1 Surgical treatment is required in approximately half of patients with IE due to severe complications.Reasons for considering surgery early in the active phase (while the patient is still receiving antibiotic treatment) are to avoid progressive heart failure and irreversible structural damage caused by severe infection, as well as to prevent systemic embolization.On the other hand, surgical therapy during the active phase of IE is associated with significant risk. 2 Prognostic scores are a reasonable estimate of the risk of death, which is important in clinical decision-making regarding indications for surgery.Estimates are needed to inform patients and their families about surgical risks, and risk stratification allows for a fair comparison of cardiac surgery outcomes, so that surgeons and hospitals treating high-risk patients do not appear worse off than others. 3The performance of traditional surgical scores (EuroSCORE and EuroSCORE II) and IE-specific scores (STS-IE, PALSUSE, AEPEI, EndoSCORE, RISK-E) in predicting surgical risk in patients undergoing cardiac surgery for IE was recently evaluated in a Brazilian cohort. 4In that study, the best performer in terms of predicting mortality risk was the EuroSCORE when taking both discrimination power and calibration (observed-to-estimated mortality ratio) into account.However, previous work showed conflicting results, with the EuroSCORE underperforming compared to an IE-specific score (STS-IE). 5These differences in the literature support the hypothesis that, to date, no single score has proven to be ideal to identify patients with IE at greater risk for in-hospital mortality.
The SHARPEN score was developed to predict in-hospital mortality in hospitalizations secondary to IE in patients undergoing or not undergoing cardiac surgery. 6Unlike other scores, the SHARPEN score is composed of clinical variables that are readily obtained, which could facilitate its use in clinical practice.Analysis of SHARPEN in a Brazilian cohort in a tertiary care center showed performance similar to that described in the original cohort. 7Within this context, the objective of the present study was to evaluate the SHARPEN score as a predictor of in-hospital mortality in hospitalized patients with IE who underwent cardiac surgery, and to compare it with traditional and IE-specific surgical scoresan analysis not performed in the literature yet.

Methods
Retrospective cohort study including all admissions between 2007 and 2016 of patients aged ≥18 years who underwent cardiac surgery due to active IE at Hospital de Clínicas de Porto Alegre (HCPA), a tertiary public teaching hospital in southern Brazil.Only patients with definitive IE, diagnosed according to the modified Duke criteria, 8 were included; those whose electronic medical records were unavailable were excluded.Patients were identified from the surgical booking system and by a keyword search in the PPV, NPV, and accuracy of the analyzed scores, according to the cutoff point.Observed mortality was 29.5%, except for the RISK-E, which was 29.0%: five cases of pulmonary/tricuspid OE were excluded, as they are not included in the analysis of this score.Error  The SHARPEN score 6 was calculated for each IE admission.Depending on the calculated score, admissions were classified as low-(2 to 6 points), moderate-(7 to 10 points), or high-risk (11 to 20 points).The parameters of interest were: • Systolic blood pressure <90 mmHg at presentation: 3 points; • Heart failure during hospitalization (Framingham criteria 9 ): 2 points; • Age: <50 years: 2 points; 50-65 years: 4 points; >65 years: 6 points; • Raised serum creatinine at admission (>2.26 mg/dL): 2 points; • Pneumonia (≥48 hours after admission): 2 points; • Elevated C-reactive protein (peak >200 mg/L during hospitalization): 2 points; • Non-intravenous drug abuser: 3 points.
The performance of the SHARPEN score was compared with that of the logistic EuroSCORE 10 and EuroSCORE II, 11 as well as the IE-specific STS-IE, 12 PALSUSE, 13 AEPEI, 14 EndoSCORE, 15 and RISK-E 16 scores.Any death during hospitalization, regardless of length of stay, was defined as in-hospital mortality.Creatinine clearance (CrCl) was estimated using the Cockcroft-Gault formula. 17eoperative critical status was defined as the presence of one of the following during the same hospitalization: ventricular tachycardia/fibrillation or aborted sudden death, cardiopulmonary resuscitation, mechanical ventilation before induction of anesthesia, administration of inotropic agents, use of intra-aortic balloon pump/ventricular assist device before induction of anesthesia, or acute kidney injury (anuria or oliguria [urinary output <10 mL/h]). 11Active IE (still on antibiotics at the time of surgery), chronic lung disease, extracardiac arteriopathy, reduced mobility (severe mobility impairment secondary to neuro-musculoskeletal dysfunction), recent myocardial infarction (≤ 90 days), severe pulmonary artery hypertension (systolic pulmonary artery pressure >55 mmHg), severe kidney injury (CrCl <50 mL/ min), and urgency of surgery were also defined according to the EuroSCORE II criteria. 11

Statistical analysis
Data were collected directly from patients' electronic medical records and analyzed using IBM SPSS 21.0, MedCalc 12.5, and OpenEpi 3.01.16software.Qualitative data were described as absolute and relative frequencies; mean (standard deviation) or median (interquartile range, IQR) were used for quantitative data as appropriate, depending on the normality of distribution, as determined by the Shapiro-Wilk test.In-hospital mortality was compared between groups using the chi-square test or Fisher's exact test, as appropriate.The optimal cutoff point for continuous scores was defined using the highest Youden's J index, as calculated by the equation "(sensitivity + specificity) -1".Calibration (expressed as the observed-to-estimated [O/E] mortality ratio, i.e., the standardized mortality ratio [SMR]) and the discriminative ability (expressed by the area under the receiver operating characteristic curve [AUC-ROC]) of the scores were evaluated.The mid-p exact test with Miettinen's modification was used to calculate the SMR with a 95% confidence interval (CI).AUC-ROC comparisons were performed using the DeLong test.P-values <0.05 were considered significant.

Results
We studied 105 hospitalizations of 101 patients (four patients had two hospitalizations each) for active IE, who underwent cardiac surgery, between 2007 and 2016.Of the 107 hospitalizations initially retrieved, two (1.9%) were excluded due to unavailability or inaccessibility to the electronic medical record.Characteristics of the sample are described in Table 1.Hemodialysis before surgery was performed in 22 hospitalizations (21.0%): 14 (13.3%) for chronic kidney disease (CKD), six (5.7%) for acute kidney injury, and two (1.9%) for acute-on-CKD.
The observed in-hospital mortality was 29.5% (95%CI: 20.8-38.2%);20.0% in admissions stratified as low or moderate risk and 38.1% in high-risk admissions (P = 0.068).Low-and moderate-risk admissions were pooled for this analysis due to the limited representativeness of the former.There was no statistically significant difference in observed vs. estimated mortality in the overall analysis, nor in moderate-risk hospitalizations.Conversely, the observed mortality for low-risk hospitalizations was higher than estimated, while that of high-risk hospitalizations was lower than estimated (Figure 1).There was wide variability in estimated mortality among the analyzed scores (Figure 2).Besides the SHARPEN score, only EuroSCORE, PALSUSE, and RISK-E did not present a statistically significant difference between observed and estimated mortality (Table 3).
The AUC-ROC for the SHARPEN score was 0.66 (95%CI: 0.54-0.79;P = 0.008) (Figure 3).In comparison with the other scores (Table 3), there was no significant difference in discrimination, with only a trend for discriminative ability to be lower than that of the logistic EuroSCORE (P = 0.098) and EndoSCORE (P = 0.110).
Statistics for the SHARPEN score, and for the other scores analyzed, are shown in the Central Figure .An increase in positive predictive value (PPV) and in accuracy can be observed when adopting a cutoff of >12 points, defined according to Youden's J statistic, instead of the threshold used in the original study (>10 points).Using the original threshold, mortality was significantly higher (51.7 vs. 21.0%,P = 0.004).However, the CIs of these statistics -as well as those of the negative predictive value (NPV) -overlap, indicating absence of a statistically significant difference between them.

Discussion
In this cohort of patients, the SHARPEN score was calibrated (P = 0.147), with no statistically significant difference in discriminative power (AUC-ROC = 0.66;  The possibility of using a tool that yields reliable prognostic information based on clinical variables alone is a potential advantage of this new score.In addition, these are objective variables, hence easy to obtain and standardize, which facilitates outcome measurement across different populations.In addition, the SHARPEN score does not include microbiological variables (which may be affected by issues such as difficult confirmation or slow-growing pathogens), unlike other STS-IEs, such as the STS-IE, 12 PALSUSE, 13 EndoSCORE, 15 and RISK-E. 16owever, as it incorporates two variables that can only be identified late in the admission (nosocomial pneumonia and peak C-reactive protein >200 mg/L), it may have limited value in early risk estimation.
In the original study cohort including 233 patients with an in-hospital mortality of 23.2%, of which only 51 (21.9%) underwent cardiac surgery, the score showed an AUC-ROC of 0.86 (95%CI: 0.80-0.91). 6In the only study that validated the SHARPEN score in Brazil, 7 including 179 hospitalizations for IE in a public hospital, with an in-hospital mortality rate of 22.3% and cardiac surgery performed in 68 patients (38.0%), the score showed an AUC-ROC of 0.76 (95%CI: 0.67-0.85);0.77 when only clinical treatment was performed and 0.72 in those requiring cardiac surgery.In this study, there was no analysis of specific mortality in the high-risk group (>10 points).
A key strength of the present analysis was the evaluation of the score performance in a cohort of exclusively hospitalizations in which cardiac surgery was performed, unlike the original cohort 6 and the subsequent Brazilian study that also evaluated the performance of the score. 7Furthermore, while in the original study only 26.2% of admissions were categorized as high risk, in the present series they represented 52.4% of the total.On the other hand, while the feared S. aureus was the main etiologic agent in the original cohort (48.1%), in this cohort it was the cause of only 8.6% of hospitalizations.In the present study, we observed a nonsignificant increase in score accuracy when we adopted a higher cutoff point, raising the hypothesis that, perhaps, for high-risk surgical patients, the cutoff point should be higher; this will need to be confirmed in future cohorts.
Unlike previous studies and comparative studies, we chose to analyze the score statistics starting from a The present study has limitations.The sample was relatively small and restricted to a single tertiary care center.The long period of analysis means that both clinical and surgical management of these patients may have changed over time.Finally, retrospective data collection can compromise the quality of the data obtained.

Conclusions
Several research groups have been searched for the optimal prognostic score for risk stratification in cardiac

Figure 1 -
Figure 1 -Observed/estimated in-hospital mortality and overall mortality by SHARPEN risk categories.CI: confidence interval; O/E: observed/estimated; Pts: points.

Figure 2 -
Figure 2 -Observed and estimated in-hospital mortality according to the tested scores.*Observed mortality was 29.5%, except for the RISK-E, which was 29.0% (five cases of pulmonary/tricuspid IE were excluded, as they are not included in the analysis of this score).Error bars denote the 95% CI.IE: infective endocarditis

index Central Illustration: Analysis of the SHARPEN Score in the Prediction of In-Hospital Mortality of Patients With Infective Endocarditis Undergoing Cardiac Surgery Int J Cardiovasc Sci. 2023; 36:e20230061
bars denote the 95% CI.
NPV: negative predictive value; PPV: positive predictive value; Pts: points.*Cutoffpoint for the high-risk category according to the original score study.†Cutoffpoint defined according to Youden's J

Table 2 -Description of the SHARPEN score and patients' (n = 105) distribution by the variables
9V: intravenous; SBP: systolic blood pressure; CRP: C-reactive protein.*According to Framingham criteria.9