ABSTRACT
This methodological study was conducted in order to carry out the adaptation and validation of the “PRE-DELIRIC score” prediction of delirium model in Türkiye among patients hospitalized in the intensive care unit (ICU).
The research involved 172 patients treated in the intensive care units of a training and research hospital between October 2019 and April 2020. The study data were collected using (1) a data collection form to determine the participants’ descriptive characteristics, (2) the PRE-DELIRIC score, and (3) the Confusion Assessment Method for the ICU (CAM-ICU). Receiver operating characteristic (ROC) analysis and diagnostic screening tests were applied for the purpose of determining cut-off points for the groups. The scores’ sensitivity and specificity were calculated. Significance was evaluated at the p<0.05 level.
A statistically significant association was determined between the cut-off point obtained for the PRE-DELIRIC score (≥7.58%) and the study groups (p=0.003). Patients with PRE-DELIRIC scores of 7.58 or higher exhibited a 7.404-fold greater risk of being CAM-ICU-positive [odds ratio: 7.404; 95% confidence interval (CI): 1.638-33.469]. The area under the ROC curve was 64.9% (95% CI: 0.538-0.760), and the standard error was 5.6% (p=0.044).
The PRE-DELIRIC score yielded reliable results in this study. It appears significant for patients with a likelihood of developing delirium within the ICU, and its use is recommended as a functional score that is easily applied and calculated.
Keywords: Critical care, PRE-DELIRIC, delirium, model
Introduction
Delirium is a neuropsychiatric disorder characterized by impaired consciousness, distraction, and disorganized thinking (1). It can be seen at rates of up to 80% in intensive care patients and causes increased morbidity and mortality, prolongation of mechanical ventilation and intensive care unit (ICU) stays, and long-term cognitive impairment (2,3).
A number of organic factors can result in delirium (4). Final diagnosis is based on the assessment of findings elicited by means of interviews. Several different scales have been developed for the screening, diagnosis and grading of symptoms. Proper diagnosis requires periodic evaluation of the diagnostic criteria and a knowledge of the patient’s initial mental state (5). The most common methods used for the evaluation of delirium are the Confusion Assessment Method for the intensive care unit (CAM-ICU) (6) and the Intensive Care Delirium Screening Checklist (7). Delirium has been detected at a rate of 30-70% in intensive care patients using these methods (8).
Appropriate interventions are extremely important in preventing delirium. Guidelines on pain, agitation and delirium management (9) strongly recommend the application of non-pharmacological methods for preventing delirium, but the evidence supporting pharmacological approaches is inadequate. The applications of such procedures is also time-consuming and entails a significantly increased workload. A number of prediction models that may be of assistance in identifying high-risk individuals have therefore been produced (4). One of these models, the PREdiction of DELIRium in ICu patients (PRE-DELIRIC), has been validated in various ICUs and described as useful (10). This model emerged from the findings of a systematic review study investigating risk factors for delirium (11). It predicts the development of the condition in the first 24 hours following admission to the ICU. This relies on a calculation containing 10 known risk factors for the development of delirium that are capable of being both objectively and precisely defined. This model used for estimating the risk of delirium (4,12,13) was also employed in a recent study from Turkey, although its predictive ability for dementia was not assessed (14).
This methodological study was conducted for the purpose of establishing the applicability of the “PRE-DELIRIC score” delirium prediction model in general ICU patients.
• Is the PRE-DELIRIC score confidential?
• Can it be used in the ICU?
Materials and Methods
This methodological study was conducted in order to carry out the adaptation and validation of the “PRE-DELIRIC score” prediction of delirium model in Turkey among patients hospitalized in the ICU.
Patients treated in the ICU of a training and research hospital between October 2019 and April 2020 were included in the research.
Participants
Patients who were hospitalized and treated for more than 24 hours in the general ICU, aged 18 years or older, with no history of chronic alcoholism, dementia, or delirium, who were not pregnant or breastfeeding, who had no communication problems, with Richmond Agitation Sedation scale (RASS) values of +4 to -2, and for whom consent to participate was obtained from a relative were included in the study. One hundred eighty-nine 189 cases were initially included, although the study was eventually conducted with 172 patients since eight cases were excluded due to dementia, three due to history of delirium, one due to history of alcoholism, and five for being aged under 18.
Study Procedure
A data collection form was applied to elicit the participants’ descriptive characteristics, together with the PRE-DELIRIC score, and the CAM-ICU as collection tools. Data collection took place during the study period and was performed by a physician and a nurse, who were also involved as researchers. In this study, the patient who was delirium negative at admission should have been included. The general data were collected in the first 24 hours. Data on the diagnostic and clinical characteristics were obtained from the patients’ relatives and patient charts. Data on the clinical course were also collected within the first 24 hours.
The data collection form was produced by the authors following a review of the literature in the field (9,15-17). It consists of 19 questions investigating sociodemographic and clinical characteristics.
Measures
PRE-DELIRIC Score
The PRE-DELIRIC Scoring System developed by van den Boogaard et al. (10) considers the patient’s age, Acute Physiology and Chronic Health Evaluation-II (APACHE-II) score, blood urea level, amount of morphine used, sedation use, metabolic acidosis, coma state, infection, planned/emergency intensive care admission, and the reason for hospitalization to provide a score. A pre-delirium score ≥50 is reported to be associated with a high incidence of delirium. The scoring system is used within the first 24 hours after admission to the ICU. The blood results were obtained from the patient’s medical chart by the researchers and recorded in the questionnaire. The PRE-DELIRIC score was determined within the first 24 hours after admission to the ICU in this study.
Once the scale had been independently translated into Turkish by three translators consisting of an English teacher, an English language linguistic specialist, and a medical doctor proficient in English language, the translators agreed on a common text in terms of the appropriateness and comprehensibility of translations. The scale thus obtained was then translated back into English by three English teachers. No change in meaning was determined in the backtranslated scale compared to the original English document and the form was finalized after preliminary administration to five intensive care nurses and 10 patients. The PRE-DELIRIC Scoring System does not include intercultural differences since it is based on objective criteria and not on patient statements or interpretation. The risk factors including these objective data have the same meaning in all languages and cultures. Determining content validity by eliciting an expert opinion was therefore not required for the PRE-DELIRIC score.
CAM-ICU
This scale is in common use and is reported to provide the best compliance with DSM IV criteria (16). It was developed by Ely et al. (6). The reliability and validity of the Turkish-language version were confirmed by Akıncı et al. (18) and the scale was found to have an acceptable level of sensitivity (65-69%), together with perfect specificity (97%) and reliability (Kappa =0.96). The scale has four domains, consisting of changes in the patient’s state of consciousness, attention disorder, impaired thought process, and level of consciousness. Sub-categories are not taken into account and a conclusion is reached in the form of the “presence” or “absence” of delirium according to the answers to the scale questions. All the first and second category answers must be negative, and one of the third and fourth categories must be present as a condition for the “presence” of delirium. This scale can be applied to all intensive care patients aged over 18 who are not comatose and who are able to communicate. It is recommended that the scale be completed within the first 24 hours following admission to the ICU. A repeat evaluation is performed during the day in case of any change in the patient’s condition. Otherwise evaluation once a day is appropriate. CAM-ICU was measured within the first 24 hours following admission to the ICU in this study.
Kırklareli University Institute of Health Sciences Ethics Committee approval (decision no: 3, date: 11.10.2019) was obtained from the institution in which the study was carried out, in addition to consent from the patients who were included in the study with the permission of the relevant institution. Written permission for the use of the PRE-DELIRIC score was obtained by e-mail from van den Boogaard.
Statistical Analysis
All statistical analyses were conducted on NCSS (Number Cruncher Statistical System) 2007 (Kaysville, Utah, USA) software. The study data were analyzed using mean, standard deviation, median, frequency, ratio, minimum, and maximum values. Normality of distribution was evaluated by means of the Shapiro-Wilk test and graphical assessments. Student’s t-test was applied to compare normally distributed variables between the two groups, and the Mann-Whitney U test in case of non-normal distribution. ROC analysis and diagnosis screening tests were employed to calculate cut-off points by the groups. The score’s sensitivity and specificity characteristics were also calculated. P values <0.05 were considered statistically significant.
Results
One hundred seventy-two patients were enrolled in the study. CAM-ICU measurement revealed that 90.1% (n=155) of the patients were negative and 9.9% (n=17) positive for delirium. The age range of the subjects was 35-94 years, and the mean age was 72.94±13.99 years. The mean age (p=0.036) and the cerebrovascular event (CVE) incidence rate (p=0.036) of the patients who were positive on CAM-ICU were statistically significantly higher than those who were negative. No statistically significant difference was found between the groups in terms of the other variables (p>0.05) (Table 1).
Patients who were positive on the CAM-ICU required statistically significantly more physical restriction (p=0.07) and developed more pressure ulcers (p=0.001) than those who were negative. Statistically significantly larger amounts of sedatives were required for patients who were negative on the CAM-ICU than those who were positive (p=0.017). A statistically significant difference was found between the groups in terms of the form of discharge from intensive care (p=0.047) and those who died were usually negative while those who were transferred were usually positive. No statistically significant difference was found between the groups in terms of the other variables (p>0.05) (Table 2).
The 1st PRE-DELIRIC score of the patients who were negative on CAM-ICU was 3.9-11.9 with a mean value of 7.85±1.91. The 1st PRE-DELIRIC score of the patients who were positive for CAM-ICU was 6.7-14.9 with a mean value of 9.04±2.09 (Figure 1). A statistically significant difference was found between the 1st PRE-DELIRIC scores of the CAM-ICU negative and positive groups (p=0.017), and the scores of the positive patients were higher than the negative patients. An increase of one unit in the PRE-DELIRIC scores increased the risk of CAM-ICU positivity 1.358 times [odds ratio (OR): 1.358; 95% confidence interval (CI): 1.047-1.761] (Table 3).
Determining the Cut-off Point for PRE-DELIRIC Scores Based on CAM-ICU Status
A statistically significant difference was found between the 1st PRE-DELIRIC scores of the CAM-ICU negative and positive patients (p=0.017), with the CAM-ICU positive patients having higher scores (Table 3). Based on this significance, the cut-off point for the PRE-DELIRIC score was calculated. ROC analysis and diagnostic screening tests were used to determine this cut-off point by group. The cut-off point for the PRE-DELIRIC score was 7.58. This PRE-DELIRIC score cut-off value exhibited sensitivity of 88.24%, specificity of 49.68%, a positive predictive value of 16.13%, a negative predictive value of 97.47%, and accuracy of 53.49% (Table 4). The area under the ROC curve was 64.9% (95% CI: 0.538-0.760) and the standard error was 5.6% (p=0.044) (Figure 2).
The cut-off point determined for the PRE-DELIRIC score (≥7.58%) was significantly associated with the groups (p=0.003). The risk of CAM-ICU positivity was 7.404 times higher among individuals with PRE-DELIRIC scores of 7.58 or above (OR: 7.404; 95% CI: 1.638-33.469). An increase of 5% or more in the PRE-DELIRIC score exhibited sensitivity of 100%, a specificity of 6.45%, a positive predictive value of 10.49%, a negative predictive value of 100%, and accuracy of 15.70%. An increase of 10% of more in the PRE-DELIRIC score exhibited sensitivity of 29.41%, a specificity of 29.41%, a positive predictive value of 17.85%, a negative predictive value of 91.67%, and accuracy of 79.65%.
Discussion
We investigated the applicability of the PRE-DELIRIC model in intensive care, and the sensitivity of the model in predicting delirium with various physical and medical parameters compared with the CAM-ICU, the current gold standard.
The rate of development of delirium in this study was 9.9%. The reported delirium rate in ICUs ranges widely, from 10% to 80%, and it is important to obtain a measurement that can easily detect delirium, since this is vital for follow-up, treatment, and care (19-21). Delirium development is affected by many factors, and its prediction and prevention will make it possible to reduce the rate of disorders capable of leading to mortality and morbidity (14). There is evidence that delirium increases the length of hospital stay, exacerbates the risk of transmission of in-hospital infections, and puts the patient at risk of pressure ulcers and injuries (22-24).
CAM-ICU exhibited sensitivity of 88.24% (95% CI: 0.538-0.760) in this study and specificity of 85.16%. In their prospective, observational study, Guenther et al. (25) determined a risk of delirium of 19.8%, with CAM-ICU exhibiting sensitivity of 71% [(CI) 44-90%] and specificity of 100%, while another study reported a risk of delirium development of 25.2%, and CAM-ICU sensitivity of 100% [(CI) 92-99%] and specificity of 98% (6). A meta-analysis of nine separate studies concluded that CAM-ICU exhibited 80.0% sensitivity (95% CI: 77.1% and 82.6%) and 95.9% specificity (95% CI: 94.8% and 96.8%) (26). These inconsistencies may be due to variations in sampling and patient diagnoses.
CAM-ICU-positive cases exhibited higher scores than negative cases in this study. Patients with PRE-DELIRIC scores of 7.58 or more were 7.404 times more at risk of CAM-ICU-positivity (OR: 7.404; 95% CI:1.638-33.469). Studies elsewhere in the literature have also noted that PRE-DELIRIC and CAM-ICU scores are superior in identifying delirium, and that PRE-DELIRIC scores are particularly important on account of their simplicity, reliability, and rapid calculation (10,27). Studies have also observed high PRE-DELIRIC scores among individuals with positive CAM-ICU values (28,29). Similarly in the present study, a positive correlation was determined between CAM-ICU and PRE-DELIRIC.
The relevant factors in patients developing delirium according to the CAM-ICU in the present study were age, previous CVE, being physically restrained, presence of a pressure ulcer, and the form of intensive care discharge. Delirium was found to develop more commonly in elderly patients, those under physical restraint, in patients with a history of CVE, and in those with pressure ulcers. The predisposing factors reported to be related to delirium in the literature are similar to those found in the present study (12,30,31). Alcohol abuse, a history of dementia, hypertension, sedation, a high APACHE-II score, mechanical ventilation, and metabolic acidosis have also been described as factors exacerbating the risk of development of delirium in other studies (4,12,30,31). Since we only included intensive care patients with no history of chronic alcohol abuse, delirium or dementia, and no communication problems, and with a RASS score of +4 to -2 in this study, we may have been unable to detect all predisposing factors. However, the question of whether the PRE-DELIRIC model should not be taken into account in individuals with histories of dementia or misuse of alcohol and those who may have significant risk factors for delirium is a controversial one. van den Boogaard et al. (10) excluded groups of patients with a history of alcohol abuse and dementia from their PRE-DELIRIC regression model due to the low prevalence rate and reported that preventive measures can be taken directly instead of predicting the delirium risk, since the present evidence shows that these patients are already at a high risk of delirium. This has been criticized as a deficiency of the PRE-DELIRIC model by many researchers (4,31-33).
The PRE-DELIRIC score is easily applied and uses objective data, without the need for the patient to be conscious. This score can also be used as an important screening tool in detecting delirium in patients who are unable to communicate. Based on the findings of this study, we suggest that the PRE-DELIRIC score can be usefully employed in determining the risk of development delirium among patients in the ICU since it is easy to use and calculate and can make a useful contribution to clinical practice.
This study cannot be generalized to the general population since it was conducted at a single center within a limited time frame. The number of patients included in the study was also quite low. In addition, the inclusion of only conscious and communicating patients in the ICU, in which delirium evaluation was not routinely performed, may have resulted in a lower incidence than usual in this study in which we observed an incidence rate at the lower limit of the range reported in the literature.
Conclusion
Routine use of the PRE-DELIRIC score will make it possible to safely and easily predicting the risk of delirium within the first 24 hours after admission to the ICU. Evidence-based literature support of whether the model provides a fully valid and reliable risk estimate will require its common use in intensive care patients in addition to further interventional and observational studies to decrease the risk of delirium. In addition, CAM-ICU can predict the presence of delirium, and the PRE-DELIRIC model is beneficial when making a preliminary prediction and evaluation. The PRE-DELIRIC score is convenient for determining the risk of delirium development in patients hospitalized in the ICU and connected to a mechanical ventilator. There is currently no suitable screening test for delirium diagnosis, especially in disorientated patients. The present delirium screening tests require evaluation using a subjective method, in other words by means of answers to questions put to conscious and communicating patients.
Ethics
Ethics Committee Approval: Kırklareli University Institute of Health Sciences Ethics Committee approval (decision no: 3, date: 11.10.2019) was obtained from the institution in which the study was carried out, in addition to consent from the patients who were included in the study with the permission of the relevant institution. Written permission for the use of the PRE-DELIRIC score was obtained by e-mail from van den Boogaard.
Informed Consent: Consent was obtained from the patients included in the study.
Peer-review: Externally peer-reviewed.
Authorship Contributions
Surgical and Medical Practices: S.S., Concept: S.S., V.Ö., Design: S.S., V.Ö., B.T., Data Collection and Process: S.S., A.A.S., Analysis or Interpretation: B.T., N.Ö., Literature Search: S.S., V.Ö., B.T., A.A.S., N.Ö., Writing: S.S., V.Ö., B.T., A.A.S., N.Ö.
Conflict of Interest: No conflict of interest was declared by the authors.
Financial Disclosure: The authors declared that this study received no financial support.
