Delirium is a common neurocognitive disorder among cancer patients in hospice and palliative care, and is characterized by disorders in attention and consciousness along with changes in cognition. Delirium occurs over a short period of time, and it tends to fluctuate during the day and coexist with disorders in memory, learning, and orientation, changes in language, distortions in perception, or perception-motor function disorder [1]. The prevalence of delirium among cancer patients in palliative care wards in South Korea and internationally is high, ranging from 41.8% to 67.3% [2-4].

Delirium causes a significant burden to patients, their families, and medical personnel. A meta-analysis on delirium and negative health outcomes among elderly patients found that patients with delirium were 2.41 and 1.95 times more likely to face hospitalization or death than patients without delirium [5]. In a study of cancer patients who received palliative care services, patients with delirium reported higher levels of depression, anxiety, loss of appetite, and nausea [6]. In addition, delirium increases the cost of treatment and decreases quality of life [7].

Making a proper assessment of delirium is important for determining the appropriate treatment [8]; however, there are various difficulties related to assessing delirium in the clinical field. A study on the assessment of patients for delirium among 601 intensive care unit (ICU) nurses found that, while delirium was experienced by 87% of patients, a majority of nurses did not assess delirium [9]. There are various reasons for this, but the primary reason is the limited awareness of the diagnostic criteria for delirium among medical personnel and a general lack of understanding and education about delirium [10,11]. In fact, it was found that primary referring teams were unable to diagnose 61% of cancer patients in a palliative care ward who had been diagnosed with delirium by palliative care specialists [12]. In another study, oncologists misdiagnosed 22 out of 60 patients with delirium with other disorders, including anxiety and acute psychosis [13]. Moreover, the time and workload burden on medical personnel [10,11] and the lack of consensus regarding guidelines on which tool should be used in a palliative care context often prevent a proper assessment of delirium [11,14].

Therefore, in order to effectively identify and assess delirium, a review of the existing screening or assessment tools for delirium is needed, particularly concerning their psychometric properties. A previous review of delirium assessment tools in the context of palliative care identified 26 screening or assessment tools. The Confusion Assessment Method (CAM), Memorial Delirium Assessment Scale (MDAS), Bedside Confusion Scale (BCS), Communication Capacity Scale (CCS), and Agitation Distress Scale (ADS) have been validated for patients in hospice and palliative care [15]. The CAM and MDAS are the most common tools for assessing delirium in cancer or palliative care patients [14,16]. The CAM is a tool developed for the rapid detection of delirium and includes a diagnostic algorithm. For patients at geriatric centers and elderly patients in general wards, the CAM demonstrated high interrater reliability (κ=0.81~1.0), face and content validity, convergent validity (Mini-Mental State Examination [MMSE]: κ=0.59~0.82), and discrimination (sensitivity=94~100%, specificity=90~95%) [17]. The MDAS is a tool that evaluates the severity of delirium, and it has been validated for use with cancer patients and AIDS patients at cancer clinics. The MDAS had high internal consistency (α=0.91), interrater reliability (r=0.92), and concurrent validity (DRS: r=0.88, MMSE: r=-0.91) in addition to good discrimination with a cutoff score of 13 (sensitivity=70.59%, specificity=93.75%) [18].

Even though screening or assessment tools have been identified that can be used in various clinical contexts including hospice and palliative care [14-16], few South Korean or international studies have validated and examined the psychometric properties of screening or assessment tools among cancer patients in hospice and palliative care settings. Moreover, since most previous reviews have been limited to studies published in English [14-16], they did not include studies with cancer patients in hospice and palliative care settings in South Korea.

This study aimed to provide information to support the selection of the appropriate screening and assessment tools for measuring delirium in cancer patients in hospice and palliative care settings by conducting a systematic review of South Korean and international studies and examining the psychometric properties of the identified screening tools.

1. Literature search and selection

This study was conducted according to the Preferred Reporting Items for Systematic Review and Meta-Analysis guidelines [19]. International and South Korean databases including PubMed, CINHAL, KoreaMed, and RISS were searched for literature published no later than December 31, 2020. The main search terms were “delirium,” “assessment tool,” and “hospice/palliative care” in addition to “cancer”, and they were connected using Boolean operators (such as “AND,” “OR,” and “NOT”). The specific search strategy is shown in Supplementary 1.

The titles and abstracts of the articles were reviewed according to the inclusion and exclusion criteria. After the title and abstract were reviewed, the final set of articles was selected after reviewing the full texts. Additional articles were added after reviewing the reference lists of the final set of articles and conducting a search of Google Scholar.

The inclusion criteria were a) articles on screening/assessment tools for delirium in cancer patients applied to hospice/palliative care, and b) articles published in English or Korean. The exclusion criteria were a) studies conducted in an intensive care setting, and b) case studies, qualitative studies, systematic reviews, and meta-analyses.

This study was conducted with the approval of the Institutional Review Board (IRB number: H-1809-105-974) of S hospital.

2. Quality assessment of literature

The systematic review and meta-analysis of diagnostic accuracy studies can produce heterogeneous results due to differences in the design and execution of studies. Therefore, to ensure accurate interpretation and integration of the study results, the quality of the chosen articles must be assessed. The quality of the articles in the present study was assessed using the revised Quality Assessment of Diagnostic Accuracy Studies (QUADAS-2). QUADAS-2 is a tool for evaluating the quality of diagnostic accuracy studies and evaluates 4 areas: patient selection, index tests, reference standards, and flow and timing. Each area is rated “low,” “high,” or “unclear” for the risk of bias and applicability concerns [20].

3. Data extraction

The author, publication year, delirium screening/assessment tool, language, study location, sample size, age of subjects, sex of subjects, and delirium diagnosis ratio were extracted from the final set of articles. Information on reliability, validity, and diagnostic accuracy were extracted from the studies that included psychometric properties. Information on diagnostic accuracy included cutoff score, sensitivity, specificity, area under the curve (AUC), positive predictive value (PPV), negative predictive value (NPV), positive likelihood ratio, and negative likelihood ratio. Internal consistency and reliability were evaluated as proper when the Cronbach’s α ranged from 0.70 to 0.95 and the weighted kappa was 0.70 or more. Criterion validity (concurrent and predictive validity) was evaluated as proper when the correlation with the gold standard was 0.70 or more. Diagnostic accuracy was considered good when the AUC was 0.70 or more [21].

1. General characteristics of the literature

A total of 704 articles were identified in the initial search of international and national databases, and 206 duplicates were removed. The titles and abstracts of the remaining 498 articles were examined for the inclusion and exclusion criteria. The full texts of the 255 articles that remained after the review of titles and abstracts were examined, and 74 articles were selected. Seven articles were added after examining the reference lists in the final set of articles, resulting in a total of 81 articles (Figure 1, Supplementary 2).

Figure 1. Flow chart of the search strategy and selection of articles.

All articles in the final selection were published between 1999 and 2020, and most were published after 2010 (n=62, 77.5%). Other than the studies about home hospice services, all studies were conducted in outpatient or inpatient (hospice, palliative care ward, and cancer centers) settings. Sample sizes ranged from 15 to 2,829 participants, and the proportion of male to female participants ranged from 37.0% to 86.7%. The ages of the participants ranged from 18 to 107 years, and a majority were elderly. The prevalence of delirium among cancer patients receiving palliative care services was 3.0%, while the prevalence among cancer patients in cancer clinics or palliative care wards was 100%, indicating a wide range in the prevalence of delirium [S1-S81].

2. Evaluation of study quality

The results of the quality evaluation of the 10 articles that reported psychometric characteristics (Table 1) showed high study quality (9/10, 90%). With regard to patient selection, the risk of bias was low and applicability concerns were either unclear or low for all studies. In all 10 studies, the risk of bias and applicability concerns related to the index test and reference standards were either unclear or low. Finally, in 9 studies, the risk of bias and applicability concerns related to flow and timing were either unclear or low. However, a study in which the accuracy of the Delirium Observation Screening Scale (DOS) was evaluated did not conduct a reference standard test for all participants and did not include all participants in the analysis [1], creating a high risk of bias in terms of flow and timing. It is possible that the characteristics of the participants included in the analysis were different from those of the participants who were not included [20]. Therefore, caution is required when interpreting the psychometric properties of the DOS.

Table 1 . Quality Assessment of the Included Studies..

No.	First author and year (reference)	Risk of bias					Applicability concerns
		Patient selection	Index test(s)	Reference standard	Flow and timing		Patient selection	Index test(s)	Reference standard
1	Kang 2019 (S2)	○*	?	○	○		○	○	○
2	Barahona 2018 (S5)	○	○	?	○		○	○	○
3	Lawlor 2000 (S6)	○	?	○	○		○	○	○
4	Noguera 2014 (S3)	○	○	○	○		○	○	○
5	Grassi 2001 (S4)	○	○	○	○		○	○	○
6	Morita 2001 (S10)	○	?	○	○		○	?	○
7	Neefjes 2019 (S1)	○	○	○	×		○	○	○
8	Shim 2020 (S7)	○	○	○	○		○	○	○
9	Benītez-Rosario 2013 (S9)	○	○	?	?		?	○	?
10	de la Cruz 2015 (S8)	○	?	○	?		○	○	○

○: Low risk, ×: High risk, ?: Unclear risk..

3. Screening and assessment tools for delirium

Across the 81 articles, 16 screening and assessment tools for delirium were identified (Table 2, Supplementary 3). The MDAS was the most commonly used tool for assessing delirium in cancer patients in hospice and palliative care (n=47), followed by the CAM (n=19), the Delirium Rating Scale-Revised 98 (DRS-R-98; n=12), and Nursing Delirium Screening Scale (Nu-DESC; n=9). Across the 10 studies that examined psychometric properties, 8 tools were identified (Table 3), and the psychometric properties of the MDAS were validated most frequently (n=5) [S1-S81].

Table 2 . Characteristics of the Included Tools..

Assessment tool (reference)	Korean version (reference)	No. of items	Score	Score range	Cutoff	Completion time	No. of studies	Notes
MDAS (18)	K-MDAS (S2)	10	0~3	0~30	13	10 min	47	- K-MDAS cutoff: 9.
CAM (17)	-	9	-	-	-	5 min	19	- Items reflect DSM-Ⅲ diagnostic criteria for delirium. - Delirium diagnosis is based on diagnostic algorithm.
DRS-R-98 (S84)	DRS-R98-K (S85)	16	0~2;0~3	0~46	15.25;17.75	20~30 min (15)	12	- Revised version of the DRS. - DRS-R98-K cutoff: 18.5 & 19.5. - Total score range of severity items: 0~39.
Nu-DESC (S86)	Korean Nu-DESC (S87)	5	0~2	0~10	1	1 min	9	- Korean Nu-DESC cutoff: 2. - Total score range of Korean Nu-DESC: 0~5.
DRS (S82)	K-DRS (S83)	10	0~2;0~3;0~4	0~32	-	-	5	- Items are derived from DSM-Ⅲ diagnostic criteria and previous studies on delirium. - K-DRS cutoff: 16.5.
RASS (S87)	-	1	-5~+4		-	<15 sec	4
ADS (S10)	-	6	0~3	0~18	-	-	4
CCS (S10)	-	5	0~3;0~5	0~17	-	-	3
DOS (S91)	-	13	0~1	0~13	3	<5 min	3	- Modified version based on clinical studies.
DMSS (S81)	K-DMSS (S93)	11	0~1	0~11	-	-	2	- Shortened version of the DMC. - Hyperactivity is defined as the presence of at least 2 responses indicating hyperactivity. - Hypoactivity is defined as the presence of at least 2 responses indicating hyperactivity.
CRS (S95)	-	4	0~2	0~8	1	-	2
CTD (S96)	-	5	0~6	0~30	18	10~15 min	2
BCS (S94)	-	2	0~1;0~4	0~5	2	-	1
DMC (S92)	-	30	0~1	0~30	-	-	1
RASS-PAL (S89)	-	1	-5~4	-5~4	-	-	1	- RASS modified for palliative care settings.
SDC (S97)	-	10	0~4	0~40	-	-	1	- Items are derived from the DSM-Ⅲ diagnostic criteria for delirium. - Lower scores indicate more severe delirium.

ADS: Agitation Distress Scale, BCS: Bedside Confusion Scale, CAM: Confusion Assessment Method, CCS: Communication Capacity Scale, CRS: Confusion Rating Scale, CTD: Cognitive Test for Delirium, DMC: Delirium Motoric Checklist, DMSS: Delirium Motor Subtype Scale, DOS: Delirium Observation Screening Scale, DRS: Delirium Rating Scale, DRS-R-98: Delirium Rating Scale-Revised-98, DRS-R98-K: Korean Version of the Delirium Rating Scale-Revised-98, Korean Nu-DESC: Korean Nursing Delirium Scale, Distress Scale, K-DMSS: Korean version of the Delirium Motor Subtype Scale, K-DRS: Korean Version of the Delirium Rating Scale, K-MDAS: Korean version of Memorial Delirium Assessment Scale, MDAS: Memorial Momentary Assessment Scale, Nu-DESC: Nursing Delirium Screening Scale, RASS: Richmond Agitation-Sedation Scale, RASS-PAL: Richmond Agitation-Sedation Scale-Palliative Version, SDC: Saskatoon Delirium Checklist..

Table 3 . Psychometric Properties of the Tools..

No.	Assessment tool (reference)	Reliability			Validity			Diagnostic accuracy
		Internal consistency	Inter-rater reliability		Structural validity	Concurrent validity		Cutoff	Sensitivity (%)	Specificity (%)	AUC	PPV (%)	NPV (%)	PLR (%)	NLR (%)
1	MDAS (S3)	α=0.82	r=0.95			- DRS-R-98: ρ=0.80. - MMSE: ρ=-0.74.
2	MDAS (S5)							7	92.9	71.8	0.93	70.2	93.9
3	MDAS (S4)	α=0.89			2 factor	- DRS: r=0.76. - MMSE: r=-0.88.		13	68	94		95	63
4	MDAS (S6)		r=0.89		2 factor	- MMSE: r=0.55.		7	97	95				19.4	0.03
5	K-MDAS (S2)	α=0.94	r=0.99			- DRS-R-98: r=0.96.		9	96	92	0.97	79	99
6	DRS (S4)	α=0.70			3 factor	- MDAS: r=0.76. - MMSE: r=-0.67.		10	95	61		80	89
								12	80	76		85	69
7	DRS-R-98 (S7)	α=0.87~0.90	ICC=0.98~0.99		2 factor
8	Nu-DESC (S8)							MDAS>7	63	67		61	68
9	RASS (S9)		χ=0.85~0.95			- Ramsay Sedation Scale: ρ=0.81~0.89. - Glasgow Coma Scale: ρ=0.81~0.85.
10	ADS (S10)	α=0.91	χ=0.78~0.95			- DRS: ρ=0.61.
11	CCS (S10)	α=0.96	χ=0.78~0.95			- Modified MDAS: ρ=0.78. - DRS: ρ=0.44. - Sedation Scale: ρ=0.86.
12	DOS (S1)							3	>99.9	99.6		100	65

AUC: Area under the ROC curve, NLR: Negative Likelihood Ratio, NPV: Negative Predictive Value, PLR: Positive Likelihood Ratio, PPV: Positive Predictive Value..

1) Memorial Delirium Assessment Scale (MDAS)

The MDAS is a tool that was developed by Breitbart et al. in 1997 to evaluate the severity of delirium in patients with cancer and AIDS based on the Diagnostic and Statistical Manual of Mental Disorders-IV (DSM-IV) diagnostic criteria for delirium. It includes 10 items for measuring patients’ arousal, level of consciousness, cognitive function, and psychomotor function. Each item is scored on a scale of 0 (none) to 3 (severe), and the cutoff score is 13 [18]. The Korean version of the MDAS (K-MDAS) was validated in a study of 102 cancer patients in a Korean palliative care ward, and the cutoff score was 9 [S2].

Internal consistency and interrater reliability were .82 and .95 in a study of 85 cancer patients in a palliative care ward in Spain, demonstrating high reliability. The correlations with the DRS-R-98 and MMSE were .80 and -0.74, respectively, indicating good concurrent validity [S3]. The results of a study of 105 terminal cancer patients at an Italian cancer and palliative care ward indicated good internal consistency for the MDAS at .89. A two-factor structure of vigilance and attention disorders and psychotic symptoms was identified, but the construct validity could not be evaluated as the fitness was not reported. The correlations with the DRS and MMSE were 0.76 and -0.88, respectively, showing high concurrent validity, and the discrimination was acceptable with a cutoff score of 13 (sensitivity=68%, specificity=94%, PPV=95%, NPV=63%) [S4]. The Spanish version of the MDAS was validated in a study of 67 cancer patients at a palliative care ward and showed good discrimination with a cutoff of 7 (sensitivity=92.9%, specificity=71.8%, PPV=70.2%, NPV=93.9%) and high diagnostic accuracy (AUC=0.93) [S5]. A study of 104 cancer patients in a palliative care ward found good concurrent validity (MMSE: r=0.55) and high discrimination with a cutoff of 7 (sensitivity=97%, specificity=95%) [S6]. When validating the K-MDAS with cancer patients in a Korean palliative care unit, good internal consistency (α=0.94), interrater reliability (r=0.99), and concurrent validity (DRS-R-98: r=0.96) were found. Moreover, the K-MDAS showed high discrimination with a cutoff of 9 (sensitivity=96%, specificity=92%, PPV=79%, NPV=99%) and good diagnostic accuracy (AUC=0.97) [S2].

2) Confusion Assessment Method (CAM)

The CAM, developed by Inuouye et al. in 1990 [17], was developed to help clinicians who are not psychiatrists rapidly detect delirium in elderly patients. The CAM consists of 9 items based on the DSM-III-R (acute onset and changes in severity, inattention, disorganized thinking, changes in levels of consciousness, disorientation, memory impairment, perceptual disturbances, psychomotor agitation or retardation, and changes in the sleep-wake cycle). The diagnostic algorithm includes acute onset and changes in severity, inattention, disorganized thinking, and changes in levels of consciousness, and delirium is diagnosed when acute onset and changes in severity and inattention are present and there are changes in the level of disorganized thinking or consciousness. The interrater reliability (κ=0.81~1.0) and concurrent validity (MMSE: κ=0.59~0.82) were good.

3) Delirium Rating Scale (DRS)/Delirium Rating Scale-Revised 98 (DRS-R-98)

Trzepacz et al. [S82] developed the DRS in 1998 in a study of 20 patients diagnosed with delirium. It consists of 10 items based on the DSM-III and existing literature (temporal onset of symptoms, perceptual disturbances, hallucination type, delusions, psychomotor behavior, cognitive status during formal testing, physical disorder, sleep-wake cycle disturbance, lability of mood, and variability of symptoms), and each item is scored 0~2, 3, or 4. The Korean version of the DRS (K-DRS) was validated in study of 104 patients who were hospitalized at a psychiatric ward and had been diagnosed with delirium, and the cutoff was 16.5 [S83].

In a study of 105 patients at an Italian cancer and palliative care ward, the tool had adequate internal consistency (α=70), and a 3-factor structure was identified. When the cutoff was set at greater than 10 or 12, the discrimination was found to be good (sensitivity=80~95%, specificity=61~76%, PPV=80~85%, NPV=69~89%) [S4].

Developed by Trzepacz et al. in 2001 [S84], the DRS-R-98 is a tool that was revised using data from hospitalized patients in teaching hospitals and long-term care hospitals that addressed the limitations of the original DRS. It consists of 13 items on severity (sleep-wake cycle, perceptual disturbance, delusions, lability of affect, language, thought process, motor agitation and retardation, orientation, attention, short-term and long-term memory, and visuospatial ability) and 3 diagnostic items (temporal onset of symptoms, fluctuation of symptom severity, physical disorder). Each item is scored from 0~2 or 3, and discrimination was good at cutoffs of 15.2 and 17.75. The Korean version of the DRS-R-98 (DRS-R98-K) was validated with patients in a general ward. DRS-R98-K had acceptable discrimination at cutoffs of 18.5, 19.5, and 20.5 [S85].

In a Korean study of 93 cancer patients in a palliative care ward, internal consistency (α=0.87~0.90) and interrater reliability (ICC=0.98~0.99) were high, and the two-factor structure of core and noncore symptoms of delirium was found to be adequate [S7].

4) Nursing Delirium Screening Scale (Nu-DESC)

Gaudreau et al. [S86] developed the Nursing Delirium Screening Scale (Nu-DESC) in 2005 in a study of 146 patients receiving treatment in hematology and oncology or internal medicine wards. Psychomotor retardation were added to the 4 Confusion Rating Scale (CRS) items (disorientation, inappropriate behavior or communication, illusions/hallucinations) to identify patients with hypoactive delirium. Each item is scored from 0 to 2, and the optimum cutoff is 1. The Korean version of the Nu-DESC was validated in a study of elderly patients. Each item on the Korean Nu-DESC receives a score of either 0 (no) or 1 (yes) according to the pilot study and the suggestions of specialists during the tool’s development, and it demonstrated good discrimination with a cutoff of 2 [S87].

Compared to the study that developed the original scale [S86], the sensitivity and specificity were 63% and 67%, respectively, for cancer patients receiving home hospice care, demonstrating relatively low discrimination [S8].

5) Richmond Agitation-Sedation Scale (RASS)/Richmond Agitation-Sedation Scale Modified for Palliative Care Inpatients (RASS-PAL)

The Richmond Agitation-Sedation Scale (RASS) is a single-item scale that was developed by Sessler et al. in 2002 to evaluate the level of agitation and sedation of ICU patients. Four points indicates that the patient made aggressive behavior and threats toward medical personnel, 3 points indicates that the patient pulled or removed catheters or tubes or displayed aggression, 2 points indicates that the patient engages in frequent nonpurposeful behaviors or acts of resistance such as refusing a ventilator, and 1 point indicates that the patient is anxious without showing aggressive behaviors. The ability to open one’s eyes while listening and maintaining eye contact for more than 10 seconds (i.e., the patient can be awake even if he or she is not fully alert) is given a score of -1 point, the ability to awaken briefly when hearing a voice but being unable to maintain eye contact for more than 10 seconds is given a score of -2 points, and the ability to move or open one’s eyes in response to a voice but being unable to maintain eye contact is given a score of -3 points. The ability to move or open one’s eyes in response to physical stimuli but being unable to react when hearing someone’s voice is given a score of -4 points, and the inability to respond to a voice or physical stimuli is given a score of -5 points [S88].

In a study of 156 cancer patients in a palliative care ward, high interrater reliability was found (κ=0.85~0.95). Face validity of the RASS was confirmed by the hospice and palliative care team, and the RASS was found to have high concurrent validity (Ramsay Sedation Scale: ρ=0.82~0.89, Glasgow Coma Scale: ρ=0.81~0.85) [S9].

The Richmond Agitation-Sedation Scale Modified for Palliative Care Inpatients (RASS-PAL) is a scale that was revised by Bush et al. in 2014 [S89] to adapt the RASS for use in palliative care contexts. Detailed descriptions such as “acts of resistance such as removing tubes and refusing a ventilator” that did not apply to the context of palliative care were removed. Patients are observed for 20 seconds and rated from 0 to 4 according to the level of consciousness or agitation. When patients are not conscious, their response to voice or physical stimulation is observed and evaluated on a scale of –1 to -5 according to the level of sedation.

6) Agitation Distress Scale (ADS)

The ADS was developed to measure agitation distress (such as behaviors or emotional hyperactivity that could cause pain) in a study of 30 terminal cancer patients who were diagnosed with delirium in a palliative care ward. The ADS includes 6 items on agitation, psychological instability, hallucination and delusions, and sleep disturbance. The items are scored from 0 to 3.

The ADS had high internal consistency (α=0.91) and interrater reliability (κ=0.72~1.00) in a study of cancer patients in a palliative care ward. Good concurrent validity was demonstrated by a correlation with the DRS of 0.61, but correlations with the MDAS (and modified MDAS) and Sedation Scale were not significant [S10].

7) Communication Capacity Scale (CCS)

The CCS was developed to measure the degree of communication capacity (i.e., the ability to understand the circumstances and express one’s intentions accurately) in patients in a study of 30 cancer patients in a palliative care ward. It consists of 5 items scored from 0~3 or 0~5.

In the study in which the original scale was developed, the internal consistency (α=0.96), interrater reliability (κ=0.78~0.95), and concurrent validity (modified MDAS: ρ=0.78, DRS: ρ=0.44, Sedation Scale: ρ=0.86) were all good [S10].

8) Delirium Observation Screening Scale (DOS)

The DOS is based on the DSM-IV and was developed by Schuurmans et al. in 2003 [S90] to identify delirium early in elderly patients with hip fractures. It initially contained 25 items but was reduced to 13 items based on the study of elderly patients with hip fractures. Each of the 13 items on the DOS is given a score of either 0 (no) or 1 (yes), and the optimum cutoff is 3 [S91].

When the discrimination was tested in a study of 187 cancer patients in hospice, university cancer centers, and oncology departments at university hospitals, the sensitivity, specificity, PPV, and NPV were 99.9%, 99.5%, 100%, 65%, respectively, with a cutoff of 3, demonstrating good discrimination [S1].

9) Delirium Motoric Checklist (DMC)/Delirium Motor Subtype Scale (DMSS)

The Delirium Motoric Checklist (DMC) is a tool for categorizing the subtypes of delirium that contains 30 items (21 items on hyperactivity and 9 items on hypoactivity) based on the psychomotor symptoms in previous studies. Each item is given a score of either 0 (no) or 1 (yes) [S92].

Meagher et al. [S81] extracted 11 items from the 30 items in the DMC to develop the Delirium Motor Subtype Scale (DMSS) in 2008, and each item is given a score of 0 (no) or 1 (yes). When more than 2 of 4 items related to hyperactivity (increased quantity of motoractivity, loss of control of activity, restlessness, wandering) are present, the patient is categorized as hyperactive. When more than 2 of 7 items related to hypoactivity (decreased amount of activity, decreased speed of action, reduced awareness of surroundings, decreased amount of speech, decreased speed of speech, listlessness, reduced alertness/withdrawal) are present, the patient is categorized as hypoactive. The Korean version of the Delirium Motor Subtype Scale (K-DMSS) was validated in a study of 145 elderly patients who received joint care from the departments of internal medicine and psychiatry [S93].

10) Bedside Confusion Scale (BCS)

The BCS is a scale developed by Stillman and Rybicki [S94] for measuring confusion in palliative care patients and assesses the level of consciousness, ability to complete an attention task (reciting the months backward from December), and a diagnostic algorithm for each patient. When assessing a patient’s level of consciousness, 0 points indicates a normal level of consciousness, and 1 point indicates hypoactivity or hyperactivity. When assessing the ability to perform attention tasks, 0 points indicates successful completion of a task within 30 seconds, 1 point indicates accurate completion of the task in more than 30 seconds due to hesitation or delays. The patient receives a score of 1 if 1 month is omitted, a score of 2 if 2 months are omitted, and a score of 3 if 3 or more months are omitted. If the patient cannot complete the task, a score of 4 is given. High discrimination was observed, with a cutoff of 2.

11) Confusion Rating Scale (CRS)

The CRS was developed based on a previous study of acute confusion among elderly patients with hip fractures by Williams [S95]. The CRS consists of 4 items (orientation to time, place, and/or person; inappropriate behavior; inappropriate communication; and hallucinations). Each item is scored from 0 (not present) to 2 (present, and pronounced), and the optimum cutoff is 1.

12) Cognitive Test for Delirium (CTD)

The CTD is based on DSM-III-R and previous studies, and it was validated by Hart et al. [S96] in a study of 22 patients with delirium at an ICU. It examines 5 areas (orientation, attention span, memory, comprehension/conceptual reasoning, and vigilance). The level of function in each area is rated on a scale of 0 and 6, and the cutoff was 18.

13) Saskatoon Delirium Checklist (SDC)

The Saskatoon Delirium Checklist (SDC) was developed by Miller et al. [S97] based on the DSM-III and includes 10 items on decreased clarity of consciousness, perceptual disturbance, incoherent speech, sleep disturbance, motor behavior slowing and agitation, disorientation, memory problems, fluctuation of above symptoms over time, and physical causes of symptoms (such as physical examinations or medications). Each item is given a score of 0 to 4.

This study examined the screening and assessment tools used for cancer patients in hospice and palliative care contexts and their psychometric properties. Sixteen tools were identified across 81 studies, and 10 articles examined the psychometric properties of 8 tools.

The psychometric properties of the MDAS were the most frequently validated (n=5) in studies about cancer patients in palliative care. The MDAS was developed to assess the delirium symptom severity of patients with cancer or AIDS, but it can be useful for evaluating delirium due to its high discrimination [18]. In fact, both international and domestic studies of cancer patients in palliative care wards found high internal consistency (α=0.82~0.94), interrater reliability (r=0.89~0.99), and concurrent validity [S2-S6]. Good discrimination (sensitivity=68~97%, specificity=71.8~95%, PPV=70.2~95%, NPV=63~99%) and diagnostic accuracy (AUC=0.93~0.97) [S2,S4-S6] also indicate the usefulness of the MDAS for evaluating delirium in cancer patients in palliative care contexts. Compared to the time it takes to complete the DRS-R-98 (approximately 20~30 minutes) [15], the duration of the MDAS is relatively short at approximately 10 minutes [18], which does not pose as much of a time or workload burden for medical personnel. However, the lack of standardized training programs and manuals can make implementation difficult. Moreover, a consensus has not been reached regarding the cutoff for diagnosing delirium, and the optimum cutoff for diagnosing delirium in cancer patients in palliative care settings needs to be validated. While the optimum cutoff was 13 in the study in which the original scale was developed and the study in which the Italian version of the MDAS was validated [18,S4], the cutoff was 7 in the English [S6] and Spanish versions [S5]. The cutoff for the K-MDAS was 9 [S2] and it excluded patients with dementia [S2], while the English and Spanish versions of the MDAS did not [S5,S6]. As such, the differences in cutoffs could reflect differences in the demographic characteristics of the study participants. The Italian version of the MDAS did not consider its application in palliative care contexts and had a cutoff of 13, like the study in which the original scale was developed for patients with cancer or AIDS at a cancer center [S4]. Therefore, the cutoff of the Italian version of the MDAS might not be the actual optimum cutoff.

The DRS and DRS-R-98 are the most frequently used assessment tools for measuring delirium in clinical practice [16] and are especially useful for measuring symptom severity [11]. In South Korea, the K-DRS [S83] and DRS-R98-K [S85] were validated in a study of 104 patients with delirium who were admitted to psychiatric units as well as 157 patients in a general ward. The results of previous studies have suggested that the DRS/DRS-R-98 can be useful for assessing delirium in terminal cancer patients in palliative care. For example, a study of cancer patients in cancer or palliative care wards in Italy reported good internal consistency (α=0.89) and discrimination (sensitivity=80~95%, specificity=61~76%, PPV=80~85%, NPV=69~89%) with DRS scores of 10 or 12 [S4]. The DRS-R-98 also showed high reliability in a South Korean study of cancer patients in a palliative care ward [S7]. However, the length of time it takes to complete the DRS-R-98 is relatively long [15], and it requires training sessions to administer [11]. The Italian version of the DRS tested the discrimination of the cutoff found in a previous study of elderly patients in a psychiatric ward without considering the context and participant demographics [S4]. Therefore, it might not reflect the actual optimal cutoff of the DRS. While the reliability of the DRS-R-98 was confirmed, its diagnostic accuracy was not validated. In the future, the discrimination of the DRS-R-98 for assessing delirium in cancer patients in hospice and palliative care contexts should be assessed, both internationally and in South Korea.

The Nu-DESC is a tool for nurses to identify delirium in patients. The Nu-DESC takes less than 1 minute to administer, and delirium is assessed through observation during a work shift, which makes it easy to perform during a busy workday [S86]. Since it was validated in a study of elderly patients in South Korea, it can be used to identify delirium in cancer patients in palliative care contexts in South Korea. However, while the sensitivity and specificity were 85.7% and 86.8%, respectively, for scores of greater than 1 in the study in which the original scale was developed [S86], the sensitivity and specificity were 63% and 67%, respectively, in a study of cancer patients in hospice, which are relatively low [S8]. This result suggests that the Nu-DESC may be limited in its ability to identify delirium in cancer patients in palliative care. Therefore, it should be used alongside tools with high sensitivity, such as the MDAS, for cancer patients in palliative care [S8].

In addition, while the psychometric properties of the tool were not validated for cancer patients in palliative care, the CAM was the most frequently used delirium screening/evaluation tool in clinical practice [16], and it was useful for evaluating delirium among cancer patients in palliative care. Compared to the amount of time it takes to complete the DRS-R-98 (approximately 20~30 minutes) and the MDAS (approximately 10 minutes), the duration of the CAM is relatively short at less than 5 minutes and is as such more likely to be used [15]. When the psychometric properties of the tool were validated in a study of 52 patients in a palliative care ward, the discrimination was found to be high with 88% sensitivity and 100% specificity [22]. However, the CAM requires training, as it relies on the skills of the evaluator [17,22]. Moreover, the diagnostic accuracy of the CAM for cancer patients in palliative care has not been determined. Therefore, future studies should validate the psychometric properties of the CAM for terminal cancer patients in hospice and palliative care settings.

The results of the literature review in this study showed that the MDAS was the most frequently used tool for assessing delirium in cancer patients in hospice or palliative care settings both internationally and nationally, and its psychometric properties were good. While the prevalence of delirium among cancer patients in palliative care is high [2-4], only 10 studies have validated the psychometric properties of the screening/assessment tools used to assess delirium, only 2 of which were from South Korea. Moreover, even though the CAM is used frequently in the clinical field and has good discrimination, there have been few studies to validate the psychometric properties of the CAM for assessing delirium in cancer patients in palliative care settings, indicating a further need to validate the psychometric properties of various screening/assessment tools for delirium.

When interpreting the results of this review, the following limitations must be considered. First, most studies other than those on the MDAS either did not suggest a cutoff for diagnosing delirium [S7-S10] or validate the diagnostic accuracy of the cutoff from the studies in which the original tools were developed [S4]. Therefore, future studies should validate the optimal cutoff of each tool for accurately screening and assessing delirium. Second, for this systematic review, the process of the literature search was not reviewed by a librarian.

Despite these limitations, this study is meaningful since it provides helpful information for selecting the most appropriate tool for evaluating delirium in cancer patients in hospice and palliative care settings through its review of such screening/assessment tools in both international and domestic studies and their psychometric properties.

Supplementary materials can be found via https://doi.org/10.14475/jhpc.2021.24.4.214.

No potential conflict of interest relevant to this article was reported.

Conception or design of the work: EJS. Data collection: EJY. Data analysis and interpretation: EJY, EJS. Drafting the article: EJY, EJS. Critical revision of the article: BJH, EJS. Final approval of the version to be published: all authors.