Measuring outcomes in randomized clinical trials of insomnia treatments

doi:10.1053/smrv.2002.0274

Sleep Medicine Reviews

Volume 7, Issue 3, June 2003, Pages 263-279

https://doi.org/10.1053/smrv.2002.0274 Get rights and content

Abstract

Significant efforts have been made in the last decade to develop evidence-based guidelines for the treatment of insomnia and other sleep disorders. Despite such progress, there are still no standard assessment methods to document outcome and no accepted criteria to define what should be a successful outcome in the treatment of insomnia. This paper reviews methodological and conceptual issues related to the measurement of outcomes in clinical trials of insomnia. Selected studies of behavioral and pharmacological therapies of insomnia are summarized to illustrate the types of dependent variables and assessment instruments used to document treatment efficacy. Additional outcome variables and assessment methods of potential interest are discussed, and criteria for interpreting and reporting outcomes are summarized. As most studies have relied on fairly narrow criteria to define an effective treatment, the need to broaden the scope of outcome assessment is highlighted. For instance, it is essential to document treatment efficacy beyond the simple reduction of insomnia symptoms and to incorporate additional indicators of success. Given that insomnia is associated with significant morbidity, an effective treatment should not only improve sleep parameters, but it should also produce clinically meaningful changes in daytime functioning, fatigue, mood, and quality of life. The need to evaluate outcome from multiple perspectives and to develop a core-assessment battery that would consider efficacy, clinical significance, and cost-effectiveness are discussed.

References (100)

DE Ford et al.
Epidemiologic study of sleep disturbances and psychiatric disorders: An opportunity for prevention?
JAMA
(1989)
F Hohagen et al.
Prevalence and treatment of insomnia in general practice: A longitudinal study
Eur Arch Psy Clin Neuro Sci
(1993)
MM Ohayon
Epidemiology of insomnia: what we know and what we still need to learn
Sleep Med Rev
(2002)
G Simon et al.
Prevalence, burden, and treatment of insomnia in primary care
Am J Psychiatry
(1997)
MJ Sateia et al.
Evaluation of chronic insomnia. An American Academy of Sleep Medicine review
Sleep
(2000)
CM Morin et al.
Nonpharmacologic treatment of chronic insomnia
Sleep
(1999)
DL Chambless et al.
Defining empirically supported therapies
J Consult Clin Psychol
(1989)
J Backhaus et al.
Long-term effectiveness of a short-term cognitive-behavioral group treatment for primary insomnia
Eur Arch Psychiatry Clin Neurosci
(2001)
SR Currie et al.
Cognitive-behavioral treatment of insomnia secondary to chronic pain
J Consult Clin Psychol
(2000)
JD Edinger et al.
Cognitive behavioral therapy for treatment of chronic primary insomnia. A randomized controlled trial
JAMA
(2001)

CA Espie et al.

The clinical effectiveness of cognitive behaviour therapy for chronic insomnia: Implementation and evaluation of asleep clinic in general medical practice

Behav Res Ther

(2001)

GD Jacobs et al.

Home-based central nervous system assessment of a multifactor behavioral intervention for chronic sleep-onset insomnia

Behav Ther

(1993)

KL Lichstein et al.

Relaxation and sleep compression for late-life insomnia: A placebo-controlled trial

J Consult Clin Psychol

(2001)

R Cluydts et al.

Comparison of continuous versus intermittent administration of zolpidem in chronic insomniacs: A double-blind, randomized pilot study

J Int Med Res

(1998)

R Elie et al.

Sleep latency is shortened during 4 weeks of treatment with zaleplon, a novel nonbenzodiazepine hypnotic

J Clin Psychiatry

(1999)

G Hajak et al.

Doxepin in the treatment of primary insomnia: A placebo-controlled, double-blind, polysomnographic study

J Clin Psychiatry

(2001)

DF Kripke et al.

Sleep evaluation in chronic insomniacs during 14-day use of flurazepam and midazolam

J Clin Psychopharmacol

(1990)

M Scharf et al.

A multicenter, placebo-controlled study evaluating zolpidem in the treatment of chronic insomnia

J Clin Psychiatry

(1994)

E Van der Kleijn

Effects of zopiclone and temazepam on sleep, behaviour and mood during the day

Eur J Clin Pharmacol

(1989)

JK Walsh et al.

A five week, polysomnographic assessment of zalepan 10 mg for the treatment of primary insomnia

Sleep Med

(2000)

PJ Hauri

Can we mix behavioral therapy with hypnotics when treating insomniacs?

Sleep

(1997)

CM Morin et al.

Behavioral and pharmacological therapies for late-life insomnia. A randomized controlled trial

JAMA

(1999)

HY McClusky et al.

Efficacy of behavioral versus triazolam treatment in persistent sleep-onset insomnia

Am J Psychiatry

(1991)

CM Morin et al.

Nonpharmacological interventions for insomnia: a meta-analysis of treatment efficacy

Am J Psychiatry

(1994)

CM Morin et al.

Psychological management of insomnia: a clinical replication series with 100 patients

Behavior Therapy

(1994)

VK Kapur et al.

The relationship between chronically disrupted sleep and healthcare use

Sleep

(2002)

D Léger et al.

SF-36: Evaluation of quality of life in severe and mild insomniacs compared with good sleepers

Psychosom Med

(2001)

GK Zammit et al.

Quality of life in people with insomnia

Sleep

(1999)

CM Morin

Insomnia: Psychological assessment and management

(1993)

DE Moul et al.

Symptom reports in severe chronic insomnia

Sleep

(2002)

A Vignola et al.

Effects of chronic insomnia and use of benzodiazepines on daytime performance in older adults

J Gerontology: Psychological Sciences

(2000)

JE Broman et al.

Subjective and objective performance in patients with persistent insomnia

Scand J Behav Ther

(1992)

N Breslau et al.

Sleep disturbance and psychiatric disorders: a longitudinal epidemiological study of young adults

Biol Psychiatry

(1996)

DJ Buysse et al.

Clinical diagnoses in 216 insomnia patients using the international classification of sleep disorders (ICSD), DSM-IV and ICD-10 categories: a report from the APA/NIMH DSM-IV field trial

Sleep

(1994)

CM Morin et al.

Sleep and psychopathology

App and Prev Psychol

(1996)

JD Edinger et al.

Insomnia and the eye of the beholder: are there clinical markers of objective sleep disturbances among adults with and without insomnia complaints

J Consult Clin Psychol

(2000)

M Perlis et al.

Behavioral treatment of insomnia: a clinical case series study

J Beh Med

(2000)

EJ Stepanski et al.

Characteristics of individuals who do or do not seek treatment for chronic insomnia

Psychosomatics

(1989)

MM Gladis et al.

Quality of life: expanding the scope of clinical significance

J Consult Clin Psychol

(1999)

TM Gill et al.

A critical appraisal of the quality of quality-of-life measurements

JAMA

(1994)

JM Graf von der Schulenburg

Measuring the unmeasurable: the role and importance of quality of life measurement in economic evaluations of sleep disorder treatment

Eur Psychiatry

(1995)

CA Moyer et al.

Quality of life in obstructive sleep apnea: a systematic review of the literature

Sleep Med

(2001)

TE Weaver

Outcome measurement in sleep medicine practice and research. Part I: assessment of symptoms, subjective and objective daytime sleepiness, health-related quality of life and functional status

Sleep Medi Rev

(2001)

MH Bonnet et al.

Hyperarousal and insomnia

Sleep Med Rev

(1977)

QR Regestein et al.

Daytime alertness in patients with primary insomnia

Am J Psychiatry

(1993)

AN Vgontzas et al.

Chronic insomnia is associated with nyctohemeral activation of the hypothalamic-pituitary-adrenal axis: clinical implications

J Clin Endocrinol Metab

(2001)

A Kazdin

Research design in clinical psychology

(2003)

M Reite et al.

The use of polysomnography in the evaluation of insomnia

Sleep

(1995)

WK Wohlgemuth et al.

Stability of PSG measures in elderly insomniacs and normals both at home and in the lab

Psychophysiology

(1999)

JD Edinger et al.

Sleep in the laboratory and sleep at home: comparisons of older insomniacs and normal sleepers

Sleep

(1997)

Cited by (145)

Can cry tolerance be increased in mothers of infants with sleep problems, and why does it matter? A quasi-experimental study
2023, Sleep Medicine
Infant sleep problems are one of the most common complaints of new parents. Research to date has demonstrated a relationship between low parental cry tolerance and infant sleep problems. The aim of this study was to explore whether three emotion regulation strategies could increase parental cry tolerance.
This study utilised a quasi-experimental design. We recruited 83 females (Mage= 32, SD= 5.26) comprising 3 groups: mothers of good sleeping infants aged 6–24 months, mothers of poor sleeping infants aged 6–24 months, and good sleeping women aged 23–40 years without children. Participants were instructed to listen to crying audio segments and indicate when they felt the child needed to be tended to (measured in seconds). This crying audio was paired with one of three emotion-regulation strategies (i.e., music, gaming, reappraisal). Each participant completed all 4 cry conditions which were counterbalanced to control for order effects.
We found that all three strategies yielded significantly longer reaction times (indicating higher cry tolerance) compared to the control condition (p <.001). We also found that mothers of poor sleepers and good sleeping women benefitted from all three emotion regulation strategies compared to control (p <.001 and p= <.05, respectively). The cry tolerance of mothers of good sleepers, on the other hand, did not differ between the control condition and any of the strategies (all ps >.05).
This demonstrates that cry tolerance can be increased using emotion regulation strategies, such as distraction via music or gaming, and reappraisal. This has clinical implications for families implementing behavioural sleep interventions.
Sleep interventions for children with attention deficit hyperactivity disorder (ADHD): A systematic literature review
2023, Sleep Medicine
Citation Excerpt :
The effect size of TST was reported to increase over 30 min at 4 weeks follow-up for children using melatonin [57] and around 15 min at 3-month follow-up after a behavioral intervention [52], which is considered as a moderate effect, although with low evidence. This conclusion is based on research showing that an improvement in TST rarely exceeds 30 min in sleep interventions [66,67] but is likely to increase linearly over time [67]. Two RCTs reviewed investigated two types of pharmaceutical treatment.
Healthy sleep is particularly important for children with attention deficit hyperactivity disorder (ADHD), as sleep disturbances might aggravate disease symptoms. This review aims to synthesize and report evidence on the effectiveness of sleep interventions in increasing sleep, quality of life (QoL), and ADHD symptoms among children with ADHD.
The systematic literature review follows the Cochrane Collaboration methodology recommendations for literature reviews. Four databases were used based on the population, intervention, control and outcome (PICO) framework. Controlled trials with minimum 20 children in each group, aged 6–18, and published from 2005 and onwards were included. Results from the studies were reported in forest plots and three of the seven review outcomes were synthesized in meta-analyses.
The search identified 7710 records; of which 4808 abstracts were screened. After fulltext-screening of 99 papers, eight papers from five studies were included. The studies included behavioral sleep interventions and pharmacological interventions using melatonin and eszopiclone. For six of the seven outcomes, the effect sizes were small to moderate and the certainty of the evidence was low. For one outcome, sleep disturbances, the effect size was a moderate −0.49 standardized mean differences (95% confidence interval −0.65;-0.33), with a moderate certainty of evidence for the behavioral interventions for children aged 5–13 years with ADHD.
This review identified few and heterogeneous studies. A moderate certainty of evidence for a moderate effect size was only obtained for sleep disturbances from the behavioral interventions. A low certainty of the evidence for a moderate effect size was found for the total sleep time from the pharmacological intervention using melatonin and one behavioral intervention, which indicates that these sleep interventions impact sleep quantity and quality among children with ADHD.
Patient-reported sleep outcomes in randomized-controlled trials in persons with substance use disorders: A systematic review
2022, Drug and Alcohol Dependence
Citation Excerpt :
Below we discuss perceived sleep quality and diagnostic criteria for sleep disturbances as potential outcomes for RCTs aimed at improving sleep in persons with SUDs. Sleep quality may be inferred by improvements in objective measures of sleep, but validated questionnaires offer unique information on patient-reported improvements in satisfaction with sleep and corresponding daytime functioning (Morin, 2003). The most commonly used measures of those reviewed here were the PSQI and the ISI (Table 3).
Sleep disturbances and disorders are a common and sometimes recalcitrant problem in persons recovering from substance use disorders (SUDs). As such, several randomized-controlled trials (RCTs) have been conducted to address sleep disturbances in a variety of SUD subpopulations and clinical scenarios. The goal of this systematic review was to collate patient-reported sleep outcomes used in past SUD-related RCTs to provide guidance for future sleep research in persons with SUDs.
This systematic review was registered in the International Prospective Register of Systematic Reviews (PROSPERO) on May 7th, 2020 (CRD42020182004). Studies were included if they were peer-reviewed manuscripts describing RCTs in an SUD population.
The initial search yielded 13,403 candidate articles, and 76 met a priori criteria and were included in this review. Thirty-five (46.1%) assessed sleep as a primary outcome (i.e., sleep improvement was the primary goal of the research) and 41 (53.9%) assessed sleep as a secondary outcome (i.e., sleep improvement was an important outcome, but not the primary outcome). The most commonly used measures included the Pittsburgh Sleep Quality Index, the Insomnia Severity Index, and sleep diaries. However, multiple additional sleep assessments were also used, including visual analogue and Likert scales.
The field of addiction medicine would benefit from a streamlined approach in assessing patient-reported sleep in RCTs, including commonly used and validated assessments of sleep quality, inserting daily or repeated measures into RCTs, and including questionnaires that assess clinically relevant insomnia or other sleep disorders.
Objective sleep outcomes in randomized-controlled trials in persons with substance use disorders: A systematic review
2022, Drug and Alcohol Dependence
Improving sleep health is an important target for substance use disorder (SUD) research. However, there is little guidance for SUD researchers regarding the use of technologies to objectively assess sleep outcomes in randomized-controlled trials (RCTs). This systematic review aimed to describe the use of technologies to objectively measure sleep outcomes in RCTs conducted in persons with SUDs, in order to inform future sleep intervention studies in SUD populations.
This study was registered in the International Prospective Register of Systematic Reviews (PROSPERO) on May 7th, 2020 (CRD42020182004). RCTs were reviewed here if they were peer-reviewed manuscripts that included objective measures of sleep in RCTs that sought to improve sleep in persons with SUDs.
The initial search yielded 13,403 potential articles, with 27 meeting a priori criteria to be included in this review. The most common SUD was alcohol use disorder (59%). The most common technology used to assess sleep was polysomnography (41%), followed by actigraphy (37%), ambulatory polysomnography or components of polysomnography (e.g., electroencephalography; 19%), and at-home sleep apnea testing (7%). The most common sleep outcome reported was total sleep time (96%).
There are a range of options to assess objective sleep outcomes. Polysomnography or ambulatory devices that directly measure brain activity are critical to advance medications through the regulatory process for the indication of improving sleep duration, continuity, and/or sleep onset latency outcomes. Actigraphy is also useful in preliminary investigations and in detecting the relationship between diurnal and SUD-related behaviors.
The Future of Sleep Measurements: A Review and Perspective
2021, Sleep Medicine Clinics
Zao Ren An Shen for insomnia: a systematic review with meta-analysis
2020, Sleep Medicine
Citation Excerpt :
Data concerning the study design, the participants type, age, gender, sample size, the interventions regimen, outcomes types and measurements, and outcome data were collected by YB with a data collection form [65]. The outcome data collected were sleep quality or similar outcomes measured with a questionnaire (eg, PSQI), objective and subjective sleep parameters such as Sleep Onset Latency (SOL), Total Sleep Time (TST), Number of AWAKenings (NAWAK) or Sleep Efficiency (SE), clinical severity, fatigue and sleepiness levels, psychological distress levels, and QoL [66] for effectiveness, all measured with a validated measurement only, and number of participant-reported adverse events, regardless of the collection procedure. The authors were contacted when relevant data were missing.
To assess the effectiveness and safety of Zao Ren An Shen (ZRAS), a Chinese herbal medicine formula, for the treatment of insomnia.
Seven databases (ie, EMBASE, PubMed, the Cochrane library, and PsycINFO, Chinese National Knowledge Infrastructure, Wanfang and Chongqing VIP) were searched from their inception to 6 November 2018. Controlled trials comparing the effectiveness or safety of ZRAS to conventional treatments, a placebo or no-treatment in an insomnia population were selected. Primary outcomes were: sleep quality (assessed with the Pittsburgh Sleep Quality Index, PSQI), and the number of adverse events at post-treatment. The risk of bias was assessed with the Cochrane Collaboration's tool and meta-analyses were performed using RevMan 5.3.
A total of 19 studies (1780 participants) were included. The effect of ZRAS on sleep quality (mean difference) was found to be superior compared to placebo in the sole placebo-controlled study located [−0.90 (-1.56, −0.24; 95% CI), p = 0.007] and similar to Benzodiazepine Receptor Agonists (BzRAs) [0.17 (-0.29, 0.64); 95% CI, p = 0.46]. The number of adverse events (relative risk) was lower for ZRAS than BzRAs [0.16 (0.12, 0.23; 95% CI), p < 0.001]. An overall high risk of bias was found in the selected studies.
The results favor ZRAS against BzRAs and placebo for the treatment of insomnia. However, the poor methodology of the studies prevents strong recommendations for ZRAS. Clinical trials with higher quality designs are required.

View all citing articles on Scopus

^f1: Correspondence should be addressed to: Charles M. Morin, Ph.D., Université Laval, École de Psychologie, Pavillon FAS, Sainte-Foy, Québec, Canada G1K 7P4. Tel.: +1-418-656-3275; Fax: +1-418-656-5152; E-mail: [email protected]

View full text

Clinical ReviewMeasuring outcomes in randomized clinical trials of insomnia treatments

Abstract

Epidemiologic study of sleep disturbances and psychiatric disorders: An opportunity for prevention?

JAMA

Prevalence and treatment of insomnia in general practice: A longitudinal study

Eur Arch Psy Clin Neuro Sci

Epidemiology of insomnia: what we know and what we still need to learn

Sleep Med Rev

Prevalence, burden, and treatment of insomnia in primary care

Am J Psychiatry

Evaluation of chronic insomnia. An American Academy of Sleep Medicine review

Sleep

Nonpharmacologic treatment of chronic insomnia

Sleep

Defining empirically supported therapies

J Consult Clin Psychol

Long-term effectiveness of a short-term cognitive-behavioral group treatment for primary insomnia

Eur Arch Psychiatry Clin Neurosci

Cognitive-behavioral treatment of insomnia secondary to chronic pain

J Consult Clin Psychol

Cognitive behavioral therapy for treatment of chronic primary insomnia. A randomized controlled trial

JAMA

The clinical effectiveness of cognitive behaviour therapy for chronic insomnia: Implementation and evaluation of asleep clinic in general medical practice

Behav Res Ther

Home-based central nervous system assessment of a multifactor behavioral intervention for chronic sleep-onset insomnia

Behav Ther

Relaxation and sleep compression for late-life insomnia: A placebo-controlled trial

J Consult Clin Psychol

Comparison of continuous versus intermittent administration of zolpidem in chronic insomniacs: A double-blind, randomized pilot study

J Int Med Res

Sleep latency is shortened during 4 weeks of treatment with zaleplon, a novel nonbenzodiazepine hypnotic

J Clin Psychiatry

Doxepin in the treatment of primary insomnia: A placebo-controlled, double-blind, polysomnographic study

J Clin Psychiatry

Sleep evaluation in chronic insomniacs during 14-day use of flurazepam and midazolam

J Clin Psychopharmacol

A multicenter, placebo-controlled study evaluating zolpidem in the treatment of chronic insomnia

J Clin Psychiatry

Effects of zopiclone and temazepam on sleep, behaviour and mood during the day

Eur J Clin Pharmacol

A five week, polysomnographic assessment of zalepan 10 mg for the treatment of primary insomnia

Sleep Med

Can we mix behavioral therapy with hypnotics when treating insomniacs?

Sleep

Behavioral and pharmacological therapies for late-life insomnia. A randomized controlled trial

JAMA

Efficacy of behavioral versus triazolam treatment in persistent sleep-onset insomnia

Am J Psychiatry

Nonpharmacological interventions for insomnia: a meta-analysis of treatment efficacy

Am J Psychiatry

Psychological management of insomnia: a clinical replication series with 100 patients

Behavior Therapy

The relationship between chronically disrupted sleep and healthcare use

Sleep

SF-36: Evaluation of quality of life in severe and mild insomniacs compared with good sleepers

Psychosom Med

Quality of life in people with insomnia

Sleep

Insomnia: Psychological assessment and management

Symptom reports in severe chronic insomnia

Sleep

Effects of chronic insomnia and use of benzodiazepines on daytime performance in older adults

J Gerontology: Psychological Sciences

Subjective and objective performance in patients with persistent insomnia

Scand J Behav Ther

Sleep disturbance and psychiatric disorders: a longitudinal epidemiological study of young adults

Biol Psychiatry

Clinical diagnoses in 216 insomnia patients using the international classification of sleep disorders (ICSD), DSM-IV and ICD-10 categories: a report from the APA/NIMH DSM-IV field trial

Sleep

Sleep and psychopathology

App and Prev Psychol

Insomnia and the eye of the beholder: are there clinical markers of objective sleep disturbances among adults with and without insomnia complaints

J Consult Clin Psychol

Behavioral treatment of insomnia: a clinical case series study

J Beh Med

Characteristics of individuals who do or do not seek treatment for chronic insomnia

Psychosomatics

Quality of life: expanding the scope of clinical significance

J Consult Clin Psychol

A critical appraisal of the quality of quality-of-life measurements

Clinical Review
Measuring outcomes in randomized clinical trials of insomnia treatments