ABSTRACT
Background: The Patient-Reported Outcomes Measurement Information System (PROMIS) was developed to provide an easily administered patient-outcome questionnaire that was adaptable to a variety of medical and surgical subspecialties. Numerous authors have examined the effectiveness of PROMIS in various areas of spine surgery. Our goal was to systematically review PROMIS scores compared with legacy patient-reported outcomes measures (PROMs) in spinal surgery and spine pathology.
Methods: A systematic search of the PubMed, EMBASE, and Cochrane databases using the Preferred Reporting Items for Systematic Reviews and Meta-Analysis (PRISMA) guidelines was performed, yielding 254 unique studies reporting on “PROMIS” in “spine.” Each study was independently reviewed. A total of 16 studies were selected for inclusion.
Results: The pooled sample size yielded a total of 4268 patients. In the cervical population, PROMIS physical function (PF; |r| = .47–.87, pain intensity (PIn; |r| = .61–.74), pain interference (PIf; |r| = .65–.88), and pain behavior (PB; |r| = .59–.74) correlated with the Neck Disability Index (NDI). PROMIS PF also strongly correlated with the modified Japanese Orthopaedic Association scale (mJOA; |r| = .61–.72). Among patients with lumbar pathology and adult spinal deformities, PROMIS PF (|r| = .53–.85), PIn (|r| = .73–.78), PIf (|r| = .59–.89), and PB (|r| = .58–.82) strongly correlated with the Oswestry Disability Index (ODI). PF (|r| = .51–.78), PIf (|r| = .60–.70), and anxiety (|r| = .73) also strongly correlated with the Scoliosis Research Society (SRS)-22 and SRS-30. When comparing measures of global health, PROMIS PF was strongly correlated with the Short Form (SF)-12 and SF-36 (|r| = .50–.85). On average, all PROMIS domains required less time to complete (49.6–56 seconds) than the ODI (176 seconds), NDI (190.3 seconds), SF-12 (214 seconds), and SF-36 physical function domains (99 seconds). The responsiveness of the PROMIS PF, PIf, and PB was comparable to that of legacy measures ODI, NDI, and SF-12.
Conclusions: The PROMIS PF, PIn, PIf, and PB demonstrated moderate to strong correlations with NDI, mJOA, ODI, SRS, and SF-12 measures in various populations of spine patients. All PROMIS domains had decreased time to completion and similar responsiveness compared with legacy measures.
Level of Evidence: 2.
Clinical Relevance: These results highlight the potential of PROMIS as a valid and reliable tool to assess patient-reported outcomes in spinal surgery patients and support more widespread use of PROMIS in spine.
INTRODUCTION
As the focus on evidence-based medicine and value-driven health care has grown over the past several decades, patient-reported outcomes measures (PROMs) have gained in popularity and importance in orthopedics.1,2 The current approach to PROMs focuses on a group of “legacy outcome measures” that measure outcomes in specific disease states and/or anatomical locations.3,4 This disease-specific approach, however, is fragmented by its nature and makes it difficult to compare patient health and response with treatment across different pathologies and interventions. Furthermore, current PROMs are burdensome and require significant time to complete.5
To remedy these challenges, in 2004 the National Institute of Health developed the Patient-Reported Outcomes Measurement Information System (PROMIS) to provide a widely reliable and valid tool to measure patient outcomes across medicine.4 PROMIS questionnaires offer a set short-form version with a fixed set of 4–10 questions or a computer adaptive test version of 4–12 questions tailored to individuals and the severity of their symptoms.6 Since its development, there has been a substantial increase in attention surrounding the validity, responsiveness, and ease of using PROMIS in spine patients.
The goal of this study was to systematically review PROMIS scores compared with legacy PROMs in spinal surgery. We hypothesize that PROMIS scores will be highly correlated with the various legacy PROMs in spinal surgery and be less burdensome to patients, requiring less time to complete.
METHODS
A systematic search of 3 databases, MEDLINE via PubMed, EMBASE, and the Cochrane Library, was performed to identify all relevant literature published between database inception and November 2018. The search strategy was created in collaboration with a professional medical librarian. Search terms included a combination of MeSH terms and keywords related to PROMIS and spine. The search was completed on November 20, 2018. The complete search strategy is available in the Appendix.
Inclusion criteria were any orthopedic-related article with level of evidence 1–4 and reporting use of PROMIS in spine pathology. Articles not related to spine, non-English articles, unpublished studies, studies with level 5 evidence, letters to the editor, editorials, basic science articles, and conference abstracts were excluded. Covidence (Melbourne, Australia) software was used for the screening process. The search yielded 254 unique studies reporting on “PROMIS” in “spine.” Each study was independently reviewed. Ultimately, 55 studies were selected for full-text review, and 16 studies were selected for final inclusion (see Figure). Study characteristics (eg, study population, level of evidence, number of included patients, mean age) were recorded. The reported outcome instruments were noted and correlations between PROMIS domains and legacy measures were documented for each study. Correlations were reported as weak (0 ≤ |r| < 3), moderate (3 ≤ |r| ≤ 5), or strong (|r| ≥ 5). Responsiveness of the instrument and time to completion were also noted when available.
RESULTS
The characteristics of the included studies are shown in Table 1. The pooled sample size from the 16 studies yielded a total of 4268 patients. Of the 16 studies, one was level 1; nine were level 2; four were level 3; and two were level 4. The mean number of patients in each study was 267 (range, 41–1607).
Cervical Spine and Neck Pain
Of the 16 studies included, 7 reported on cervical pathology and the respective legacy measures (Table 2). In this population, PROMIS physical function (PF; |r| = .47–.87, pain intensity (PIn; |r| = .61–.74), pain interference (PIf; |r| = .65–.88), and pain behavior (PB; |r| = .59–.74) all moderately to strongly correlated with the Neck Disability Index (NDI). In Table 2, correlations may be reported as negative, given that higher PROMIS PF scores indicating an improvement in physical functioning would correlate with lower disability scores such as the NDI. Owen et al16 also showed PROMIS physical function strongly correlated with the mJOA (|r| = .61–.72). In a population of nonsurgical patients, Moses et al15 demonstrated a strong correlation between PROMIS PF and the NDI (|r| = .771) but only moderate correlations with the visual analog scale (VAS) instruments (|r| = .302–.428).
Lumbar Spine and Adult Spinal Deformity
There were 11 studies that reported on patients with lumbar pathology or adult spinal deformity. In this population, the most common legacy measures included the Oswestry Disability Index (ODI), Scoliosis Research Society (SRS)-30 and SRS-22r, and VAS Back/Leg (Table 3). Among these patients, PROMIS PF (|r| = .53–.85), PIn (|r| = .73–.78), PIf (|r| = .59–.89), and PB (|r| = .58–.82) all strongly correlated with the ODI. The PROMIS PF (|r| = .51–.78) and PIf (|r| = .60–.70) also strongly correlated with the SRS-22 and SRS-30, whereas PROMIS PF demonstrated a wide range of correlations with VAS Back (|r| = .09–.69) and with VAS Leg (|r| = .33–.64).
In addition, when comparing PROMIS measures with ODI measures at different follow-up time points, PROMIS PF and PIf showed greater correlation as follow-up time increased. PROMIS PF demonstrated increasing correlation with ODI, from baseline or preoperative scores (|r| = .53–.76) to 3-month postoperative scores (|r| = .74–.85) to 6-month postoperative scores (|r| = .80–.84). PROMIS PIf demonstrated a similar pattern of increasing correlation with ODI with greater follow-up time, from baseline or preoperative scores (|r| = .59–.61) to 3-month postoperative scores (|r| = .79) to 6-month postoperative scores (|r| = .83).
Global Health and Mental Health
When comparing PROMIS with legacy measures of global health (Table 4), PROMIS PF was moderately to strongly correlated with the Short Form (SF)-12 (|r| = .50–.85) and strongly correlated with the SF-36 physical function domains (PFD; |r| = .807), whereas the SF-12 moderately strongly correlated with PROMIS PIn (|r| = .34–.67) and PB (|r| = .44–.47).
Three studies reported on PROMIS anxiety and depression scores compared with legacy mental health scores General Anxiety Disorder (GAD)-7, Patient Health Questionnaire (PHQ)-8, and modified Zung Depression Index (mZDI; Table 5). PROMIS depression scores strongly correlated with depression risk assessments PHQ-8 (|r| = .74–.79) and mZDI (|r| = .67), whereas PROMIS anxiety scores strongly correlated with the GAD-7 (|r| = .71–.76).
Time to Completion
All studies that have reported on the time needed to complete PROM questionnaires in spine patients have shown that PROMIS requires significantly less time to complete than legacy measures (Table 6).8–10,17,18 On average, individual PROMIS domains required less time to complete (49.7–56 seconds) than ODI (176 seconds), NDI (190.3 seconds), SF-12 (214 seconds), and SF-36 PFD (99 seconds).
Responsiveness
The responsiveness of the PROMIS PF (effect size [ES] = 0.35–1.42, standard response mean [SRM] = 0.31–1.31), PIf (ES = 0.8–1.39, SRM = 0.78–1.16), and PB (ES = 0.7–1.09) was comparable to that of legacy measures ODI (ES = 0.96–1.03, SRM = 0.7–1.33), NDI (ES = 074–0.76, SRM = 0.58–1.18) and SF-12 (ES = 0.675).8,13,18,19,21,22
Floor and Ceiling Effects of PROMIS
Multiple included studies reported on floor and ceiling effects of various PROMIS domains. Reported floor effects for PROMIS PF (0.0%–3.86%),9,10,18,23 PIf (0.0%–0.44%),7,15,18 and PB (0.0%–1.0%)9,18 were comparable or less than those reported for SF-36 PFD (23.65%),10 ODI (0%–44.24%),9,10,18 NDI (7.10%),15 SF-12 (0.0%),9,18 and SRS-22r (0.88%–1.32%).7 Similarly, ceiling effects were relatively low for PROMIS PF (0.0%–1.7%),9,10,18,23 PIf (0.0%–0.88%),7,15,18 and PB (0.0%).9,18
DISCUSSION
As health care becomes more reliant on PROMs to evaluate treatments, numerous general and disease-specific measures have been put forth as means to assess the impact of various interventions across orthopedic pathologies. Nevertheless, these instruments are often cumbersome to complete and/or lack the ability to compare across populations and pathologies. PROMIS is promoted as a fast, simple instrument with broad applicability, and it has become increasingly used in research and to assess clinical outcomes in orthopedics.2,4 The goal of this study was to systematically review the literature available comparing the psychometric properties of PROMIS and legacy measures in spinal populations.
Across all populations included in this review, we found PROMIS to have strong correlation with legacy outcome measures. In the cervical population, PROMIS PF, PIn, PIf, and PB all moderately to strongly correlated with the NDI (|r| = .47–.88) and PF strongly correlated with the mJOA (|r| = .61–.72). Among patients with lumbar pathology and adult spinal deformity, the PROMIS PF, PIn, PIf, and PB all strongly correlated with the ODI (|r| = .70–.76), and PROMIS PF and PIf also strongly correlated with the SRS-22 and SRS-30 (average |r| = .65). Similarly, PROMIS measures correlated strongly with measures of global health (SF-12 and SF-36), as well as those assessing depression and anxiety, such as the PHQ-8, GAD-7, and mZDI.
It is important for the providers and reseachers in spine to be able to coordinate the use of PROMs in outcome research so that results can be compared between different studies. Guzman et al5 reported in 2016 that the top 6 most frequently used PROMs in spine practices were the VAS, ODI, SF-36, mJOA, NDI, and SRS-22, highlighting that the use of these different disease-specific legacy measures resulted in inconsistencies and author-dependent modifications that could not be standardized.5 Similarly, Winebrake et al24 evaluated outcome reporting in the setting of fusion for lumbar spinal stenosis, noting substantial variability across the literature and recommending efforts to standardize reporting of outcomes to facilitate comparison across surgical interventions and pathologies. The high variability in PROMs used in research led the National Institute of health to develop PROMIS. The findings in this study support the use of PROMIS in diverse spine populations and anatomic locations.
In addition to standardized use and broad applicability, another important factor in determining the effectiveness and utility of PROMs is the burden of the instrument to the patient. Because most of these PROMs forms are administered during clinic visits, ease of use and time to completion are critical components of their success. On average, the PROMIS PF required 49.7 seconds to complete, PROMIS PIn required 52 seconds, and PROMIS PB required 56 seconds. Comparatively, legacy measures ODI (176 seconds), NDI (190.3 seconds), SF-12 (214 seconds), and SF-36 PFD (99 seconds) required more time to complete. The findings in this study demonstrate the possible reduced burden of administration of the PROMIS domains, highlighting this key advantage over legacy measures. However, investigators may want to administer multiple questionnaires to assess different domains of health and should be aware of the additive time of multiple questionnaires.
With these findings in mind, we believe that PROMIS has several advantages over other instruments. Including PROMIS data consistently in spine outcomes will allow practitioners and researchers to easily review literature and compare outcomes across interventions and pathologies.25 As seen in this review, the PROMIS domains focused on physical function and pain were the most commonly studied because these address symptoms often primarily targeted in spine patients. However, if investigators wish to study other domains such as global or mental health, they may still be able to compare results across the different domains because the PROMIS scoring system is a T-score metric with a score of 50 being the mean for a reference population and a 10-point standard deviation in either direction. PROMIS can be administered using the short form or computer adaptive testing, allowing practitioners to use technology to easily distribute questionnaires, increase response rates, and further reduce the administrative burden in their clinics.
There are several limitations of our study. As a systematic review, our study is limited by the quality of the underlying studies that were analyzed. Several studies that were examined included heterogeneous populations undergoing widely varied treatments, limiting the applicability to any 1 pathology or intervention. Nevertheless, the goal of this study was to examine the use of PROMIS in the spine literature, comparing its use with legacy outcome measures generally, and where possible we attempted to provide study details and organize by pathology and type of intervention (eg, surgical vs nonsurgical). Furthermore, there are multiple studies included in this review that were published by the same group of authors, raising the possibility of observer bias among these studies. Last, given that PROMIS was developed relatively recently, there are a limited number of studies providing direct comparisons of PROMIS to legacy measures. Therefore, although this review covers a range of different spinal pathologies, numerous diagnoses are not included. Future studies reporting PROMIS are needed to provide a more comprehensive review of the spine literature.
CONCLUSIONS
PROMIS PF, PIn, PIf, and PB demonstrated strong correlations with disease-specific legacy measures NDI, mJOA, ODI, and SRS-22 and global health measures SF-12 and SF-36 in cervical, lumbar, and spinal deformity patients. The decreased time to completion and comparable responsiveness of PROMIS domains support more widespread use of PROMIS in spine outcome research.
APPENDIX. SEARCH STRATEGIES FOR RESPECTIVE ACADEMIC DATABASES
PubMed Strategy
(“Promis”[tw] OR “Patient-Reported Outcomes Measurement Information System”[tw] OR “Patient-Reported Outcome Measurement Information System”[tw]) AND (“Spine” [Mesh] OR “Spinal Diseases”[mesh] OR “Back Pain”[mesh] OR “neck”[mesh] OR “Spine”[tw] OR “Spinal”[tw] OR “lumbar”[tw])
EMBASE Strategy
(promis:ti,ab,de,tn,kw OR ‘patient-reported outcomes measurement information system':ti,ab, de,tn,kw OR ‘patient-reported outcome measurement information system':ti,ab,de,tn,kw OR ‘patient reported outcomes measurement information system'/exp OR ‘patient reported outcome measurement information system'/exp) AND (‘spine'/exp OR ‘spinal disease'/exp OR ‘backache'/exp OR ‘neck'/exp OR spine:ti,ab, de,tn,kw OR spinal:ti,ab,de,tn,kw OR lumbar: ti,ab,de,tn,kw)
Cochrane Library Strategy
(Promis:ti,ab,kw OR “Patient-Reported Outcomes Measurement Information System”: ti,ab,kw OR “Patient-Reported Outcome Measurement Information System”:ti,ab,kw) AND ([mh Spine] OR [mh “Spinal Diseases”] OR [mh “Back Pain”] OR [mh neck] OR Spine: ti,ab,kw OR Spinal:ti,ab,kw OR lumbar: ti,ab,kw)
Resources for Selecting PROMIS Forms
A full list of PROMIS adult measures can be found here:
A manual for the PROMIS physical function instruments including characteristics of the short form, computer adaptive tests, and scoring guide can be found here:
A manual for the PROMIS pain interference instruments including characteristics of the short form, computer adaptive tests, and scoring guide can be found here:
Footnotes
Disclosures and COI: Han Jo Kim has received royalties from Zimmer Blomet and K2M-Stryker and is a consultant for Alphatec. All other authors received no funding for this study and report no conflicts of interest.
- This manuscript is generously published free of charge by ISASS, the International Society for the Advancement of Spine Surgery. Copyright © 2021 ISASS