Research ArticleArticles

Paired Comparison Survey Analyses Utilizing Rasch Methodology of the Relative Difficulty and Estimated Work Relative Value Units of CPT® Code 27279

Morgan Lorio, Melissa Martinson and Lisa Ferrara
International Journal of Spine Surgery January 2016, 10 40; DOI: https://doi.org/10.14444/3040

Abstract

Background Minimally invasive sacroiliac joint arthrodesis (“MI SIJ fusion”) received a Category I CPT® code (27279) effective January 1, 2015 and was assigned a work relative value unit (“RVU”) of 9.03. The International Society for the Advancement of Spine Surgery (“ISASS”) conducted a study consisting of a Rasch analysis of two separate surveys of surgeons to assess the accuracy of the assigned work RVU.

Methods A survey was developed and sent to ninety-three ISASS surgeon committee members. Respondents were asked to compare CPT® 27279 to ten other comparator CPT® codes reflective of common spine surgeries. The survey presented each comparator CPT® code with its code descriptor as well as the description of CPT® 27279 and asked respondents to indicate whether CPT® 27279 was greater, equal, or less in terms of work effort than the comparator code. A second survey was sent to 557 U.S.-based spine surgeon members of ISASS and 241 spine surgeon members of the Society for Minimally Invasive Spine Surgery (“SMISS”). The design of the second survey mirrored that of the first survey except for the use of a broader set of comparator CPT® codes (27 vs. 10). Using the work RVUs of the comparator codes, a Rasch analysis was performed to estimate the relative difficulty of CPT® 27279, after which the work RVU of CPT® 27279 was estimated by regression analysis.

Results Twenty surgeons responded to the first survey and thirty-four surgeons responded to the second survey. The results of the regression analysis of the first survey indicate a work RVU for CPT® 27279 of 14.36 and the results of the regression analysis of the second survey indicate a work RVU for CPT® 27279 of 14.1.

Conclusion The Rasch analysis indicates that the current work RVU assigned to CPT® 27279 is undervalued at 9.03. Averaging the results of the regression analyses of the two surveys indicates a work RVU for CPT® 27279 of 14.23.

Introduction

Effective January 1, 2015, a new Category I Current Procedural Terminology (CPT®) code 27279 was created to report “Arthrodesis, sacroiliac joint, percutaneous or minimally invasive (indirect visualization), with image guidance, includes obtaining bone graft when performed, and placement of transfixing device” (“MI SIJ fusion”). Diagnosing sacroiliac joint pain consists of medical history, physical examination including provocative maneuvers, imaging studies, and confirmatory intra-articular joint injection as defined within the International Society for the Advancement of Spine Surgery (“ISASS”) Policy Statement1 and the North American Spine Society (“NASS”) Policy Statement.2

In April 2014, the American Medical Association (“AMA”) Specialty Society Relative Value Scale Update Committee (“RUC”) recommended to the Centers for Medicare and Medicaid Services (“CMS”) that CPT® code 27279 be valued at 9.03 work relative value units (“RVU”). In valuing the new code, the RUC noted3 that the survey process was interfered with by an outside party prompting the dismissal of the survey data in favor of a crosswalk methodology to CPT® code 62287 (Decompression procedure, percutaneous, of nucleus pulposus of intervertebral disc, any method utilizing needle based technique to remove disc material under fluoroscopic imaging or other form of indirect visualization, with the use of an endoscope, with discography and/or epidural injection(s) at the treated level(s), when performed, singe or multiple levels, lumbar). Surgeon members of ISASS felt that this was not a comparable procedure and questioned the methodology and rationale. Members queried the Society as to options available to examine this valuation and supported Society action to address the valuation. For background purposes, ISASS became an approved member of the AMA House of Delegates at the June 2014 meeting. As such, ISASS was ineligible to participate in the April 2014 RUC meeting when CPT® 27279 was presented and discussed for valuation.

In order to address the work RVU assigned to CPT® 27279, ISASS requested refinement of the code in December 2014 through a process convened by CMS. CMS grants requests for refinement panel for codes with interim values where new data, not available at the time the code went through the RUC process, might inform CMS before the value is finalized via the final Physician Fee Schedule rulemaking. In order to present new data to CMS, ISASS utilized an alternative process to evaluate the work RVU of CPT® 27279 by conducting two separate paired-comparison surveys which were analyzed using Rasch methodology. The goal of both surveys was to determine whether the work RVU assigned to CPT® 27279 was appropriate or misvalued based on the input of spine surgeon members who perform the procedure.

Rasch methodology has been described and validated by neurosurgeon Robert A. Florin, MD who served on the RUC and has used the methodology to find the relative work effort of healthcare procedures.4, 5 The method also has a well-established track record in education, where it is used to find the relative difficulty of each of a set of test items6; in marketing, where it is used to find the preferences among consumer products7; and in health economics, where it is used both to find the desirability of health states (medical conditions).8

The Rasch method for healthcare procedures involves the use of paired comparisons. In a traditional Rasch analysis, each CPT® code in a set is paired with every other code in that set, and each respondent indicates which of each pair requires the greater work effort. Then logistic regression methods are used to estimate the probability that each procedure is more work than the others, and to estimate the difficulty or work-effort score for each procedure in the set. The scores are logits (log of the odds ratio) and they are values on a cardinal scale with one (arbitrary) procedure set to a score of 0.54; (Nominal scales have classes without order, [e.g. New England, mid-Atlantic, mid-west]; ordinal scales have order but no meaningful interval [e.g. always, sometimes, never]; cardinal scales have order and the interval is meaningful [e.g. 30°, 45°, 60°]). After the difficulty/work effort scores are generated, RVUs are estimated by regression analysis using existing RVUs.

Methods

Two separate paired comparison surveys were conducted and analyzed using the Rasch method; the first paired comparison survey was conducted by ISASS in December 2014. This study used a minor modification of the Rasch analysis method; instead of comparing all pairs of CPT® codes in a set of spine surgeries, CPT® code 27279 was compared in pairs with CPT® codes reflective of common surgeries spine surgeons performed with a 90-day global period or represented work RVUs that ranged from below 9.03 to above it. Table 1 provides the CPT® codes used in this study, a description of each, and the physician work RVUs assigned to each by CMS in 2014. This was a reasonable modification because it was assumed that all of the other CPT® codes were valued approximately correctly, and the only work RVU needing evaluation was that assigned to CPT® 27279. (Note that physician work is the same in the facility and non-facility settings.)

View this table:
Table 1

CPT® Codes Used in the First Survey.

A survey was posted on Survey Monkey (www.surveymonkey.com ) and an email message requesting completion of the survey and providing the internet link to do so was sent to 93 surgeons representative of the ISASS Board of Directors, Coding & Reimbursement Task Force, and Minimally Invasive Spine Surgery Committee. Non-U.S. surgeons were omitted from the list. The survey was completed by 20 surgeons who perform MI SIJ fusions.

The survey presented each comparator CPT® code with its code descriptor as well as the description of CPT® 27279, and asked the surgeon to indicate the more difficult procedure in terms of work effort. For the purpose of this analysis, work effort was defined as the total of the time and skill level required to perform the procedure. Importantly, the work RVUs of the procedures were not provided as part of the survey instrument and the surgeons were not asked to estimate work RVU of any procedure.

Responses were coded 0 if CPT® 27279 was easier than the comparator, and 1 if CPT® 27279 was harder. Respondents were permitted to indicate that both procedures were approximately equivalent; in this case, the response was randomly assigned a 0 or 1 value, each with probability 0.5. That is to say, if two procedures are deemed equal by some respondents, and these individuals were forced to choose a procedure as the harder one, approximately half would choose one procedure and half would choose the other. This modification was an important one, as “equivalent” is actually not a missing response.

The scores from Rasch analysis include an arbitrary item assigned a score of 0 (not to be confused with the raw data value of 0). Since the other CPT® codes were chosen to provide a range above and below CPT ® 27279, CPT® 27279 was analyzed as the arbitrary 0 score. To enable this feature of the analysis, it was necessary to create a paired “dummy” comparison of CPT® 27279 with itself. Since CPT® 27279 is equivalent to itself in work effort, respondents’ comparisons were randomly assigned values of 0 and 1 in equal proportion, as described in the previous paragraph. Using these data, the difficulty of each other CPT® codes relative to CPT® 27279 was estimated. Table 1 provides the CPT® codes used in this study, a description of each, and the physician work RVUs assigned to each by CMS in 2014.

The Rasch analysis was conducted in STATA 13 (Stata Corp., College Station, TX; www.Stata.com) using the raschtest procedure. The Rasch analysis estimated the difficulty/work effort scores. According to the method of Florin4, the scores of the accepted CPT® codes were then regressed on their RVUs to generate a regression line. The line provided the best-fit estimate of the RVUs per score point (i.e., the slope of the line). Then the RVUs for CPT® 27279 were estimated by finding the RVUs that coincided with a score of 0, which was the score assigned to CPT® 27279. The simple linear regression analysis and graphics were generated by Excel 2013 (Microsoft Corp., Redmond, WA).

The second paired-comparison survey was conducted by ISASS in March 2015. The survey was posted on Survey Monkey for fourteen days (www.surveymonkey.com) and a message requesting participation in the survey and a direct link to the survey was emailed to 557 U.S. surgeon members of ISASS and 241 U.S. surgeon members of the Society for Minimally Invasive Spine Surgery (“SMISS”).

The design and methodology of the second survey mirrored the design and methodology of the first survey with the exception of the utilization of a broader code comparator list, which included 27 CPT® codes compared to 10 in the first survey. Table 2 provides the CPT® codes used in this study, a description of each, and the work RVU assigned to each by CMS in 2014.

View this table:
Table 2

CPT Codes Used in the Second Survey.

Finally, it was of interest to parse out the relative contribution of individual time components to the total work RVU of CPT® 27279 and the 27 comparator procedures. We obtained the values of the following time components from data collected by the AMA and made available by CMS9: pre-service evaluation, dressing and scrubbing, pre-service “other”, intra- operative service, same-day post-operative followup, post-operative follow-up, and office visit. We conducted a backwards-selection linear regression analysis. Full models included all time components, with serial removal of each time component with the highest explanatory power (i.e., largest p-value) until all model variables had p-values <.05. The resulting regression line was used to estimate the work RVUs for CPT®27279 by substituting the values of the time components into the variables of the regression equation.

Results

First Survey

Twenty surgeons completed the first survey. Table 3 shows the results of the Rasch analysis. Each CPT® code is listed along with its estimated difficulty score. The difficulty scores are relative to CPT® 27279, which has a score of 0. Negative score indicate a procedure that is less difficult than CPT® 27279; positive scores indicate a more difficult procedure.

View this table:
Table 3

Difficulty Scores Estimated by the Rasch Analysis.

The RVUs from Table 1 (except those of CPT® 27279) and difficulty scores from Table 3 were analyzed by simple linear regression. The regression analysis of RVUs (dependent variable) on difficulty score (independent variable) estimated an intercept of 14.36 and a slope of 2.47. Figure 1 shows the results of this analysis. The points are labeled with their CPT® codes and RVUs. The estimated work RVU for CPT® 27279 is 14.36. This is derived by solving the equation:

Fig. 1
Fig. 1

Results of Regression Analysis to Estimate Work RVU of CPT® 27279

RVUs = intercept + slope * difficulty

The R2 of the regression analysis was .379, which means that less than 40% of the variation in work RVU was explained by the difficulty of the procedure. In a system in which work RVUs were determined solely by work effort, all of the data points would lie on a straight line and R2 would equal 1.0.

Second Survey

Thirty-four (34) surgeons met the required criteria and completed the second survey. The specialties of the physicians are summarized in Table 4. Approximately 25% represented each neurosurgery and orthopedics, and just under 50% from spine surgery. They represented 17 different states and the District of Columbia, rural practices (12%), suburban practices (44%) and urban practices (44%). All stated that they had performed a MI SIJ fusion surgery within the past 24 months.

View this table:
Table 4

Specialties of Participants in the Second Survey.

Table 5 shows the survey responses and the results of the Rasch analysis. Each CPT® code is listed along with its estimated difficulty score. The difficulty scores are relative to CPT® code 27279, which has a score of 0. Negative scores indicate a procedure that is less difficult than CPT® 27279; positive scores indicate procedure that is more difficult than CPT® 27279.

View this table:
Table 5

Difficulty Scores Estimated by the Rasch Analysis.

The RVUs from Table 2 (except CPT® 27279) and difficulty scores from Table 5 (except CPT® 27279) were analyzed by simple linear regression. The regression analysis of RVUs (dependent variable) on difficulty score (independent variable) estimated an intercept of 14.1 and a slope of 1.34. Figure 2 shows the results of this analysis. The points are labeled with their CPT® codes and RVUs. The estimated work RVU for CPT® 27279 is 14.1. This is derived by solving the equation:

Fig. 2
Fig. 2

Results of Regression Analysis to Estimate RVUs of CPT® 27279.

RVUs = intercept + slope * difficulty

The R2 of the regression analysis was .27, which means that less than one-third of the variation in work RVUs was explained by the difficulty of the procedure.

Time Component Analysis

The results of the time component analysis show that three of the seven individual time components have a significant influence on the total work RVU of CPT® 27279. Table 6 lists the time components with a significant influence on the total work RVU of CPT® 27279 and the level of influence for each. The three time components in Table 6 were shown to have the greatest influence on total work RVU, most likely because they capture the bulk of physician time and intensity related to the patient’s care. The other time components have a small (but important) influence on the work RVU that was not detectable in this small data set (of 28 CPT® codes). The overall time component analysis model had a p-value of 0.0037 and an adjusted R2 of .35, meaning that about 35% of the variation in total work RVU is explained by these three predictor variables.

View this table:
Table 6

Results of the Time Component Regression Analysis.

None of the other time variables were significant predictors of work RVU and thus did not improve the R2 value.

This analysis indicates a total work RVU associated with CPT® 27279 at 1.00 + .139*40 + .0704*60 + .057*62 = 14.3.

Discussion

The Medicare Physician Fee Schedule was implemented in 1992 by CMS. The Physician Fee Schedule mandated a fixed payment system administered by CMS, and private health insurers often look to the Physician Fee Schedule when negotiating payment rates with physicians. The RUC was established in 1991 to act as an expert panel to develop relative value recommendations to CMS. Procedure codes are assigned a value by adding together three components: work, practice expense, and malpractice expense (Total RVU= Work RVU + Practice Expense RVU + Malpractice Expense RVU). The work RVU is calculated using measures of time and intensity based on Harvard research by Hsiao et al.10

Measuring work effort is subjective; time is the foremost predictor of physician work assessment and the most objective metric. Measuring intensity is more subjective and includes the following: 1) technical skill and physical effort, 2) mental effort and judgement, and 3) psychological stress. The work RVU is subdivided into pre-service work, intra-service work, and post-service work determined through a RUC survey process of the time and intensity required to perform the procedure; intra-service work is usually the most variable as far as intensity.11

The current work RVU assigned to CPT® 27279 using a crosswalk methodology is 9.03. The Rasch analysis of the first survey discussed in this paper indicates a work RVU of 14.36 and the Rasch analysis of the second survey discussed in this paper indicates a work RVU of 14.1. The time-component regression analysis was consistent with these estimates, at 14.3 work RVUs. The data from the two surveys was used by ISASS to request refinement of the work component of CPT® 27279 to CMS. CMS grants requests for refinement panel for codes with interim values where new data, not available at the time the code went through the RUC process, might inform CMS before the value is finalized via the final Physician Fee Schedule rulemaking. In addition to the survey data, a retrospective review of prospectively collected data12 examining the surgical work effort of MI SIJ fusion was shared with CMS during the refinement process. The results of this study indicate open primary lumbar microdiscectomy (CPT® 63030) as a more comparable procedure with a work RVU of 13.18. The 2015 Multi-Specialty Refinement Panel convened by CMS in August 2015 unfortunately maintained the work RVU of CPT® 27279 developed by utilizing a crosswalk methodology to CPT® code 62287. The drastic difference between the results of this study and current work RVU assigned to CPT® 27279 leads us to question the validity of the crosswalk methodology utilized to value CPT® 27279.

There are several negative implications for misvalued procedures including decreased patient access to surgeons who will perform the procedure and the impact to innovative technologies that advance medicine as a whole. The general movement in surgical spine care is from inpatient to outpatient and the development of new technology that reduces operating times, decreases blood loss, risk for infection, and recovery time has certainly made this possible in many surgical cases. There are large costs incurred in emerging technology including but not limited to patent development, FDA approval, procedure code development, valuing the procedure, insurance coverage of the procedure, education, and advocacy. Procedures that are inappropriately valued now have a large impact on the ability for future emerging technologies.

Disclosures & COI

This study was funded by the International Society for the Advancement of Spine Surgery (ISASS) and contributions by Globus, SI Bone, and Zyga. Melissa Martinson has a consulting arrangement with ISASS and is a paid consultant on this project.

References

  1. ↵
    1. Lorio MP
    ISASS Proposed Recommendations/Coverage Criteria for Minimally Invasive Sacroiliac Joint Fusion 2016 – Coverage Indications, Limitations, and/or Medical Necessity, Updated July 2016.
  2. ↵
    1. NASS Coverage Committee
    (May, 2015) Percutaneous Sacroiliac Joint Fusion.
  3. ↵
    1. AMA/Specialty RVS Update Committee Meeting Minutes
    (April 24-27, 2015) https://download.ama-assn.org/resources/doc/rbrvs/april-2014-ruc-minutes.pdf .
  4. ↵
    1. Florin RE
    (2000) Rasch analysis in measurement of physician work. J Outcome Meas 4(2):564–78.
  5. ↵
    1. McCall NT,
    2. Cromwell J,
    3. Griggs ML
    (June 15, 1999) Report by Health Economics Research, Inc, Five Year Review of Work Relative Value Units: Final Report. Section 2.2: The Identification of Potentially Misvalued Work RVUs, A Rasch Paired Comparison of Total Physician Work. Downloaded 15 DEC2014 from http://www.rasch.org/florin.htm .
  6. ↵
    1. Boone WJ,
    2. Staver JR,
    3. Yale MS
    (2014) Rasch Analysis in the Human Sciences (Springer, Inc, New York).
  7. ↵
    1. Bechtel GG
    (1985) Generalizing the Rasch model for consumer rating scales. Marketing Science 4(1):62–73.
  8. ↵
    1. Young T,
    2. Yang Y,
    3. Brazier JE,
    4. et al.
    (2008) The first stage of developing preference-based measures: constructing a health-state classification using Rasch analysis. Qual Life Res 18:253–265.
  9. ↵
    1. RBRVS DataManager
    (2015) Online subscription (American Medical Association).
  10. ↵
    1. Hsiao W.C,
    2. Braun P,
    3. Kelly N.L,
    4. Becker E.R
    (1988) Results, Potential Effects, and Implementation Issues of the Resource-Based Relative Value Scale. JAMA 260(16):2429–2438.
  11. ↵
    1. Mabry C.D,
    2. Barton C.M,
    3. Harris J.A,
    4. et al.
    (2005) The Use of Intraservice Work per Unit of Time (IWPUT) and the Building Block Method (BBM) for the Calculation of Surgical Work. Ann Surg 241:929–940.
  12. ↵
    1. Garber T,
    2. Ledonio C.G.T
    (2015) Polly, D.W. How much work effort is involved in minimally invasive sacroiliac joint fusion? International Journal of Spine Surgery, 9.
Back to top

In this issue

Print
Download PDF
Article Alerts
Email Article
Citation Tools
Paired Comparison Survey Analyses Utilizing Rasch Methodology of the Relative Difficulty and Estimated Work Relative Value Units of CPT® Code 27279
Morgan Lorio, Melissa Martinson, Lisa Ferrara
International Journal of Spine Surgery Jan 2016, 10 40; DOI: 10.14444/3040
Twitter logo Facebook logo Mendeley logo

Jump to section

Keywords