Abstract
The worlds of spinal surgery and computational science are intersecting at the nexus of the operating room and across the continuum of patient care. As medicine moves toward digitizing all aspects of a patient’s care, immense amounts of patient data generated and aggregated across surgeons, procedures, and institutions will enable previously inaccessible computationally driven insights. These early insights from artificial intelligence (AI) and machine learning (ML)–enabled technologies are beginning to transform medicine and surgery. The complex pathologies facing spine surgeons and their patients require integrative, multimodal, data-driven management strategies. As these data and the technological tools to computationally process them become increasingly available to spine surgeons, AI and ML methods will inform patient selection, preoperatively risk-stratify patients based on myriad factors, and inform interoperative surgical decisions. Once these tools enter early clinical practice, their use creates a virtual flywheel whereby the use of these tools generates additional data that further accelerate the evolution of computational “knowledge” systems. At this digital crossroads, interested and motivated surgeons have an opportunity to understand these technologies, guide their application toward optimal care, and advocate for opportunities where these powerful new tools can deliver step changes in efficiency, accuracy, and intelligence. In the present article, we review the nomenclature and basics of AI and ML and highlight the current and future applications of these technologies across the care continuum of spinal surgery.
- artificial intelligence (AI)
- machine learning (ML)
- natural language processing
- convolutional neural networks
- computer vision
- generative adversarial networks
- electronic medical record
Introduction
Today, spine surgery is experiencing a transformational moment happening as the worlds of spinal surgery and computational science intersect at the nexus of the operating room and across the continuum of patient care. As medicine moves toward digitizing all aspects of patient care, immense amounts of data will be generated for individual patients and aggregated across surgeons and institutions. Precision medicine—the ability to customize care for an individual patient—will not only be driven by expert opinion but also by data, which will help guide patients and clinicians to make informed decisions. As datasets expand, the unique aspects of each patient and pathology will be deconstructed to variables and correlated with others with similar genotypical and phenotypical expression. Data of high quality in high quantity can provide valuable insights for making informed decisions to optimize patient care.
A deeper understanding of individual patient complexity has been limited by a lack of data and analytical tools to aggregate, study, and decipher useful information for the clinician to use to support timely clinical decisions. Cognitive association and memories of prior experience have driven medicine and surgery since its inception. The evolution from handwritten notes and journals to paper medical records and now electronic medical records (EMRs) has enabled a new opportunity to allow for the use of more advanced statistical methods and help generate novel questions. Rapidly advancing capabilities of artificial intelligence (AI) and machine learning (ML) portend a new era where a single patient’s data recorded across the entire care continuum is compared against thousands or millions of related cases to diagnose conditions, personalize approaches, assess the risks associated with the care options available, and, with high certainty, predict the outcome of a given intervention.
Although AI/ML will offer precise and clear options for care, the impact on trainees, surgeons, payers, and health care systems will be equally immense, with value derived in varying ways before, during, and after the surgical procedure. The opportunities available by leveraging comprehensive datasets have already been recognized by the largest technology companies that have made multibillion-dollar investments in AI and health care.
In this article, we review the technical aspects of AI and ML, illustrate current and future applications across the care continuum, and discuss future directions, priorities, and risks as we use and apply these technologies. AI/ML is an incredibly rich field, and our goal is to equip the reader with a firm basis of understanding to help bring a better future to fruition.
Understanding AI/ML Concepts
AI is defined as intelligence demonstrated by machines.1 Among the various subfields of AI are reasoning, knowledge representation, planning, learning, natural language processing (NLP), and perception. Artificial neural networks are computational models inspired by biological neural systems and take various forms, including probabilistic and convolutional neural networks (CNNs). Probabilistic neural networks are used for classification, pattern recognition, and recommender systems (eg, “watch this, based on your history”), whereas CNNs are designed for processing visual and multidimensional data, allowing for image and video recognition, object segmentation, and NLP.
ML, a fundamental concept within AI, is the study of computer algorithms that improve through experience. The field of AI/ML is continuously evolving and has become fundamentally important to the evolution of computer science (see the Figure for a hierarchy of the main ML branches). Unsupervised learning finds patterns in a stream of input data without labels. Supervised learning relies on labeled input data. ML methods typically perform two tasks: classification and numerical regression. Classification is used to determine the category (class) to which a data input belongs: the program learns patterns within input categories that define the differences between classes and learns to classify new inputs. Numerical regression attempts to produce a function that describes the (continuous) relationship between inputs and outputs and predicts how the output varies with the input. Deep learning is part of a broader family of ML methods based on artificial neural networks within a large number of “hidden” internal layers that may be difficult for an external observer to understand or measure. Deep-learning architectures are so named because of the many additional layers that input data flow through compared with traditional neural networks and include methods such as deep neural networks, deep belief networks, deep reinforcement learning, recurrent neural networks, and CNNs.
Hierarchy of machine learning.
For supervised learning, the most common instance of AI/ML in medical applications, neural networks “learn” (ie, are trained) by processing examples, each of which contains a known “input” and “result.” Training updates probability-weighted associations between the inputs and outputs, which are stored within the data structure of the net itself. The training of a neural network from a given example is typically conducted by computing an “error value” between the currently processed output of the network (called the inference) and a known target output. The network then adjusts its weighted associations according to a learning policy and this error value. Successive training examples lead the neural network to produce output that converges to the target output.
Data Science: Traditional Statistics vs AI/ML
Studies in medicine have traditionally used statistical methods to determine correlations with an a priori hypothesis evaluated via univariant and/or multivariant analyses. These methodologies have served us well when we have a finite number of variables to assess and specific hypotheses to validate or invalidate. However, a key limiting factor of traditional statistical methods is that statistical power diminishes as the dimension of the multivariate analysis grows. The novelty of deep learning and CNNs (collectively referred to here as AI/ML) is that no predetermined hypothesis is necessarily proposed at the onset of the study. Algorithms can correlate information and associations that may have been otherwise overlooked or unnoticed due to their complexity and multifactorial origins. In these ways, AI can reduce bias when analyzing complex datasets.
Rapid Growth in Medicine
Although few AI/ML products have been deployed in patient-facing use cases, interest, investment, and research leading up to the first commercial entrants have grown exponentially over the past decade. Most of the AI publications in medicine were published in the past 5 years. There are now more than 3300 published articles relating to AI and ML in orthopedics, most of which were published in the past few years. By mid-2022, there were already 212 publications listed on PubMed with the keywords “AI” and “spine” and 136 publications with the keywords “ML” and “spine.” Organizations such as the Medical Image Computing and Computer-Assisted Intervention society continue to introduce broader AI/ML research trends into the medical space. In 2021, over $30 billion in venture capital was invested in health care focusing on AI/ML in areas such as drug, cancer, and molecular therapies. It is clear that technology-enabled services in health care will transform all aspects of care discovery, delivery, and the business of medicine. Surgeons are paying attention and researching to determine where AI and ML will provide value. AI/ML is poised to drive value across the entire care continuum for spine surgery patients and change aspects of how we diagnose, operate, care, and monetize these surgical strategies. The focus of AI/ML should always tie back to the patient and drive the highest quality and most efficient value-based care. In the following paragraphs, we describe examples of how these technologies will touch aspects of everything we do, with an emphasis on spine surgery.
Preoperative Applications
Optimizing the care continuum requires optimizing the allocation of patients to various possible treatments, including medical management and/or surgical intervention. Surgeons are challenged to individualize treatment approaches using the entirety of available patient data due to human reliance on heuristics and decision rules. The ability of AI/ML to take large data aggregates and understand the preoperative state of the patient relative to the outcome desired will inform patient selection. These models will refine the matching of patient variables to treatments that have the highest potential to lead to a good outcome. The interplay between physical findings and radiographic imaging is a promising area where AI/ML is already being used and holds the potential to greatly refine surgical selection criteria and improve access by enabling various providers to evaluate complex anatomic patterns and disease presentations. AI is well suited for tasks such as scene or image identification and as such is increasingly being applied for automating radiographic assessment.2 In spine surgery, the proper utilization of individual quantitative metrics derived from patient-specific imaging data (eg, Cobb angles, sagittal balance, and bone density) is of paramount importance for treatment selection and surgical planning. Manual calculation of each of these metrics is extremely time consuming, even though the process of calculating spinal alignment measurements can be performed by trained observers with minimal subject matter expertise. Recent studies have used AI/ML to automate these measurements and make predictions regarding the suitability of patients for surgical referral.3–5 Computation assessment of radiographic imaging using AI/ML to assess for bone quality may help risk-adjust the surgery or potentially aid in planning ideal screw placement relative to distributed bone density in a given vertebra.6 Studies by Ames and others have shown that AI/ML can be used to assess large datasets and understand the presenting features and patient characteristics that are more likely to lead to improved outcomes or heightened complications such as adjacent segment disease or proximal junctional kyphosis.7–9 Understanding the individual relative to a large paired cohort with similar physical characteristics and radiographic imaging will allow us to better understand modifiable risks that can be addressed before surgery. Perhaps equally importantly, such information may change the surgical plan or how we counsel patients regarding the risk/benefit ratio of operating.10–20
Although clinicians prize data-driven decision-making, surgeons are often too time constrained to manually quantify their patient assessments. Surgeons will find their expertise supplemented by automated AI/ML systems that reliably, explainably, and rapidly generate quantitative metrics from preoperative data. Knowing ahead of time which construct portends the best outcome in an individual patient is one way AI/ML will drive value to the patient through informing clinical practice. A single surgeon’s experience, training, habits, and bias can be normalized as individual decision-making about a patient can be vetted or compared with data on millions of patients with similar history, pathology, and outcomes. A surgeon’s experience and outcomes will be amplified exponentially as AI/ML will enhance the decision-making process by identifying and offering validated care options throughout our workflows.21
Intraoperative Applications
Surgery today is a singular event driven by one lead surgeon with physical, mental, and emotional support from a larger team. That surgeon is tasked with understanding the patient’s medical history and imaging, deriving the operative plan, intraoperatively executing the plan, adjusting to any anomalous variables that arise, and adjudicating the success of the operation. The knowledge, skill, experience, and decision-making capabilities of humans are variable and are achieved over time, circumstances, tutelage, positive and negative feedback mechanisms, and luck. Spinal surgery is one of the most complex endeavors in medicine requiring a mastery beyond anatomy, including complex physical and mechanical properties of the spine: the calculus must include the postoperative musculoskeletal dynamics when the patient is upright and ambulating. In 2023, spinal surgeons are solving a multidimensional, real-time problem with myriad variables, relying only on their internal abilities.
AI/ML has the potential to impact the operating surgeon in many ways. Bringing AI/ML to preoperative planning will bring the understanding of the optimal spinal construct informed by the “expanded” experience of aggregated large datasets tied to the outcome.22,23 Intraoperatively, AI has the potential to augment the current navigation technologies and robotics. Alignment and the ultimate construct placement will be guided by computer vision to track and report the progress and ultimate position of the spine.24 Finite element modeling running in real time against the final construct will give information about robustness to gravity in the upright position.23 Such AI/ML tools will allow the surgeon to configure constructs to obtain an optimized result and reduce stress at adjacent levels, leading to better patient outcomes. Through the use of computer vision and applied AI/ML programs, the surgical scene can also be captured and used to provide real-time clinical decision support (scene augmentation, overlays, and imagery) to the surgeon and to extract labeled data that begins to form a “surgical EMR.”25 Each segment of the surgery can be extracted, step by step, and archived. Cases with optimal outcomes will be captured, and those with suboptimal outcomes and complications will be used to inform the progression of AI/ML to identify best practices leading to the best outcomes. In the near future, the live surgical scene will be “read” by the computer, and the surgeon will be presented with clinical decision support that is correlated with outcomes and the experience of thousands of cases.5,26,27 The promise of AI/ML is to give a single surgeon the benefit of hundreds of cumulative years of experience to augment the judgment that is derived from one’s own experience and repetition. AI/ML will supplement the surgeon in determining what is the best thing to do at a particular moment, in a specific type of operation, on a given patient.
Postoperative Applications
AI/ML will touch all aspects of the care continuum, including the postoperative period. Outcome metrics and patient physical characteristics can be objectively recorded using emerging technologies within the hospital setting and in the patient’s home. Tools are being developed to “watch” patients in the postoperative period and predict which patients are on track and which are falling behind the recovery curve or showing signs of complications.28 Early warning systems take various patient vital signs and nursing assessments and create AI/ML-based scoring systems to alert and forewarn which patients are most likely to require attention in the immediate postoperative period.29,30 Similarly, numerous sensors in cell phones are creating patient “digital phenotypes” that can be used to survey, engage, and predict aspects of the preoperative patient state and the postoperative course.31,32 A picture of a wound evaluated through AI/ML can identify erythema or other findings suggestive of wound infection or dehiscence. Simple postoperative questions may be presented via phone-based programs and refined to determine whether a patient is “on course” or failing at home based on predetermined metrics compared against matched controls from similar procedures. Physical therapy may be recorded, analyzed, and graded to determine the outcome, which is then fed back into the database to inform the patient selection and preoperative variables, informing a feedback cycle to refine the AI/ML and human decision-making processes. High-quality, high-volume data allows for continued refinement of the computational algorithms, which in turn leads to improved accuracy of the desired outcome metrics upon which early management decisions may be guided.
Future Directions
As the costs to provide health care continue to grow with climbing labor costs and accompanying scarcity of skills, hospital systems will look to do more with less. Augmenting and scaling human capabilities will serve as means of meeting the needs of a growing and aging population. AI/ML and the databases upon which these technologies source “knowledge” will provide for our future surgical paradigm. These data will also be leveraged by data scientists to mitigate the most burdensome aspects of care delivery for all providers and staff across the care continuum while feeding into larger and better-informed public health initiatives.33 The ability of AI/ML to automate mundane aspects of care will allow surgeons to focus on providing surgical care vs time spent on purely administrative tasks. Automated note-taking, billing, coding, intake, and image analysis are immediately obtainable with AI/ML tools and could reduce the burden of EMR usage and documentation for the surgeon while reducing facility costs of revenue cycle management. Maximizing the margin contribution and reducing the cost of care delivery will be essential as the financial strain on the health care ecosystem worsens with a growing and aging populous and shortage of adequately trained surgeons.34
The optimized future state afforded by AI/ML requires data and the means with which to organize it at vast scope and scale. As acclaimed AI researcher Pieter Abbeel and New York Times correspondent Cade Metz note,35 the creation of a common task framework surrounding the 2010 ImageNet dataset by Fei-Fei Li36 and its ensuing organized challenges were instrumental in the rapid development of computer vision systems. Data are the progenitor in AI/ML, and the field of spinal surgery lacks sufficient accessible, high-quality, reproducible, and actionable data. Elyan et al37 reviewed the existing medical image datasets available for computer vision analysis (summarized in Table), yet spine and orthopedic pathologies are notably absent. Data in all forms—visual, radiographic, verbal, written, and otherwise—provide the foundation upon which AI/ML will drive surgery into the next S-curve of innovation in health care. Creating these datasets must be a priority for our field for both research and clinical applications. As the industry builds and enables devices to capture the breadth of data created in surgery, clinicians, as key opinion leaders, must lead the charge to advocate for and enable the archival of the information upon which AI programs are built.
Applications of AI/ML for medical imaging.
The most influential technology companies in the world have the deepest pools of AI talent and are bringing these capabilities to bear in medicine. Microsoft’s $19.7 billion acquisition of Nuance has substantial beyond-dollar value in its recognition of the practical application of NLP in health care. Nuance, through its Dragon speech transcription tool, has gained access to nearly 80% of the hospitals. A treasure trove of data like this, when analyzed by NLP and ML, can automatically generate Common Procedural Terminology codes from operative note transcripts, automating processes and leading to significant time savings.38
Surgeons must take the lead in clearly defining the clinical problems that will motivate the next generation of surgical tools in partnership with scientific and technical collaborators. The interface between the computer scientist and clinician will be critical to build systems that solve real-world problems, and these partnerships must expand beyond academia into the industry to create virtuous cycles of innovation, clinical adaptation, and value creation. The continued evolution of AI/ML architecture and techniques must always point back to the patient outcome as a fundamental tenet for any technology that engages within the surgery and care continuum. Surgeons must also identify clinical needs where AI/ML can provide access to better, faster, and cheaper care. The computational sciences are innovating and evolving at a tremendous speed as demand is driven largely outside of medicine by other industries and the consumer space. The field of spinal surgery can capitalize on increasing data availability and broader societal trends if surgical leaders engage with emerging technologies, define use cases for AI/ML, form multidisciplinary teams, and motivate the adaptation and evolution of AI/ML to meet the demands of patient care, regulatory policies, and the surgical continuum.
Limitations
Although the above concepts make the capabilities of AI/ML seem limitless, these advancements are constrained unless several impediments related to function and scale are resolved. Computational speed, high data transfer rates, and storage of immense datasets are necessary for these systems to “gain” experience. Although the cost of data storage drops according to Kryder’s law, the volume of health care data is growing even faster.39,40 Thus, clinically relevant value will only be fully achieved once physicians, hospitals, and the entire health care ecosystem make the foundational investments required to allow the capture and sharing of these immense datasets. Data privacy and security represent another area that needs updated rules and regulations. All learning-based methods experience the risk of “over-fitting” due to small or homogeneous datasets. The ability to transmit and share data beyond a single hospital, system, or vendor is critical to enable these AI/ML systems to learn from broadly representative data and avoid bias against underserved and at-risk populations. Data privacy can be solved through novel methods of data deidentification and encryption, but US Health Insurance Portability and Accountability Act laws need to be modernized to allow the full breadth of these technological advancements to reach their potential as a rising tide for all.41
The US Food and Drug Administration and other regulatory bodies will need to adopt strategies to deal with these emerging computational processes that will underpin technologies in surgery and across the care continuum.42 Dynamic software that evolves with data is hampered by current regulations and the need for code to be “locked” before approval. “Learning” cycles for AI/ML should be rapid as real-time clinical decision support is brought to fruition. How the code is tested/validated and maintains regulatory compliance needs attention. How we approach software documentation, verification, and validation will need to be rethought before some of the cutting-edge technologies can reach commercial use.
Beyond patient privacy and regulatory technical issues are cultural concerns: the surgeon must understand and have comfort in opening the “black box” of surgery by recording the operation and component pieces of the care continuum. Concerns about litigation and discoverable information from the surgery will need to be overcome to allow all surgeries to be captured, processed, and archived to fully realize the benefits of AI/ML in surgery. Likewise, the fear of “big brother” and continuous oversight needs to be considered and concerns mitigated before the full potential of AI/ML can be realized. The most recent surgeon cohorts (born in the 1990s) have high digital literacy and are now graduating residency and entering practice. With generational shifts and increased comfort in sharing personal and professional activities via social media and other platforms, younger, emerging surgeons often look differently at privacy, liability, recording of their surgical performance, and various aspects of patient engagement. However, the average age of an orthopedic surgeon is 57 years, and technological advances happen faster than generational turnover, so lifelong learning and adaptation of new technologies are paramount to the evolution of our field.
Conclusions
AI/ML holds promise to impact every aspect of the health care value chain. From optimizing surgical patient selection to augmenting the surgeon’s native skill and intellect, providing heightened postoperative surveillance, and reducing errors, costs, and administrative burden, AI/ML stands to revolutionize the performance and delivery of health care across the continuum of spine care. AI/ML has arrived and will impact our practices moving forward. It is incumbent on all clinicians to understand the benefits and potential risks of these new technologies in order to meet the new challenges of providing our patients with the best possible care in the age of digitized surgery.
Acknowledgments
The authors thank James Youngquist, Gabriel Jones, and Kristin Kraus for editorial support.
Footnotes
Funding Dr. Donoho’s work is supported by NIH K23EB034110-01.
Declaration of Conflicting Interests The authors report no conflicts of interest in this work.
Disclosures Dr. Browd is co-founder and has equity and intellectual property interests in Proprio, Inc.
- This manuscript is generously published free of charge by ISASS, the International Society for the Advancement of Spine Surgery. Copyright © 2023 ISASS. To see more or order reprints or permissions, see http://ijssurgery.com.
References
In this issue
Jump to section
Related Articles
Cited By...
- No citing articles found.