19 Harley St, London, W1G 9QJ, UK

Assessing the Accuracy and Reliability of AI-Generated Responses to Patient Questions Regarding Spine Surgery – Lumbar Spinal Stenosis

The article examines the accuracy and completeness of Bing’s responses to patient questions regarding spine surgery. The study utilized SearchResponse.io to determine the most popular patient questions on four specific spine surgery topics. Two spine surgeons assessed the accuracy and completeness of Bing’s answers, and re-queried inaccurate responses two weeks later. The study found that Bing’s answers were generally accurate and complete, with incorrect responses rectified upon re-querying. Commercial websites were the most prevalent source of information, and the type of source, number of sources, and mean JAMA benchmark score were not significantly correlated with answer accuracy. The study highlights the importance of ongoing evaluation and improvement of large language models to ensure reliable and informative results for patients searching for spine surgery information online

Summarised by Mr Mo Akmal – Lead Spinal Surgeon
The London Spine Unit : best rated treatment clinic in London

Published article

CONCLUSIONS: Bing’s answers were generally accurate and adequately complete, with incorrect responses rectified upon re-querying. The plurality of information was sourced from commercial websites. The type of source, number of sources, and mean JAMA benchmark score were not significantly correlated with answer accuracy. These findings underscore the importance of ongoing evaluation and improvement of large language models to ensure reliable and informative results for patients seeking…

Lumbar Decompression Surgery Expert. Best Spinal Surgeon UK
J Bone Joint Surg Am. 2024 Feb 9. doi: 10.2106/JBJS.23.00914. Online ahead of print.ABSTRACTBACKGROUND: In today’s digital age, patients increasingly rely on online search engines for medical information. The integration of large language models such as GPT-4 into search engines such as Bing raises concerns over the potential transmission of misinformation when patients search for,

J Bone Joint Surg Am. 2024 Feb 9. doi: 10.2106/JBJS.23.00914. Online ahead of print.

ABSTRACT

BACKGROUND: In today’s digital age, patients increasingly rely on online search engines for medical information. The integration of large language models such as GPT-4 into search engines such as Bing raises concerns over the potential transmission of misinformation when patients search for information online regarding spine surgery.

METHODS: SearchResponse.io, a database that archives People Also Ask (PAA) data from Google, was utilized to determine the most popular patient questions regarding 4 specific spine surgery topics: anterior cervical discectomy and fusion, lumbar fusion, laminectomy, and spinal deformity. Bing’s responses to these questions, along with the cited sources, were recorded for analysis. Two fellowship-trained spine surgeons assessed the accuracy of the answers on a 6-point scale and the completeness of the answers on a 3-point scale. Inaccurate answers were re-queried 2 weeks later. Cited sources were categorized and evaluated against Journal of the American Medical Association (JAMA) benchmark criteria. Interrater reliability was measured with use of the kappa statistic. A linear regression analysis was utilized to explore the relationship between answer accuracy and the type of source, number of sources, and mean JAMA benchmark score.

RESULTS: Bing’s responses to 71 PAA questions were analyzed. The average completeness score was 2.03 (standard deviation [SD], 0.36), and the average accuracy score was 4.49 (SD, 1.10). Among the question topics, spinal deformity had the lowest mean completeness score. Re-querying the questions that initially had answers with low accuracy scores resulted in responses with improved accuracy. Among the cited sources, commercial sources were the most prevalent. The JAMA benchmark score across all sources averaged 2.63. Government sources had the highest mean benchmark score (3.30), whereas social media had the lowest (1.75).

CONCLUSIONS: Bing’s answers were generally accurate and adequately complete, with incorrect responses rectified upon re-querying. The plurality of information was sourced from commercial websites. The type of source, number of sources, and mean JAMA benchmark score were not significantly correlated with answer accuracy. These findings underscore the importance of ongoing evaluation and improvement of large language models to ensure reliable and informative results for patients seeking information regarding spine surgery online amid the integration of these models in the search experience.

PMID:38335266 | DOI:10.2106/JBJS.23.00914

The London Spine Unit : best rated treatment clinic in London

Read the original publication:

Assessing the Accuracy and Reliability of AI-Generated Responses to Patient Questions Regarding Spine Surgery

Related Posts

0/5 (0 Reviews)

Trusindex Reviews

London Spine Unit Harley Street Hospital

A Focus on High Quality Specialised Care

We are a specialist Private Hospital based on Harley Street, London UK The Harley Street Hospital, Day Surgery Hospital

We provide exclusive health services for individuals seeking Advanced medical, non-surgical or minimally invasive treatments. We are covered by All Insurance Companies apart from AXA PPP

Our Medical Director and Lead Spinal Surgeon Mr Mo Akmal MD is a world renowned Spine Specialist Consultant with over 20 years of experience. He and his team have developed revolutionary techniques to perform all types of Spinal Surgery as a Day Case procedure without traditional General Anaesthetic.

We are constantly improving our techniques for treatment and improving facilities for our patients.

Book your Appointment Now 
Check out our Reviews 
Check out our Patient Videos 
Check our Mr Akmal’s Profile

 

What our patients say ...

J Bone Joint Surg Am. 2024 Feb 9. doi: 10.2106/JBJS.23.00914. Online ahead of print.ABSTRACTBACKGROUND: In today's digital age, patients increasingly rely on online search engines for medical information. The integration of large language models such as GPT-4 into search engines such as Bing raises concerns over the potential transmission of misinformation when patients search for

Revolutionary Keyhole surgical technique to vaporise bulging discs

Dr Mo Akmal Medical Director
Dr Mo Akmal MD - Lead Spinal Surgeon

Laser Disc Surgery can be performed under local anaesthetic at The Harley Street Hospital.

Initial Consultation

with Consultant Spine Surgeon
£ 250
  • No Waiting Times
  • Top NHS affiliated Consultant
  • Includes Clinical Review and Report
  • Multidisciplinary discussion
  • Review of Previous Scans and Reports

Follow up Consultation

any appointment after initial consultation
£ 180
  • Top NHS affiliated Consultant
  • Includes Clinical Review and Report
  • Multidisciplinary discussion

High Resolution MRI Scan

any Single Region (3.0 Tesla)
£ 600
  • No waiting times
  • Includes Full Radiologist Report
  • Open or Closed MRI scan types
  • Copy of Scan on CD

Website Offer

Pre-Booked Online
£1130
£ 800
  • Initial Consultation
  • MRI Scan (Single Region)
  • Follow Up consultation
  • Same Day One Stop Visit
  • Full Medical and MRI scan Report
  • Copy of scan on CD
Popular

If you have any emergency Doctor’s need, simply call our 24 hour emergency

Your personal case manager will ensure that you receive the best possible care.

Call Now 

+44 844 589 2020
+44 203 973 8810