Pilot study for the clinical validation of an artificial intelligence algorithm to optimize the appropriateness of dermatology referrals.

Conclusions

Primary care doctors exhibit a notably low sensitivity of approximately 25% when it comes to the crucial task of deciding whether to refer a patient to secondary care. This is especially true in referrals in the field of Dermatology.

On the other hand, they maintain a high specificity rate of 96%, meaning that when they do decide to make a referral, they are highly likely to be correct in their judgment that specialist care is necessary.

This pattern reflects a cautious approach, as primary care physicians seem to prefer minimizing the risk of false negatives, even if it means that some patients who could benefit from secondary care might not be promptly referred. This cautiousness impedes an optimal utilization of specialist resources.

This study reveals that approximately 29% of the referrals involve common and easily diagnosable conditions, even those from teledermatology. About half of them are related to seborrheic keratosis. Another example of conditions that can be confidently identified and managed without referrals includes skin tags, which the device can reliably confirm, and other entities are unlikely to misdiagnose.

The quality of the images significantly influences the performance of the system. This is a well-established fact in the field because image quality not only impacts the effectiveness of algorithms but also hinders dermatologists from making diagnoses through teledermatology systems. Specifically, poor-quality images of nevi often require an in-person consultation, causing unnecessary delays for specialists.

It is typically complex to calculate precise costs, but we can estimate that algorithms like the device could have a substantial impact on cost optimization while simultaneously reducing waiting times and expediting urgent cases.

In terms of the waiting list, the analysis assumes that patients could have received treatment earlier, and the appointment delays were a result of the hospital's waiting list.

The current analysis primarily focuses on malignancy, but there may be other criteria to consider when referring patients. While not addressed in this study, the device incorporates additional algorithms that focus on the severity of chronic skin conditions. Moreover, in certain cases, referrals can be based on the possibility of a specific disease that may be complex to manage.

Summary

• Code: LEGIT.HEALTH_DAO_Derivación_O_2022
• Status: Ongoing
• Start date: April 7th, 2022
• Acceptance criteria:
  • Improve the adequacy of referrals to dermatology
  • A reduction of waiting lists (at least 30% Warshaw et al. 2011)
  • A reduction of the costs in secondary care

Background and rationale

Skin-related conditions account for a significant portion of primary care visits, representing about 5% of all consultations, predominantly among working-age populations. However, discrepancies between primary care physicians and dermatologists in diagnosing these conditions are common, with diagnostic agreement ranging between 57% and 65%. The lack of dermatological expertise among general practitioners, coupled with time constraints and limited access to specialists, poses a challenge in effectively managing skin diseases, particularly in smaller communities where access to dermatologists is scarce. This results in misdiagnoses and unnecessary referrals, leading to inefficient healthcare delivery.

Recent advances in artificial intelligence (AI) and image recognition have shown promising potential in improving diagnostic accuracy for skin conditions. Computer-aided diagnosis (CAD) systems, which combine AI and digital image processing, allow for more accurate interpretation of medical images. Studies demonstrate that AI algorithms can classify skin lesions with a level of competence comparable to that of expert dermatologists. The use of AI-based tools offers a significant opportunity to improve clinical workflows, enhancing the accuracy of patient assessments and reducing the burden on healthcare professionals.

This study aims to clinically validate an innovative AI tool for grading the severity of skin conditions. By improving diagnostic precision and referral decisions, this technology could enhance medical practice, particularly in the early detection of skin cancer, while also opening new avenues for research on treatment effectiveness and disease subtypes.

Design

Prospective observational analytical study of a longitudinal clinical case series. It is intended to assess if Legit.Health is a valid tool to improve referrals to dermatology. This investigation includes a population of 400 patients treated in their health centres and referred to dermatology of Cruces and Basurto Hospitals. Data collection will include photograph analysis, data about clinical flow and the possible reduction of costs. The study adhered to strict ethical guidelines, ensuring patient confidentiality and compliance with international standards. Patients were provided with detailed information and informed consent. The study's robust methodology aimed to assess the clinical utility and usability of the device.

Product Identification

	Information
Device name	Legit.Health Plus (hereinafter, the device)
Model and type	NA
Version	1.0.0.0
Basic UDI-DI	8437025550LegitCADx6X
Certificate number (if available)	MDR 792790
EMDN code(s)	Z12040192 (General medicine diagnosis and monitoring instruments - Medical device software)
GMDN code	65975
Class	Class IIb
Classification rule	Rule 11
Novel product	FALSE
Novel related clinical procedure	FALSE
SRN	ES-MF-000025345