Improving diagnosis of skin pathologies in primary care and dermatology in a live environment
Conclusions
Legit.Health significantly enhanced diagnostic accuracy across all healthcare professionals, increasing it from 68.08% to 88.78%. Notably, primary care physicians experienced a substantial improvement in diagnostic accuracy, rising from 62.90% to 89.92%, with the integration of Legit.Health. Similarly, dermatologists saw their diagnostic accuracy improve from 76.47% to 86.93% with the utilization of Legit.Health.
The impact of Legit.Health varied across different skin conditions, demonstrating significant improvements in cases such as tinea, granuloma annulare, and seborrheic keratosis. It's noteworthy that all conditions included in the test showed increased accuracy.
Approximately 58.1% of cases did not necessitate a referral, with minor differences observed between primary care physicians and dermatologists. Additionally, a substantial portion of cases (55.11%) across all specialities could be effectively managed remotely, with primary care physicians exhibiting a slightly higher percentage compared to dermatologists.
When examining the results categorized by pathology, notable discrepancies emerged between primary care doctors and dermatologists in certain cases. It's worth mentioning that the limited number of dermatologists impedes reaching a definitive conclusion at the pathology level. For instance, unanimous agreement exists among experts on the feasibility of remote management for conditions like acne, herpes, and tinea. Conversely, there's a consensus favouring in-person consultations for melanoma, granuloma annulare, and nevus.
Furthermore, dermatologists recommend referring patients with nevus, melanoma, and alopecia to their care, while suggesting that primary care doctors can manage most other pathologies effectively.
Regarding healthcare professional (HCP) feedback, Legit.Health is viewed as a useful and user-friendly tool for its intended purpose, with users highlighting its role in diagnosis support during remote consultations. 87% of HCPs believe they could manage a patient in less than 10 minutes, despite the average consultation time typically being at least 15 minutes.
Reduction of referral and use of remote consultation
Previous works reported that 66% of the patients visiting primary care HCPs are referred to dermatology, with very low (1%) remote consultation rates (González-López et al., 2019). In terms of urgent referral and triage, some institutions have reported that 76.8% of the patients referred from primary HCPs to dermatology result in benign diagnoses (Pagani et al., 2023).
In this experiment, as reported above, we found out that according to the primary HCP with the information provided by the device 39.11% of the cases should be referred, which is a 41.34% lower than the aforementioned referral rates. In addition, our results improve the remote consultation rates, which suggests that diagnosis support tools can help foster remote consultations.
Summary
- Code: LEGIT.HEALTH_SAN_2024
- Status: Finished
- Start date: November 15th, 2023
- Finish date: February 17th, 2024
- Acceptance criteria:
- An improvement of diagnostic accuracy on both primary care physicians and dermatologists
- A positive view of Legit.Health regarding diagnosis support
- A reduction of 30% in referral to dermatology (Warshaw et al. 2011)
- An improvement in remote consultations
Results
The results of diagnostic accuracy are summarized in the table below:
| HCP | Accuracy (%) | Accuracy with Legit.Health (%) | Difference (%) |
|---|---|---|---|
| All specialties | 68.08 | 88.78 | 20.70 |
| Primary care | 62.90 | 89.92 | 27.02 |
| Dermatologist | 76.47 | 86.93 | 10.46 |
An analysis by pathology identified significant impacts for certain conditions, as detailed in the table below:
| Condition | Accuracy (%) | Accuracy with Legit.Health (%) | Difference (%) | p-value |
|---|---|---|---|---|
| Pressure ulcer | 76.92 | 100.00 | 23.08 | 0.25000 |
| Urticaria | 85.71 | 100.00 | 14.29 | 0.50000 |
| Tinea | 62.96 | 100.00 | 37.04 | 0.00195 |
| Seborrheic keratosis | 33.33 | 73.33 | 40 | 0.03125 |
| Psoriasis | 40.00 | 77.50 | 37.5 | 0.00006 |
| Onychomycosis | 76.92 | 88.46 | 11.54 | 0.25000 |
| Nevus | 70.37 | 83.33 | 12.96 | 0.01562 |
| Melanoma | 66.67 | 85.71 | 19.04 | 0.00781 |
| Herpes | 100.00 | 100.00 | 0 | 1.00000 |
| Granuloma annulare | 33.33 | 93.33 | 60 | 0.00391 |
| Dermatitis | 68.06 | 93.06 | 25 | 0.00004 |
| Alopecia | 96.55 | 100.00 | 3.45 | 1.00000 |
| Acne | 65.38 | 69.23 | 3.85 | 1.00000 |
We separated the results per pathology into two tables, one for primary care doctors and another for dermatologists.
Primary care doctors
| Condition | Accuracy (%) | Accuracy with Legit.Health (%) |
|---|---|---|
| Pressure ulcer | 75.00 | 100.00 |
| Urticaria | 88.89 | 100.00 |
| Tinea | 58.82 | 100.00 |
| Seborrheic keratosis | 22.22 | 77.78 |
| Psoriasis | 24.00 | 72.00 |
| Onychomycosis | 81.25 | 100.00 |
| Nevus | 72.73 | 84.85 |
| Melanoma | 65.38 | 92.31 |
| Herpes | 100.00 | 100.00 |
| Granuloma annulare | 11.11 | 88.89 |
| Dermatitis | 53.33 | 91.11 |
| Alopecia | 94.44 | 100.00 |
| Acne | 68.75 | 75.00 |
Dermatologists
Because of the quantity of images per pathology and the total number of dermatologists involved, the evidence is inconclusive and may be biased.
| Condition | Accuracy (%) | Accuracy with Legit.Health (%) |
|---|---|---|
| Pressure ulcer | 80.00 | 100.00 |
| Urticaria | 80.00 | 100.00 |
| Tinea | 70.00 | 100.00 |
| Seborrheic keratosis | 50.00 | 66.67 |
| Psoriasis | 66.67 | 86.67 |
| Onychomycosis | 70.00 | 70.00 |
| Nevus | 66.67 | 80.95 |
| Melanoma | 68.75 | 75.00 |
| Herpes | 100.00 | 100.00 |
| Granuloma annulare | 66.67 | 100.00 |
| Dermatitis | 92.59 | 96.30 |
| Alopecia | 100.00 | 100.00 |
| Acne | 60.00 | 60.00 |
Referral
In assessing the impact of Legit.Health on referrals, our findings revealed that 58.1% of cases did not necessitate a referral. However, this percentage varied slightly to 60.89% for primary care doctors and 53.59% for dermatologists. These results are summarized in the table below:
| HCP | Should refer | Should refer (%) | Should not refer | Should not refer (%) |
|---|---|---|---|---|
| All specialties | 168 | 41.9 | 233 | 58.1 |
| Primary care | 97 | 39.11 | 151 | 60.89 |
| Dermatologists | 71 | 46.41 | 82 | 53.59 |
Background and rationale
Dermatological conditions represent a significant portion of primary care consultations, constituting approximately 5% of all visits. However, discrepancies between diagnoses made by primary care physicians and dermatologists remain substantial, with concordance rates between 57% and 65.52%. This gap in expertise often leads to misdiagnoses, incorrect referrals, and delays in appropriate treatment, particularly in rare and severe conditions. The limited availability of dermatologists, especially in rural areas, further complicates patient care, underscoring the need for innovative solutions to optimize resource allocation and improve diagnostic accuracy.
Teledermatology has shown promise in reducing the pressure on in-person consultations by enabling remote assessments. However, the use of artificial intelligence (AI) presents a transformative opportunity to enhance the diagnostic capabilities of primary care physicians. Legit.Health, an AI-based medical device, has already been validated in the diagnosis of skin conditions and offers advanced tools, such as the automatic scoring of diverse pathologies. This pilot study aims to evaluate whether the use of the Legit.Health medical device can increase the true accuracy of healthcare professionals (HCPs) in the diagnosis of multiple dermatological conditions.
Design
Prospective observational analytical and cross-sectional study. It is designed so as to assess if the use of the medical device Legit.Health by dermatologists and primary care physicians can increase the accuracy in the diagnosis of multiple dermatological conditions, who will be presented with 29 images of patients with different skin conditions. In this case, the data collection will include the diagnosis accuracy for different dermatological pathologies. The study adhered to strict ethical guidelines, ensuring patient confidentiality and compliance with international standards. Patients were provided with detailed information and informed consent. The study's robust methodology aimed to assess the clinical utility and usability of the device.
Product Identification
| Information | |
|---|---|
| Device name | Legit.Health Plus (hereinafter, the device) |
| Model and type | NA |
| Version | 1.0.0.0 |
| Basic UDI-DI | 8437025550LegitCADx6X |
| Certificate number (if available) | MDR 792790 |
| EMDN code(s) | Z12040192 (General medicine diagnosis and monitoring instruments - Medical device software) |
| GMDN code | 65975 |
| Class | Class IIb |
| Classification rule | Rule 11 |
| Novel product | FALSE |
| Novel related clinical procedure | FALSE |
| SRN | ES-MF-000025345 |