Improving the diagnosis of Generalized Pustular Psoriasis (GPP)
Conclusions
Legit.Health significantly improved healthcare professionals' diagnosis of generalized pustular psoriasis (GPP), doubling the diagnosis rate with the use of the medical device. In primary care, this effect was even greater, with a 120% increase in GPP diagnoses.
On average, diagnostic accuracy increased from 47.91% to 62.81%, a relative increase of 31%. For primary care doctors, the improvement was even more pronounced, with a 40% relative increase in correct diagnoses.
For other conditions like hidradenitis suppurativa and palmoplantar pustulosis, the results were also impressive. Primary care doctors correctly diagnosed 12.43% more cases, while dermatologists showed similar improvements with the device. For palmoplantar pustulosis, primary care doctors demonstrated an outstanding 146% increase, with dermatologists maintaining their performance.
Regarding the dermatologist's data, the results were not statistically significant at the pathology level due to two factors we identified. First, the lower number of participants. We had only four dermatologists, while primary care had eleven participants. This is not a design flaw because the focus was on primary care, but it is a limitation when conducting a detailed analysis. Additionally, the dermatologists had a high level of expertise, especially in hidradenitis suppurativa (HS). This, combined with the relatively average complexity of the HS cases in the experiment, may explain why the tool was not useful for dermatologists in the case of HS. However, this does not mean it is not useful for other dermatologists or dermatologists in general.
In summary, Legit.Health had a substantial impact on the diagnostic capabilities of healthcare professionals, particularly primary care doctors who have less specialization in dermatology. It also significantly benefited dermatologists when diagnosing rare conditions like GPP, which they encounter infrequently in their practice.
Summary
- Code: LEGIT.HEALTH_BI_2024
- Status: Finished
- Start date: May 1st, 2024
- Finish date: August 29th, 2024
- Acceptance criteria:
- An improvement in diagnosis accuracy of generalized pustular psoriasis (GPP)
- An improvement of diagnosis accuracy of other skin conditions such as hidradenitis suppurativa or palmoplantar pustulosis on both, primary care physicians and dermatologists
Background and rationale
Dermatological conditions represent a significant portion of primary care consultations, constituting approximately 5% of all visits. However, discrepancies between diagnoses made by primary care physicians and dermatologists remain substantial, with concordance rates between 57% and 65.52%. This gap in expertise often leads to misdiagnoses, incorrect referrals, and delays in appropriate treatment, particularly in rare and severe conditions like generalized pustular psoriasis (GPP) and hidradenitis suppurativa (HS). The limited availability of dermatologists, especially in rural areas, further complicates patient care, underscoring the need for innovative solutions to optimize resource allocation and improve diagnostic accuracy.
Teledermatology has shown promise in reducing the pressure on in-person consultations by enabling remote assessments. However, the use of artificial intelligence (AI) presents a transformative opportunity to enhance the diagnostic capabilities of primary care physicians. Legit.Health, an AI-based medical device, has already been validated in the diagnosis of skin conditions and offers advanced tools, such as the automatic scoring of GPP severity using the AGPPGA system. This pilot study aims to evaluate whether the use of the Legit.Health medical devices can improve the diagnosis of complicated pathologies with a low incidence but a great impact on the patient's quality of life, such as GPP or also HS.
Design
Prospective observational analytical and cross-sectional study. It is designed so as to assess if the use of the medical device Legit.Health by dermatologists can increase the accuracy in the diagnosis of pathologies as the Generalized Pustular Psoriasis. In this research, we pretend to involve 15 HCPs (primary care physicians or dermatologists), who will be presented with 100 images of patients with GPP, HS or pathologies that can be consumed with these. In this case, the data collection will include the diagnosis accuracy for different pathologies and above all for the target pathologies, such as GPP or HS. The study adhered to strict ethical guidelines, ensuring patient confidentiality and compliance with international standards. Patients were provided with detailed information and informed consent. The study's robust methodology aimed to assess the clinical utility and usability of the device.
Product Identification
| Information | |
|---|---|
| Device name | Legit.Health Plus (hereinafter, the device) |
| Model and type | NA |
| Version | 1.0.0.0 |
| Basic UDI-DI | 8437025550LegitCADx6X |
| Certificate number (if available) | MDR 792790 |
| EMDN code(s) | Z12040192 (General medicine diagnosis and monitoring instruments - Medical device software) |
| GMDN code | 65975 |
| Class | Class IIb |
| Classification rule | Rule 11 |
| Novel product | FALSE |
| Novel related clinical procedure | FALSE |
| SRN | ES-MF-000025345 |
Results
The results of diagnostic accuracy are summarized in the table below:
| HCP | Accuracy (%) | Accuracy with Legit.Health (%) | Difference (%) |
|---|---|---|---|
| All specialties | 47.91 | 62.81 | 14.90 |
| Primary care | 44.29 | 61.71 | 17.42 |
| Dermatologist | 57.25 | 65.65 | 8.40 |
An analysis by pathology identified significant impacts for certain conditions, as detailed in the table below:
| Condition | Accuracy (%) | Accuracy with Legit.Health (%) | Difference (%) | p-value |
|---|---|---|---|---|
| Generalised pustular psoriasis | 23.70 | 46.67 | 22.97 | 0.00001 |
| Eczematous dermatitis | 71.34 | 73.17 | 1.83 | 0.62906 |
| Acute generalized exanthematous pustulosis | 5.00 | 5.00 | 0.00 | 1.00000 |
| Acne | 37.50 | 54.69 | 17.19 | 0.00739 |
| Acné conglobata | 18.40 | 37.60 | 19.20 | 0.00000 |
| Severe inflammatory acne | 10.61 | 43.94 | 33.33 | 0.0000 |
| Seborrheic keratosis | 94.67 | 96.00 | 1.33 | 1.00000 |
| Seborrheic dermatitis | 75.34 | 90.41 | 15.07 | 0.00098 |
| Palmoplantar pustulosis | 45.31 | 79.69 | 34.38 | 0.00002 |
| Plaque psoriasis | 91.89 | 97.30 | 5.41 | 0.12500 |
| Pemphigus vulgaris | 28.77 | 56.16 | 27.39 | 0.0000 |
| Impetigo | 57.43 | 75.68 | 18.25 | 0.0000 |
| Hidradenitis suppurativa | 85.48 | 93.60 | 8.12 | 0.00195 |
| Subcorneal pustular dermatosis | 2.67 | 2.67 | 0.00 | 1.00000 |
| Tinea corporis | 35.96 | 62.50 | 26.54 | 0.00004 |
We separated the results per pathology into two tables, one for primary care doctors and another for dermatologists.
Primary care doctors
| Condition | Accuracy (%) | Accuracy with Legit.Health (%) |
|---|---|---|
| Generalised pustular psoriasis | 20.20 | 44.44 |
| Eczematous dermatitis | 68.33 | 70.83 |
| Acute generalized exanthematous pustulosis | 0.00 | 0.00 |
| Acne | 36.36 | 59.09 |
| Acné conglobata | 21.18 | 47.06 |
| Severe inflammatory acne | 10.87 | 50.00 |
| Seborrheic keratosis | 92.73 | 94.55 |
| Seborrheic dermatitis | 69.81 | 88.68 |
| Palmoplantar pustulosis | 32.61 | 80.43 |
| Plaque psoriasis | 88.89 | 96.30 |
| Pemphigus vulgaris | 22.64 | 43.40 |
| Impetigo | 50.00 | 71.30 |
| Hidradenitis suppurativa | 82.02 | 92.22 |
| Subcorneal pustular dermatosis | 0.00 | 0.00 |
| Tinea corporis | 29.23 | 59.38 |
Dermatologists
Because of the quantity of images per pathology and the total number of dermatologists involved, the evidence is inconclusive and may be biased.
| Condition | Accuracy (%) | Accuracy with Legit.Health (%) |
|---|---|---|
| Generalised pustular psoriasis | 33.33 | 52.78 |
| Eczematous dermatitis | 79.55 | 79.55 |
| Acute generalized exanthematous pustulosis | 18.75 | 18.75 |
| Acne | 40.00 | 45.00 |
| Acné conglobata | 12.50 | 17.50 |
| Severe inflammatory acne | 10.00 | 30.00 |
| Seborrheic keratosis | 100.00 | 100.00 |
| Seborrheic dermatitis | 90.00 | 95.00 |
| Palmoplantar pustulosis | 77.78 | 77.78 |
| Plaque psoriasis | 100.00 | 100.00 |
| Pemphigus vulgaris | 45.00 | 90.00 |
| Impetigo | 77.50 | 87.50 |
| Hidradenitis suppurativa | 94.29 | 97.14 |
| Subcorneal pustular dermatosis | 10.00 | 10.00 |
| Tinea corporis | 54.17 | 70.83 |
Target pathologies
If we focus on primary care doctors and target conditions like generalized pustular psoriasis (GPP), hidradenitis suppurativa, and palmoplantar pustulosis, we will find the following results.
For GPP, with a p-value of 0.00015:
- Legit.Health supports doctors' diagnoses in 12.12% of cases.
- Legit.Health improves doctors' diagnoses in 32.32% of cases.
- Legit.Health has no impact on doctors' diagnoses in 47.47% of cases.
- Legit.Health negatively affects doctors' diagnoses in 8.08% of cases.
For hidradenitis suppurativa, with a p-value of 0.00391:
- Legit.Health supports doctors' diagnoses in 82.02% of cases.
- Legit.Health improves doctors' diagnoses in 10.11% of cases.
- Legit.Health has no impact on doctors' diagnoses in 7.87% of cases.
- Legit.Health does not affect negatively doctors' diagnoses.
For palmoplantar pustulosis, with a p-value of 0:
- Legit.Health supports doctors' diagnoses in 32.61% of cases.
- Legit.Health improves doctors' diagnoses in 47.83% of cases.
- Legit.Health has no impact on doctors' diagnoses in 19.57% of cases.
- Legit.Health does not affect negatively doctors' diagnoses.