Summary​

• Study type: Multireader multicase (MRMC) study
• Journal: JMIR Dermatology (2026)
• DOI: https://doi.org/10.2196/82030
• Readers: 15 health care practitioners (11 primary care practitioners and 4 dermatologists)
• Cases: 100 images of GPP and visually similar conditions, reviewed first without device assistance and then with the device's top 5 predictions
• Objective: Assess whether the Legit.Health device improves the accuracy of health care practitioners in diagnosing generalized pustular psoriasis

Background and rationale​

Generalized pustular psoriasis is a rare, chronic, systemic inflammatory disease with an unpredictable clinical course. Its rarity and visual similarity to other dermatologic conditions make it challenging to diagnose, which can delay appropriate treatment. Artificial intelligence presents a transformative opportunity to enhance the diagnostic capabilities of health care practitioners, particularly in primary care settings where specialized dermatologic expertise may not be available.

Legit.Health is an AI-based medical device incorporating a deep neural network trained on more than 200 skin conditions. This study assessed whether the device, used as a diagnosis support tool, can increase the accuracy of health care practitioners in identifying GPP.

Design​

The device was fine-tuned on 4397 GPP images and then evaluated by 15 health care practitioners (11 primary care practitioners and 4 dermatologists). Each reader reviewed 100 images of GPP and visually similar conditions in two stages: first without device assistance, and then with the device's top 5 predictions. Diagnostic accuracy was compared between the two stages to measure the impact of the device as a diagnosis support tool.

Results​

Device standalone performance​

The device was fine-tuned on 4397 GPP images. The table below reports its diagnostic performance as a standalone classifier before and after fine-tuning.

After fine-tuning, the device reached top-1, top-3, and top-5 sensitivity of 0.80, 0.86, and 0.90, respectively, while keeping specificity above 0.90 across all thresholds.

Improvement in GPP diagnostic accuracy​

When health care practitioners used the device as a diagnosis support tool, their accuracy in identifying GPP improved significantly (P<.001) across every group.

Diagnostic accuracy for generalized pustular psoriasis before and after using the device, by reader group. The largest relative gain was observed among primary care practitioners.

Conclusions​

Use of Legit.Health was associated with a significant improvement in the accuracy of health care practitioners when diagnosing GPP and other dermatologic conditions such as hidradenitis suppurativa. Overall diagnostic accuracy for GPP increased by 22.97%, with primary care practitioners improving by 24.24% and dermatologists by 19.45%.

As a standalone classifier, the device reached a top-1, top-3, and top-5 sensitivity of 0.80, 0.86, and 0.90, respectively, with specificity exceeding 0.90 across all thresholds.

These results highlight that the device could be particularly beneficial in primary care settings, where specialized dermatologic expertise may not be available, helping to reduce misdiagnoses and streamline decision-making.

Product Identification​

ALADIN was validated retrospectively against IGA and Global Acne Grading System (GAGS) scores annotated by expert dermatologists, using both a public dataset of predominantly severe cases and a private dataset of mild-to-moderate cases. Its performance aligned well with the interobserver correlation of dermatologists, supporting its use as a standardized, explainable grading tool.

Figure 2: The ALADIN acne lesion density calculator

Results​

Table 2: Performance of ALADIN (Acne Lesion And Density INdex) with regard to Global Acne Grading System scores

Correlation of ALADIN scores with expert-annotated Global Acne Grading System (GAGS) scores, reported for two model versions (AcneSeverity-V1 and AcneSeverity-V2) using Spearman and Pearson coefficients. The dermatologist row reflects the interobserver correlation between expert raters, used as the reference benchmark. ALADIN's correlation with GAGS approaches and, for AcneSeverity-V2, matches the agreement observed between dermatologists themselves.

ALADIN demonstrates consistent performance and excellent interpretability. Compared with manual scoring systems, it provides a streamlined, data-driven alternative capable of minimizing observer bias. When integrated into a medical device software, it can further aid in standardizing acne severity assessment and enhance patient monitoring.

Figure 1: Samples compiled for evaluation

Results​

Table 1: Evaluation results​

These results show the performance of our proposed area measurement methodology when applied to different body parts. MAE, mean absolute error (lower is better); SRPE, symmetric root percentage error (higher is better).

Table 2: Performance comparison with pixel-to-area baseline​

These results show the performance of our proposed area measurement methodology in comparison with a standard methodology for area measurement. MAE, mean absolute error (lower is better); SRPE, symmetric root percentage error (higher is better).

The proposed approach reduces Mean Absolute Error by more than half (47 mm² → 20 mm²) versus conventional pixel-to-area mapping, with consistent gains across all SRPE thresholds.

The APASI represents a breakthrough in psoriasis assessment technology, as highlighted in recent scientific validation:

APASI provides a robust AI-driven framework for psoriasis severity assessment, delivering rapid, objective, and precise evaluations. Its integration into clinical and research workflows could enhance disease monitoring, improve treatment assessment and reduce evaluation costs.

Legit.Health significantly enhanced primary care physicians' diagnostic accuracy, increasing it from 72.96% to 82.22%.

The impact of Legit.Health varied across different skin conditions, demonstrating significant improvements in hidradenitis suppurativa, urticaria, and actinic keratosis. However, p-values were not statistically significant for all cases, due to the low number of samples per pathology.

Approximately 49% of cases did not necessitate a referral. Additionally, 60.74% of cases across all specialities could be effectively managed remotely.

Reduction of referral and use of remote consultation​

Previous studies reported that 66% of patients visiting primary care HCPs are referred to dermatology, with very low (1%) remote consultation rates (González-López et al., 2019). In terms of urgent referral and triage, some institutions have reported that 76.8% of patients referred from primary HCPs to dermatology result in benign diagnoses (Pagani et al., 2023).

In this experiment, we found that 49% of cases should be referred according to the primary HCP with the information provided by the device, which is 17% lower than the aforementioned referral rates. Additionally, our results improve the remote consultation rates, suggesting that diagnostic support tools can help foster remote consultations.

Summary​

• Code: LEGIT.HEALTH_PH_2024
• Status: Finished
• Start date: June 4th, 2024
• Finish date: September 13th, 2024
• Acceptance criteria:
  • An improvement of diagnostic accuracy on both primary care physicians and dermatologists.
  • A reduction of 30% of referrals to dermatology (Warshaw et al. 2011).
  • An improvement in remote consultations.

Results​

An analysis by pathology identified significant impacts for certain conditions, as detailed in the table below:

Referral​

In assessing the impact of Legit.Health on referrals, our findings revealed that 48.89% of cases did not necessitate a referral.

Remote consultations​

Furthermore, we examined the feasibility of handling cases remotely through teledermatology. The results show that 60.74% of the cases can be handled remotely.

Conducting a Pearson's chi-squared test on the necessity for referrals and teleconsultations, we concluded with 95% confidence that a strong association exists between referrals and remote consultations. Specifically:

• 36.67% of the cases do not require a referral and can have follow-up remotely
• 12.22% of the cases do not require referral but require an in-person appointment
• 24.07% of the cases require referral and remote consultation
• 27.04% of the cases require a referral in addition to an in-person appointment

Background and rationale​

Dermatological conditions represent a significant portion of primary care consultations, constituting approximately 5% of all visits. However, discrepancies between diagnoses made by primary care physicians and dermatologists remain substantial, with concordance rates between 57% and 65.52%. This gap in expertise often leads to misdiagnoses, incorrect referrals, and delays in appropriate treatment, particularly in rare and severe conditions. The limited availability of dermatologists, especially in rural areas, further complicates patient care, underscoring the need for innovative solutions to optimize resource allocation and improve diagnostic accuracy.

Teledermatology has shown promise in reducing the pressure on in-person consultations by enabling remote assessments. However, the use of artificial intelligence (AI) presents a transformative opportunity to enhance the diagnostic capabilities of primary care physicians. Legit.Health, an AI-based medical device, has already been validated in the diagnosis of skin conditions and offers advanced tools, such as the automatic scoring of diverse pathologies. This pilot study aims to evaluate whether the use of the Legit.Health medical device can increase the true accuracy of healthcare professionals (HCPs) in the diagnosis of multiple dermatological conditions.

Design​

Prospective observational analytical and cross-sectional study. It is designed so as to assess if the use of the medical device Legit.Health by dermatologists and primary care physicians can increase the accuracy in the diagnosis of multiple dermatological conditions, who will be presented with 29 images of patients with different skin conditions. In this case, the data collection will include the diagnosis accuracy for different dermatological pathologies. The study adhered to strict ethical guidelines, ensuring patient confidentiality and compliance with international standards. Patients were provided with detailed information and informed consent. The study's robust methodology aimed to assess the clinical utility and usability of the device.

Product Identification​

Legit.Health significantly enhanced primary care physicians' diagnostic accuracy, increasing it from 72.96% to 82.22%.

The impact of Legit.Health varied across different skin conditions, demonstrating significant improvements in hidradenitis suppurativa, urticaria, and actinic keratosis. However, p-values were not statistically significant for all cases, due to the low number of samples per pathology.

Approximately 49% of cases did not necessitate a referral. Additionally, 60.74% of cases across all specialities could be effectively managed remotely.

Reduction of referral and use of remote consultation​

Previous studies reported that 66% of patients visiting primary care HCPs are referred to dermatology, with very low (1%) remote consultation rates (González-López et al., 2019). In terms of urgent referral and triage, some institutions have reported that 76.8% of patients referred from primary HCPs to dermatology result in benign diagnoses (Pagani et al., 2023).

In this experiment, we found that 49% of cases should be referred according to the primary HCP with the information provided by the device, which is 17% lower than the aforementioned referral rates. Additionally, our results improve the remote consultation rates, suggesting that diagnostic support tools can help foster remote consultations.

Summary​

• Code: LEGIT.HEALTH_PH_2024
• Status: Finished
• Start date: June 24th, 2022
• Finish date: January 10th, 2024
• Acceptance criteria:
  • An improvement of diagnostic accuracy of 10% (Ferri et al. 2020) in primary care physicians and dermatologists.

Results​

An analysis by pathology identified significant impacts for certain conditions, as detailed in the table below:

Referral​

In assessing the impact of Legit.Health on referrals, our findings revealed that 48.89% of cases did not necessitate a referral.

Remote consultations​

Furthermore, we examined the feasibility of handling cases remotely through teledermatology. The results show that 60.74% of the cases can be handled remotely.

Conducting a Pearson's chi-squared test on the necessity for referrals and teleconsultations, we concluded with 95% confidence that a strong association exists between referrals and remote consultations. Specifically:

• 36.67% of the cases do not require a referral and can have follow-up remotely
• 12.22% of the cases do not require referral but require an in-person appointment
• 24.07% of the cases require referral and remote consultation
• 27.04% of the cases require a referral in addition to an in-person appointment

Background and rationale​

Dermatological conditions represent a significant portion of primary care consultations, constituting approximately 5% of all visits. However, discrepancies between diagnoses made by primary care physicians and dermatologists remain substantial, with concordance rates between 57% and 65.52%. This gap in expertise often leads to misdiagnoses, incorrect referrals, and delays in appropriate treatment, particularly in rare and severe conditions. The limited availability of dermatologists, especially in rural areas, further complicates patient care, underscoring the need for innovative solutions to optimize resource allocation and improve diagnostic accuracy.

Teledermatology has shown promise in reducing the pressure on in-person consultations by enabling remote assessments. However, the use of artificial intelligence (AI) presents a transformative opportunity to enhance the diagnostic capabilities of primary care physicians. Legit.Health, an AI-based medical device, has already been validated in the diagnosis of skin conditions and offers advanced tools, such as the automatic scoring of diverse pathologies. This pilot study aims to evaluate whether the use of the Legit.Health medical device can increase the true accuracy of healthcare professionals (HCPs) in the diagnosis of multiple dermatological conditions.

Design​

Prospective observational analytical and cross-sectional study. It is designed so as to assess if the use of the medical device Legit.Health by dermatologists and primary care physicians can increase the accuracy in the diagnosis of multiple dermatological conditions, who will be presented with 29 images of patients with different skin conditions. In this case, the data collection will include the diagnosis accuracy for different dermatological pathologies. The study adhered to strict ethical guidelines, ensuring patient confidentiality and compliance with international standards. Patients were provided with detailed information and informed consent. The study's robust methodology aimed to assess the clinical utility and usability of the device.

Product Identification​

Our work on the AUAS was presented at congresses such as the AEDV annual meeting in the year 2021. The following video shows a short explanation of the AUAS in the congress mentioned above:

The AIHS4 has been echoed in recent scientific publications, such as the following article by the National Research Council of Italy, and the Universities of Palermo and Messina:

(...) to overcome the IHS4, which is time-consuming and subject to variability, the AIHS4 is introduced, using a DL model, Legit.Health-IHS4net, for lesion detection (...). This evidence highlights the utility of AI in evidence-based dermatology, offering a potential tool to empower dermatologists in daily practice and clinical trials.

The medical device demonstrated high performance in malignancy detection and pathology diagnosis, performing at a level comparable to that of expert dermatologists both for the retrospective and prospective analysis. This performance was achieved despite the inherent bias in the dataset, which only includes lesions deemed suspicious enough to warrant a biopsy.

The device algorithms demonstrate moderate accuracy in predicting the Ludwig score for FAA. The overall accuracy was 47% in the prospective analysis, improving to 53% in the latest model. There is a low incidence of predicted grades differing by two grades from the investigator's score and a 50% correlation between the alopecia percentage and the investigator's score. These results indicate the potential of the device solution as a tool for estimating the Ludwig score for the FAA. Besides that, expanding the dataset and incorporating more diverse image samples could enhance the model's robustness and generalizability.

Sumary​

• Code: LEGIT.HEALTH_IDEI_2023
• Status: The first part of the study is finished. The second part will start in Q1, 2025
• Start date: February 2nd, 2024
• Finish date: August 7th, 2024
• Acceptance criteria:
  • An improvement of diagnostic accuracy of 10% (Ferri et al. 2020)
  • Scores equal to or greater than 70 on the System Usability Scale (SUS)
  • An AUC equal to or greater than 0.8 detecting malignancy
  • A sensitivity of 80% detecting malignancy
  • A specificity of 70% detecting malignancy

Background and rationale​

The use of image-based artificial intelligence (AI) holds significant potential for improving diagnostic accuracy in medical visual assessments. The COVID-19 pandemic, which limited access to in-person healthcare, has accelerated the adoption of telemedicine, highlighting the importance of AI in triaging and supporting decision-making processes. In dermatology, conditions such as pigmented lesions, acne, and alopecia represent high-demand cases requiring significant in-person resources and specialist attention. AI tools can play a crucial role in optimizing these processes, reducing the workload, and improving efficiency in patient management.

Advancements in image recognition and AI technologies have led to innovations in diagnosing skin conditions, with Computer-Aided Diagnosis (CAD) systems proving their ability to classify lesion images at a level comparable to expert clinicians. This study aims to evaluate the Legit.Health tool, developed by AI Labs Group S.L., which utilizes AI to enhance clinical workflow and patient care for dermatological conditions. The tool seeks to automatically prioritize patients based on urgency, assign appropriate consultations (dermatological or aesthetic), improve diagnostic capabilities in non-specialist staff, and provide a visual record for expert review.

The primary goal of this study is to validate that the AI-based Legit.Health tool improves clinical workflow efficiency and patient care processes by diagnosing and determining the severity of skin lesions. This would result in reduced in-person consultation time and healthcare costs per patient, while assigning the correct type of consultation from the outset. Secondary objectives include decreasing patient wait times based on medical urgency, reducing the number of initial dermatology consultations, and improving both specialist satisfaction and patient usability, ultimately benefiting the clinic's economic outcomes.

Design​

Prospective observational study with both longitudinal and retrospective case series to assess if Legit.Health is a validated tool to improve the clinical flow and patient care process. This investigation encompasses a diverse cohort of patients with different pathologies. On one hand, prospectively, a minimum of 60 cases will be included: 30 with pigmented lesions, 15 with androgenic alopecia and 15 with inflammatory acne. On the other hand, retrospectively, 60 patients with pigmented lesions, 15 with androgenic alopecia and 15 with inflammatory acne will be included. Data collection will include questionnaires, photograph analysis, and patient satisfaction surveys. The study adhered to strict ethical guidelines, ensuring patient confidentiality and compliance with international standards. Patients were provided with detailed information and informed consent. The study's robust methodology aimed to assess the clinical utility and usability of the device.

Product Identification​

Primary care doctors exhibit a notably low sensitivity of approximately 25% when it comes to the crucial task of deciding whether to refer a patient to secondary care. This is especially true in referrals in the field of Dermatology.

On the other hand, they maintain a high specificity rate of 96%, meaning that when they do decide to make a referral, they are highly likely to be correct in their judgment that specialist care is necessary.

This pattern reflects a cautious approach, as primary care physicians seem to prefer minimizing the risk of false negatives, even if it means that some patients who could benefit from secondary care might not be promptly referred. This cautiousness impedes an optimal utilization of specialist resources.

This study reveals that approximately 29% of the referrals involve common and easily diagnosable conditions, even those from teledermatology. About half of them are related to seborrheic keratosis. Another example of conditions that can be confidently identified and managed without referrals includes skin tags, which the device can reliably confirm, and other entities are unlikely to misdiagnose.

The quality of the images significantly influences the performance of the system. This is a well-established fact in the field because image quality not only impacts the effectiveness of algorithms but also hinders dermatologists from making diagnoses through teledermatology systems. Specifically, poor-quality images of nevi often require an in-person consultation, causing unnecessary delays for specialists.

It is typically complex to calculate precise costs, but we can estimate that algorithms like the device could have a substantial impact on cost optimization while simultaneously reducing waiting times and expediting urgent cases.

In terms of the waiting list, the analysis assumes that patients could have received treatment earlier, and the appointment delays were a result of the hospital's waiting list.

The current analysis primarily focuses on malignancy, but there may be other criteria to consider when referring patients. While not addressed in this study, the device incorporates additional algorithms that focus on the severity of chronic skin conditions. Moreover, in certain cases, referrals can be based on the possibility of a specific disease that may be complex to manage.

Summary​

• Code: LEGIT.HEALTH_DAO_Derivación_O_2022
• Status: Ongoing
• Start date: April 7th, 2022
• Acceptance criteria:
  • Improve the adequacy of referrals to dermatology
  • A reduction of waiting lists (at least 30% Warshaw et al. 2011)
  • A reduction of the costs in secondary care

Background and rationale​

Skin-related conditions account for a significant portion of primary care visits, representing about 5% of all consultations, predominantly among working-age populations. However, discrepancies between primary care physicians and dermatologists in diagnosing these conditions are common, with diagnostic agreement ranging between 57% and 65%. The lack of dermatological expertise among general practitioners, coupled with time constraints and limited access to specialists, poses a challenge in effectively managing skin diseases, particularly in smaller communities where access to dermatologists is scarce. This results in misdiagnoses and unnecessary referrals, leading to inefficient healthcare delivery.

Recent advances in artificial intelligence (AI) and image recognition have shown promising potential in improving diagnostic accuracy for skin conditions. Computer-aided diagnosis (CAD) systems, which combine AI and digital image processing, allow for more accurate interpretation of medical images. Studies demonstrate that AI algorithms can classify skin lesions with a level of competence comparable to that of expert dermatologists. The use of AI-based tools offers a significant opportunity to improve clinical workflows, enhancing the accuracy of patient assessments and reducing the burden on healthcare professionals.

This study aims to clinically validate an innovative AI tool for grading the severity of skin conditions. By improving diagnostic precision and referral decisions, this technology could enhance medical practice, particularly in the early detection of skin cancer, while also opening new avenues for research on treatment effectiveness and disease subtypes.

Design​

Prospective observational analytical study of a longitudinal clinical case series. It is intended to assess if Legit.Health is a valid tool to improve referrals to dermatology. This investigation includes a population of 400 patients treated in their health centres and referred to dermatology of Cruces and Basurto Hospitals. Data collection will include photograph analysis, data about clinical flow and the possible reduction of costs. The study adhered to strict ethical guidelines, ensuring patient confidentiality and compliance with international standards. Patients were provided with detailed information and informed consent. The study's robust methodology aimed to assess the clinical utility and usability of the device.

Product Identification​

The study conducted to evaluate the clinical performance, efficacy, and safety of the device has yielded promising results. The comprehensive analysis of the CUS, Data Utility questionnaire, SUS, and Patient Satisfaction questionnaire has provided valuable insights into the tool's effectiveness in supporting dermatologists in their clinical practice.

The observed sample mean of 76.67 on the CUS suggests that the device has been positively received by the participating specialists. Noteworthy is the unanimous agreement on the ease of use and the high rating for optimizing time according to each patient's needs. It's also worth noting that, despite that the medical device was positively rated by the specialists, the goal of achieving a mean of 80.00 on the CUS was not reached. This result was due to the lower sample size of specialists who completed the questionnaire. In this way, an outlier, and due to the small sample size, impacted disproportionately the overall result, especially for questions 7, 8, 9 and 11. As can be seen by the higher standard deviation and lower mean average in these specific questions. We need to take this fact into account for the following studies and implement measures to mitigate this effect, such as a larger sample size, which could have diluted the effect of the outliers over the statistical outcomes, or predefined management for outliers.

Additionally, the device demonstrated efficiency in generating reports, receiving high ratings from the specialists. These outcomes affirm the device's potential to streamline clinical workflows and enhance patient care.

The Data Utility questionnaire revealed unanimous agreement among specialists regarding the usefulness of a device to facilitate their regular practice. Moreover, the majority expressed a preference for utilizing a device to identify the severity of cases, indicating its potential as an aid in diagnostic support.

The System Usability Scale assessment further underlines the positive reception of the device. Specialists found the tool to be user-friendly, with high scores indicating ease of navigation and minimal complexity. The unanimous agreement on the ease of use and the absence of perceived expertise required to navigate the device emphasize its accessibility and suitability for clinicians.

Patient satisfaction is a crucial aspect of any medical tool or platform. The results of the Patient Satisfaction questionnaire indicate a generally positive response from patients. They found the device to be easy to use, useful in monitoring their condition, and were satisfied with the care provided through the device.

In conclusion, the device has demonstrated notable clinical utility, usability, and safety in the evaluation of dermatological pathologies. The positive responses from both specialists and patients affirm its potential to serve as a valuable clinical decision-support tool. Further research and real-world application are warranted to explore the device's broader impact on dermatological practice and patient care.

Summary​

• Code: LEGIT_COVIDX_EVCDAO_2022
• Status: Finished
• Start date: March 3rd, 2022
• Finish date: October 23rd, 2023
• Acceptance criteria:
  • A score of 8 or higher in the Clinical Utility Score (CUS) filled by the medical staff

Background and rationale​

The COVID-19 pandemic has disrupted healthcare systems globally, particularly in managing non-urgent medical conditions, including dermatology. The reduction of in-person consultations has left many dermatological patients without timely care, exacerbating conditions such as skin cancer and chronic diseases like psoriasis and eczema. The pandemic has underscored the critical need for an efficient, remote diagnostic system to ensure continuous monitoring and timely intervention for dermatological conditions, particularly in countries with an already imbalanced ratio of dermatologists to patients.

Currently, the diagnosis and monitoring of dermatological conditions heavily rely on subjective human assessments, which are prone to inconsistencies and biases. Physicians face challenges in quantifying lesions or disease severity accurately, and patient-reported outcomes often lack reliability. This situation has been further complicated by COVID-19, as patients avoid in-person visits, leading to delayed diagnoses and worsening of conditions. There is an urgent need for reliable tools that can support remote diagnosis and activity tracking of skin pathologies, which would reduce the risks of transmission while maintaining high standards of care.

This study seeks to address these challenges by clinically validating an innovative artificial intelligence (AI) tool designed for remote and continuous monitoring of dermatological conditions. Leveraging the Legit.Health platform, this AI-powered tool can enhance diagnostic precision, reduce human error, and improve the management of chronic skin diseases. By enabling patients to be evaluated from home and providing physicians with a more objective measurement tool, this technology has the potential to improve both patient outcomes and healthcare efficiency.

Design​

A prospective, observational and analytical study designed to evaluate the effectiveness of the device in remotely monitoring chronic dermatologic pathologies. The research encompassed a diverse cohort of 180 patients, which represent the studied population with various dermatological conditions. Data collection will include questionnaires, photograph analysis, and patient satisfaction surveys. The study adhered to strict ethical guidelines, ensuring patient confidentiality and compliance with international standards. Patients were provided with detailed information and informed consent. The study's robust methodology aimed to assess the clinical utility and usability of the device.

Product Identification​

This work is further acknowledged in recent scientific literature, highlighting its potential to revolutionize AD severity assessment.

(...) very promising is the attempt to arrive at an automatic definition of AD severity by using CNNs (...) to achieve a scoring accuracy of erythema, papulation, excoriation, and lichenification severity comparable to that of dermatologists (...). Computational applicative advances in this direction have led to the more recent design of Automatic SCORing Atopic Dermatitis (ASCORAD).

Discover more about ASCORAD from its authors in this webinar (in Spanish).

The device has demonstrated an excellent performance in terms of malignancy prediction, which turns it into a valuable tool to prioritize patients according to their risk of presenting malignancy.

The AUC metric for the malignancy prediction was 0.8983, which is comparable to that of expert healthcare professionals (HCP) and speaks to the potential of using the device to improve clinical workflows.

Regarding skin lesion recognition in general terms, the Top-5 accuracy was 84.22%, which supports the device's intended use as a clinical decision-support tool. Specifically in melanoma, the AUC metric was 84.82% which is considerably high and means the consecution of the goals set out in the hypotheses of the study. On the downside, the Top-1 accuracy was 55.01% in the multiple ICD classification task, but the Top-3 accuracy increased to 75.69%. However, it's important to keep in mind that the Top-1 accuracy metric was not a relevant metric to this study, nor the performance of the device, because the device is designed to always output at least the top five predicted classes. This is aligned with its intended purpose as a clinical decision-support tool.

Given the results of the image dataset collected in this study, it is clear that images of better quality would have improved the functioning of the device and provided more valuable insights. Also, as the raw images were usually taken far away from the skin lesion, the cropping resulted in images of suboptimal resolution, which would have also been corrected by higher quality images - or by actually taking the image closer to the lesion.

Summary​

Background and rationale​

The incidence of cutaneous melanoma (CM), the deadliest form of skin cancer, has been rising significantly, with a sharp increase in both new cases and mortality rates over the past decades. Melanoma metastasizes quickly to distant organs and becomes resistant to conventional therapies, making treatment difficult. However, when diagnosed early, it can be effectively treated through simple surgical excision. The main challenge in early detection lies in differentiating melanoma from benign pigmented skin lesions, as they often share visual similarities, especially during an initial visual examination. Limited access to dermatologists and a lack of public awareness further complicate timely diagnoses, often leading to delayed detection when tumours are already advanced.

Given these challenges, early melanoma diagnosis and prevention have become critical. Recent technological advancements in image recognition and artificial intelligence (AI) have shown promise in improving diagnostic accuracy. Studies have demonstrated that AI algorithms can classify skin lesion images, including melanomas, with a competency level comparable to that of dermatologists. These AI-driven computer-aided diagnosis (CAD) systems present a valuable opportunity for improving early detection and survival rates by assisting healthcare professionals and potentially empowering non-experts to detect melanomas earlier.

This study aims to clinically validate an AI-based system for diagnosing cutaneous melanoma, leveraging artificial vision and machine learning to differentiate between melanomas and benign skin lesions. By developing and validating this technology, the study seeks to improve diagnostic precision and contribute to earlier interventions, potentially enhancing patient outcomes and survival rates.

Product Identification​

Limitations​

Additionally, we believe another factor that limits the results is that, in some cases, malignant cases can not be easily analyzed simply by observing the image. Indeed, some cases require a biopsy to ascertain a diagnosis, regardless of the experience of the observer. This is not a problem for the performance or the safety of the device because whenever there is a suspicion of melanoma, clinicians universally adhere to the protocol of conducting a biopsy to confirm a diagnostic suspicion. This established clinical practice is rooted in the fundamental understanding that the removal of a melanoma is a minor procedure compared to the significant risks associated with the disease. Consequently, practitioners will never rely solely on the device for information when it comes to identifying melanoma, ensuring a comprehensive and cautious approach to diagnosis and treatment.