Conclusions​

Legit.Health significantly enhanced primary care physicians' diagnostic accuracy, increasing it from 72.96% to 82.22%.

The impact of Legit.Health varied across different skin conditions, demonstrating significant improvements in hidradenitis suppurativa, urticaria, and actinic keratosis. However, p-values were not statistically significant for all cases, due to the low number of samples per pathology.

Approximately 49% of cases did not necessitate a referral. Additionally, 60.74% of cases across all specialities could be effectively managed remotely.

Reduction of referral and use of remote consultation​

Previous studies reported that 66% of patients visiting primary care HCPs are referred to dermatology, with very low (1%) remote consultation rates (González-López et al., 2019). In terms of urgent referral and triage, some institutions have reported that 76.8% of patients referred from primary HCPs to dermatology result in benign diagnoses (Pagani et al., 2023).

In this experiment, we found that 49% of cases should be referred according to the primary HCP with the information provided by the device, which is 17% lower than the aforementioned referral rates. Additionally, our results improve the remote consultation rates, suggesting that diagnostic support tools can help foster remote consultations.

Summary​

• Code: LEGIT.HEALTH_PH_2024_NIPPLE
• Status: Finished
• Start date: June 4th, 2024
• Finish date: September 13th, 2024
• Acceptance criteria:
  • An improvement of diagnostic accuracy on both primary care physicians and dermatologists.
  • A reduction of 30% of referrals to dermatology (Warshaw et al. 2011).
  • An improvement in remote consultations.

Conclusions​

Legit.Health significantly enhanced primary care physicians' diagnostic accuracy, increasing it from 72.96% to 82.22%.

The impact of Legit.Health varied across different skin conditions, demonstrating significant improvements in hidradenitis suppurativa, urticaria, and actinic keratosis. However, p-values were not statistically significant for all cases, due to the low number of samples per pathology.

Approximately 49% of cases did not necessitate a referral. Additionally, 60.74% of cases across all specialities could be effectively managed remotely.

Reduction of referral and use of remote consultation​

Previous studies reported that 66% of patients visiting primary care HCPs are referred to dermatology, with very low (1%) remote consultation rates (González-López et al., 2019). In terms of urgent referral and triage, some institutions have reported that 76.8% of patients referred from primary HCPs to dermatology result in benign diagnoses (Pagani et al., 2023).

In this experiment, we found that 49% of cases should be referred according to the primary HCP with the information provided by the device, which is 17% lower than the aforementioned referral rates. Additionally, our results improve the remote consultation rates, suggesting that diagnostic support tools can help foster remote consultations.

Summary​

• Code: LEGIT.HEALTH_PH_2024
• Status: Finished
• Start date: June 24th, 2022
• Finish date: January 10th, 2024
• Acceptance criteria:
  • An improvement of diagnostic accuracy of 10% (Ferri et al. 2020) in primary care physicians and dermatologists.

The Automatic Urticaria Activity Score (AUAS) system has been published in the Journal of Investigative Dermatology (JID) Innovations, showcasing our deep learning-based approach for urticaria severity assessment.

The AIHS4, our novel system for scoring Hidradenitis Suppurativa, is detailed in Skin Research and Technology. This study exemplifies our commitment to developing practical AI solutions for complex dermatological conditions.

Conclusions​

The medical device demonstrated high performance in malignancy detection and pathology diagnosis, performing at a level comparable to that of expert dermatologists both for the retrospective and prospective analysis. This performance was achieved despite the inherent bias in the dataset, which only includes lesions deemed suspicious enough to warrant a biopsy.

The device algorithms demonstrate moderate accuracy in predicting the Ludwig score for FAA. The overall accuracy was 47% in the prospective analysis, improving to 53% in the latest model. There is a low incidence of predicted grades differing by two grades from the investigator's score and a 50% correlation between the alopecia percentage and the investigator's score. These results indicate the potential of the device solution as a tool for estimating the Ludwig score for the FAA. Besides that, expanding the dataset and incorporating more diverse image samples could enhance the model's robustness and generalizability.

Sumary​

• Code: LEGIT.HEALTH_IDEI_2023
• Status: The first part of the study is finished. The second part will start in Q1, 2025
• Start date: February 2nd, 2024
• Finish date: August 7th, 2024
• Acceptance criteria:
  • An improvement of diagnostic accuracy of 10% (Ferri et al. 2020)
  • Scores equal to or greater than 70 on the System Usability Scale (SUS)
  • An AUC equal to or greater than 0.8 detecting malignancy
  • A sensitivity of 80% detecting malignancy
  • A specificity of 70% detecting malignancy

The Dermatology Image Quality Assessment (DIQA) technology, which ensures the clinical utility of images for remote consultations and clinical trials. This was published in the Journal of the American Academy of Dermatology.

Conclusions​

Primary care doctors exhibit a notably low sensitivity of approximately 25% when it comes to the crucial task of deciding whether to refer a patient to secondary care. This is especially true in referrals in the field of Dermatology.

On the other hand, they maintain a high specificity rate of 96%, meaning that when they do decide to make a referral, they are highly likely to be correct in their judgment that specialist care is necessary.

This pattern reflects a cautious approach, as primary care physicians seem to prefer minimizing the risk of false negatives, even if it means that some patients who could benefit from secondary care might not be promptly referred. This cautiousness impedes an optimal utilization of specialist resources.

This study reveals that approximately 29% of the referrals involve common and easily diagnosable conditions, even those from teledermatology. About half of them are related to seborrheic keratosis. Another example of conditions that can be confidently identified and managed without referrals includes skin tags, which the device can reliably confirm, and other entities are unlikely to misdiagnose.

The quality of the images significantly influences the performance of the system. This is a well-established fact in the field because image quality not only impacts the effectiveness of algorithms but also hinders dermatologists from making diagnoses through teledermatology systems. Specifically, poor-quality images of nevi often require an in-person consultation, causing unnecessary delays for specialists.

It is typically complex to calculate precise costs, but we can estimate that algorithms like the device could have a substantial impact on cost optimization while simultaneously reducing waiting times and expediting urgent cases.

In terms of the waiting list, the analysis assumes that patients could have received treatment earlier, and the appointment delays were a result of the hospital's waiting list.

The current analysis primarily focuses on malignancy, but there may be other criteria to consider when referring patients. While not addressed in this study, the device incorporates additional algorithms that focus on the severity of chronic skin conditions. Moreover, in certain cases, referrals can be based on the possibility of a specific disease that may be complex to manage.

Summary​

• Code: LEGIT.HEALTH_DAO_Derivación_O_2022
• Status: Ongoing
• Start date: April 7th, 2022
• Acceptance criteria:
  • Improve the adequacy of referrals to dermatology
  • A reduction of waiting lists (at least 30% Warshaw et al. 2011)
  • A reduction of the costs in secondary care

Conclusions​

The study conducted to evaluate the clinical performance, efficacy, and safety of the device has yielded promising results. The comprehensive analysis of the CUS, Data Utility questionnaire, SUS, and Patient Satisfaction questionnaire has provided valuable insights into the tool's effectiveness in supporting dermatologists in their clinical practice.

The observed sample mean of 76.67 on the CUS suggests that the device has been positively received by the participating specialists. Noteworthy is the unanimous agreement on the ease of use and the high rating for optimizing time according to each patient's needs. It's also worth noting that, despite that the medical device was positively rated by the specialists, the goal of achieving a mean of 80.00 on the CUS was not reached. This result was due to the lower sample size of specialists who completed the questionnaire. In this way, an outlier, and due to the small sample size, impacted disproportionately the overall result, especially for questions 7, 8, 9 and 11. As can be seen by the higher standard deviation and lower mean average in these specific questions. We need to take this fact into account for the following studies and implement measures to mitigate this effect, such as a larger sample size, which could have diluted the effect of the outliers over the statistical outcomes, or predefined management for outliers.

Additionally, the device demonstrated efficiency in generating reports, receiving high ratings from the specialists. These outcomes affirm the device's potential to streamline clinical workflows and enhance patient care.

The Data Utility questionnaire revealed unanimous agreement among specialists regarding the usefulness of a device to facilitate their regular practice. Moreover, the majority expressed a preference for utilizing a device to identify the severity of cases, indicating its potential as an aid in diagnostic support.

The System Usability Scale assessment further underlines the positive reception of the device. Specialists found the tool to be user-friendly, with high scores indicating ease of navigation and minimal complexity. The unanimous agreement on the ease of use and the absence of perceived expertise required to navigate the device emphasize its accessibility and suitability for clinicians.

Patient satisfaction is a crucial aspect of any medical tool or platform. The results of the Patient Satisfaction questionnaire indicate a generally positive response from patients. They found the device to be easy to use, useful in monitoring their condition, and were satisfied with the care provided through the device.

In conclusion, the device has demonstrated notable clinical utility, usability, and safety in the evaluation of dermatological pathologies. The positive responses from both specialists and patients affirm its potential to serve as a valuable clinical decision-support tool. Further research and real-world application are warranted to explore the device's broader impact on dermatological practice and patient care.

Summary​

• Code: LEGIT_COVIDX_EVCDAO_2022
• Status: Finished
• Start date: March 3rd, 2022
• Finish date: October 23rd, 2023
• Acceptance criteria:
  • A score of 8 or higher in the Clinical Utility Score (CUS) filled by the medical staff

Our ASCORAD (Automatic SCORing of Atopic Dermatitis) study, a collaboration with Dr. Ramon Grimalt, was published in the Journal of Investigative Dermatology (JID) Innovations. This study details our approach to automating severity assessment of atopic dermatitis and eczema.

Conclusions​

The device has demonstrated an excellent performance in terms of malignancy prediction, which turns it into a valuable tool to prioritize patients according to their risk of presenting malignancy.

The AUC metric for the malignancy prediction was 0.8983, which is comparable to that of expert healthcare professionals (HCP) and speaks to the potential of using the device to improve clinical workflows.

Regarding skin lesion recognition in general terms, the Top-5 accuracy was 84.22%, which supports the device's intended use as a clinical decision-support tool. Specifically in melanoma, the AUC metric was 84.82% which is considerably high and means the consecution of the goals set out in the hypotheses of the study. On the downside, the Top-1 accuracy was 55.01% in the multiple ICD classification task, but the Top-3 accuracy increased to 75.69%. However, it's important to keep in mind that the Top-1 accuracy metric was not a relevant metric to this study, nor the performance of the device, because the device is designed to always output at least the top five predicted classes. This is aligned with its intended purpose as a clinical decision-support tool.

Given the results of the image dataset collected in this study, it is clear that images of better quality would have improved the functioning of the device and provided more valuable insights. Also, as the raw images were usually taken far away from the skin lesion, the cropping resulted in images of suboptimal resolution, which would have also been corrected by higher quality images - or by actually taking the image closer to the lesion.

Summary​