Conclusions​

Legit.Health significantly enhanced primary care physicians' diagnostic accuracy, increasing it from 72.96% to 82.22%.

The impact of Legit.Health varied across different skin conditions, demonstrating significant improvements in hidradenitis suppurativa, urticaria, and actinic keratosis. However, p-values were not statistically significant for all cases, due to the low number of samples per pathology.

Approximately 49% of cases did not necessitate a referral. Additionally, 60.74% of cases across all specialities could be effectively managed remotely.

Reduction of referral and use of remote consultation​

Previous studies reported that 66% of patients visiting primary care HCPs are referred to dermatology, with very low (1%) remote consultation rates (González-López et al., 2019). In terms of urgent referral and triage, some institutions have reported that 76.8% of patients referred from primary HCPs to dermatology result in benign diagnoses (Pagani et al., 2023).

In this experiment, we found that 49% of cases should be referred according to the primary HCP with the information provided by the device, which is 17% lower than the aforementioned referral rates. Additionally, our results improve the remote consultation rates, suggesting that diagnostic support tools can help foster remote consultations.

Summary​

• Code: LEGIT.HEALTH_PH_2024_NIPPLE
• Status: Finished
• Start date: June 4th, 2024
• Finish date: September 13th, 2024
• Acceptance criteria:
  • An improvement of diagnostic accuracy on both primary care physicians and dermatologists.
  • A reduction of 30% of referrals to dermatology (Warshaw et al. 2011).
  • An improvement in remote consultations.

Conclusions​

Legit.Health significantly improved healthcare professionals' diagnosis of generalized pustular psoriasis (GPP), doubling the diagnosis rate with the use of the medical device. In primary care, this effect was even greater, with a 120% increase in GPP diagnoses.

On average, diagnostic accuracy increased from 47.91% to 62.81%, a relative increase of 31%. For primary care doctors, the improvement was even more pronounced, with a 40% relative increase in correct diagnoses.

For other conditions like hidradenitis suppurativa and palmoplantar pustulosis, the results were also impressive. Primary care doctors correctly diagnosed 12.43% more cases, while dermatologists showed similar improvements with the device. For palmoplantar pustulosis, primary care doctors demonstrated an outstanding 146% increase, with dermatologists maintaining their performance.

Regarding the dermatologist's data, the results were not statistically significant at the pathology level due to two factors we identified. First, the lower number of participants. We had only four dermatologists, while primary care had eleven participants. This is not a design flaw because the focus was on primary care, but it is a limitation when conducting a detailed analysis. Additionally, the dermatologists had a high level of expertise, especially in hidradenitis suppurativa (HS). This, combined with the relatively average complexity of the HS cases in the experiment, may explain why the tool was not useful for dermatologists in the case of HS. However, this does not mean it is not useful for other dermatologists or dermatologists in general.

In summary, Legit.Health had a substantial impact on the diagnostic capabilities of healthcare professionals, particularly primary care doctors who have less specialization in dermatology. It also significantly benefited dermatologists when diagnosing rare conditions like GPP, which they encounter infrequently in their practice.

Summary​

• Code: LEGIT.HEALTH_BI_2024
• Status: Finished
• Start date: May 1st, 2024
• Finish date: August 29th, 2024
• Acceptance criteria:
  • An improvement in diagnosis accuracy of generalized pustular psoriasis (GPP)
  • An improvement of diagnosis accuracy of other skin conditions such as hidradenitis suppurativa or palmoplantar pustulosis on both, primary care physicians and dermatologists

Conclusions​

Legit.Health significantly enhanced diagnostic accuracy across all healthcare professionals, increasing it from 68.08% to 88.78%. Notably, primary care physicians experienced a substantial improvement in diagnostic accuracy, rising from 62.90% to 89.92%, with the integration of Legit.Health. Similarly, dermatologists saw their diagnostic accuracy improve from 76.47% to 86.93% with the utilization of Legit.Health.

The impact of Legit.Health varied across different skin conditions, demonstrating significant improvements in cases such as tinea, granuloma annulare, and seborrheic keratosis. It's noteworthy that all conditions included in the test showed increased accuracy.

Approximately 58.1% of cases did not necessitate a referral, with minor differences observed between primary care physicians and dermatologists. Additionally, a substantial portion of cases (55.11%) across all specialities could be effectively managed remotely, with primary care physicians exhibiting a slightly higher percentage compared to dermatologists.

When examining the results categorized by pathology, notable discrepancies emerged between primary care doctors and dermatologists in certain cases. It's worth mentioning that the limited number of dermatologists impedes reaching a definitive conclusion at the pathology level. For instance, unanimous agreement exists among experts on the feasibility of remote management for conditions like acne, herpes, and tinea. Conversely, there's a consensus favouring in-person consultations for melanoma, granuloma annulare, and nevus.

Furthermore, dermatologists recommend referring patients with nevus, melanoma, and alopecia to their care, while suggesting that primary care doctors can manage most other pathologies effectively.

Regarding healthcare professional (HCP) feedback, Legit.Health is viewed as a useful and user-friendly tool for its intended purpose, with users highlighting its role in diagnosis support during remote consultations. 87% of HCPs believe they could manage a patient in less than 10 minutes, despite the average consultation time typically being at least 15 minutes.

Reduction of referral and use of remote consultation​

Previous works reported that 66% of the patients visiting primary care HCPs are referred to dermatology, with very low (1%) remote consultation rates (González-López et al., 2019). In terms of urgent referral and triage, some institutions have reported that 76.8% of the patients referred from primary HCPs to dermatology result in benign diagnoses (Pagani et al., 2023).

In this experiment, as reported above, we found out that according to the primary HCP with the information provided by the device 39.11% of the cases should be referred, which is a 41.34% lower than the aforementioned referral rates. In addition, our results improve the remote consultation rates, which suggests that diagnosis support tools can help foster remote consultations.

Summary​

• Code: LEGIT.HEALTH_SAN_2024
• Status: Finished
• Start date: November 15th, 2023
• Finish date: February 17th, 2024
• Acceptance criteria:
  • An improvement of diagnostic accuracy on both primary care physicians and dermatologists
  • A positive view of Legit.Health regarding diagnosis support
  • A reduction of 30% in referral to dermatology (Warshaw et al. 2011)
  • An improvement in remote consultations

Conclusions​

Legit.Health significantly enhanced primary care physicians' diagnostic accuracy, increasing it from 72.96% to 82.22%.

The impact of Legit.Health varied across different skin conditions, demonstrating significant improvements in hidradenitis suppurativa, urticaria, and actinic keratosis. However, p-values were not statistically significant for all cases, due to the low number of samples per pathology.

Approximately 49% of cases did not necessitate a referral. Additionally, 60.74% of cases across all specialities could be effectively managed remotely.

Reduction of referral and use of remote consultation​

Previous studies reported that 66% of patients visiting primary care HCPs are referred to dermatology, with very low (1%) remote consultation rates (González-López et al., 2019). In terms of urgent referral and triage, some institutions have reported that 76.8% of patients referred from primary HCPs to dermatology result in benign diagnoses (Pagani et al., 2023).

In this experiment, we found that 49% of cases should be referred according to the primary HCP with the information provided by the device, which is 17% lower than the aforementioned referral rates. Additionally, our results improve the remote consultation rates, suggesting that diagnostic support tools can help foster remote consultations.

Summary​

• Code: LEGIT.HEALTH_PH_2024
• Status: Finished
• Start date: June 24th, 2022
• Finish date: January 10th, 2024
• Acceptance criteria:
  • An improvement of diagnostic accuracy of 10% (Ferri et al. 2020) in primary care physicians and dermatologists.

The Automatic Urticaria Activity Score (AUAS) system has been published in the Journal of Investigative Dermatology (JID) Innovations, showcasing our deep learning-based approach for urticaria severity assessment.

The AIHS4, our novel system for scoring Hidradenitis Suppurativa, is detailed in Skin Research and Technology. This study exemplifies our commitment to developing practical AI solutions for complex dermatological conditions.

Conclusions​

The medical device demonstrated high performance in malignancy detection and pathology diagnosis, performing at a level comparable to that of expert dermatologists both for the retrospective and prospective analysis. This performance was achieved despite the inherent bias in the dataset, which only includes lesions deemed suspicious enough to warrant a biopsy.

The device algorithms demonstrate moderate accuracy in predicting the Ludwig score for FAA. The overall accuracy was 47% in the prospective analysis, improving to 53% in the latest model. There is a low incidence of predicted grades differing by two grades from the investigator's score and a 50% correlation between the alopecia percentage and the investigator's score. These results indicate the potential of the device solution as a tool for estimating the Ludwig score for the FAA. Besides that, expanding the dataset and incorporating more diverse image samples could enhance the model's robustness and generalizability.

Sumary​

• Code: LEGIT.HEALTH_IDEI_2023
• Status: The first part of the study is finished. The second part will start in Q1, 2025
• Start date: February 2nd, 2024
• Finish date: August 7th, 2024
• Acceptance criteria:
  • An improvement of diagnostic accuracy of 10% (Ferri et al. 2020)
  • Scores equal to or greater than 70 on the System Usability Scale (SUS)
  • An AUC equal to or greater than 0.8 detecting malignancy
  • A sensitivity of 80% detecting malignancy
  • A specificity of 70% detecting malignancy

The Dermatology Image Quality Assessment (DIQA) technology, which ensures the clinical utility of images for remote consultations and clinical trials. This was published in the Journal of the American Academy of Dermatology.

Conclusions​

Primary care doctors exhibit a notably low sensitivity of approximately 25% when it comes to the crucial task of deciding whether to refer a patient to secondary care. This is especially true in referrals in the field of Dermatology.

On the other hand, they maintain a high specificity rate of 96%, meaning that when they do decide to make a referral, they are highly likely to be correct in their judgment that specialist care is necessary.

This pattern reflects a cautious approach, as primary care physicians seem to prefer minimizing the risk of false negatives, even if it means that some patients who could benefit from secondary care might not be promptly referred. This cautiousness impedes an optimal utilization of specialist resources.

This study reveals that approximately 29% of the referrals involve common and easily diagnosable conditions, even those from teledermatology. About half of them are related to seborrheic keratosis. Another example of conditions that can be confidently identified and managed without referrals includes skin tags, which the device can reliably confirm, and other entities are unlikely to misdiagnose.

The quality of the images significantly influences the performance of the system. This is a well-established fact in the field because image quality not only impacts the effectiveness of algorithms but also hinders dermatologists from making diagnoses through teledermatology systems. Specifically, poor-quality images of nevi often require an in-person consultation, causing unnecessary delays for specialists.

It is typically complex to calculate precise costs, but we can estimate that algorithms like the device could have a substantial impact on cost optimization while simultaneously reducing waiting times and expediting urgent cases.

In terms of the waiting list, the analysis assumes that patients could have received treatment earlier, and the appointment delays were a result of the hospital's waiting list.

The current analysis primarily focuses on malignancy, but there may be other criteria to consider when referring patients. While not addressed in this study, the device incorporates additional algorithms that focus on the severity of chronic skin conditions. Moreover, in certain cases, referrals can be based on the possibility of a specific disease that may be complex to manage.

Summary​

• Code: LEGIT.HEALTH_DAO_Derivación_O_2022
• Status: Ongoing
• Start date: April 7th, 2022
• Acceptance criteria:
  • Improve the adequacy of referrals to dermatology
  • A reduction of waiting lists (at least 30% Warshaw et al. 2011)
  • A reduction of the costs in secondary care

Conclusions​

The study conducted to evaluate the clinical performance, efficacy, and safety of the device has yielded promising results. The comprehensive analysis of the CUS, Data Utility questionnaire, SUS, and Patient Satisfaction questionnaire has provided valuable insights into the tool's effectiveness in supporting dermatologists in their clinical practice.

The observed sample mean of 76.67 on the CUS suggests that the device has been positively received by the participating specialists. Noteworthy is the unanimous agreement on the ease of use and the high rating for optimizing time according to each patient's needs. It's also worth noting that, despite that the medical device was positively rated by the specialists, the goal of achieving a mean of 80.00 on the CUS was not reached. This result was due to the lower sample size of specialists who completed the questionnaire. In this way, an outlier, and due to the small sample size, impacted disproportionately the overall result, especially for questions 7, 8, 9 and 11. As can be seen by the higher standard deviation and lower mean average in these specific questions. We need to take this fact into account for the following studies and implement measures to mitigate this effect, such as a larger sample size, which could have diluted the effect of the outliers over the statistical outcomes, or predefined management for outliers.

Additionally, the device demonstrated efficiency in generating reports, receiving high ratings from the specialists. These outcomes affirm the device's potential to streamline clinical workflows and enhance patient care.

The Data Utility questionnaire revealed unanimous agreement among specialists regarding the usefulness of a device to facilitate their regular practice. Moreover, the majority expressed a preference for utilizing a device to identify the severity of cases, indicating its potential as an aid in diagnostic support.

The System Usability Scale assessment further underlines the positive reception of the device. Specialists found the tool to be user-friendly, with high scores indicating ease of navigation and minimal complexity. The unanimous agreement on the ease of use and the absence of perceived expertise required to navigate the device emphasize its accessibility and suitability for clinicians.

Patient satisfaction is a crucial aspect of any medical tool or platform. The results of the Patient Satisfaction questionnaire indicate a generally positive response from patients. They found the device to be easy to use, useful in monitoring their condition, and were satisfied with the care provided through the device.

In conclusion, the device has demonstrated notable clinical utility, usability, and safety in the evaluation of dermatological pathologies. The positive responses from both specialists and patients affirm its potential to serve as a valuable clinical decision-support tool. Further research and real-world application are warranted to explore the device's broader impact on dermatological practice and patient care.

Summary​

• Code: LEGIT_COVIDX_EVCDAO_2022
• Status: Finished
• Start date: March 3rd, 2022
• Finish date: October 23rd, 2023
• Acceptance criteria:
  • A score of 8 or higher in the Clinical Utility Score (CUS) filled by the medical staff

Our ASCORAD (Automatic SCORing of Atopic Dermatitis) study, a collaboration with Dr. Ramon Grimalt, was published in the Journal of Investigative Dermatology (JID) Innovations. This study details our approach to automating severity assessment of atopic dermatitis and eczema.

Conclusions​

The device has demonstrated an excellent performance in terms of malignancy prediction, which turns it into a valuable tool to prioritize patients according to their risk of presenting malignancy.

The AUC metric for the malignancy prediction was 0.8983, which is comparable to that of expert healthcare professionals (HCP) and speaks to the potential of using the device to improve clinical workflows.

Regarding skin lesion recognition in general terms, the Top-5 accuracy was 84.22%, which supports the device's intended use as a clinical decision-support tool. Specifically in melanoma, the AUC metric was 84.82% which is considerably high and means the consecution of the goals set out in the hypotheses of the study. On the downside, the Top-1 accuracy was 55.01% in the multiple ICD classification task, but the Top-3 accuracy increased to 75.69%. However, it's important to keep in mind that the Top-1 accuracy metric was not a relevant metric to this study, nor the performance of the device, because the device is designed to always output at least the top five predicted classes. This is aligned with its intended purpose as a clinical decision-support tool.

Given the results of the image dataset collected in this study, it is clear that images of better quality would have improved the functioning of the device and provided more valuable insights. Also, as the raw images were usually taken far away from the skin lesion, the cropping resulted in images of suboptimal resolution, which would have also been corrected by higher quality images - or by actually taking the image closer to the lesion.

Summary​