3 publicaciones etiquetados con "Malignancy"

Published in European Archives of Oto-Rhino-Laryngology, this prospective study evaluated Legit.Health as a diagnosis support tool in the head and neck surgery clinic of Donostia University Hospital, on the two most common skin cancers: basal cell carcinoma (BCC) and cutaneous squamous cell carcinoma (cSCC).

Conclusions​

In the prospective arm of IDEI_2023, Legit.Health’s malignancy score achieved an AUC of 96.69% (89.00%-100.00%) for distinguishing malignant from benign pigmented lesions, using histopathology as the reference. The interval shown is the 95% confidence interval. This result comes from a small prospective sample; confirmation in an independent population is planned.

The device algorithms demonstrate moderate accuracy in predicting the Ludwig score for FAA. The overall accuracy was 47% in the prospective analysis, improving to 53% in the latest model. There is a low incidence of predicted grades differing by two grades from the investigator's score and a 50% correlation between the alopecia percentage and the investigator's score. These results indicate the potential of the device solution as a tool for estimating the Ludwig score for the FAA. Besides that, expanding the dataset and incorporating more diverse image samples could enhance the model's robustness and generalizability.

Summary​

• Code: LEGIT.HEALTH_IDEI_2023
• Status: Finished
• Start date: January 25th, 2024
• Finish date: August 23rd, 2024
• Acceptance criteria:
  • An improvement of diagnostic accuracy of 10% (Ferri et al. 2020)
  • Scores equal to or greater than 70 on the System Usability Scale (SUS)
  • An AUC equal to or greater than 80.00% detecting malignancy
  • A sensitivity equal to or greater than 80.00% detecting malignancy
  • A specificity equal to or greater than 84.00% detecting malignancy

Conclusions​

The device has demonstrated an excellent performance in terms of malignancy prediction, which turns it into a valuable tool to prioritize patients according to their risk of presenting malignancy.

The AUC metric for the malignancy prediction was 0.8983, which is comparable to that of expert healthcare professionals (HCP) and speaks to the potential of using the device to improve clinical workflows.

Regarding skin lesion recognition in general terms, the Top-5 accuracy was 84.22%, which supports the device's intended use as a clinical decision-support tool. Specifically in melanoma, the AUC metric was 84.82% which is considerably high and means the consecution of the goals set out in the hypotheses of the study. On the downside, the Top-1 accuracy was 55.01% in the multiple ICD classification task, but the Top-3 accuracy increased to 75.69%. However, it's important to keep in mind that the Top-1 accuracy metric was not a relevant metric to this study, nor the performance of the device, because the device is designed to always output at least the top five predicted classes. This is aligned with its intended purpose as a clinical decision-support tool.

Given the results of the image dataset collected in this study, it is clear that images of better quality would have improved the functioning of the device and provided more valuable insights. Also, as the raw images were usually taken far away from the skin lesion, the cropping resulted in images of suboptimal resolution, which would have also been corrected by higher quality images - or by actually taking the image closer to the lesion.

Summary​