2 articles tagués avec « Malignancy »

Conclusions​

The medical device demonstrated high performance in malignancy detection and pathology diagnosis, performing at a level comparable to that of expert dermatologists both for the retrospective and prospective analysis. This performance was achieved despite the inherent bias in the dataset, which only includes lesions deemed suspicious enough to warrant a biopsy.

The device algorithms demonstrate moderate accuracy in predicting the Ludwig score for FAA. The overall accuracy was 47% in the prospective analysis, improving to 53% in the latest model. There is a low incidence of predicted grades differing by two grades from the investigator's score and a 50% correlation between the alopecia percentage and the investigator's score. These results indicate the potential of the device solution as a tool for estimating the Ludwig score for the FAA. Besides that, expanding the dataset and incorporating more diverse image samples could enhance the model's robustness and generalizability.

Sumary​

• Code: LEGIT.HEALTH_IDEI_2023
• Status: The first part of the study is finished. The second part will start in Q1, 2025
• Start date: February 2nd, 2024
• Finish date: August 7th, 2024
• Acceptance criteria:
  • An improvement of diagnostic accuracy of 10% (Ferri et al. 2020)
  • Scores equal to or greater than 70 on the System Usability Scale (SUS)
  • An AUC equal to or greater than 0.8 detecting malignancy
  • A sensitivity of 80% detecting malignancy
  • A specificity of 70% detecting malignancy

Conclusions​

The device has demonstrated an excellent performance in terms of malignancy prediction, which turns it into a valuable tool to prioritize patients according to their risk of presenting malignancy.

The AUC metric for the malignancy prediction was 0.8983, which is comparable to that of expert healthcare professionals (HCP) and speaks to the potential of using the device to improve clinical workflows.

Regarding skin lesion recognition in general terms, the Top-5 accuracy was 84.22%, which supports the device's intended use as a clinical decision-support tool. Specifically in melanoma, the AUC metric was 84.82% which is considerably high and means the consecution of the goals set out in the hypotheses of the study. On the downside, the Top-1 accuracy was 55.01% in the multiple ICD classification task, but the Top-3 accuracy increased to 75.69%. However, it's important to keep in mind that the Top-1 accuracy metric was not a relevant metric to this study, nor the performance of the device, because the device is designed to always output at least the top five predicted classes. This is aligned with its intended purpose as a clinical decision-support tool.

Given the results of the image dataset collected in this study, it is clear that images of better quality would have improved the functioning of the device and provided more valuable insights. Also, as the raw images were usually taken far away from the skin lesion, the cropping resulted in images of suboptimal resolution, which would have also been corrected by higher quality images - or by actually taking the image closer to the lesion.

Summary​