Skip to main content

Known Limitations

Clinical trial sponsors need to understand not just what an AI scoring system can do, but where its boundaries are. This page documents the known limitations of the automated acne severity scoring technology, explains why each limitation exists, and describes how it is managed. Transparency about limitations is a regulatory expectation (ISO 14971, EU AI Act Article 13) and a prerequisite for informed protocol design.

Clinical agreement in a prospective study has not yet been measured​

The evidence for ALADIN (Acne Lesion And Density INdex) is a retrospective, peer-reviewed study on two image datasets. Its results are descriptive, with no acceptance criterion set in advance. The formula was fitted on the larger dataset, so its results there are in-sample, and the independent dataset is small: on it, ALADIN's agreement with the expert consensus was similar to the dermatologists', while its average error was higher. See Clinical evidence.

Impact: Agreement between ALADIN and the treating investigator's IGA, on the same patients at the same visit, has not yet been measured, and neither has ALADIN's sensitivity to change over a course of treatment.

Mitigation: A prospective post-market clinical follow-up study with a pre-specified agreement threshold is planned. Agreement with the study's own investigators can be built into a trial as a sub-study during protocol design, and sponsors should weigh the current evidence when choosing whether ALADIN supports a primary or a secondary endpoint.

ALADIN is supportive data, not a primary endpoint​

ALADIN is not a clinically validated primary efficacy endpoint, and it does not replace the investigator's IGA. In a trial it gives objective, supportive data alongside the IGA: an independent, reproducible second measurement of every visit. The certified Legit.Health device detects the inflammatory lesions on each photograph. The lesion count, the density and the IGA score and ALADIN formulas are applied to those detections by an integration that Legit.Health develops for clinical trials, outside the certified device.

Mitigation: The inflammatory lesion count can be delivered on its own, without the composite score, for protocols that only need the count, and agreement between ALADIN and the study's own investigators can be measured in a sub-study.

Inflammatory lesions only​

The detector counts inflammatory lesions as a single class: papules, pustules and nodules are counted together and not told apart. Comedones (blackheads and whiteheads) are not counted.

The FDA guidance asks for inflammatory and noninflammatory lesions to be counted separately, so a trial that needs a noninflammatory count, or a count by lesion type, cannot take it from the platform today.

Mitigation: Noninflammatory counts and lesion types are assessed by the investigator, as a separate endpoint. This should be agreed during protocol design.

Facial acne only​

The method is designed and evaluated for facial acne, on diagonal photographs of the face. Acne on the chest, back and shoulders is not assessed. The FDA guidance recommends that efficacy assessment be limited to the face, the most frequent site of involvement (FDA, 2018).

Mitigation: If a protocol needs truncal acne assessed, it is handled as a separate endpoint, and its feasibility is discussed during protocol design.

Skin type and severity representation​

Most of the images behind the detector and the ALADIN study come from people with lighter skin. The darkest phototypes (V and VI) are represented by very few images: the detector met its criterion for them on the point estimate, but with a wide confidence interval that reaches below it, and the ALADIN study included only two such images. The study's training data also leaned towards some severities more than others (Medela et al., 2026).

Mitigation: The detector's performance is reported by Fitzpatrick group and by severity, so sponsors can judge it against their study population; see Lesion detector performance. If the study population includes many patients with darker skin, this should be discussed during protocol design.

Two photographs, scored one at a time​

The published evaluation scored one photograph at a time. In a trial, each visit has two photographs, one for each half of the face, and the platform combines their results into a score for the visit. The two views meet in the middle of the face, so a lesion on the nose, the centre of the forehead or the chin can appear in both.

Impact: The agreement reported in the study applies to single photographs; the score for a visit made of two photographs has not been evaluated against a dermatologist's grade of the whole face, and adding the counts of the two views would count a lesion in the overlap twice.

Mitigation: Lesion counts are reported per view, as the Hayashi criteria count them on each half of the face, and the score for the visit is the mean of the two half-face ALADIN scores, the same rule for every patient and every visit.

What a photograph cannot show​

The AI analyses photographs of the face, not the patient. What only an in-person examination shows, such as the depth of a nodule felt on palpation, cannot be read from an image, and lesions hidden by hair, make-up, glasses or the camera angle are not counted. On faces with very little acne, the detector can find lesions that dermatologists would not count (Medela et al., 2026).

Mitigation: The imaging protocol standardises the two views, the lighting and the patient preparation (no make-up, hair tied back, no glasses), and the DIQA quality gate rejects images that do not meet minimum quality standards for focus, lighting, framing and resolution. The investigator reviews every report, with each counted lesion marked on the image, alongside the clinical examination.

Reference standard​

The reference ALADIN is measured against is the consensus IGA of several expert dermatologists. It is the best available approximation of the true severity, but the IGA rests on broad descriptors and dermatologists disagree on it, so the reference is not an objective measurement, and the score cannot be more accurate than the reference it was designed and tested against.

Mitigation: Using the consensus of several dermatologists reduces the bias of any single one, and each dermatologist's own agreement with the same consensus is reported alongside ALADIN's.

Model version specificity​

All performance metrics reported in this documentation apply to a specific validated model version. Model updates, including retraining, architecture changes or threshold adjustments, require full re-validation under IEC 62304 before deployment.

Mitigation: The model version is locked at study initiation. No mid-study model updates occur, so every patient in a trial is scored by the same model and endpoint integrity is preserved throughout the study.

Decision support, not autonomous diagnosis​

The system provides severity scoring to support clinical decisions. It does not replace clinical judgement, and it does not make autonomous diagnostic or treatment decisions. All AI-generated scores should be interpreted by qualified healthcare professionals within the context of the patient's overall clinical presentation.

How limitations are managed​

All limitations documented on this page are tracked within the formal risk management process (ISO 14971) and the software development lifecycle (IEC 62304). Each limitation has been assessed for clinical risk, and mitigations have been implemented where the residual risk is not already acceptable.

The post-market clinical follow-up (PMCF) programme under MDR Annex XIV continuously monitors real-world performance. Any new limitation identified through post-market surveillance triggers a formal risk assessment and, if necessary, corrective action.