Skip links
severity assessment with dermatology API

Creating a brand-new scoring system, the 5 key steps every scientist should follow

Table of Contents

    Scoring systems are the unsung heroes of day-to-day dermatological practice. Disliked by many for the additional work they can bring to the table, they help train the clinical eye that is so often used to replace them.

    Despite that, there’s no denying the crucial role scoring systems play in the clinical development of the field, as they are essential tools to bring precision, objectivity and reliability to clinical trials.

    However, on many occasions, the scoring system is seen as a tool to be used for a greater purpose, and once a physician chooses a preferred one for a specific pathology, it is rare for that choice to be re-evaluated. 

    In order to better use and understand the tools at our disposal, we have to start at the beginning…

    What is a Dermatological scoring system?

    In Dermatology, a Scoring system is a methodology that allows the doctor to assess the severity of a condition by observing and quantifying objective parameters such as redness, the affected area, the quantity and density of lesions, and so on and so forth.

    Through this project, it has been possible to create, for the first time in history, an artificial intelligence system that is in charge of this work. We are very satisfied with the results that will allow clinical trials and more precise diagnoses.

    Dr. Ramón Grimalt

    The main goal of a scoring system is to provide a tool to record data precisely and consistently for routine evaluations and clinical studies. 

    APASI, A dermatological scoring system
    APASI, A dermatological scoring system

    Although many doctors still use the traditional versions of these scoring systems, the key opinion leaders of the dermatological world are pushing for digitalization and automatization of the practice, allowing greater precision, reliability and objectivity to the measurements.

    How is a scoring system created?

    Step 1: Identify a need

    In this video (in Spanish), Dr Ramon Grimalt and Alfonso Medela, both co-authors of the ASCORAD scoring system, explain the need that pushed them to create a new scoring system for atopic dermatitis.

    Even before starting the development process of a new scoring system, the researchers at Legit.Health make sure they are fulfilling an existing need. Staying up-to-date with the state of the art in dermatology and extensively reviewing the pertinent literature is a key step prior to any research project. 

    In the case of scoring systems, Legit.Health’s Chief Artificial Intelligence Officer, Alfonso Medela, has some insight into the thought process behind the early stages of development.

    “Often, for each condition, you can find several pre-existing scoring systems, each one with some strengths and weaknesses. It is by analysing them that we can find out which parameters are the most important ones for the condition”

    Alfonso Medela CAIO at Legit.Health

    Do you want to see the clinical A.I. technology in action?

    Selecting the dataset

    As is often the case in medical science, one of the biggest issues our researchers have to face is selecting an optimal dataset.

    As a rule of thumb, the more parameters we want to include in the new scoring system, the more images we need to have in the dataset. But of course, the more images the better

    One thing that can be done in this type of study to better optimise the usage of the dataset is to start using the images gradually, looking for the moment when the results stabilise, thus deciding the N of the first time of the trial as you perform it, without ever falling short or using too many images.

    Step 2: Building a model

    Once all the parameters are known, the most relevant ones are selected and analysed, looking for the most optimal combination of them. A good example of this would be the case of ALEGI, the acne severity scoring system developed by Legit.Health’s scientific team. 

    In the case of Acne, the literature suggested that aside from the number of lesions (a parameter that showed in every other scoring system), the density of said lesions was also noted as a very prominent sign of a severe condition. 

    Do you want to see the clinical A.I. technology in action?

    The selected parameters will then be measured and identified in a selection of clinical pictures, both by specialized doctors and by any technology that is being developed to assist in this process (as is usually the case with the automatic algorithms used by Legit.Health), and these values will be compared with those of the gold standard scoring system for that same array of images. 

    This allows researchers to correlate the values of the previously existing method with the newly defined parameters.

    Step 3: Optimising the model

    Once the researchers have all the parameters defined, it is time to create an equation that combines them all to best represent the severity of the different clinical images used in the study. 

    This is known as an optimisation problem, a mathematical term that describes the process of finding the best possible solution to a specific problem: in this case, how to represent the severity of a condition. 

    Do you want to see the clinical A.I. technology in action?

    Without delving too deep into the mathematical intricacies of the process, different combinations of operations are tested on the parameters: adding, subtracting or multiplying them to obtain a single combined value that represents the severity of the condition. 

    Then, the results of each of these proposed equations for every single image in the set are compared with the results given by the selected gold standard, looking for the best possible correlation with the previously existing problems. 

    Step 4: Prove that it works

    One crucial step in the development of a scoring system is to prove that it works. To do this, you must put the model into practice.

    Legit.Health research team presenting the AIHS4 poster
    Legit.Health research team presenting the AIHS4 poster

    This is done with a subset of clinical pictures that have not been used to optimise or train the model in the previous steps. After all, your equation has been created to perform perfectly when tested against those images, so to demonstrate that it can work with any picture of the condition, we have to test it with some new ones. 

    Simply put, our researchers work closely with specialised doctors to label the images using both the gold standard method and the newly developed model and will compare the results to see if the new model performs better than the original one and still finds a correlation with it. 

    Step 5: Put it into practice in a clinical environment

    Finally, the model is brought to the forefront of medical experimentation and compared with the results of dermatologists observing the same affection in clinical practice. 

    Caption of a full report from the CADx system. The chart at the top right shows the evolution of the urticaria, by plotting the AUAS scores across time.
    Caption of a full report from the CADx system. The chart at the top right shows the evolution of the urticaria, by plotting the AUAS scores across time.

    This step is key to ensure that the method can keep up with the reality of the day-to-day practice of medicine, as a good and reliable scoring system is not only more objective and less error-prone than the clinical eye but also needs to be fast and not overload doctors with more work. 

    That’s why at Legit.Health we link our scoring systems to computer vision algorithms, to put the speed and precision of artificial intelligence in the hands of doctors and help them help their patients.

    Get access now

    This free 23-day trial of Legit.Health gives clinics and hospitals a hands-on look at how to drive increased adherence and improve patient outcomes, as well as improving efficiency and overall quality of life.

    white block

    We are here to help

    What can you expect from contacting us?

    • Demo of the platform: See the inside of the application first-hand
    • Understand the solution: find the right features for you
    • Pricing: discover the plan that suits you best
    • Connect with helpful resources
    Creating a brand-new scoring system, the 5 key steps every scientist should follow