On 1 January 2008, Professor Cliff Rosendahl made a decision to photograph every mole he cut out. It was a painstaking task for a solo GP. “It took about an hour every night for the last 10 years,” the Queensland-based skin cancer doctor tells The Medical Republic.
“It involved downloading images from multiple cameras, put them on a holding file and put them on PowerPoints,” he said. Then, Professor Rosendahl would go through each image individually and document all the relevant clinical information.
While most GPs engage in fastidious record-keeping, the 20,000 lesions photodocumented by Professor Rosendahl are something else entirely.
This rich data set has been used in several research projects and was even used to train 141 different algorithms for a world competition on AI melanoma detection last year.
“Nowadays, I don’t use the big cameras anymore except sometimes when I particularly want to photograph something,” says Professor Rosendahl. “I use FotoFinder. I find that very, very convenient for organising and storing and retrieving images.”
FotoFinder is a German company that sells large dermatology imaging units with a display monitor, a handheld dermatoscope and software.
“FotoFinder is a big machine with a camera attached where you very quickly take the clinical photo and then it organises the photo in the patient’s file on the machine,” says Professor Rosendahl.
“You can do total body photography with FotoFinder. You can then repeat that and upload the image, and the software compares it with the previous image and identifies new and changing lesions.”
FotoFinder has also developed a deep learning algorithm, called Moleanalyzer pro, which provides a risk-of-malignancy evaluation and allows doctors to ask for a second opinion from international skin cancer experts.
Dr Robert Miller, a dermatologist in North Queensland, used to have thousands of patients on the surveillance system called MoleMap, system but he eventually gave it away.
“The MoleMap system I used for quite a while,” he says. “That does a lot of direct comparison of images side-by-side with a software program. I found that very good.
“But I don’t use it anymore. It’s actually too expensive for most of my patients. I live in an area where there’s a lot of people who are unemployed or on low income so sometimes the technology is great, but it’s not necessarily affordable.”
Dr Miller now uses a digital camera with a dermatoscope attachment, which magnifies moles 10 times and shows some sub-surface features.e then
He then uploads the photos onto his practice management software, called Genie.
“And I just go to the entry a year, or two, or three years ago when I photographed something and then I look at the patient and I make a direct comparison that way,” he says.
“A lot of dermatologists do that. It’s simple. It is low cost and it works well. The other systems range from $10,000 to $60,000.”
Lots of GPs are in a similar boat. There might be beautiful software offerings that could speed up their workflow, but it doesn’t make financial sense to invest.
While the AI for melanoma diagnosis will rapidly improve over time, it’s likely to remain out of reach for many clinicians and patients, says Dr Miller.
“I am a great advocate for us expanding photography and AI in melanoma diagnosis,” says Dr Miller. “About the only issue I see is that as the technology gets more and more complex and better, it’s going to cost more.”
Dr Chris Miller, a GP at the Spot Check Skin Cancer Clinic in Melbourne, switched to DermEngine two years ago.
DermEngine is different to most of the other providers. It’s a bit more like Netflix or Spotify in that clinics pay a subscription of $75 to $580 per month, depending on the number of consultations, instead of an upfront payment of tens to hundreds of thousands of dollars.
DermEngine can integrate with lots of different practice management systems and dermatoscopes, including simple mobile device attachments.
Images stored on the system can also be securely shared with other practitioners, including plastic surgeons and dermatologists who do not have DermEngine.
“The reason we switched to DermEngine originally was because the equipment we had for doing imaging of skin lesions was really getting a bit rundown and was very difficult to maintain, very expensive to maintain,” says Dr Miller.
“DermEngine’s system was good because it is hosted in the cloud so there is no need to install a server to host all of the images. It gave us the flexibility to be able to examine people in different locations. So, we could use it at different branches of our clinic or during site visits to workplaces.
“The other thing that was good about it was it was able to use existing hardware that we had for taking photos. So, if you have a dermatoscope that you really like you can actually use that for magnifying and illuminating the lesions. You don’t have to buy special equipment that comes with it, which a lot of the medical imaging systems require.”
DermEngine also gives patients access to all their own photos and pathology results.
“So, from the patient’s point of view, you can log into DermEngine and you can see the picture of the spot that we talked about today,” says Dr Miller. “You can also see what my diagnosis was and if we removed it. Then you can see what the pathology result was, together with my own explanation of what that means and what you need to do about it.”
Dr Miller says that some doctors are still using mobile phones to take clinical photos of moles without having specialised medical software installed on their phones.
The disadvantage of this approach is that the images are not automatically associated with a particular patient or a body location, so it can be a lot of work to manually record that information in the patient’s file, he says.
There’s also a risk that the photos will be automatically uploaded to a service such as iCloud, which may be a breach of Australian privacy legislation, says Dr Miller.
The AI installed on DermEngine can compare images of moles with thousands of other images in the system and present a pie chart showing how many of these other moles turned out to be malignant or benign.
This data visualisation can be a good educational tool for patients during the consultation, but the AI isn’t as good as a doctor’s trained eye and it isn’t approved as a diagnostic tool, says Dr Miller.
For some patients, the AI results can cause concerns. Some of these patients can be reassured by a doctor interpreting what the risk calculation actually means, whereas others will insist on having the mole removed.
“It’s not usually a really big deal to remove a weird looking spot,” says Dr Miller. “Even if you don’t think it’s a melanoma, if that’s what the patient really wants. So, they can sleep at night, then you just do it, as long as they can live with the small risks involved.”
DermEngine, however, is quite expensive and it’s probably not a good return on investment if you’re a GP who doesn’t work in a dedicated skin cancer practice, says Dr Miller.
Dr Annika Smith, a dermatologist at St Vincent’s Hospital in Sydney, has been using DermEngine since March last year.
What she finds particularly useful is the software’s ability to detect which moles are new, or have changed the most in terms of size or colour, via the MoleMatch function.
Around 70% of melanomas are now thought to arise as new lesions on the skin, whereas 30% may arise within a pre-existing mole, she says.
When a patient is covered in moles “from top to toe”, it’s very difficult for the patient or the dermatologist to detect changes without the aid of technology, she says.
“In the context of high-risk patients with a large naevus burden and multiple atypical naevi, I don’t know what lesions are new or changing,” she says.
“The patient may not know what’s new or changing. We need an objective baseline to compare to in order to increase our diagnostic sensitivity for melanoma, and that’s what comparative analysis with total body photography allows us to do.
“I can count several cases in the last few months in young high-risk patients where the aid of total body photography has assisted with the diagnosis of melanoma.
“These were melanomas that were what we call featureless.
These were very early melanomas that were very much benign-looking clinically and dermatoscopically that did not have the typical features of melanoma, and the only thing that alerted us to be concerned about these spots was the fact that they were new.”
Most skin cancer doctors don’t bother using the AI functions built into dermatology software, because they haven’t been tested in real-life clinical practice.
In experimental settings, however, the computers are already beating human experts, says Professor Rosendahl.
Last year, 139 different AI algorithms competed against 511 doctors from around the globe to diagnose skin disease from images as part of the International Skin Imaging Collaboration (ISIC) challenge.
The participants had to decide whether the dermatoscopic images were benign or fell into one of seven skin disease categories, including melanoma.
Around one third of these 11,210 images came from Professor Rosendahl’s own collection and the rest came from the Medical University of Vienna.
“It’s the first time a very large study on dermatoscopic images shows that machines are better than humans,” says Professor Rosendahl.
In the study, humans got 18 out of 30 diagnoses correct on average, while the best machine-learning algorithms got 25 out of 30 correct.
The top AI performers were MetaOptima, DAISYLab, and Medical Image Analysis Group.
On average, the AI algorithms had greater sensitivity and specificity than the human readers. For melanoma, the top three algorithms had a specificity of 96% and a sensitivity of 82%, while the top experts had a specificity of 94% and a sensitivity of 68%.
“Machine learning algorithms outperformed human readers with respect to most outcome measures,” the study, published in Lancet Oncology, states.
“Now, that’s in an experimental setting,” says Professor Rosendahl. “That doesn’t mean that’s it’s the same in the real world.
“But what it does mean, for example, is if someone has a lesion that they are looking at and they want to send me a photo and ask my opinion, they will have a choice of actually taking a picture and getting the machine to interpret it.
“The machine works 24 hours a day, seven days a week. It doesn’t get tired. And the machines will have a known specificity and sensitivity, whereas humans can vary.”
The 2020 ISIC challenge will be organised, in part, by Associate Professor Pascale Guitera, a dermatologist at the Melanoma Institute Australia who helps treat around 7,000 melanoma patients a year.
This challenge will focus on whether AI can detect changing moles, she says.
“Change is major information for us,” she says. “So, if you have a really ugly, dysplastic mole that you have all your life and is not changing, I will not remove it.
“But if you had a very small, pinkish lesion that is growing, I will jump on it and remove it because I’m worried it could be a melanoma. So, change is a major way for us to say, ‘Oh, is this is biologically active’.”
AI is already better than humans at diagnosing melanoma from standardised images. But dermatologists don’t just depend on images during a consultation; they can ask the patient questions, physically touch the lesion, move the lighting around or look at the mole from a different angle.
“So, it’s not a real life setting at all,” says Associate Professor Victoria Mar, a dermatologist and the director of the Victorian Melanoma Service at the Alfred Hospital.
A clinical trial that is just kicking off in Victoria will compare MoleMap’s AI against a dermatologist doing face-to-face consultations.
The Acceleration Fund of the Victorian Department of Health and Human Services is funding this research, which is being led by the Alfred Hospital and Monash University.
High-tech 3D imaging systems are also being trialled in Queensland, NSW and Victoria.
This is funded by a $9.9 million grant from The Australian Cancer Research Foundation and involves the installation of 15 Vectra skin imaging systems (Canfield Scientific) across the three states, including regional centres.
The aim of this is to improve access to specialist level care by establishing a telehealth network and eventually integrating AI diagnostic support into these imaging systems, says Associate Professor Mar.
At the moment, total body mapping of moles is a very slow process.
Patients have to adopt several different poses while images are taken of every part of their body. Clinics often need to hire an extra staff member to take the photos.
With the 3D imaging technology, a person stands motionless for a short time as the 92 cameras simultaneously photograph every part of their body, including hard-to-see areas that may have been missed in the past.
The computer stitches these photos together and creates a 3D avatar of the person. The clinician can then zoom in and spin the avatar around on the computer screen.
“So, you can get an incredibly detailed representation of the person’s whole skin, not just individual moles,” says Associate Professor Mar.
MetaOptima (the company behind DermEngine) has a slightly different vision for the future.
It is developing a drone that can fly around a patient, automatically locate moles and record images.
So, will your practice be using a drone or AI as a melanographer anytime soon?
It’s hard to imagine when even basic record-keeping software is still too pricey for many doctors.
While industry dreams up ultramodern AI diagnostics, many doctors are just dreaming of affordable software for securely processing dermoscopy images.
