COVID-19 has revealed how vulnerable economies and businesses are to contagion. The virus has caused death, chaos and economic destruction. Yet, it is not as dangerous as it might have been. COVID-19 provides us with a salient warning of just how unprepared we are.
In early 2020, a team of researchers at the University of South Australia (UniSA) started to talk to the Canadian drone technology company, Draganfly inc., about the ways that drone may facilitate non-contact health monitoring. Staff at Draganfly had seen a journal article by the UniSA team from in 2017 that illustrated the ways that vital signs could be measured from a hovering drone [1].
Drone operators can measure a human’s heart rate from using technology known as imaging photoplethysmography (iPPG). iPPG uses a visible light camera to detect the changes in the volume of blood in the face following each heartbeat. Such changes are invisible to the human eye. Wrist-worn heart rate monitors use a similar technique and work very reliably, but they require skin contact and a light source.
In contrast, iPPG doesn’t rely on skin contact, but it does require serious image processing and artificial intelligence. Putting the technology on a drone adds additional challenges, because the computer must be small, the camera is moving, distances may be 10 metres or more and the system has to determine what to look at.
Another challenge is to track breathing from the sky. Measuring the breathing rate requires different advanced processing to extract a meaningful signal. These researchers extracted details about the tiny movements of the chest in the video using a computer vision technique known as optical flow.
As Draganfly and UniSA planned and replanned with the news cycle, the virus disrupted the entire world. The team in Australia and Canada commenced an ambitious product development project using their hastily prepared home offices and laboratories. It was clear that non-contact technology options to survey and monitor for illness were few. The appeal of the drone was the possibility to position a pandemic screening perimeter far from critical facilities.
The team started a review of COVID-19 symptoms and possible computer vision techniques for detecting them in March 2020 [2]. It revealed what we all now know: that COVID-19 often causes higher body temperatures and heart rates. Later, lung function might be compromised, leading to reduced blood oxygen saturation (SpO2) and sometimes increased breathing rate.
Clinical and epidemiological data started emerging from reliable jurisdictions in April and May. It became clear that the drone application, designed to triage crowds, was unlikely to be required for COVID-19. By June 2020, the drone software was working well, but the mission of the team had changed to static screening kiosks that would allow business and education to continue to operate in countries with high rates of infection.
A thermal camera was added to the video camera, with an artificial intelligence visual system that could take an approximate core body temperature from infrared radiation emitted by the inner canthus, or tear ducts.
Nevertheless, there were some challenges. Thermal cameras authorised for human thermography by the US Food and Drug Administration were hard to obtain. Supply chain problems plagued the project in mid-2020. Yet, by late August 2020, commercial screening kiosks based on the technology had been installed in multiple locations across the United States.
It became clear that SpO2 estimates would be necessary at screening points. The team devised an algorithm that estimated it from faces using the changes of ratio of red to green with each heartbeat. By October 2020, Draganfly was installing uniquely capable screening systems that included an SpO2 estimate as quickly as they could manufacture them.
Work continues on the software, with the aim of improving the its accuracy for COVID-19 screening. Longer term, this will evolve into a capability that can optically detect probable symptoms of diseases with pandemic potential from a significant distance.
The University of South Australia received funds from Draganfly inc. to undertake the COVID-19 screening research.
Professor Javaan Chahl joined the University of South Australia as Defence Chair of Sensor Systems in 2012, prior to that he has been a Professor of Aerospace and a Public Servant with Defence. He does research on novel sensing applications.
[1] Al-Naji, Ali, Asanka G. Perera, and Javaan Chahl. “Remote monitoring of cardiorespiratory signals from a hovering unmanned aerial vehicle.” Biomedical engineering online 16.1 (2017): 1-20.
[2] Khanam, Fatema-Tuz-Zohra, et al. “Noncontact Sensing of Contagion.” Journal of Imaging 7.2 (2021): 28.
