“If you can’t measure it, you can’t manage it”. It’s a cliché that more than a few of my old bosses in corporate used on me during strategy, reforecasting and budget sessions. It was usually the smarter Dutch ones that pulled me up when I pitched something that sounded great but didn’t have the data to back it up. The Dutch have a big thing for numbers that align. Ones you can measure against the past and benchmark your progress into the future.
Following my interview with the COO of The Australian Digital Health Agency (ADHA) Bettina McMahon I was struck by just how much progress she outlined, versus a very vocal and largely negative backdrop of local industry leaders, commentators (including a few expert blogs) , independent experts and ,eek, the media (yes, us) .
If you read through everything McMahon talks about in the secure messaging program of the Agency, between the lines of ‘everything is going swimmingly’, there’s actually a fair bit of work that has been done by the ADHA in the last two years.
How much of it constitutes meaningful progress though? And what are we measuring ourselves against?
Naysayers, of which there are many, would argue that the last two years has been a lot of things ticked off and paraded to the public and ministers as progress (aka the My Health Record) , but systemically there hasn’t much meaningful progress at all.
But that’s a bit hard to believe if you pick through all that has been attempted and you look at the detail of the work McMahon outlines.
I think the truth lies in the middle somewhere. On the one hand the My Health Record (MHR) might be an eventual dead end, as far as a properly connected digital health eco system is concerned in 5 to 10 years time. On the other, without it, we wouldn’t have legislation that provides us a fair bit of leeway to get things done with our citizens which might not normally be doable (this could be a good or bad thing of course), and we wouldn’t have a population and many more healthcare professionals now primed to the potential of digital health transformation. Isn’t that some form of progress? It’s not how a lot of people would have gone about progress, and it might not be up to the standards of a lot of critics in terms of overall progress, but it is progress.
If you ask a much more practical question around progress, like, have we made any progress at all in secure messaging, the technical answer has to be ‘yes, we have’. A bit. The major problem again is that the progress is nothing like what we’d all love to have seen or indeed nowhere near what anyone had been predicting. The ADHA cops a lot of the blame for that. But if you were measuring this objective and you were applying a reasonable degree of difficulty scale, you’d know now that the degree of difficulty was always in the high nines. So ‘a bit’, might actually be not so bad.
A lot of the industry discourse is that the whole show has been slow and tortuous through a combination of inexperience, discontinuity, playing politics and out and out blunder. But our perception of reality here might have been a little less harsh had we been more fastidious about how we measured our progress, I suspect.
In her interview, McMahon answered a lot of hard questions in quite a bit of detail, and admitted clearly that progress, whatever it has been, had come with a lot of heartache, mistakes, and effort. Much of that heartache has been industry based. But she acknowledged industry for how hard the work is for them given their relative commercial situations, and how much effort and time they put in. She puts the persistent negative vibes down to a group of people who really care and are passionate so are naturally disappointed at not going faster.
She also said that the ADHA was learning. She was a lot more forthright (and convincing) than your average ADHA press release in the assessment of progress.
Distilling McMahon’s conversation on progress into just three points I think you could say the following positive things:
- We have a lot of peak bodies and industry players, and groups coming together to work on issues in a more collaborative manner than two years ago. I’m not saying, as the ADHA does occasionally, that these groups are all facing forward and marching in a united line. Far from it. There are still issues, as the walkout of Medical Objects on the Secure Messaging program suggests. But there is a general mood and agreement that there needs to be more ‘burning of the boats’ and a ‘one community sentiment’ for the greater good.
- Read between the lines on McMahon’s words and you see that the ADHA is becoming more pragmatic and realistic about how they approach the marketing of progress. If they didn’t know they were attempting to eat an elephant in one sitting before, they do now. McMahon is clear that even secure messaging progress will be slow still, and only in selected pockets of the sector. Notably she excuses some big parts of the system which simply can’t be assuaged for commercial reasons to jump enthusiastically on board today. Big pathology is its own elephant, best left alone until some other goals are kicked. Her logic is that if you move some parts of the system, others will have to follow eventually.
- There are some good examples of interesting technical progress which the ADHA initiated and oversaw. The FHIR API between secure messaging vendor directories is one. There are others. These are only small parts of a giant puzzle, but some of it is clearly in the ambit of ‘progress’. It might be that this is a game of inches.
No matter where the MHR ends up in our digital health history – and there are many bets on that being nowhere or worse – the maniacal push to get the project to the stage it is, with opt-out, pharmacy systems, pathology results, increasing hospital participation, and some GP traction, has set an agenda that has digital health on the minds of most healthcare professionals. It also has bought time and the trust of the pollies for the current management of the ADHA. The pollies love it. That isn’t such a bad thing for now.
If you look at New Zealand and compare their digital health progress you seem to see a lot more tangible progress with less effort, money and fuss. What is happening there?
A few things come to mind. It is a much more tightknit community generally, it has a point to prove against bigger players (think The All Blacks) and it has a better structural set up in that it is less federated for funding and management.
But if you have to pick two things that seem to make it work better I’d say there is a much more established culture of collaboration in New Zealand – which is maybe the small community needing to punch above their weight thing – and they do a lot more around system measurement. They are data nuts. It’s partly their funding system which focuses more on outcomes that creates this need for data. But there is something else intangible in the culture there – something a little Dutch perhaps.
If you’re a GP in New Zealand and you’re treating someone with diabetes you know a lot about them. You have lots of data in a longitudinal line on all important points of management. If you don’t have that of course you don’t get paid properly. In Australia, you have mostly your memory and a few notes on your patient management system to help you. Outside of that you’re fairly lost. The difference does have a lot to do with how funding varies between the countries, but again, there it’s more systemic than that. You can’t measure progress without transparent and meaningful data to compare in time.
The ADHA tends not to set itself KPIs (key performance indicators) that are detailed, hard and fast. In fact in one interview I had there once I was told that goal setting was not there job but the job of their masters in the Department of Health and the various state health departments. Their job is to meet those goals.
See if you can find any hard and fast KPIs for the ADHA in the National Digital Health Strategy. It feels almost deliberate. It’s goals are grand and visionary. My Dutch bosses would say flowery and without much meaning because there aren’t any properly realistic numbers that have been thought about carefully in the context of the job to do, and the track record in place.
Some of the goals – less deaths through medication misadventure – aren’t practical at all in terms of measurement. They are surprising goals because existing measurement of this statistic is so misunderstood and poor to start with. You can’t measure the future without a hard line on the past. But this goal is held up as one of the biggest ones that justify a lot of the work of ADHA. It’s a fluffy figure not many people understand now. How are we going to benchmark to that as we move forward? Where is our score now? Where should it be?
McMahon interview provides detail and figures. It feels fairly clear that we’ve had progress of sorts. And mostly it’s measurable progress, if we measured it properly.
It would be great if the ADHA now (perhaps with their DoH masters) could sit down, and come up with some numbers which we could meaningfully measure progress against going forward. We promise not to be too negative if they miss these numbers. But if you think about this process, if they do provide numbers, they are meaningful, and they meet them, maybe that will dampen some of the negative noise out there.
We know this game isn’t easy now. And we know such math based KPIs will need to be constantly looked and revised.
If we don’t start measuring at a more systemic level from the top, it isn’t likely to become imbedded in the culture of our evolving healthcare system further down.
One related and very important issue to measurement is common governance and standards by which digital health can be managed. Detailed numbers that align in time matter a lot. In the management of businesses and in the business of health.
