Clinical AI in the new world of ChatGPT

Clinical AI in the new world of ChatGPT

For many years, we have developed niche AI to help our hospital clients. Two years ago, we released our first platform (CAC) powered by a metered (fee-for-use) external AI service. Specifically, we use Microsoft's Cognitive Services platform and its Natural Language Processing (NLP) API to translate freeform clinical text into medical concept codes. This AI cannot be used for patient care, but it is very useful for other purposes including assistance in the medical coding process.

‍

‍

With the mainstream adoption of ChatGPT and other Large Language Models (LLMs), it is worth reviewing the current landscape and examining where things are headed.

‍

‍

The current state of clinical NLP

While commercial clinical NLP is available from vendors such as Microsoft, Google, IBM, and Amazon, the recent exposure of Large Language Models (LLMs) in the media has captured everyone's imagination. ChatGPT has made it clear that AI will transform many professions, including medical transcription and clinical coding.

‍

‍

WARNING: Do not enter any sensitive data into any Language Model, including GPT-4. It is not intended for commercial use, does not comply with privacy standards, and is not accurate enough for general medical use.

‍

‍

‍

‍

The diagnosis codes seem reasonably accurate. The intervention codes are not. GPT-4 was not specifically trained for the purpose of interpreting clinical data; however, it did correctly identify three diagnosis and two intervention concepts from the text and it was familiar with Canadian-specific medical codes. This is a remarkable feat for a general-purpose model.

‍

‍

Practical barriers to medical coding with AI

Data privacy and security

The handling of sensitive patient data is of the utmost importance. Hospitals must ensure that any AI systems they use comply with data privacy regulations such as PIPEDA. Commercial vendors like Microsoft offer this level of security, whereas public language models like GPT-4 do not.

‍

‍

Accuracy and reliability

Although clinical AI systems have shown high levels of accuracy in translating clinical text to standardized medical codes, they still have a long way to go. In the near-term, AI will continue to be a coding assistance tool rather than a complete replacement for professional coders. Our CAC platform is built on this coding assistance principle and focuses on enhancing the overall productivity of the coding workflow.

‍

‍

Limited understanding of context

GPT-4 generates text based on the context provided to it; however, it may not always understand the context of medical codes and their relation to a patient's medical history, diagnosis, and treatment. That being said, AI's ability to consider context is increasing at an astonishing rate.

‍

‍

It can only code what it can see

Within the healthcare system, the availability of comprehensive medical data has been a persistent limitation for accurate medical coding. To ensure quality medical records, we must address several upstream issues, such as comprehensive integration of all available medical data and the thoroughness of clinical documentation.

‍

‍

Where things are going (and when)

No one knows. Everything is too new. If one were to logically predict the likely workflow using highly accurate AI, it is reasonable to conclude that EMRs will eventually have sophisticated transcription and automated medical coding functionality built into them. In this scenario, coding software, as we know it, would be redundant.

‍

‍

However, please keep in mind that there are many practical limitations to this becoming a reality. Many hospitals currently use paper charts and we do not yet have a consolidated electronic patient record in Ontario — change in healthcare is very slow.

‍

‍

What should our hospital do to prepare?

Form an AI team

It is a good idea to form an AI team to monitor the use of AI in healthcare and determine if there are new tools that can be used to immediately improve the effectiveness of care in your organization. It will also help you better assess what is coming down the pipeline.

‍

‍

Improve your IT infrastructure

The use of cloud-based providers will become the norm; however, it takes time for a hospital to become comfortable with securely using cloud services that involve transmitting sensitive data to a third-party provider. It requires knowledge of advanced topics such as cybersecurity and API usage.

‍

‍

Focus on data integration and data quality

The value of AI will come from the quantity and quality of data available to it. Comprehensive data integration and data quality initiatives are expensive and laborious, and there are few shortcuts.

‍

‍

Focus effort and find good partners

There are many novel ways to use this technology and not all of them bring concrete benefits. Have a process to define the actual value derived from any of the many options, and bring in experts to help in the domain, as it may be impractical to internally develop certain advanced skills.

‍

‍

Summary

Interactive AI that provides accurate, detailed answers to extremely complex questions will soon be available to everyone. The commoditization of AI will enable entrepreneurs and software engineers to solve practical problems and build new business models.

‍

‍

It will be messy, and it will take time for healthcare providers and their partners to adapt. The best short-term approach will be to find where the current value is and be adaptable as everything evolves, while managing the risks and distractions that will inevitably appear.