Hospitals everywhere are are developing and deploying knowledge resources and CDSSs integrated into many of their existing silo systems.
Are we doing it right? Don't know...
Is there a best way? Definitely not!
Is there a better way? Probably...
On reflection, these are my main conclusions:
CDSS has an extremely broad range of use cases.
Many rules are extremely very simple (so there is no requirement to deploy a complex GELLO interpreter before taking the first baby-steps):
CDSS can only use the information available to it. The less a CDSS engine knows about a patient the more cautious its conclusions must be.
Reliable CDSS will require structured quality clinical data. Deploying the CDSS maybe necessary before this data is available reliably in order to give clinical teams an incentive to collect and correct this data (as well as allowing the necessary cultural evolution to begin).
A virtual medical record (VMR) provides a series of standardised structured containers for aggregating and sharing all the available data on a patient:
Beginning to populate each of the VMR slots allows exponential growth of the potential rule-base scope.
If you only have a problem list, you cannot refine conclusions based on disease severity (as measured by selected observations).
Pooling KR into a unified repository will facilitate the necessary configuration management and quality assurance.
In the future it is easy to imagine that a typical tertiary hospital will have CDSS rules count in the 100’s of thousands, conceivably millions. Most of these rules will have been developed at other institutions. Test case libraries will be a critical component of the quality assurance process.
CDSS conclusions will vary widely on three axes: certainty, urgency, and importance. Classifying the conclusion along these axes will itself be the subject of complex rules. eg “it looks like the patient’s heart is in VF (so has stopped beating)” but maybe the ECG dot has just peeled off and is reading the pillow’s electrical activity rate – urgent, important, but not certain.
Selecting the appropriate target for CDSS conclusions (RMO? Ward nurse? Dietician? Consultant? GP? Patient?) may also be a complex decision.
Automated collection of feedback from the coal-face will greatly speed the process of rule development and refinement. This could be as simple as having four closing buttons on the bottom of a report “Thanks!”, “Useless”, and “This is inaccurate”, “This is dangerously inaccurate”. If it isn’t possible to deploy a button then a set of tracked URLs would allow a central server to collate the feedback.
Possible causes of “dangerously inaccurate” CDSS conclusions maybe: poorly worded conclusions, errors (bugs) in the logic, errors (bugs) in the underpinning semantic networks (eg SNOMED), inaccurate or incomplete information about the patient, system integration failures, failure of the user to read the conclusion properly, a bug in CDSS engine.
Doctors deal well with unreliable information. They even retain respect for the sources of unreliable information providing the source confesses its uncertainty. (Radiologists are gifted at expressing their uncertainty while conveying the possible conclusions.) So we should not be afraid to present doctors with cautious interpretations – as long as conclusions are labelled with appropriate disclaimers doctors will cope, and retain confidence in the system's value.
Tracking the computation cost of every rule will be important: The system must automatically collect statistics on how often each rule is fired and how many CPU seconds it used. Hardware is cheap but 100000 rules running on half a million patients has the potential to unnecessarily consume ridiculous computational resources if not managed.
Computationally expensive rules will need to be examined. It maybe that a rule is taking five minutes (or five days) to run because its performing extraordinarily sophisticated and important calculations… but its far more likely the rule just needs to be re-written or optimised. Similarly, inane rules of only moderate complexity maybe being run with excessive frequency. Keeping statistics on this computational cost will allow the technical team to tweak rule algorithms, and negotiate with clinical teams.
Finally, this is all, by nature of it not being done (much) before, experimental. Caution is a good thing.
Are we doing it right? Don't know...
Is there a best way? Definitely not!
Is there a better way? Probably...
On reflection, these are my main conclusions:
CDSS has an extremely broad range of use cases.
Many rules are extremely very simple (so there is no requirement to deploy a complex GELLO interpreter before taking the first baby-steps):
- Patients taking X drug need Y screening test every Z days.
- Patients of W age should have had procedure/drug X at least Y times or within the last Z days
- If a patient is booked for procedure X then please perform action Y
CDSS can only use the information available to it. The less a CDSS engine knows about a patient the more cautious its conclusions must be.
Reliable CDSS will require structured quality clinical data. Deploying the CDSS maybe necessary before this data is available reliably in order to give clinical teams an incentive to collect and correct this data (as well as allowing the necessary cultural evolution to begin).
A virtual medical record (VMR) provides a series of standardised structured containers for aggregating and sharing all the available data on a patient:
- Age and gender
- Observation values
- Problems
- Medications (past, current and future)
- Procedures (past, current and future)
- Allergies and alerts
Beginning to populate each of the VMR slots allows exponential growth of the potential rule-base scope.
If you only have a problem list, you cannot refine conclusions based on disease severity (as measured by selected observations).
Pooling KR into a unified repository will facilitate the necessary configuration management and quality assurance.
In the future it is easy to imagine that a typical tertiary hospital will have CDSS rules count in the 100’s of thousands, conceivably millions. Most of these rules will have been developed at other institutions. Test case libraries will be a critical component of the quality assurance process.
CDSS conclusions will vary widely on three axes: certainty, urgency, and importance. Classifying the conclusion along these axes will itself be the subject of complex rules. eg “it looks like the patient’s heart is in VF (so has stopped beating)” but maybe the ECG dot has just peeled off and is reading the pillow’s electrical activity rate – urgent, important, but not certain.
Selecting the appropriate target for CDSS conclusions (RMO? Ward nurse? Dietician? Consultant? GP? Patient?) may also be a complex decision.
Automated collection of feedback from the coal-face will greatly speed the process of rule development and refinement. This could be as simple as having four closing buttons on the bottom of a report “Thanks!”, “Useless”, and “This is inaccurate”, “This is dangerously inaccurate”. If it isn’t possible to deploy a button then a set of tracked URLs would allow a central server to collate the feedback.
Possible causes of “dangerously inaccurate” CDSS conclusions maybe: poorly worded conclusions, errors (bugs) in the logic, errors (bugs) in the underpinning semantic networks (eg SNOMED), inaccurate or incomplete information about the patient, system integration failures, failure of the user to read the conclusion properly, a bug in CDSS engine.
Doctors deal well with unreliable information. They even retain respect for the sources of unreliable information providing the source confesses its uncertainty. (Radiologists are gifted at expressing their uncertainty while conveying the possible conclusions.) So we should not be afraid to present doctors with cautious interpretations – as long as conclusions are labelled with appropriate disclaimers doctors will cope, and retain confidence in the system's value.
Tracking the computation cost of every rule will be important: The system must automatically collect statistics on how often each rule is fired and how many CPU seconds it used. Hardware is cheap but 100000 rules running on half a million patients has the potential to unnecessarily consume ridiculous computational resources if not managed.
Computationally expensive rules will need to be examined. It maybe that a rule is taking five minutes (or five days) to run because its performing extraordinarily sophisticated and important calculations… but its far more likely the rule just needs to be re-written or optimised. Similarly, inane rules of only moderate complexity maybe being run with excessive frequency. Keeping statistics on this computational cost will allow the technical team to tweak rule algorithms, and negotiate with clinical teams.
Finally, this is all, by nature of it not being done (much) before, experimental. Caution is a good thing.
No comments:
Post a Comment