Research Article - (2015) Volume 23, Issue 3
Director of Teaching and Learning, co-Deputy Director, Centre for Health Policy
Gary L Freed MD, MPH
Percy and Mary Murphy Professor of Pediatrics School of Medicine and Professor of Health Management and Policy, School of Public Health, University of Michigan and Professor, co-Deputy Director, Centre for Health Policy
Matthew J Spittal B Sci (Hons), MBio, PhD
Senior research fellow, Centre for Mental Health
Michelle Williamson B Health Sci (Hons)
Research fellow, Centre for Health Policy
Melbourne School of Population and Global Health, The University of Melbourne, Australia
Telephone triage and advice services (TTAS) operate in many countries to help manage demand for primary care and hospital emergency departments and improve access for those who experience barriers to obtaining healthcare. Simulated patients have been used to evaluate safety, appropriateness and adequacy of communication of TTAS. This short report presents a quality improvement model based on the simulated patient method that will be implemented in Australia’s national nurse and out of hours GP helpline in 2015. It consists of five steps: random selection of a subset of commonly presenting case scenarios with an uncommon level of acuity in some scenarios; an annual cycle of simulated patient calls presenting the selected scenarios; robust clinical and communications assessment using outcomes ascribed by a multidisciplinary Clinical Advisory Panel and clinical communication standards; rapid feedback to the service provider on strengths, weaknesses and gaps, and provision of targeted professional development and reflective practice opportunities for helpline clinicians. Implementing the model is expected to improve the safety and appropriateness of telephone health advice offered to Australian consumers.
telephone triage, quality improvement, simulated patients
Telephone triage and advice services (TTAS) operate in many countries, aiming to help manage demand for primary care and hospital emergency departments and improve access for those who experience locational, physical or social barriers to obtaining healthcare. [1-9] A nurse TTAS, healthdirect, was first established in Australia in 1999 and similar services have been established in every Australian state and territory in the intervening period. In 2011 the Australian government established an after-hours GP helpline as a supplement to existing nurse TTAS. [10] Out-ofhours callers to the nurse helpline triaged as needing to see a GP immediately or within 24 hours are transferred to a GP on the line for assessment and advice. A government funded company, Healthdirect Australia, commissions the service from a private provider.
There is a reasonable community expectation that telephonebased health consultation should be as safe as face-to-face consultation. While studies have shown that TTAS are generally safe, it has also been found that there is room for improvement in assessment of patients with high-risk symptoms. [11,12] Telephonebased assessment of an unseen and unknown patient poses risks of both over assessment and under assessment of the urgency of a patient’s condition. [13] Assessment of urgency as higher than actual can lead to unnecessary and costly service utilisation, while assessment of urgency as lower than actual may result in delayed commencement of treatment, longer or more intensive treatment if the condition has escalated and potentially poorer health outcomes for the patient. Safety, effectiveness (and efficiency) and patient experience, important dimensions of quality in primary care, [14] may be compromised. Health system efficiency and patient experience are the main casualties of over assessment and patient safety and clinical effectiveness are likely to be diminished by under assessment of urgency.
Communication between the telephone clinician and the caller is also an important consideration in quality of TTAS. [15,16] Poor communication in telephone assessment has been found to contribute to patient non-compliance with advice [17,18] unsafe dispositions and self-care advice [9,20] and adverse events. [21] Clear communication is an essential element of a high quality telephone consultation. [15, 22]
The simulated patient method has been widely used in evaluating safety, clinical appropriateness and adequacy of communication in TTAS, usually in one-off research studies or evaluations, not as part of ongoing quality improvement.[9,12-14] The standardised nature of the simulated patient method minimises variability in presentation and increases reliability when testing clinical assessment. [23] Using simulated patients has been found to be a feasible method for assessing performance of TTAS. Simulated patients can be trained to consistently present a clinical scenario in multiple calls over time and detection of fictitious calls made by trained simulated patients is rare. [24]
Despite the demonstrated value of the method, to our knowledge a standardised approach to using simulated patients for ongoing quality assessment of TTAS has not been developed. This short report presents a quality improvement model based on the simulated patient method that will be implemented in Australia’s national nurse and out of hours GP helpline in 2015.
The aim of the model is to systematically assess and improve the safety, clinical appropriateness and adequacy of communication provided by healthdirect, a 24-hour nurse triage service, and the supplementary after hours GP helpline. The commissioner has a Clinical Governance Framework that specifies quality, efficiency and operational standards for the helpline services. [25] The quality improvement model aligns with the objectives of the Clinical Governance Framework.
Development of the model
We developed the model in a three-phase process.
Firstly, we reviewed the literature on use of simulated patients to assess quality and safety in delivery of telephone triage and health advice. We investigated the number and type of scenarios presented, the number of calls and statistical reasoning underpinning this; the professional background and number of simulated patients, the professional background and number of assessors of the calls and the methods used to test and confirm the reliability of those assessments.
Secondly, we established a seven-member multidisciplinary Clinical Advisory Panel comprising a paediatrician with both clinical and academic roles, two GPs, two nurses (an Emergency Department triage nurse and a community home care nurse), an Emergency Department physician and the Clinical Governance manager of the service commissioner. The panel advised on selection and scripting of scenarios and determined, using a consensus approach, the appropriate disposition or outcome that should be applied to a given scenario.
Thirdly, we addressed logistical and statistical issues associated with the model. Logistical issues related to the time frame of the calling period, sourcing of simulated patients from clinical education agencies and the development of clinical scoring and communication assessment instruments. Statistical issues related to the number of scenarios and calls that would be required to achieve precision in testing and generalisability of results. We also considered the number and background of assessors needed and methods to test reliability between assessors.
The resulting model has five steps, as shown in Figure 1.
Figure 1: Quality improvement model for telephone nurse triage and GP assessment
1. Select common case scenarios with an uncommonly high level of acuity in some scenarios. (For example, gastrointestinal discomfort is a common, usually nonurgent presenting case, but we included a scenario of stomach discomfort with symptoms indicating intestinal bleeding, which is uncommonly urgent.)
The starting point in the model is a Clinical Scenario Resource based on the 20 most common conditions presenting to the nurse triage line and after hours GP helpline over a two-year period. The literature on simulated patients and published telephone triage training manuals 22 were used to flesh out clinical scenarios for each case, with further input from the Clinical Advisory Panel. The Clinical Advisory Panel ascribed an appropriate outcome for each scenario, with reference to outcomes specified in the literature and telephone triage training manuals. The scenarios have varying levels of urgency and cover paediatric, adult and aged presentations. A database of unique and feasible simulated patient identities is attached to each scenario. Each year five scenarios will be randomly selected from the Clinical Scenarios Resource. Box 1 provides examples of two scenarios and ascribed outcomes. Helpline outcomes that are more urgent than that ascribed by the Clinical Advisory Panel will be scored as “over triage” and helpline outcomes that are less urgent than the ascribed outcome will be scored as “under triage”. For example a paediatric scenario ascribed as needing “self-care at home” that is assessed by the helpline clinician as needing to see a GP within 4 hours would be scored as over triage.
2. Annual cycle of simulated patient calls
An annual call schedule will be initiated, with the five selected scenarios presented 50 times each over a 14-week period by trained simulated patients. The total number of calls (N=250) was chosen because it would ensure the workforce of approximately 200 nursing staff would each receive a minimum of one call on average. The number of scenarios (five) was selected because it represents a reasonable balance between precision (i.e. standard error) and generalisability when assessing the accuracy of the triage service. The call pattern will match the typical volume and timing of calls to the helpline across the week. Clinicians working in the service will be advised of the purpose and broad timeframe in which simulated patient calls will be made.
3. Assess clinical and communications quality
The digitally recorded calls will be assessed using the clinical outcomes ascribed by the Clinical Advisory Panel and a customised brief primary care communications assessment instrument. The communications assessment includes four quality domains of clinical communication identified in the literature: client-centred interaction, active listening, language clarity and advice clarity. [17,18] Two clinical assessors (a GP and a nurse) will listen to call recordings to determine if the advice given is in accord with the ascribed outcomes (appropriate) or demonstrates over or under assessment of urgency. Two communications assessors (an experienced simulated patient and a communications professional trained in the use of the assessment tool) will also rate the calls. A subset of all simulated patient calls (50 calls or 10 per scenario) will be rated by both assessors of clinical quality and both assessors of communication quality. Cohen’s Kappa will be used to measure inter-rater reliability of the assessors.
4. Provide rapid feedback
To ensure timely action for quality improvement, findings will be provided to the service provider within 12 weeks of completion of the round of simulated patient calls, identifying strengths, weaknesses and specific areas for service improvement and action in relation to clinical and communication performance. Findings will not be used to inform individual clinician performance review.
5. Deliver targeted professional development and reflective practice
The service operator will provide professional development opportunities and reflective practice groups for clinical staff to address weaknesses or gaps identified.
Implementation of a quality improvement model based on the use of simulated patients is expected to enhance the safety and appropriateness of telephone triage and advice offered to Australian consumers.
Thanks to Clinical Advisory Panel members: Mary Byrne RN and policy analyst, Dana Carpenter RN, Dr George Cerchez GP, Dr Liz Crock RN, Dr Jonathan Knott Emergency Department Director, Dr Kelvin Lau GP and Maureen Robinson Clinical Governance Manager. Professor Gary L. Freed chaired the Clinical Advisory Panel.
Funding to develop the quality improvement model was provided by Healthdirect Australia.
