Quality in Primary Care Open Access

Key words

appraisal, audit, general practice, revalidation, significant events

Introduction

Significant event analysis (SEA) is a qualitative method of audit, which is now heavily promoted and encouraged in general practice as a  exible and important quality improvement tool.[1–6] The SEA technique allows for a single event or related events that are considered ‘significant’ by individual practitioners and healthcare teams to be highlighted, discussed and analysed in a formal and structured manner. Learning issues are then identified and change can be implemented to minimise the risk of the event recurring, where applicable. Most highlighted events will tend to be of a clinical or administrative nature and can typically be classified as adverse or critical incidents, errors or near misses. A noted feature of the SEA technique is that it also encourages ‘significant’ events that demonstrate ‘good practice’ to be identified and re ected upon by the healthcare team.

The regular application of SEA by a healthcare team can potentially help to facilitate re ective learning, highlight needs assessment, address risk management issues, enhance patient safety and improve patient care.[3,7–9] There is also some evidence that SEA can improve team dynamics in a practice and increase respect and trust between team members.[10] However, there is limited research about the range and underlying causes of many significant events that occur in general practice, or of the adequacy of the analyses performed and the subsequent impact on practitioner behaviour and patient care. More specifically, knowledge about the personal attitudes of healthcare professionals and perceived barriers towards formally addressing and analysing significant events as part of everyday clinical practice is also limited.[11]

Gaining some insight into the attitudinal beliefs of general practitioners (GPs) about the application of SEA is vitally important for a number of practical and professional reasons. There is increasing pressure on GPs to show that they understand and can demonstrate use of the SEA technique. For example, general practices in Scotland seeking Royal College of General Practitioners (RCGP) Practice Accreditation must provide documentary evidence of involvement in this area. In the west of Scotland regional deanery, evidence of participation in SEA is now a compulsory requirement for all GP training practices and this is periodically checked and verified. The recent formation of the National Patient Safety Agency (NPSA) and its role in co-ordinating a national system of mandatory adverse incident reporting is another important reason why significant events are of potential relevance to all GPs and their teams.[12] But perhaps the most compelling reason is that participation in SEA is now a compulsory condition of the nascent system of appraisal and revalidation for all doctors in the UK.[13] It is now abundantly clear that the identification and subsequent analysis of significant events are firmly embedded in the NHS quality agenda and the professional regulation of medical practitioners in the UK.

Taking these various top-down examples into consideration, GPs and their teams may have little option but to cultivate a positive attitude towards the identification and analysis of significant events or, at the very least, be seen to take part in this activity even if unconvinced by the purported benefits.

We aimed to determine the attitudes of principals in general practice in Greater Glasgow to addressing significant events and participating in their analyses. In particular, perceived barriers to and beliefs about the SEA technique and its application were explored. Our null hypothesis was that there would be no di¡erence in attitudes between GPs, regardless of their training practice status, membership of the RCGP or length of service as a principal in general practice.

Methods

Ideally this study would have involved a mix of sta¡ from the primary care healthcare team, but it was restricted to principals in general practice because of the availability and easy access to an accurate and upto- date local database with details of these doctors.

A postal questionnaire was sent to all 617 principals in general practice in Greater Glasgow Primary Care Trust (PCT). Non-respondents were sent two additional reminders and the survey was completed in January 2002. The questionnaire was devised by the authors and pre-tested amongst five departmental colleagues who are part-time GPs. A pilot survey involving twelve principals in general practice based in the Eastern Glasgow Local Health Care Cooperative (LHCC) was undertaken in October 2001. Minor amendments were made to the design of the questionnaire based on the feedback received from pilot participants.

Data were collected on a variety of GPs’ professional and practice characteristics. Respondents were also asked whether they agreed or disagreed with a series of 11 attitudinal statements specifically concerned with perceived barriers to and beliefs about Data were collected on a variety of GPs’ professional and practice characteristics. Respondents were also asked whether they agreed or disagreed with a series of 11 attitudinal statements specifically concerned with perceived barriers to and beliefs about

Calculations of 95% confidence intervals (CIs) were carried out to quantify the di¡erences in the proportions of respondents agreeing to each statement by both training practice status and RCGP membership. Relationships between attitude and length of time as a principal in general practice were examined using a chi squared test of association

Results

A total of 617 principals in general practice were surveyed and 466 responded (76%), with 162 responses (35%) from training practice principals and 304 (65%) from non-training practice principals. GPs from training practices were more likely to respond (162/186, 87%) than colleagues from the non-training practice environment (304/431, 71%). The characteristics of respondents and their practices are outlined in Table 1.

Table 1: Principals in general practice: summary of respondents’ and practice characteristics

The proportions of GPs who agreed with the various statements about perceived barriers to addressing and analysing significant events are outlined in Table 2. Sizeable minorities of respondents were in agreement with most of the barrier statements, but in particular they agreed with the two statements about the di¤culty in determining when an event is significant and being uncertain how to analyse a significant event. In addition, a minority of respondents also agreed that significant events can cause problems between sta¡ and that they sometimes avoided dealing with them because of the complexities involved. Lack of time was agreed by a majority of GP respondents to be a specific barrier to the discussion of significant events.

Table 2: Barriers to SEA: proportion of GPs in agreement with attitudinal statements, by training practice status and RCGP membership

Further analysis of the results showed that there were clear di¡erences in some of the reported attitudes of GPs from the training and non-training environments. Similar, but less statistically significant di¡erences were also evident between RCGP members and non-members. In particular, training practice GPs were less likely to agree with certain barrier statements than colleagues from the nontraining environment. For example, there were statistical di¡erences in the levels of agreement with the statements about confidence in identifying a significant event, about being uncertain how to analyse a significant event and about the lack of time to discuss events. There were similar but less significant di¡erences towards these particular statements depending on whether respondents are RCGP members or not, with non-members tending to agree more with the barrier statements.

The vast majority of respondents were in agreement that they can always learn something from a significant event after it has occurred and also that SEA should be undertaken by all GPs as part of the revalidation process (see Table 3). Similar numbers agreed that SEA is not threatening or that it should only be undertaken by clinical sta¡. A significant minority agreed that they would require further training to analyse a significant event, although significantly fewer training practice respondents were in agreement with this statement. RCGP membership or non-membership did not appear to be an important factor in determining the proportion of responses to the various belief statements, although there were minor di¡erences between both groups.

Table 3: Beliefs about SEA: proportion of GPs in agreement with attitudinal statements, by training practice status and RCGP membership

Using length of time as a principal in general practice as a proxy for ‘experience’, there are clear associations between two of the selected attitudinal statements and depth of experience (see Table 4). Less experienced respondents (those who had been a GP principal for five years or less) were statistically more likely to agree that they have di¤culty in determining when an event is ‘significant’ and also agreed that they avoid dealing with events because of the complexity involved. Overall, the same age group is proportionately more likely to agree with the majority of the barrier and belief statements outlined than their more experienced colleagues, although there is a lack of strong statistical evidence to add further support.

Table 4: Principals in general practice: proportion in agreement with selected `barriers’ and `beliefs’ attitudinal statements and length of time as a principal

Discussion

A very good survey response rate was achieved from the large group of GPs who were identified for the study. However, cross-sectional surveys of this type have a number of methodological limitations. In particular we relied on GP respondents to self-report, which potentially limits the reliability of the data collected as there is no independent means of verifying their responses. A slight bias may have been introduced because GPs in training practices responded in greater numbers than colleagues from non-training practices. This may re ect their local knowledge and experience of SEA and audit in general.

The survey focused on the views of the principal in general practice, who is, of course, only one member of the primary care team, in order to gather attitudinal information about SEA and its potential role in the revalidation process. In doing this it is acknowledged that SEA is essentially a team-based activity, which has the potential to enhance both team working and practice learning and as such it should be seen in this context.

The mounting pressure on GPs from a variety of sources to undertake regular SEA and provide documentary evidence of participation appears to be dependent on the assumption that there are no apparent barriers to being involved in this activity. There is a further assumption that individual GPs possess the inherent ability and knowledge to instinctively identify a significant event, bring it to the attention of the practice team in a suitable forum and perform a structured analysis. In addition, the independent nature of general practice culture may point to a lack of consistency in the approaches taken to quality and patient safety issues. In reality, this might make it di¤cult for some practice teams to actually meet and sit down together, never mind discuss the implications of potentially sensitive significant events.

However, it is possible our findings may have identified some specific barriers to GPs’ potential participation in SEA. The ability to determine when an event is ‘significant’ is an integral part of the SEA process and the potential link to local adverse incident reporting systems and the national system co-ordinated by the NPSA. It is clear that a sizeable minority of GPs agree that they have di¤culty in this particular area, which may impact on their ability to potentially report relevant adverse incidents and also participate in SEA for revalidation purposes, although this may be highlighted as part of the appraisal process. The issue or definition of ‘significance’ is an area that perhaps merits further research if practical guidance is to be developed to assist GPs in determining which events are relevant and require in-depth analyses or even reporting.

Recent research has highlighted a potential educational issue for a large number of GPs in terms of the quality of significant event analyses they have performed. Approximately one-third of completed SEA reports voluntarily submitted in confidence by motivated GPs (as part of their postgraduate education) to be independently assessed by peer review were considered unsatisfactory by their GP colleagues.[14,15] GPs who could have implemented change as part of their analyses, but failed to do so, or who were inexperienced in SEA in comparison to GP trainers, were much more likely to have their SEA report assessed as unsatisfactory.

Our results appear to support these findings by highlighting that one-quarter of GPs agreed that they are uncertain how to properly analyse a significant event, with one-fifth of all respondents admitting that they would require training in SEA. The SEA technique is relatively simple to put into practice and a structured method of doing this from an educational perspective has previously been described.[15] However, it is clear that there is a potential education and training issue for many GPs if SEA is to be adequately understood and applied in a robust manner. An additional problem may be passivity in the process of SEA (similar to what can happen in criterion-based audit), whereby some individuals do not highlight any significant events but ‘take part’ in the discussion of others. These are perhaps areas that primary care organisations may wish to consider and tackle as part of their ongoing clinical governance agenda. Unfortunately, it appears that simple documentary evidence of participation in SEA rather than the quality or e¡ectiveness of the analyses undertaken by the GP will be monitored during the appraisal process. It could be argued that this is potentially a missed opportunity to provide formative educational feedback to GPs on the quality of SEA performed.

The vast majority of respondents agreed that SEA should be undertaken as part of the revalidation of doctors in the UK, although around one-quarter disagreed with this statement. This generally positive attitude augurs well for the revalidation process and it is hoped that those who disagreed may see the benefits and need for SEA when participating in annual formative appraisal. Failure to do so may potentially impact on their fitness to practise as a doctor in the future.

The findings highlight significant di¡erences in the level of responses to certain statements between GPs from training and non-training practices. Training practice respondents were significantly less likely to agree with many of the barrier statements compared with colleagues from the non-training environment. This is exemplified by the di¡erence in attitude between these respondents to the statement about there being a lack of time to discuss significant events. An explanation for this and the other di¡erences may be related to the fact that GP training practices in the west of Scotland (which covers six PCT areas) have participated in a regional audit programme since 1996, part of which involves undertaking SEA. Evidence of participation is checked and verified on a periodic basis. It is possible that taking part in a compulsory, rolling programme of verifiable audit activity may have an impact on GPs’ attitudes, as well as their knowledge and ability to perform SEA, compared with those whose participation in audit may be on an ad hoc and unchecked basis.

The RCGP provides leadership to its members on a range of professional and quality matters amongst which the need to participate in regular SEA is made clear. We reported some proportional di¡erences in the responses to certain statements between RCGP members and non-members. However, this may also be partly explained by the responses of those GPs from training practices, many of whom are RCGP members, and so it is di¤cult to state with confidence that RCGP membership or non-membership were major factors in the formation and development of GP attitudes towards SEA. With hindsight, including RCGP membership as an independent comparative variable may have been misplaced.

The finding that those GPs who have less experience as principals are more likely to agree with certain statements about potential barriers to SEA than those more experienced colleagues is perhaps not surprising. Given the complexity and uncertainty associated with modern general practice, it is possible that exposure to significant events and dealing with them over time may become less daunting as experience of general practice is gained and attitudes change accordingly. However, a potential issue is still raised for those less experienced GPs in this specific area and it may be that a combination of training and additional support from more experienced colleagues could assist these particular GPs where this is deemed necessary.

The study set out to gauge the attitudes of a large group of GPs towards identifying and analysing significant events in light of the current requirements from di¡erent professional and organisational bodies for GPs and their teams to participate in SEA and provide verifiable evidence. In summary, GPs are generally positive about SEA being part of the revalidation process and do not view the technique as threatening. However, educational issues may have been raised for a number of GPs, particularly with regard to identifying and analysing ‘significant’ events and blocking o¡ the necessary time to do this in the practice. There were significant di¡erences in the reported attitudes from di¡erent groups of respondents and it is clear that the training practice environment, certainly in the west of Scotland, may have been a strong in uence on the development of more positive attitudes towards SEA amongst the particular GPs surveyed. The local training practice environment may be better placed to guide and support GPs with the SEA technique because of the existing quality agenda and educational infrastructure. However, it is possible that lessons can be learned by others from the quality model in use and the approach to audit taken.

Further study is required into the whole area of significant events and their analysis in general practice. In terms of exploring in detail the variation in attitudes to dealing with significant events amongst GPs and the potential areas this could impact on, it may be that in-depth qualitative research is required to gain a deeper understanding of these issues before potential solutions can be realised.

Recommendations

Our study may have identified an important clinical governance and educational issue. Primary care organisations appear to be under the assumption that clinicians instinctively support the use of the SEA technique, are knowledgeable about it and are able to put it into practice satisfactorily. However, our findings show this is not necessarily the case. If primary care organisations are serious about improving risk management procedures and enhancing patient safety then local education and training in related areas should be provided (see Box 1).

Box 1: Suggested recommendations

As a first step, consideration should be given to winning the ‘hearts and minds’ of those unaware of or unconvinced by the SEA technique as a key teambased activity. Next, education and guidance is required in the identification of significant events and in assisting sta¡ to develop a threshold for when these are ‘significant’ enough to necessitate a structured analysis or not. The role of SEA in identifying and promoting examples of good practice should also be highlighted. Importantly, training in performing an adequate structured analysis (perhaps using real or simulated significant events) should also be o¡ered. Clinicians and other sta¡ need to be confident that they can apply SEA properly so as to make best use of the technique both as a re ective learning and as a potential change management tool.

Acknowledgements

We would like to thank the GPs in Greater Glasgow who kindly participated in this survey.

References

1. Pringle M, Bradley CP, Carmichael CM et al. (1995) Signi? cant Event Auditing: a study of the feasibility and potential of case-based auditing in primary medical care. Royal College of General Practitioners, Occasional Paper Number 70. RCGP: London.
2. Department of Health and National Patient Safety Agency (2001) Doing Less Harm: improving the safety and quality of care through reporting, analysing and learning from adverse incidents involving NHS patients. Department of Health: London.
3. Harrison P, Joesbury H et al. (February, 2002) Signi? cant Event Audit and Reporting in General Practice. Commissioned Report by the School of Health and Related Research, University of She¤eld. ScHARR: She¤eld. www.shef.ac.uk/uni/academic/R-Z/ scharr/hpm/research.html
4. Stead J, Sweeney G and Westcott R (2001) Signi? cant event audit – a key tool for clinical governance. Clinical Governance Bulletin 1: 13–14.
5. Wilson T, Pringle M and Sheikh A (2001) Promoting safety in primary care – research, action and leadership are required. British Medical Journal 323: 583–4.
6. Wilson T, Smith F and Lakhani M (2002) Patient safety in primary healthcare – an overview of current developments in risk management and implications for clinical governance. The Journal of Clinical Governance 10: 25–30.
7. Bradley CP (1992) Turning anecdotes into data – the critical incident technique. Family Practice 9: 98–103.
8. Pringle M (1998) Preventing ischaemic heart disease in one general practice: from one patient, through clinical audit, needs assessment, and commissioning into quality improvement. British Medical Journal 317: 1120–3.
9. Royal College of General Practitioners (1999) Appendix 5. Memorandum by The Royal College of General Practitioners (ACI 90). Select Committee on Health, Appendices to the Minutes of Evidence. RCGP: London.
10. Sweeney G, Westcott R and Stead J (2000) Thebene? ts of signi? cant event audit in primary care: a case study. The Journal of Clinical Governance 8: 128–34.
11. Westcott R, Sweeney G and Stead J (2000) Signi? cant event audit in practice: a preliminary study. Family Practice 17: 173–9.
12. Mayor S (2001) NHS introduces new patient safety agency. British Medical Journal 322: 1013.
13. Scottish Executive, NHS Education for Scotland, RCGP (Scotland) and BMA (Scotland) (2003) GP Appraisal: a brief guide. Scottish Executive: Edinburgh.
14. McKay J, Bowie P and Lough M (2003) Evaluating signi? cant event analyses: implementing change is a measure of success. Education for Primary Care 14: 34–8.
15. Bowie P, McKay J and Lough M(2003) Peer assessment of signi? cant event analyses: being a trainer confers an advantage. Education for Primary Care 14: 338–44.