Quality Improvement Report - (2008) Volume 16, Issue 6
Associate Professor and Deputy Director, Postgraduate Studies School of Nursing, University of Nottingham, UK
Ian Grayling BA (Hons) PGDip (Training Management)
Chief Executive East Midlands Centre for Excellence in Teacher Training, Leicester, UK
Mayur Lakhani CBE
General Practitioner and Past Director of the Royal College of General Practitioners and Current Chair of the National Council for Palliative Care, UK
Received date: 25 August 2008; Accepted date: 30 September 2008
Making Things Happen (MaTH) was devised as a six-module educational programme to support clinical governance leads (CGLs) in primary health care. The aim of the programme was to help develop practice CGLs’ awareness, understanding and ability to initiate and manage change. The objectives of the training programme were: . to promote an increased and sophisticated understanding of clinical governance by building on knowledge gained already . to develop skills in delivering and sustaining improvements . to improve personal effectiveness in bringing about change. A pilot training programme was planned and delivered twice in 2003/2004 to two separate cohorts of clinical governance leads (n = 30 and n = 15) from two pilot primary care trusts (PCTs). The six training modules were delivered to the two groups by the same staff but at different times and in two separate locations. The effectiveness of the programme was evaluated in both PCTs at three levels. First the content of the programme was evaluated by the CGLs that attended the programme. Secondly, the value of the programme to the PCT was considered by the PCT co-ordinators. Thirdly, the effectiveness of the programme was considered by the presenting team. All three groups felt that the objectives were met, that the principles of MaTH training were sound and that it would be a valuable resource to offer nationally. It was also noted that training for PCT local co-ordinators in how to support clinical governance would be a useful training co-opportunity.
clinical governance, educational inter-ventions, leadership, primary care
Clinical governance aims to achieve high performance in health care through quality improvement which, of its nature, involves the processes of change and learn-ing. For the quality of any service or product to be improved, people have to change what they do and/or the way that they do it. This process needs to be evidence based, and the impact of any resulting change needs to be validated and seen to ‘add value’. Evidence-based quality improvement is a learning process for those involved and, arguably, results in ‘organisational learning’ at the practice level. It therefore follows that clinical governance leads (CGLs) at practice level need to be skilled ‘change agents’, communicating the need for change, facilitating engagement, influencing its direc-tion, and supporting the learning that underpins it.
There is a significant view, and considerable evi-dence to support it, that quality improvement and high levels of organisational performance flourish best where teams – exhibiting a diversity of knowledge and skill within their membership – rather than individ-uals, are given responsibility to ‘own’ a problem and to solve it. The government publication, Clinical Govern-ance in the New NHS, appears to echo this view (see Figure 1).2... for clinical governance to be successful, all healthcare organisations must demonstrate features such as an ethos of multi-disciplinary team working at all levels in the organisation (p.7).
Figure 1: Multiprofessional team working
In the USA, Stephen Shortell, a vociferous proponent of ‘high performing clinical care teams’, has expressed the need for a new type of logic that moves away from ‘autonomous professionals providing largely self-directed expert care within organisational, payment, and regu-latory environments involving conflicting incentives, goals and objectives’ and towards ‘patient-centred teams providing evidence-based medicine in supportive organ-isational, payment and regulatory environments’.3
The Chartered Institute of Personnel and Develop-ment, in its review of the growing international dis-course on high performance,4 noted that its emergence characteristically accompanies:
•decentralised, devolved decision making, made by those closest to the customer – so as constantly to renew and improve the offer to customers
•development of people capacities through learning at all levels, with particular emphasis on self-management, team capabilities and project-based activities – to enable and support performance improvement and organisational potential.
Substituting the word ‘patients’ for ‘customers’ in the bulleted points above enables us to more easily apply the paradigm of ‘high performance’ to general practice and health care in general. General practices (and clusters of practices) are clearly capable of functioning as high-performing, multiprofessional teams. The recently pub-lished National Audit report on the progress of clinical governance in primary care notes that where manage-ment of quality issues in primary care is required, primary care trusts (PCTs) must endeavour to ‘ensure that people with leadership, strategic planning and organisational skills are recruited’.5 Providing train-ing in these areas is therefore an acknowledged need.also became clear that, within this agenda, the devel-opment of so-called ‘soft skills’ would need to feature prominently. The developers of the programme there-fore were faced with a considerable challenge: to devise a training programme to build the confidence and competence necessary for CGLs, irrespective of their medical background, to implement the clinical gov-ernance agenda.
The type of training that would help primary care CGLs consider and reflect on their abilities to initiate and manage change was thought to be best presented in experiential and interactive exercises providing tools for attendees to take away and use at practice level. This type of training therefore was seen as distinct from the work of, for example, the National Primary Care Development Team,6 or clinical governance sup-port team,7 where the former concentrates on specific healthcare initiatives and the latter on providing sup-port for trust managers and associated managing executives. This initiative was aimed to plug the gap specifically between these two levels.
Both of the PCTs that agreed to take part in the pilot training had a strong track record of innovation and change around clinical governance. In line with The NHS Plan,8 the two organisations were firmly com-mitted to patient-centred services led by local clinicians. The two PCTs were keen to augment their develop-mental programmes with specific work for CGLs at practice level. Both PCTs agreed that practice-level CGLs were being expected to undertake potentially complex and challenging work, and they both supported the view that their development needs should be consid-ered by a programme such as MaTH.
The training programme was designed to be delivered in five distinct but linked experiential workshops, and an evaluation would be conducted on the sixth meet-ing (see Appendix). A specific challenge was to see if such a programme could be delivered as part of the existing forum of meetings that the two PCTs already ran for their CGLs. This was an additional focus for evaluation: i.e. whether it was possible to deliver highly focused modules in the course of bimonthly regular meetings and the effectiveness of this approach. The issue is important, as in any roll-out, getting add-itional protected time is likely to be a challenging issue. Since CGLs from each practice included nurses and practice managers, as well as general practitioners (GPs), it was also important that the training was designed to be delivered to such a multidisciplinary group.
The first five modules represent a basic programme. The planned programme is summarised below and was delivered to a potential 30 delegates in PCT1 and 15 delegates in PCT2.
•Module 1: Exploring the role of a CGL.
•Module 2: Understanding practice culture and its impact on a CGL.
•Module 3: Making improvements – understanding change and its barriers.
•Module 4: Personal effectiveness – facilitating high performance team-working.
•Module 5: Improving presentation skills.
•Module 6: Evaluation of learning.
The six modules were delivered bimonthly in PCT protected time set aside for educational sessions. Each module was planned as a 3-hour session.
Training should be evaluated. However, there is no generally accepted definition of the word training. Three definitions are listed by Bramley,9 and are set out below:
•The systematic development of the attitude/know-ledge/skill/behaviour pattern required by an indi-vidual to perform adequately a given task or job.
•Any organisational initiated procedures which are intended to foster learning among organisational members in a direction contributing to organisa-tional effectiveness.
•The acquisition of skills, concepts or attitudes that result in the improved performance in an on-the-job environment.
The differences in these definitions indicate the diffi-culties that evaluators of training have. The first defin-ition suggests evaluation of individuals and their ability to perform a specific job. The second stresses the organ-isation and the benefits that training brings to the organisation. The third level emphasises the acquisition of specific skills.
In this case, where training is being provided to individuals from a variety of backgrounds to improve their confidence across a set of management skills, it is quite difficult to agree on a criterion for evaluation of success. Hamblin10 and Kirkpatrick11 provide well-established frameworks for the evaluation of training, and both emphasise the need to recognise that evalu-ation can be thought of as working at four levels.
First is the level of the recipient. Did the recipient feel the training was valuable to them? This infor-mation is normally gathered at the conclusion of a training event and is normally in the form of a simple questionnaire focused on the performance of the trainer.
Second is an evaluation of what the trainee retained or remembered about the training. This can be gath-ered at the conclusion of the training or at some defined period after the conclusion of the training.
Third is an evaluation that focuses on change in personal performance as reported by the recipient of training. Recipients can list things that they do differ-ently as a result of the training experience. Examples of activities carried out can be provided and listed as objective evidence of the success of training.
A fourth evaluation could be the improvement in performance and output of the organisation of which the recipient of training is a member. This improve-ment is noted by a third party and acts as a measure of real improvement in output traceable to the training provided.
The evaluation carried out on the MaTH pro-gramme attempts to provide self-report information linked to the first three levels of the framework described above. The fourth type could be alluded to through self-report of the CGLs, but a more objective measure would require the PCT to report on increased types of activity that demonstrated significant organ-isational progress. However, two further elements of evaluation were carried out. Interviews were carried out with the local PCT educational leads who had observed the training, and the views of the members of the presenting team were also considered.
Evaluation 1: How was the programme received by the participants?
Each module was individually assessed by the 30 partici-pants. The evaluations were predominantly positive with more than 90% of responses favourable to the content and style of delivery of the material (see Table 1).
Table 1: Respondents evaluations of the MaTH programme
Attendance was greater and more consistent at the PCT1 centre. The two PCTs used different days and times for their training. PCT1 chose a Wednesday/Thursday evening 7.00–9.30, whereas PCT2 chose Friday afternoons 1.30–4.30. Attendance was supported with an attendance payment provided by both PCTs, and both centres’ meetings were sponsored by pharma-ceutical companies. The greater percentage attendance at PCT1 was attributed to the choice of day and time that suited the delegates better.
•Satisfaction with content was clearly evident in both centres.
•Satisfaction with delivery was clearly evident in both centres.
•An improvement in personal skills was self-reported by 90% of attendees.
•All delegates attending the final session felt the five modules had been a worthwhile programme.
Evaluation 2: What changes did the participants feel they noticed in themselves?
Each participant attending the final session (n = 30) was asked to evaluate how much improvement they felt had occurred, if any, on a list of 17 personal effective behaviours thought to be associated with clinical gov-ernance. Participants were given a list of the 17 qual-ities and asked to indicate the confidence they had in each skill before the course and then to indicate how confident they now felt after the course. The before and after scores were compared for significance using a Wilxcoxon matched pairs non-parametric test of difference (see Table 2).
Table 2: Improvements in confidence reported by attendees
All 17 aspects of effectiveness were shown to have significantly improved ratings as reported by the 30 attendees. The evaluation seems to show similar degree of improvement in both cohorts of trainees, suggesting that the exercises were valued by both groups irres-pective of the particular workshop environment and mix of trainees they experienced. The delegates were also asked to list the particular aspects of training that they felt had the greatest effect upon them. These are listed below, along with the module in which they were featured.
•Understanding what clinical governance means: module 1
•Presenting skills: module 5
•Speaking in meetings: module 5
•Chairing skills: module 4
•Listening and dialogue: module 3
•Resistance to change practice culture: module 2
•Multidisciplinary awareness: module 3
•Understanding accountability: module 1
•Importance of sharing information: module 1
The selection of training skills by the attendees from all five modules provides a form of validity to the con-struction of the overall programme content.
Evaluation 3: Was there any evidence of change in delegates’ clinical governance performance?
What changes at practice level had participants attempted in their role as clinical governance lead that could be linked to increased confidence gained on the course? Participants were asked to list activities attempted during the course that had been stimulated or influenced by attendance at the MaTH course. The list included the following:
•understanding the need to organise meetings
•running the practice culture questionnaire
•involving more staff at practice meetings
•developing a problem-solving approach to meetings
•becoming generally more assertive
•understanding resistance as a psychological defence
•recognising the need to provide training where necessary
•understanding the principles of dialogue
•better understanding the purpose of meetings
•involving patients in designing change to appoint-ments system
•understanding the need to practise presentation skills
•understanding the importance of staff perspectives
•feeling confident enough to undertake audit of diabetic care.
These outcomes suggest that the training was indeed having an effect that the programme had been designed for. Here was evidence of activities that were linked to the training aims of:
• promoting an increased and sophisticated under-standing of clinical governance by building on know-ledge gained already
•developing skills in delivering and sustaining im-provements
•improving personal effectiveness.
Evaluation 4: What were the views of the PCT clinical governance support team?
The PCT representatives at both delivery sites were interviewed, and their views on the content and delivery of the training were sought. In both cases the PCT representatives felt that the experiential small group work that was included in each session, and the con-fidence of the presenters in running the sessions, had contributed towards the success of the training.
‘It’s funny but although we know about the value of small group working we never have the confidence to work with it as well as the MaTH presenters did. We find it really difficult to keep the delegates on task and we have learnt a lot from watching how these presenters did it.’ (PCT1)
‘I have been quite amazed at the way in which our GPs have taken to this type of training. Normally we have a speaker and slides and they all doze off in the corner, but with these MaTH sessions it was different ... it was incredible to watch them all engage with the material and have fun doing it.’ (PCT2)
Both of the PCT clinical governance support facili-tators saw the value of the active involvement in the training and were struck by the way the experiential tasks engaged the delegates. The training therefore may have more effect when delivered by trained pre-senters comfortable with experiential learning activi-ties. This issue will be discussed further below.
Evaluation 5: Evaluation by Royal College of General Practitioners’ project team
The planning and presenting team of KS, IG and ML met to discuss the MaTH programme and how in their view it had been presented and received, and how well they fitted the task they were asked to do.
The discussion considered the following questions:
• was the programme coherent?
•how did the delegates respond to the material?
•were the activities appropriate?
• how could the programme be improved?
•how could the programme be extended?
All three presenters felt the programme had hit on five important elements that underpin personal effective-ness of CGLs. Module 1, ‘Understanding the role of the clinical governance lead’, was well received and seemed to be a useful way for the delegates to explain in a relatively non-threatening environment what their perceptions of their role were. The task of creating a job specification for a clinical governance lead al-lowed the session to remain focused on the human element that is required to make clinical governance effective.
The second session on understanding the concept of practice culture, and having a tool to measure the culture of resistance to change, proved to be concept-ually more challenging for some delegates. The idea of measuring resistance to change within a practice was felt by some delegates to be too intrusive. The principle of resistance to change culture and separate cultures within practices was recognised, however, and referred to by delegates increasingly throughout the programme.
The third session introduced ideas about change and what facilitates or helps resist change, and it was well received. The ideas were largely conceptual but delegates readily identified with the models of resist-ance presented to them.
The fourth session on personal qualities of change agents was also well received. The experiential exercises where delegates were forced to listen to each other proved an enlightening session for all concerned. The importance of ‘dialogue’ as a mechanism for improv-ing communication between potentially hostile parties was well understood. The experiential sessions were invaluable, and the presenters felt that delegates would certainly have benefited from more time to develop their ‘dialogue’ skills.
The fifth session on presenting skills was hugely enjoyed by the delegates as they struggled to present topics to each other within defined time limits. Pre-senting to peers is an important clinical governance skill. It was clear that the delegates benefited from the opportunity to practise some basic techniques.
The presenters discussed at length the appropriate-ness of this type of training programme. There was a view that providing training at this level, while ap-preciated by the delegates, is still missing an important link in the governance chain of command. There seems to be a need to focus training not only at the practice level but also at the PCT clinical governance lead level. The importance and effectiveness of clinical govern-ance could be enhanced with training directed at PCT co-ordinator level. PCTs need to be to be clearer about their role in monitoring and supporting clinical gov-ernance in practices and what this means to both parties.
The Royal College of General Practitioners’ (RCGP’s) view that this type of training is needed at practice lead level is supported in the Audit Commission report where the need for leadership development in the management of quality in primary care is clearly identified.5
Another advantage of the MaTH training approach is that it provides an opportunity for PCTs to support practices to develop leaders of change who can, as they develop the skills, work on improving quality in their practices on their own premises with their own staff.
While evaluations of the MaTH programme were all positive, it is difficult to absolutely determine whether this type of general training really works. The evalu-ations of the CGLs that attended were positive in relation to the value of the training exercises and an improved understanding of their role. But had their ability to initiate and manage change improved? There is some self-report evidence to this effect which is encouraging and can be seen both in evaluation 2 and evaluation 3. However, it is still at the self-report stage. Further presentations of the programme could be designed where changes in performance by CGLs could be more objectively recorded.12
It may be that this type of training might be used as ‘a taster’ where CGLs can experience a number of types of activity that are linked to successful leadership. Attendees may recognise that they could develop a particular leadership skill and, where there is felt to be a realisable benefit, an attendee could sign up for further appropriately targeted training perhaps to be delivered online and supported by certification.
The MaTH programme as presented and reported here clearly was well received. If as the National Audit report on progress of clinical governance in primary care recommends there is a need for leadership train-ing in initiating and managing change,5 then delivering courses such as the MaTH programme would be one way of beginning to meet that need.
Summary
MaTH was devised as a pilot training programme to support CGLs develop skills of initiating and leading change in primary health care. The Quality Unit at the RCGP responded to the suggestion that such a pro-gramme was needed, and contracted experts in train-ing and personal effectiveness to design and help deliver the programme. The aim of the programme was to develop CGLs as leaders for change. The objectives of the programme were to:
• promote an increased and sophisticated under-standing of clinical governance by building on knowledge gained already
• develop skills in delivering and sustaining im-provements
• improve personal effectiveness.
The training programme was designed to support these developments, and delivered in a way designed to engage the delegates who attended. The evaluation listed above indicates that the programme was valued by delegates and PCT co-ordinators. The presenting team felt that the objectives were met and that the principles of MaTH training were sound and it would be a valuable resource to offer nationally. It was also noted that providing training forPCT local co-ordinators in how to support CGLs would be a useful training co-opportunity.
Commissioned, based on an idea from the author; not externally peer reviewed.
None.
The programme concept was developed by Dr Tim Wilson (RCGP) in conjunction with Dr Mike Gogarty (Director of Public Health). The programme strategy and content was further developed by ML. This was refined and prepared for delivery by training experts (KS and IG) contracted for that purpose.
Appendix : The six MaTH modules
First module: Understanding the role of clinical governance lead
• How are things going?
• What is it like being a clinical governance lead?
• What would you like to do better?
• Successes and difficulties
• Understanding the role more
The module was supported by small group work on typical issues of clinical governance described in specially prepared case studies. The output from this first module was a written job description and person specification for a clinical governance lead. It was designed to support delegates’ understanding of clinical governance.
Second module: Making improvements – understanding practice culture
This session involved an introduction to the concept of practice culture and its relationship to clinical performance. The CGLs were encouraged to consider the notion of team and individual resistance to change as an aspect of practice cultures. An example of a practice culture questionnaire was provided and discussion of the value of the questionnaire as a useful tool to measure practice culture was encouraged.
Third module: Barriers to change: understanding the theory and science of clinical change behaviour
Why is it so hard to make improvements in practice? This work was based around concepts of change management, and involved practical exercises to identify and overcome barriers.
Fourth module: Understanding some of the skills of personal effectiveness and dealing with conflict
The session was designed to equip leads with specific skills to influence and make changes. Working in teams and committees requires specific skills of listening and supporting others. The session was designed to give attendees real opportunities to practise some of these skills.
Fifth module: Pr
