Quality in Primary Care Open Access

Introduction

The modernisation reforms of the National Health Service (NHS) are both extensive and far-reaching. A quality improvement agenda is to drive the development of the NHS using the framework of clinical governance. A major outcome of clinical governance is the ‘changing [of] organisational culture in a systematic and demonstrable way, moving away from a culture of ‘blame’ to one of learning’.[1] Modernisation and clinical governance documents make repeated reference to the need for involving staff in developing the organisation, embracing constructive criticism and new ideas, breaking down barriers between professional groups, learning from error, personal development of staff and lifelong learning.[1–3] These cultural changes envisaged as central to the success of the reforms are embodied in the management approach known as ‘the learning organisation’, defined by Pedler et al. as ‘an organisation which facilitates the learning of all its members and thus continually transforms itself .[4]

In fact, the Framework for Lifelong Learning for the NHS states explicitly that ‘all NHS organisations need to be learning organisations’ and that ‘there is now unparalleled support for all NHS organisations to develop and sustain a learning and knowledge sharing culture’.[3]

The smallest organisational unit within NHS primary care is the general practice. Yet it is in primary care where nine out of ten NHS patients are seen and where the largest number of patients will experience the success or failure of modernisation.[5] Thus, if the modernisation reforms are to be successful, then it is general practices that have to become learning organisations.

While there has been much exhortation to develop learning organisations within the NHS, and much effort to produce the infrastructure to support a learning culture, there has been little to facilitate NHS organisations in understanding and developing their cultures.[3]

This paper reports the development of a learning organisation diagnostic tool for use in general practice and its application to practices within the North Tees Primary Care Trust (PCT).

Method

Development of the instrument

A 40-statement questionnaire using a five-point Likert scale was developed for completion by the practice-employed staff of general practices. The statements explored staff perceptions of the extent to which their employing practice displayed eight characteristics of learning organisations. There was a mixture of positive and negative statements with five statements pertaining to each of the eight character-istics. The eight characteristics were chosen following an examination of the management literature on the learning organisation, particularly drawing on the work of Morgan and Kinston.[6,7] The eight character-istics explored by the instrument are presented in Box 1.

Responses were indexed with values from 1 to 5. The more positive the response to a statement (in terms of the practice displaying that particular learning organisation feature), the higher the value it was accorded. This method of scoring permitted a cumulative score to be computed for each of the eight learning organisation characteristics being measured. Each characteristic had five related statements in the questionnaire; thus the score for each characteristic had a range of 5 to 25.

The reliability, construct validity and internal consistency of the questionnaire were checked by a test–retest method involving 23 staff members from five practices outside the North Tees PCT area (see Appendix 1).

Applying the instrument

The subjects of the study were general medical practices which are members of the North Tees PCT. The North Tees PCT represents 24 practices serving a combined population size of approximately 180 000. In the study only medical practices employ-ing at least five staff members (administration and nursing staff) were included. The reason for this arbitrary cut-off point was the confidential nature of the inquiry and the risk of anonymity being compromised where only a small number of staff is asked to participate. By involving larger practices, it was hoped that staff would feel more willing to express their perceptions honestly. Of the 24 general practices in the North Tees PCT, 16 employed five or more staff and were invited to participate.

All the employed staff (administrative and nur-sing) in each participating practice were asked to complete a questionnaire each. They were assured of anonymity and confidentiality and their responses were sealed in an envelope prior to being returned to the investigator via their practice manager. Practice managers and doctors were excluded from the study. While the views and experience of these two groups are important, the study sought to ‘give voice’ to the experience of practice members who are unlikely to have major influence in determining the practice culture. Responses to the questionnaire were entered onto a database (EpiInfo). During data entry, if a respondent did not respond to a question or their response was not clear, the rest of their responses for that particular characteristic were excluded from analysis as well. Analysis was carried out using the computer program to produce, for each participating practice, a composite score for each of the eight characteristics being measured. Comparisons were made between practices and the mean scores for all the practices were calculated. Cumulative responses to each of the 40 statements were also analysed. 

Fifteen of the 16 eligible practices agreed to participate in the study. Of the 290 questionnaires distributed, 248 were returned (an overall response rate of 85.5%). In one practice the staff response rate was 51.7%, in one it was 70% and in the remaining 13 practices the staff response rates were all greater than 80%. practices the staff response rates were all greater than 80%. 

Practice size

Practice size, as judged by number of staff, varied from five to 37 with a mean size of 19.3 staff.

Learning organisation characteristics

The mean practice scores and ranges for each of the eight characteristics are presented in Table 1. The distribution of responses to the 40 statements of the questionnaire is shown in Tables 2 and 3. Figures 1 to 8 illustrate the results graphically.

Learning

Two practices were in definite agreement (a score of 20 or greater) that this characteristic was being expressed. The responses to the individual related statements identified ‘double loop’ learning and ‘knowledge mobility’ as the least developed areas in practices.

People in the practice

Two practices expressed definite agreement and four definite disagreement (a score of less than 15) that this characteristic was being expressed. Recognition for a job well done scored lowest among the individual statements.

Creativity

Two practices expressed de? nite agreement that this characteristic was being expressed. In only one practice was there a less than neutral perception. Thirteen out of 240 staff claimed to be afraid of admitting their mistakes. On the other hand, 76 out of 240 staff felt willing to take risks in their job rather than toe the line.

Values and beliefs

One practice expressed a definite positive perception with regard to this particular characteristic. Responses to individual statements were consistently near a neutral perception.

Change

Two practices expressed definite positive and two definite negative perceptions regarding this charac-teristic. Out of 240 respondents, 105 disagreed that their practice changed only when forced to by circumstances.

Feedback

Four practices expressed a definite negative percep-tion with regard to feedback. Most respondents (202 out of 244) perceived the practice to be bothered about what others thought. Out of 244 respondents, 129 felt able to express an honest view of the practice.

Connectedness

One practice expressed a definite positive perception, but no practices had negative perceptions with regard to this characteristic. Out of 238 respondents, 179 felt that their practice was aware of its dependence on external in• uences and agencies.

Teamwork

Two practices expressed definite positive and three definite negative perceptions with regard to team-work. Out of 242 respondents, 193 described their practice as ‘friendly’. However, perceptions of con• ict avoidance were almost equally split (91 out of 242 agreeing and 96 out of 242 disagreeing with the relevant statement) and 134 out of 242 respondents agreed that their practice worked as a team.

Learning in North Tees PCT practices

While only two practices agreed that this character-istic was being expressed, the mean practice score of 17.39 suggests that perceptions tended to be positive. This ought to encourage practices in their endea-vours. The responses to the individual statements suggest that there is greater scope for development in the areas of double loop learning and knowledge mobility.

Double loop learning refers to the practice of responding to error, not merely by recognising the error and adjusting performance to prevent recur-rence, but by using the opportunity of error to review or challenge the assumptions that set the rule in the first place. The results suggest that overall, staff barely perceive evidence of double loop learning in their practices. The busy-ness of general practice may militate against formal opportunities for double loop learning, yet this feature is regarded as key for learning organisations and its practice is not so much an add-on, but an approach which underpins the way people think in a learning organisation.[5]

Knowledge mobility refers to the development of skills and knowledge across the practice which allows not only a greater understanding (and therefore support) of one another’s roles, but which distributes knowledge and skills within the organisation thereby allowing staff to adopt different roles at times of turbulence and change. Morgan refers to this as pluripotentiality within the organisation.[5] A basic prerequisite for knowledge mobility is the opportun-ity to find out about the roles of other staff members and the results suggest that the staff questioned have limited opportunity for this. If knowledge mobility is to develop, practices will need to find creative ways of enhancing mutual understanding and support of roles.

Discussion

This questionnaire-based survey sought to measure the extent to which primary care general medical practices in the North Tees PCT expressed the characteristics of learning organisations. The excel-lent response rates by practices (93.8% of eligible practices) and their staff (85.5% of staff in participat-ing practices) suggest that there is a strong desire among practices to understand themselves as organ-isations and for their staff to feed back to their managers their perceptions of their own experiences. In the present climate of change and quality improvement in the NHS, this openness is to be welcomed and should prompt PCTs to encourage and support primary care practices in examining and developing their organisational culture.

People in North Tees PCT practices

Overall, staff did not feel valued and supported by their employing practices. In particular, staff per-ceived that they lacked recognition and appreciation for the work they did well (only 97 out of 242 expressed a positive view of this). The responses to all the statements related to this characteristic make it plain that practices who wish to be learning organ-isations have some way to go as far as the humanistic elements of that ideal are concerned.

Creativity in North Tees PCT practices

This characteristic was one of the better expressed among the North Tees practices. However, when examining the distribution of the responses to the five statements regarding creativity, there is an interesting paradox. On the one hand staff felt unafraid to admit mistakes, suggesting an open, blame-free culture. Yet, on the other hand, they expressed an inclination ‘to toe the line and not take risks’ (only 97 disagreed with the statement). This possibly indicates that staff are prepared to own up to error, even though they perceive their job as allowing little room for innovation.

Values and beliefs in North Tees PCT practices

The responses to the statements regarding values and beliefs appear fairly consistent. The overall scoring was low when it is considered that 14 of the 15 participating practices claimed to have a mission statement or similar written statement of their goals. It must be questioned who was involved in the drawing up of these statements and how practices use these statements to guide their operation.

Change in North Tees PCT practices

Fairly uniform views were expressed about this characteristic of the learning organisation. The lowest scoring statement, concerning changing only when forced to do so by circumstances is interesting in the present climate of reform in which most of the change occurring in primary care practices is externally driven and the response to this statement may be an expression of this.

Feedback in North Tees PCT practices

This characteristic scored least well across the PCT. The responses to the individual statements suggest that staff perceive their practices as being better at seeking and valuing feedback from outside the practice than from inside it. This relative devaluing of staff opinion accords with the perceptions expressed under the ‘people in the practice’ char-acteristic.

Connectedness in North Tees PCT practices

Although this characteristic scored highest among the participating practices, as with feedback, staff perceived a greater sense of being linked into things outside the practice than being linked into an interdependent network within the organisation.

Teamwork in North Tees PCT practices

The findings bear out the truth that a group of people working together, however friendly, does not auto-matically constitute a team. The responses to the individual statements confirm this, where, although the working atmosphere in practices was described as ‘friendly’ by 79.3% of respondents, only 55.4% agreed that they worked as a team in their practice. Brushing over rather than resolving conflict is one of the ways in which teams malfunction. The fact that only 39.7% disagreed with the statement: ‘if there is bad feeling between people, we generally avoid the issue’, suggests that this is an area in which primary care teams struggle.

These results suggest that, overall, there is scope for significant development of the culture within prac-tices in the PCT. This applies to all eight character-istics that were measured, but it has been possible to identify particular areas of underdevelopment and these are listed in Box 2.

The use of a culture diagnostic tool such as that described can assist practices in their development, both in setting objectives and in identifying appro-priate resources to support their development. Used across a PCT, the tool can help identify priority areas for organisational development support. These find-ings have prompted the development of a practice toolkit to assist practices who wish to act on the findings of their ‘culture diagnosis’.

Acknowledgements

This study formed part of a dissertation for the degree of Master of Medical Education from the Centre for Medical Education, Dundee, Scotland. The author received funding from the Postgraduate Institute for Medicine and Dentistry, University of Newcastle upon Tyne and the Research Practices scheme of the NHS Executive.

References

1. Department of Health (1999) Clinical Governance: quality in the new NHS. NHS Executive: London.
2. Department of Health (2000) An Organisation with a Memory. HMSO: London.
3. Department of Health (2001) ‘Working Together, Learning Together’: a framework for lifelong learning for the NHS. Department of Health Publications: London.
4. Pedler M, Burgoyne J and Boydell T (1991) The Learning Company: a strategy for sustainable development. McGraw-Hill: London.
5. Department of Health (2001) Primary Care, General Practice and the NHS Plan. Department of Health: London.
6. Morgan G (1998) Images of Organisation. Berrett- Koehler Publishers Inc: San Francisco.
7. Kinston W (1994) Strengthening the Management Culture: phasing the transformation of organisations. The Sigma Centre: London.
8. Altman DG (1990) Practical Statistics for Medical Research. Chapman and Hall/CRC Statistics and Mathematics: London.
9. Landis JR and Koch GG (1977) The measurement of observer agreement for categorical data. Biometrics 33: 159–74.
10. Mintzberg H (1979) The Structuring of Organizations. Prentice Hall: Englewood Cli¡s, NJ.

Appendix 1

Reliability testing

The stability of the questionnaire was examined using a test–retest method.

Five practices outside the North Tees area (the area to be tested with the final questionnaire) were contacted and asked to nominate a sample of their employed staff to participate in the pilot study. Contact was made through the practice manager of each practice and the method of staff selection was left to them. In total, 29 practice members were recruited. Each was sent a copy of the questionnaire to complete and return. Between 10 and 12 weeks later, each participant completed the same ques-tionnaire a second time.

Individuals’ responses to each question were categorised as agreement (includes ‘strongly agree’ and ‘agree’ responses), neutral (‘neither agree nor disagree’) and disagreement (includes ‘disagree’ and ‘strongly disagree’ responses). When comparing paired responses from each individual, the extent to which responses remained in the same category was measured using the measurement of agreement known as kappa (K).[8] A value of K = 1 will occur when there is perfect agreement of scores. A value of K suggests that the agreement between pairs of responses is no better than chance. Intermediate values of K  were interpreted according to Landis and Koch as shown in Table 4.[9]

Of the 29 participants, 23 completed a second questionnaire within the time scale and these were analysed by the method above. The results of the test– retest reliability after 10 to 12 weeks are displayed in Table 5.

The reliability of the questionnaire may be considered acceptable for the following reasons:

 the period between the questionnaires (10 to 12 weeks) was long, given that test–retest studies were usually carried out within intervals of four weeks or less

in the analysis of the final questionnaire, indi-vidual scores were added in groups of five (thus diluting the effect of less reliable questions)

the three least reliable questions fell into different groups of questions, thus distributing them among more reliable questions.

Internal consistency

Although there were five statements testing each of the eight learning practice characteristics, the five statements were not necessarily testing the same aspect of the characteristic. For example, the statements: ‘in this practice, staff from different disciplines meet up regularly to discuss their work in the practice’ and ‘people show little concern for one another in this practice’, both test for evidence of teamwork in a practice, but are not measuring the same concept of teamwork. In order to test for internal consistency, one pair of statements that explored the same concept was selected from each of the eight characteristic groups. The responses within each pairing were compared and the extent of their agreement was measured using K , as had been used for the test–retest reliability of the questionnaire. The pairs of statements selected from each of the eight characters along with the k  value for each pairing are shown in Table 6. For this estimate of internal consistence the first questionnaire by staff in the pilot practices was used (n = 23). The results suggest a moderate to good degree of consistency within each characteristic group.

Validity testing

The way in which the items were derived was described earlier in this section. The statements were produced to link with the specific learning practice characteristics derived from the work of Morgan, Mintzberg and Kinston.[6,7,10] The face validity of the questionnaire is related to the relev-ance of each statement to the descriptors derived for each learning practice characteristic.

Construct validity

Construct validity was tested during the pilot study by comparing the scores from two practices known to be very different. One had a reputation for teamwork and high-quality achievement and the other was going through a very di¤cult period of low morale and organisational di¤culties. The scores comparing  these two practices are given in Table 7. As the results show, the ‘struggling’ practice scores are appreciably lower (by 41.2% on average) than those of the highachieving practice. This suggests that the question-naire is capable of detecting significant differences between practices.