Research Paper - (2005) Volume 13, Issue 1
General Practitioner and Chairman, Fellowship by Assessment, Garswood Surgery, Wigan, UK
Alison Kay FRCGP
General Practitioner and past Chair, Fellowship by Assessment, Cumbria, UK
Received date: 08 June 2004 Accepted date: 28 July 2004
Fellowship by Assessment (FBA) is an award for the Royal College of General Practitioners’ (RCGP) members who can demonstrate high standards of patient care in their practices. Since 1989 nearly 300 general practitioners (GPs) have been awarded Fellowship, the College’s highest award, by this route. Patient care improves through FBA, in part through a culture of continuing education. FBA is a considerable achievement for GPs but requires great energy. FBA has developed considerably over the past 15 years and is now aligned with Good Medical Practice for General Practice. It has influenced, and in turn has been influenced by: General Medical Council statements on good medical practice, other RCGP awards and programmes, and the new General Medical Services contract.
audit, general practitioner, peer assessment, self-assessment
It is practices with effective leadership, responding to local needs and doing the work themselves, willing and able to devote the necessary time and resources, that manage to achieve desired change.[1]
Fellowship by Assessment (FBA) was developed in the late 1980s following the What Sort of Doctor initiative which proposed 35 criteria for assessing the quality of care in general practice.[2] At that time itwas considered that ‘until recently almost nothing was known about the quality of care in general practice’.[3] It was the first College-run, on-site, practice-based assessment leading to a major professional award in the world.[4]
The first FBAs were awarded in 1989 based on the achievement of over 60 criteria. Those same criteria can still be traced today, albeitwith substantially changed wording and generallymore rigorous evidence required. The main areas of change have been the introduction of complete audit cycles rather than surveys, the incorporation of the views of patients, health and safety requirements, and refinement of the consultation assessment.
Two-hundred and ninety-five general practitioners (GPs), both full- and part-time, have now been awarded Fellowship by Assessment. They have come from a wide range of practices, including singlehanded, rural, inner-city, armed forces and private practice. Annual study days were introduced at the outset in 1989, initially as joint sessions for assessors and candidates, but latterly shared with the Quality Practice Award (QPA) and Fellowship of the Institute of Health Care Managers (IHM).
An initial review after the first six assessments in 1990 concluded that the applicants had ‘found the scheme fair though challenging. Many assessors and applicants were enthusiastic after the visit, finding it rigorous, hard work, educational and stimulating’. These themes have recurred consistently through the years as candidates are asked their opinions after their visits.
Peer assessments of practising doctors are not common, and FBA may be the most broadly based of any that are available anywhere.[5]
FBA is described in Box 1.
Administration was devolved to the Vale of Trent Faculty from 1993 to 2003 but has returned to the College in London. Feedback from candidates regarding administration has shown a high level of satisfaction.
Income from candidates, study days and the sale of materials does not cover the cost of running the award and performing assessments, despite each candidate being currently charged £1000. The FBA budget is subsidised by the RCGP as a service tomembers and in recognition of both the educational stimulus and contribution to improved patient care.
Over 100 RCGP Fellows, usually FBAs themselves, act as advisors and assessors. They give considerable time and expertise to prospective candidates and receive basic reimbursement of their expenses. They normally attend a retraining day every 2–3 years.
An assessment of competence or performance should be valid (it measures important aspects of care); reliable (produces the same results on different occasions with different assessors); and feasible (the costs to the different parties must be proportionate to the anticipated benefit).[9]
Development of the criteria involves literature reviews and widespread consultation within the profession. They are endorsed each year by the College Council. Having over 60 criteria is an assurance that a wide variety of aspects of patient care is being assessed.
Members of the RCGP Patients’ Liaison Group are involved in the consultation process, and over 300 people are asked for their opinion of what constitutes high-quality general practice each year. Successful candidates have been surveyed each year since 1995, and over 90% have agreed that FBA covers the right areas, and that the standards are set at the right level.
The criteria have been linked to other, independently reviewed RCGP awards, the General Medical Council’s Good Medical Practice, and those skills identified by the RCGP’s working party on ‘The nature of general medical practice’.[10,11]
Endorsement of FBA in the Chief Medical Officer’s review of continuing professional development in general practice provided external validity for the award.[12] The current Chief Medical Officer concluded that health care in this century requires professionals who can lead, manage and work effectively in a team, and who can practise safe, high-quality care while also creating opportunities for improvement.[13] These skills are at the core of FBA.
FBA is assessed on essential, open criteria that are supported by guidance regarding the evidence required. All are freely available on the RCGP website. Candidates submit evidence in writing, which is assessed independently by three assessors, one of whom is the candidate’s advisor or mentor, one a lay assessor or local GP, and the third an external assessor. The FBA Chairman and National Working Group resolve any conflicts of opinion, which are rare.
Each assessment team is unique. While perhaps keeping assessors fresh, this adds to the challenge of conducting a detailed assessment of a GP almost from a standing start. Inter-rater reliability is not formally assessed, but depends to a large degree upon the experience of at least one of the assessors having carried out at least five, and usually ten or more assessments. Satisfactory participants in an assessor’s study is essential for new assessors and strongly encouraged for all. We are now piloting self-assessment within each visiting team.
From the annual candidate’s survey, 98% agreed the assessors had been professional and thorough, and 91% said they had correctly identified both the candidate’s strengths and weaknesses.
The candidate’s advisor also acts as an assessor. The possibility of introducing bias is considered to be outweighed by having someone with detailed local knowledge who also acts as a guarantor of the process. This mentorship role is central to the scheme (see Figure 1).
Figure 1: From your perspective as a candidate, how important was the one-to-one mentoring system to you?.
It is noteworthy that FBA practices have often achieved other external markers of quality such as: Beacon Status, ISO 9000, Investors in People, Charter Mark. Over a quarter of FBA practices have achieved one of these awards, and the first four practices to be awarded ISO 9000 all had Fellows by Assessment.
The continuing support of candidates and assessors is evidence that FBA is feasible and acceptable. Yet most RCGP members, although usually potential FBA candidates, have not chosen to undertake the award, which still requires unusual dedication and commitment.
Assessors and advisors subsidise the system through giving considerable amounts of their own time reading written material and attending meetings. These are only partially reimbursed. Although this time and expertise is greatly appreciated, FBA faces competition from other awards, and its future requires a secure source of funding.
The direct financial cost to the candidates is higher than for Fellows by Nomination (£1000 compared to £708), but low in comparison to other educational and quality awards. Some external support is available (see ‘Fellowship by Assessment’ on the RCGP website 14), but seems to be used by a minority of candidates. Although we are not aware that any prospective Fellow has been put off because of financial constraints we realise that financial costs could be a deterrent.
Shaw described 11 features of effective external assessment programmes (see Box 2).[15] FBA appears to do well against most of these, but in common with other assessment programmes could have more effectively co-ordinated its activity, standards and methods with other awards. Within the RCGP programmes there is now a major attempt to harmonise criteria, and FBA and Membership by Assessment of Performance (MAP) are likely to have their criteria much more closely aligned from April 2005.
Feedback from the annual candidate’s questionnaire stresses the educational importance of FBA, both for the individual GP candidate and the wider practice team. Most FBAs have had significant subsequent academic or career developments (see Figure 2), many in the form of further degrees or research. FBA influenced many of these, in a third ‘very strongly or directly’.
Figure 2: To what extent do you think FBA played a role in influencing your career changes or developments?
Fellowship by Assessment improves the care of patients and empowers the doctors by improving their confidence.[17]
Mentors and local FBA groups often enhance the educational experience for candidates through such things as mutual accountability, motivation, maintaining impetus, dealing with practical problems, and overcoming isolation.[17]
Two universities, Exeter and Northampton, have allowed credits to FBAs for their MSc degrees. This usually relates to the experiential learning components of the courses. Educational development is not confined to candidates, who often state that their whole team benefited from the process.
FBA is available to doctors who may not be known to RCGP faculties and might thus be unlikely to be nominated for fellowship. Prior to undertaking FBA, 41% of Fellows were involved in College activities, mostly at the faculty level; afterwards the figure rose to 65%. Some have become the Chairs of UK and Scottish Council, and many have become actively involved in supporting FBA, QPA and MAP.
FBA has been a stimulus to many of these activities. It is also common to find FBAs actively supporting and leading efforts to improve quality of care through clinical governance in their districts.[18]
The award has also contributed to the development of MAP,[19] QPA, awards for general dental practitioners and the fellowship for managers in both general practice and elsewhere in the health service (through the Institute of Healthcare Management).
At the present time, practices and primary care organisations are adapting to the huge implications of the new General Medical Services Contract. This change, like many others, needs effective leadership, and a willingness and ability to devote the necessary time and resources to it.[1]FBAs and their practices should, for the first time, reap financial rewards for the efforts they have made to improve patient care.
During the year 1999–2000, a trial of lay assessors was undertaken.[20] There was widespread support for the abilities of the lay people and the broader perspective that they brought to the visits. FBA strongly supports the philosophy of including lay assessors at every visit, although practical issues at present do not always allow this. Further assessor development will be required, and this will probably continue to be run alongside that for the QPA, with whom there is a considerable overlap of methods and personnel.
The College Council is awaiting a report from a working group on the future of Fellowship (January 2005). After 15 years of FBA there is considerable experience of assessing potential new Fellows. However the College decides to do this in future, we hope FBA is not allowed to wither away unless something better can be put in its place.
The College Strategic Plan for 2004–2007 states ‘In the past the College has concentrated on setting standards in general practice. In future we intend to focus on enabling general practitioners to implement those high standards’.[21] FBA is about achieving and demonstrating high standards of patient care.Webelieve most College members should be encouraged and enabled to undertake the award.[22] If this were achieved it would profoundly improve British general practice.
None.
