Discussion Paper - (2012) Volume 20, Issue 3
Head of Interprofessional Education
Chris Seenan BSc (Physiotherapy)
Lecturer in Physiotherapy
Gordon Morlan BA (Hons) PCEd (Learning Technology)
Learning Technologist
Wendy Smith MSc Bsc (Podiatry)
Lecturer in Podiatry
School of Health and Life Sciences, Glasgow Caledonian University, Glasgow, UK
Received date: 4 March 2012; Accepted date: 4 April 2012
Inter-professional education (IPE) has ‘learning to work effectively in inter-professional teams’ as one of its central learning outcomes. Whilst much is made in IPE of allocating health and social care students into inter-professional teams and setting them a task to complete, it has proved difficult to find a fair and equitable method of assessing how effective each individual has been in contributing to the task. This difficulty is compounded when the module is delivered predominantly online. This paper describes the recent push to establish meaningful educational outcomes for those involved in delivering IPE to pre-registration health and social care students. It then describes the use of a web-based peer assessment tool (Web PA) developed at Lough borough and Hull Universities (UK) which has been adapted by Glasgow Caledonian University (UK) for assessing the outcome of contributing effectively to IPE related online group tasks. The paper outlines how the process of web-based peer assessment operates in theory and how it has been received in practice. An illustration is given that shows how the process successfully discriminates between those that are working effectively in inter-professional teams and those that are not. The value of the process is discussed.
Quality in Primary Care
Volume:20 Issue:3
assessing teamwork, interprofessional education, interprofessional learning outcomes, online peer assessment
Hugh Barr assessing the development of interpro-fessional education (IPE) in health and social care courses in the UK[1] in the opening decade of the 21st century cites ‘looseness of definition of IPE as a concept’ as being a major cause of difficulties that faced educators involved in delivering and assessing IPE. He writes that ‘as IPE gained momentum from 2000 onwards there was much confusion of termin-ology’. Confusion, he suggests, existed both in what was called interprofessional learning and what was delivered. This confusion was linked to the assump-tion associated with what the actual measurable out-comes of IPE were. Barr suggests it was simply assumed that IPE ‘filtered into professional education would somehow help to equip professionals for multi-dimensional collaboration’.[2]
Despite the spread of interest in teaching health and social care students interprofessionally, there remained a lack of agreement on what constituted appropriate content and what the definitive ‘outcomes’ and ‘com-petencies’ of IPE actually were. Early IPE outcomes were reported as improving teamworking and increas-ing understanding of, or attitudes towards, other professions, but these were very vague and not pre-cisely defined.[1,3]
Sheffield Hallam University and the University of Sheffield developed their definition of IPE around the term ‘capability’. Working as the Combined Univer-sities Interprofessional Learning Unit (CUILU),[4] the IPE team produced a set of capabilities that were precisely defined and represent the most comprehen-sive and widely used UK statement of learning out-comes relating to pre-registration IPE.[1]
In the Sheffield Capability framework there are three specified outcomes relating to working effect-ively in teams which are reproduced below.
The practising professional should be able to:
• lead and participate in the interprofessional team consistently communicate sensitively in a respon-sive and responsible manner, demonstrating effec-tive interpersonal skills in the context of patient/ client focused care share uniprofessional knowledge with the team in ways that contribute to and enhance service pro-vision.
The framing of capability in these terms provides the interprofessional educator with something concrete to work with. Teaching scenarios can be developed where these capabilities can be discussed and con-sidered by students and then their professional com-petence on the specific learning outcome can be assessed. The work in Sheffield is finding favourable comparison with others, especially the national move-ment towards agreeing IPE outcomes in Canada (Canadian Interprofessional Health Collaborative, 2010)[5] and as we shall see there is considerable agreement within the two approaches.
The Canadian Interprofessional Health Collab-orative (CIHC), with Funding from Health Canada, established a working group in 2008 whose mandate was to review the literature related to competencies, review existing competency frameworks for IPE and develop a Canada-wide competency framework for interprofessional collaboration. The review of the literature regarding IPE competencies identified seven core competencies from various papers.
These were:
• problem solving
• decision making
• respect
• communication
• shared knowledge and skills
• patient-centred practice
• working collaboratively as a team.
The CIHC review also identified a lack of clarity in defining what constitutes competencies and how they become absorbed by a learner. The Canadian team acknowledge Tilman’s[6] work on defining competence and also Roegiers[7] who suggests that competencies are best developed by students integrating the knowledge, skills, attitudes, values and judgements within learn-ing or practice contexts and applying these to each situation. The Canadian starting point, therefore, for developing a competence framework included Roegiers’s overarching goal of a set of competencies that ‘enable the learner to master those situations he or she will have to deal with in his or her professional life’.
With outcomes comes assessment
The School of Health and Life Sciences at Glasgow Caledonian University (GCU) is one of the largest providers of IPE in the UK and has helped lead the way in the development of IPE practice. It has led the development and introduction of interprofessional modules directed at encouraging innovative learning and assessment addressing the requirements of all the professions involved.
In 2002, responding to an identified service need, government policy drivers and a growing inter-national lobby, plans for delivering IPE began. In September 2004, the first year one module in IPE was delivered to 300 interprofessional students from seven distinct health and social care professional student groups. In 2009, the approach was moved one stage further with a planned IPE undergraduate pre-regis-tration framework of modules spanning all four levels of the health and social care students’ four-year degree programmes. The pre-registration health and social care students now take six IPE modules as they pro-gress towards graduation. One of the underpinning learning objectives in all the modules is the require-ment to contribute effectively to interprofessional team activities. In the first-year modules, students are invited as a group to assess the contribution of their team towards the submission of one tutor-assessed presentation. The views of the students con-tribute to 10% of the presentation mark.
This year, after it had been agreed to introduce a learning objective of ‘contributing effectively to online group discussion’, the staff tutors on a second-level, largely online IPE module have increased the peer assessment to apply to five module-related online group tasks. The system that has been adopted is based on a web-based peer assessment system termed ‘Web PA’ developed at Loughborough and Hull Univer-sities. Web PA is an online automated system that allows students to rate the performance of themselves and their group in contributing to the completion of the group task. ‘A weighting factor is generated for each individual group member which is derived from each student’s input against defined criteria. Based on the total mark given to the group task, assessed and allocated by the tutor in the usual way, the weighting factor is then used to moderate marks providing an individual mark for each student.’[8]
Loughborough and Hull Universities made the online assessment tool available as an open source application in 2007. The tool was awarded an IMS Learning Impact Award in 2008 and by 2009 there were 17 higher education institutes in UK reported to be using it, including some uptake in Australian universities. (A full descripton of the development of the tool can be found in Loddington et al.)[9]
At GCU, in the second-level IPE Research Methods module, students are supported online with learning material and are asked to contribute to five research methods-related group tasks as part of continuous assessment which contributes 50% to their final mod-ule mark. The other 50% is provided through a multiple choice question test of their knowledge of research methods which the students sit in the last week. Students are allocated to online interprofes-sional groups of five students and over the 12 weeks that the course runs there are five group tasks for them to complete. Each group is allocated a group tutor who monitors their online discussions and marks their submitted group work. Each group task is released to the students, giving them two weeks to discuss and complete it. It is expected that each member will contribute to all the discussions and take the lead in coordinating and submitting the agreed group answers in one of the five tasks.
The online peer assessment process is provided to students on the completion and submission of each group task. Students complete four online grids that allow a 1–5 Likert scale judgement to be made on all members of the group including themselves across four criteria.
• GRID 1. Contribution to discussion.
• GRID 2. Quality of contribution.
• GRID 3. Contribution to final answer.
• GRID 4. Contribution to organising the effective-ness of the group.
How Web PA works
The submitted piece of work is allocated a mark for content by the tutor and that mark is then subjected to adjustment by the peer assessment adjustment factor, as suggested by the Web PA calculation. For a student
who does not contribute very much to the task, and the group all agree that the student has not contrib-uted very much and rate his contribution as poor, the peer assessment process will operate to produce a factor that adjusts that individual’s mark downwards. If the students all agree that an individual has not contributed at all across all four domains the peer adjustment process produces a zero factor score which results in an individual score of zero or close to zero depending on the unanimity of the judgements. Con-versely, for students who the group agree have worked more than others on the task their mark will be adjusted upwards from the group mark to reflect their effective contribution.
The peer assessment grids are open to students at submission date of the group task and for two days thereafter. The gradings allocated are logged and then once the system has closed they are analysed auto-matically and each student’s peer assessment weighting factor is applied to the group mark and the student’s adjusted individual score then appears in the online gradebook. A copy of the group submission with the group mark is returned with tutor feedback to each group member three days after submission. Students are then able to view and discuss their peer-adjusted mark with their tutor and learn from it with adequate time to consider how to contribute more effectively in the next task.
At the time of writing, a cohort of 280 students on this module split into 56 groups (five students per group) have completed and been assessed on three of the five group tasks. On all occasions, the majority of students (95%) successfully completed the peer as-sessment grids and the introduction of the system has generated a marked improvement in students’ contri-bution to online group discussion in comparison with previous years.
An illustration of the practical effect of peer assess-ment is provided by the analysis of the net change to the tutor awarded mark by the peer adjustment process for this cohort of 275 students on their third peer-assessed pieces of work (Table 1).
Peer assessment is benefitting the majority of these students with improved marks that reflect their effec-tive contribution to the group task. Those that have experienced being marked down by the group are encouraged to think how to contribute more effect-ively and students who are not engaging at all with the module and its group tasks are quickly identified and they can be contacted by their programme lead and progress on the module discussed.
In conclusion, the process that we have described here is an attempt to reward the efforts of those students who demonstrably work effectively in online group tasks and identify those who do not pull their weight and do not contribute effectively. The process is easily understood and appeals to students who wish to be assessed fairly for the efforts they put in to a group task and they like to see a system that legitimately penalises those students who do not contribute. The benefits of the process are that it is confidential; it involves students in the assessment process; the students receive feedback in a timeframe that helps them prepare for the next task; it definitely increases the interest in taking part in group tasks and students find it to be a fairer way of assessing their input.
We expect this approach to become the way for-ward in preparing students to work effectively in interprofessional health and social care teams.
Unfunded.
Commissioned, not externally Peer Reviewed.
None.
