Trainees’ Guide to Workplace Based Assessment

Trainees’ Guide to Workplace Based Assessment Trainees’ Guide to Workplace Based Assessment Introduction The College curriculum for specialist training in Psychiatry is supported by an extensive assessment programme comprising both workplace based assessments (WPBA) and formal MRCPsych examinations. Full details of the MRCPsych examinations appear on the College website (www.rcpsych.ac.uk). The guide describes WPBA’s, outlines their purposes, tells you how many you need to get done and tells you how to plan and organise them. You should read it in conjunction with the guidance about the specific tools that is available on the HcAT / RCPsych website (www.hcat.nhs.uk/rcpsych). The College Curriculum is the basis of the Specialty Training Programme in Psychiatry. It has been approved by PMETB and comprises the Core and General module, 6 specialty modules, and 3 sub-specialty modules. It uses the same WPBA methods throughout. The entire curriculum is mapped to Good Medical Practice. The Core and General module indicates the phase of training through the use of colour: red = ST1; gold = ST2&3; violet + ST4&5; and green = ST6. The colour codes are used in all documents concerned with a specific phase of training, including the WPBA forms. Documents that are not specific to a particular phase of training (such as the Core and General module and some of the WPBA forms) are colour coded blue. You should use the curriculum as an electronic hyperlinked document. There is also a list of competencies for each training level. Within each list there are links back to the curriculum. This list of competencies is a key document. You must familiarise yourselves with this in order to manage your training with the help of your educational supervisor. Underlying Principles Workplace based assessments must comply with the following: , They must focus on performance (i.e. what you actually do in the workplace) , They must be evidence based , Evidence must be triangulated whenever possible (i.e. it must be provided by different assessors, on different occasions, and if possible using different methods) , Records must be permanent – assessment forms must not be lost or destroyed (this is achieved by online storage). Purpose The purpose of the workplace based assessments is threefold. Firstly, they have a formative function as the basis for feedback and educational planning. This includes helping identify any specially focused teaching and learning that might be necessary to fill gaps or resolve weaknesses. You should have WPBA’s completed regularly throughout your placement to demonstrate effective review and appraisal. Therefore, do not leave them until the end of each stage of training or clinical placement. Secondly, although individually they have no ‘pass’ or ‘fail’ score taken together WPBAs contribute to the Annual Review of Competence Progression (ARCP), which has a summative function providing evidence of the attainment of competencies. Thirdly, participating in a specific number of WPBAs is mandatory for eligibility for the MRCPsych examinations (see later). Assessments There are 8 workplace based assessments available. These are described in detail in the trainee and assessor guidance for each WPBA tool and are merely outlined below. o Assessment of Clinical Expertise (ACE) The assessor observes a whole new patient encounter in order to be able to assess your ability to take a full history and mental state examination and arrive at a diagnosis and management plan. o Mini-Assessed Clinical Encounter (mini-ACE) The assessor observes part of a patient interaction, for example history taking or negotiating a treatment plan, and rates your performance. o Case-based Discussion (CbD) 2 Trainees’ Guide to Workplace Based Assessment October 2007 ? Royal College of Psychiatrists 2007 You select two sets of notes of patients you have recently seen and the assessor picks one to discuss. The discussion will allow demonstration of clinical decision-making and the application of clinical knowledge. o Case Presentation (CP) This tool can be used when you give clinical presentations and involves assessment of domains such as presentation skills and interpretation of evidence. o Journal Club Presentation (JCP) This can be used when you present a journal article and covers domains such as analysis and critique and answering questions. o Directly Observed Procedural Skills (DOPS) This has more limited use in psychiatry compared to other areas of medicine but can used in situations such as administering ECT. o Mini-Peer Assessment Tool (mini-PAT) It allows co-workers to assess your attitudes and behaviours and ability to work well with colleagues. o Assessment of Teaching (AoT) This is a new tool that has been developed after feedback from the pilot programme. It allows an assessment to be made of teaching skills and may relate to a lecture, tutorial or small group teaching session that you lead. Requirements The requirements for the number of WPBAs to be completed in ST 1-3 are: WPBA Minimum number required per year ACE 2 in ST 1, 3 in ST 2 and 3 in ST 3 mini-ACE 4 CbD 2 3 Trainees’ Guide to Workplace Based Assessment October 2007 ? Royal College of Psychiatrists 2007 mini-PAT 2 WPBA Recommended number per year CP 1 JCP 1 DOPS As the opportunity arises AoT As the opportunity arises The guidelines in the above tables should be regarded as the minimum number of WPBA’s: many trainees will need to do more, for example to provide additional evidence of achievement in a particular area of professional practice. It is important to again note that there is no ‘pass’ or ‘fail’ score for WPBA. Therefore for exam eligibility purposes you are only required to demonstrate that you have undertaken the required WPBA, not that you have achieved a specific ‘score’. WPBA and Examination eligibility The requirements for each examination are summarised in the table below. These requirements may be subject to change and it is essential that you consult the College website for the most up to date information. Trainees who enter ST 1 must fulfil the requirements below to be eligible to sit the MRCPsych examinations. There are transitional arrangements for eligibility for the MRCPsych examinations which can be found in full on the College website. All trainees who enter run through training at ST 2 or ST 3 and have passed Part 1 may take Paper 3 and the OSCE under the previous eligibility criteria (duration of training completed not WPBA). All trainees who enter run through training at ST 2 or ST 3 who have not passed Part 1 may take Paper 1 and Paper 2 under the previous 4 Trainees’ Guide to Workplace Based Assessment October 2007 ? Royal College of Psychiatrists 2007 eligibility criteria but must complete the new criteria for the OSCE to take Paper 3 and the OSCE. Exam WPBA requirement Paper 1 4 mini-ACE at ST 1 level 2 CbD at ST 1 level Paper 2 4 mini-ACE at ST 2 / 3 level 2 CbD at ST 2 / 3 level Paper 3 Cumulative criteria of Paper One and Paper Two (8 mini-ACE and 4 CbD) OSCE 8 ACE across a range of specialties (2 in ST 1, 3 in ST 2, 3 in ST 3) For trainees at the level of ST4 and above tools are currently being developed and adapted to assess higher level performance and competencies such as leadership skills. The existing WPBA tools should be used to guide educational development and as evidence for your ARCP. You should complete two rounds of mini-PAT a year and in addition aim to undertake one WPBA a month. You must note that, since they form the basis for feedback and educational planning, you must arrange WPBA’s at appropriate intervals throughout your training rather than leaving them until the end of each stage. In particular, we strongly suggest that within the early stages of each phase of training or clinical placement (i.e. within the first month) you have a mini-ACE or ACE and a CbD. This is because these tools reflect closely the expected day-to-day work of a junior doctor in Psychiatry both in terms of the assessment of cases and clinical reasoning and judgement. The purpose of these early assessments is to identify what you are doing well and in which areas you need to develop. They contribute valuable information to the educational planning of your placement. Therefore it is most unlikely that assessments made early will produce ratings consistent with performance at the standard 5 Trainees’ Guide to Workplace Based Assessment October 2007 ? Royal College of Psychiatrists 2007 expected for the completion of the stage of training. Scale Ratings All the WPBA forms use a 6-point Likert-type rating scale. The standard for completion of each stage of training (ST1, 3, 5 and 6) corresponds to a rating of 4. Higher ratings indicate performance above the minimum standard for completion of that stage; lower ratings indicate performance below the required completion standard. Since when you are in the early part of any phase of training you will probably not be performing at the standard required for completion of that phase, (and will, therefore, often receive ratings below 4), most assessment forms have an additional item where the assessor can indicate your global performance relative to your stage of training. On the other hand, there might be some occasions when your performance exceeds that indicated for your current stage of training. In such circumstances you can choose to be assessed using criteria (and rating forms) for a higher stage of training and, if you achieve ratings of 4 or more, can record this in the appropriate portfolio. Thus, for example, an exceptional ST1 Trainee might choose to be assessed on their consultation skills at ST2/3 level and, if rated at 4 or better, would record this in their ST2/3 portfolio as well as in their ST1 portfolio. Electronic list of competencies (formerly, ‘Portfolio of Progress and Attainment’) Each phase of training (ST1; 2/3; 4/5; and 6) has a designated List of Competencies: Log of Progress and Attainment. You must keep the Curricula and associated documents in a single folder with their names unaltered and you must enable macros for the hyperlinks to work. The name that you give the folder is irrelevant. Although any of the documents can be printed, they are large (especially the core curriculum) and incomprehensible unless viewed in colour. You can use the boxes in the List of Competencies to describe the evidence you wish to use to demonstrate your achievement of that competence. The evidence may consist of one of the WPBA tools but may also include other evidence such as reference to supervision notes or to reflective notes. You can print the electronic list of competencies annually and filled it in by hand, but it is better to complete it electronically and print it out annually. Then you can add it to your paper portfolio prior to your ARCP. This means that for stages of training that span two years (e.g. ST2-3) you should 6 Trainees’ Guide to Workplace Based Assessment October 2007 ? Royal College of Psychiatrists 2007 have a half completed list of competencies printed out at the end of ST2 and a completed one at the end of ST3. Paper portfolio This replaces the previous College logbook. It allows you to collate other evidence in addition to your WPBA. It will be available to download free from the College website or a commercially printed copy can be purchased (further details on the College website). It has several sections including: , self-appraisal , professional development plan , reflective learning , record of publications, audit, research, teaching etc. Competence Attainment All trainees will have an Annual Review of Competence Progression (ARCP) organised by their deanery. This is described in the Gold Guide and replaces the RITA process. The key elements of this review are: , appraisal o Educational o NHS , assessment of performance o workplace based assessments o experience demonstrated e.g. from portfolio including audit, research and other aspects of non-clinical competence , planning. Achievement of competencies will be considered in the Educational Supervisor’s Report and the educational supervisor will consider WPBA and other evidence such as examinations passed and experiential outcomes recorded in the portfolio. The competencies will be considered under 11 domains: , providing a good standard of practice and care , decisions about access to care , treatment in emergencies , maintaining good medical practice , maintaining performance , teaching and training, appraising and assessing , relationships with patients 7 Trainees’ Guide to Workplace Based Assessment October 2007 ? Royal College of Psychiatrists 2007 , dealing with problems in professional practice , working with colleagues , maintaining probity , ensuring that health problems do not put patients at risk The Educational Supervisor’s Report and the College Tutor’s Report will be submitted for consideration by the ARCP. It is the ARCP that validates the achievement of competence after considering the presented portfolio information. It is important to note that as the ARCP considers a range of evidence, it is possible for you to have performed well in your WPBA’s but be considered by the ARCP panel to require further development. The outcomes for ARCP are described in the Guide to Postgraduate Specialty Training in the UK (Gold Guide), www.mmc.nhs.uk/download/Gold_Guide_140607.pdf. Carrying out workplace based assessments You are responsible for organising your assessments. You should discuss the areas in which you need to be assessed with your Educational Supervisor and plan your WPBA’s accordingly. Remember that you will need to be assessed by a range of different assessors in different situations to ensure triangulation of evidence. Don’t rely on your Educational Supervisor to perform all the assessments! Also remember that you must ask for the patient’s agreement when you perform a WPBA that involves a patient. There is an online system for the completion and storage of WPBA’s. You can register for this through the College website. After you register you will be able to access the forms relevant to the mandatory WPBA’s (i.e. ACE, mini-ACE, CbD and mini-PAT). Your assessors must register with the system and complete these assessments online. Your assessors can continue to use the paper forms of the other WPBA tools, which may be downloaded from the College website. You may already have completed a number of the mandatory WPBA’s on paper; you must now ask your assessors to enter these assessments online, so they may be stored in the system. You can keep other assessments that have been completed as paper forms and can store them in your portfolio. 8 Trainees’ Guide to Workplace Based Assessment October 2007 ? Royal College of Psychiatrists 2007