Direct observation of procedural skills

Introduction

The Direct observation of procedural skills (DOPS) is a RANZCP-approved Workplace-based Assessment (WBA) tool. The purpose of this tool is to promote learning for trainees by providing structured feedback on performance within an authentic workplace context.

What is a DOPS?

A DOPS is a concise, validated method of assessment consisting of a supervisor observing a trainee conducting a procedural skill (e.g. psychotherapy, ECT, supervision, physical examination) and providing feedback to the trainee about their performance. The most important part of the DOPS is the feedback given to the trainee.

A DOPS, like all WBAs, should contribute to the evidence base that informs a supervisor’s judgement as to whether a trainee can be entrusted with a particular Entrustable Professional Activity (EPA) and/or for the mid-rotation In-Training Assessment (ITA) Form and end-of-rotation ITA Report.

Forms and documents

You should complete the DOPS form via Intrain. If you do not have access to Intrain, use the PDF form.

Direct observation of procedural skills form [PDF; 238 KB]

Choosing an encounter

 The DOPS may be used during a range of activities and settings to assess different procedural skills. When conducting a DOPS with a patient, the patient must be informed of the DOPS assessment process.

Supervision

 The supervisor must be familiar with the DOPS assessment process. Supervisors must be clinically competent in the area of psychiatry being assessed.

Assessment criteria

The aim of the DOPS is to enable supervisors to provide structured feedback across the following areas:

• communication skills (including therapeutic relationship/approach)

• demonstrated knowledge of procedure

• procedural, technical or supervision skills (including provision of feedback)

• organisation, time management and documentation

• management of any issues arising (transference, risks, conflicts, adverse reactions, etc.)

• boundaries and professionalism

• another skill at the supervisor’s discretion (optional).

 Prior to conducting each DOPS, the trainee and supervisor should determine together which of the assessment criteria to focus on during the formative assessment. The feedback should focus on the agreed specific tasks rather than on the trainee’s general performance.

How to undertake a DOPS

 The trainee is responsible for planning when a DOPS will occur in consultation with the supervisor.

1. The trainee makes arrangements with a supervisor to carry out the DOPS and organises all administration required (including providing the supervisor with the DOPS form).

2. The trainee and supervisor determine the assessment criteria to be considered during the DOPS.

3. The supervisor observes the trainee conducting a procedural skill.

   • The supervisor should not participate unless intervention is required for the patient’s safety.

4. The feedback session occurs immediately after the observation.

   • The feedback should be constructive and address: areas that were especially good, suggestions for improvement, agreed actions and goals.

   • The total time required for the DOPS and feedback session will usually be 30–40 minutes.

5. The supervisor rates the trainee’s performance for the relevant assessment criteria using the 3-point scale on the DOPS form.

   • The cumulative weight of the feedback comments and ratings helps determine a defensible judgement of a trainee’s competence at their stage of training.

   • Please note that not all assessment criteria on the form are required to be rated during each DOPS. Not applicable criteria are rated with the N/A option.

Table 1  Standard guide for rating scale – see Developmental Descriptors document for more detail

Please note: standards are at the level expected on completion of the relevant stage of training. A DOPS conducted at the beginning of a stage may typically include ratings of below ‘Meets standard for the end of stage’. This approach highlights areas for improvement and allows trainees to view their progress along a continuum.

1. The trainee and supervisor discuss and agree upon the next steps to progress learning. Both the supervisor and trainee sign the DOPS form.

2. The trainee is required to maintain a portfolio of their Fellowship Program forms, including their WBA forms. Trainees are required to provide this portfolio to their next supervisor for review at the start of each new rotation.

Feedback

The feedback given after a DOPS is important. Constructive and useful feedback should be given to the trainee on:

• areas that were especially good

• areas that need improvement

• potential ideas to gain further experience and skill in the areas requiring development.

For more detail on WBA requirements, please refer to the Workplace-Based Assessment Policy and Procedure.