Continuing Competence: Case Study of Self​/​Peer Assessment and Clinical Audit to Improve Quality

Everyone has differing concepts of how continuing competence and quality service should be assessed and improved.

In the Netherlands, the government was responsible for managing a register focusing on the certification and relevant expertise of physiotherapists. The professional association managed a quality register, established minimum standards of performance based on professional development activities, and supported the development and translation of practice guidelines. The health insurers conducted a more comprehensive performance assessment system and increasingly became the driver of quality practice changes based on:

• Assessments of process and outcome data
• Client satisfaction
• Clinic audits

The feedback provided to physiotherapists created changes in practice, but also had negative consequences. The Royal Dutch Society for Physical therapy (KNFG) wanted to take a leadership role in implementing self-regulated quality-improvement initiatives of physiotherapists. KNFG believed that a grassroots program had the potential to be more effective and sustainable than top-down approaches initiated by third parties. Furthermore, they were interested in the phenomena of peer-assessment and feedback from trustworthy sources and their effects on practice improvement.

A comprehensive quality improvement plan was developed (MKIB) that included both a self and peer audit as well as a clinical audit. The KNFG also engaged researchers to conduct staged studies to examine the impact and feasibility of proposed program components.

The pilot program

64 physiotherapists from the Royal Dutch Society for Physical Therapy and four physiotherapy clinics piloted a quality-improvement program intended for nationwide use.

The focus of the initiative was on quality practice domains of client-centeredness, effectiveness, and transparency of physiotherapy services though an examination of:

1. Record-keeping
2. Client communication
3. Organization and management practices

In cycle 1, Physiotherapist and their peers group reviewed the clinical record and video recordings using standardized forms (supplement 1 page 5-6) and entered their assessment and feedback comments into an online repository specifically developed to support the KNGF program. Self-assessments were conducted first and followed by peer assessments. Results were collated and presented back to the small groups for discussions facilitated by physiotherapist coaches. After the discussion, each physiotherapist developed a personal learning plan to address relevant issues identified through the process. When the cycle (cycle 2) was repeated 4-6 weeks later, the discussion also included the physiotherapists reflections on improvements made.

Clinic audits occurred following the completion of the self and peer assessment cycles. The audits focused on the organization’s structure and processes in the following domains:  

• Quality management
• Client management
• Communication and collaboration
• Physical setting
• Privacy and safety
• Innovation and entrepreneurship (supplement 1 page 7-8)

Outcomes of the pilot program

1. A change in a mindset that quality improvement is helpful rather than an administrative burden

Initially, physiotherapists were skeptical of the pilot program and its aims. They perceived the MKIB program as an additional administrative burden on top of the demands of health insurers. Their views changed because the program helped them focus on their core business and uncover what happens behind closed doors.

2. Using video to assess communication skills promoted learning

At first, physiotherapists were reluctant to participate in recording their patient interactions and most underestimated patients’ receptivity to this performance improvement strategy. Concerns over privacy dissipated when it became apparent patients were eager to participate in a recording. Initially, most physiotherapists did not want to be recorded and opted to have communication skills assessed using role play. However, during the discussion of cycle 1 assessments of submitted videos, physiotherapists realized the value of recording interactions, resulting in many more physiotherapists submitting their video-recorded interactions for cycle 2. Physiotherapists valued video recordings as they were real life demonstrations of what physiotherapists do rather than what they say they do. The videos allowed for modelling professional behaviors of skillful colleagues and unwanted behaviors became transparent, triggering suggestions for alternative behaviors.

Overall, physiotherapists were cautious about critically appraising peers, resulting in higher ratings for the videotape ratings. When compared to self-ratings, the higher ratings from peers were not significantly different. Quality of feedback improved during the face-to-face group discussions because it provided participants with an opportunity to discuss differences between peer and self-ratings and clarify quality standards of performance. The video recordings enhanced physiotherapists awareness of quality standards and their own communications styles.

3. Charting practices improved between cycle 1 and 2

Assessment of record-keeping was valued because, unlike videotapes, patients’ clinical records detail the process of patient management, and allows the assessment of clinical reasoning and decision-making, such as the application of clinical practice guidelines and use of client-reported outcomes and performance outcome measures. Face-to-face discussion was critical to developing a shared understanding of quality indicators listed in the assessment tools and qualifying the peer ratings and narrative comments. Peer feedback was valued for guiding self-improvement.

Differences in clinical record assessment results between cycle 1 and cycle 2 showed physiotherapists significantly improved their charting practices in the intervening 6 weeks.

4. Clinic audits helped organizations understand standards and identify areas for improvement

Naturally, the 4 clinics were nervous about a colleague auditing their organization’s structures and processes. This mindset changed as a safe setting was created to discuss the clinic audit results and that discussion helped clinics better understand the quality standards and gave them ownership and direction on where to focus improvement efforts.

Unique features of the KNGF program

The KNGF program is unique because it:

• Focuses on peer assessment, feedback and facilitated group discussion
• Uses recordings to assess communication competencies
• Assess both individual and organization performance
• Covers the career continuum of physiotherapists

The pilot study demonstrated that the MKIB Quality in Motion program was feasible. Following the pilot, other studies on the impacts of the program were conducted. Since the conclusion of the research around 2015, the Quality in Motion pilot program has been implemented nationally and has expanded its purpose to include clinical reasoning, interprofessional collaboration, and the use of aggregated clinical and patient-reported outcome data for quality-improvement purposes.

Relevance to Alberta physiotherapists

Physiotherapists participating in a quality-improvement initiatives should consider whether their participation meets the criteria for the DO.LEARN.GROW Self-Selected Activity and if it does, whether they want to use their quality improvement endeavors for their 2021 Self-Selected Activity.

In addition to having a Continuing Competence Practice Standard, the College of Physiotherapists of Alberta also has a Quality Improvement Standard. If you are contemplating a quality improvement initiative in your facility, consider whether any of the strategies and tools used by the Royal Dutch Society of Physiotherapists might be of value to you. The DO.LEARN.GROW. list of Self-Selected Activities contains several tools that could be used for benchmarking practice (i.e., Competency Self-Assessment, Patient Care Reflection, Chart Stimulated Recall, Hazard Assessment and Management Audit). Moreover, the Competency Profile for Physiotherapists in Canada and/or the College of Physiotherapists of Alberta’s Standards of Practice contain plenty of performance descriptors that can be repurposed for any assessment tools you develop.

Comments and questions? Contact the Continuing Competence Program 1-800-291-2782 ext. 349.

Page updated: 20/04/2022