What to observe: Use purposeful observation.
Purposeful watching begins with the relevant milestones and your learner goals in mind. Using these tools as a lens, you should prospectively choose the aspects of the learner’s clinical performance on which to focus the majority of your attention. You can further focus your coach’s lens based on by the unique opportunities afforded by a given setting or patient presentation. Using this intentional and systematic approach to observation ensures that that you do not miss important details. Withou having a systematic approach, coaches tend to refert to a more global assessment and learner ‘pattern recognition,’ relying primarily on their own familiar lenses (such as learner’s use of open-ended questioning, or use of a given physical exam technique), which may not be the learner's greatest area of need.
When to observe: Whenever possible!
You can enhance your coaching capacity by creating opportunities to observe learners performing the agreed upon skills. Strategies to create opportunities for observation include selecting patients that will be the most instructive and comfortable with the observation, focused observations during specific aspects of a clinical encounter that are most critical, or creating a simulated scenario designed to highlight specific skills. Observing a full encounter, or the learner performing a specific procedure, may require advanced planning. To balance patient and learner needs, you will need to continuously consider how observation will best fit into the routine flow of patient care for each learner. Take advantage of natural opportunities, but also be sure to deliberately choose or seek out opportunities when needed or when insufficient opportunities arise organically.
How to observe: Systematically and objectively.
The learner should take the time to prepare patients and families by explaining the coach’s role in the observation. Ask your learner if they feel comfortable doing this or if they would prefer you to model it for them the first time. Position yourself in the least intrusive place in the room (ideally outside of the patient’s line of vision), and should try to minimize interactions with the patient and family. Direct the patient's attention back to the learner as much as possible. When possible, use body language that encourages the learner to respond, but at times it may be necessary to be more directive: “Dr. Smith, what do you think?”
Record your observations using milestones or your own list to ensure that you are systematically focusing on everything you need to observe. Note discrete components of the performance, focusing on specific learner actions and making note of related outcomes (when the learner did “X” how did the patient respond?). Pay attention to your own personal biases by choosing your lenses intentionally and suspending judgment during observation to avoid premature closure.
How often to observe: The more the better.
As a general rule, the more direct observation the better. While a single observation may provide some objective data for formative feedback, more observations may be needed to ensure sufficient reliability. If you only have a single observation opportunity, you must be more intentionally comprehensive to pick up on more subtle learner behaviors. Multiple or longitudinal observations of a given learner will allows you to fine tune your observations, and to further explore the relationship of the observed details to the bigger picture of the learner’s overall performance. Many studies have shown that more than ten observations are needed to provide valid summative evaluation.
Prepare for assessment: Organize, edit and refine observations.
Following each observation, review and compare your notes to the milestones list and learner goals, filtering out less relevant observations. Compare discrete components of the observed performance against the learner’s current developmental milestone level. Which components of the skills were fully completed, only partially completed, or not completed at all? What components have you not yet observed? What questions remain? Reflect upon these initial impressions to ensure that important elements of the learner's skill performance have not been overlooked, and re-observe the learner as needed to fill in important gaps in your understanding.
Keeping your observations organized as you do more of them over time will save you time in the next step of Assessing Performance.