Introduction

Programmatic assessment is a fascinating approach to assessment that seems like it would address many of the challenges we experience in health professions education. I also think that it's very difficult (impossible?) to implement without a complete restructuring, not only of the assessment tasks in a programme, but of the entire programme itself. So many decisions are driven by the assessment task (and student expectations around assessment) that implementing programmatic assessment would require a massive cultural change as well. Which is why it's fascinating.

Podcast

Frank, J., Sherbino, Jonathan, & Snell, L. (n.d.). Can Assessment Get with the Program? (No. 211).

So, what is this “programmatic assessment” stuff? The promise of PA is that it is a supposed to be better way of organizing assessment. PA is meant to address all the many documented problems with how HPE assessment was done in the 20th century.

This episode discusses Bok, H. G. J., de Jong, L. H., O’Neill, T., Maxey, C., & Hecker, K. G. (2018). Validity evidence for programmatic assessment in competency-based education. Perspectives on Medical Education, 7(6), 362–372.

See more extensive show notes here.


Article

Van Der Vleuten, C. P. M., Schuwirth, L. W. T., Driessen, E. W., Govaerts, M. J. B., & Heeneman, S. (2015). Twelve tips for programmatic assessment. Medical Teacher, 37(7), 641–646.

Programmatic assessment is an integral approach to the design of an assessment program with the intent to optimise its learning function, its decision-making function and its curriculum quality-assurance function. Individual methods of assessment, purposefully chosen for their alignment with the curriculum outcomes and their information value for the learner, the teacher and the organisation, are seen as individual data points. The information value of these individual data points is maximised by giving feedback to the learner. There is a decoupling of assessment moment and decision moment. Intermediate and high-stakes decisions are based on multiple data points after a meaningful aggregation of information and supported by rigorous organisational procedures to ensure their dependability. Self-regulation of learning, through analysis of the assessment information and the attainment of the ensuing learning goals, is scaffolded by a mentoring system. Programmatic assessment-for-learning can be applied to any part of the training continuum, provided that the underlying learning conception is constructivist. This paper provides concrete recommendations for implementation of programmatic assessment.

There's a lot going on in this abstract, which is reasonable when you consider that the paper is trying to deliver on 12 related, but separate ideas. The benefit of this paper is not so much what it describes directly, but in the extensive reading list provided in the references.

Resource

Northeastern University. (n.d.). Programmatic assessment. Center for Advancing Teaching and Learning Through Research.

An overview of some of the main components of a programme of programmatic assessment. Including:
  • What is programmatic assessment?
  • Identifying programmatic learning outcomes.
  • Developing a curriculum map.
  • Identifying direct and indirect measures of learning.
  • Determining who is responsible for assessment.
  • Collecting and analysing data.
  • Sharing results and making decisions.
  • Finalising the assessment plan: Additional considerations.
  • Iterating on your assessment plan.
I think that this overview is written from the perspective of a statistics course. But I'm not sure because the language is generic (I saw statistics mentioned twice). I found it helpful for its extensive list of questions to consider when planning for programmatic assessment.


In programmatic assessment, you can imagine assessment information as pixels, where each assessment is only a single pixel in a picture; the more information (pixels) you have, the better the image will be.
Cees van der Vleuten (2018)
If you received this newsletter from someone else and would like to get it directly in the future, you can sign up here. Alternatively, if you know someone who might find content like this interesting, consider passing this email on to them.
Email Marketing Powered by MailPoet