…employees of the future will be selected based on their portfolio of actual work rather than based on proxies like degrees and certificates.
Stephen Downes
Introduction
Portfolio assessment in health professions education isn’t new, although there seems to be some renewed interest in the form of making more effective use of digital and/or online portfolios as part of clinical practice (side-note: I dislike the term ‘eportfolio’, along with every other prepending of ‘e’ to a word, signalling a digital component).
Portfolios – whether ‘electronic'(!) or ‘analogue’ serve two main functions:
- they capture a process of learning and reflection
- they serve as a product that can be evaluated and assessed as part of the documentation of achievement and accountability
The most common use of portfolios is to gather artifacts of student work to demonstrate the fulfilment of standards. In other words, they document a student’s progress through a programme of study and present evidence of the achievement of certain milestones as part of the demonstration of competence as determined by a regulatory body. While I think of this as the generally less interesting purpose of a portfolio, it is worth noting the potential for automating the evaluation of this progression, which would be profoundly consequential.
In my opinion, the portfolio for learning (i.e. portfolio as evidence of a process of growth and change) allows for a more creative, insightful, and inspiring use of the tool. Having said that, I should also note that these portfolios are easier to create than to read, and that when portfolios are used as a workspace for learning and reflection, they can be time-consuming and difficult to work through. This may be unfair and onerous for those responsible for assessing them.
Anyhow, regardless of whether you use them to document a process of transformation, or to capture the achievement of milestones as part of a progression towards competence, I think that portfolio assessment will form an increasingly important part of health professions education in the future.
Podcast
Snell, L., Sherbino, J., & Frank, J. (n.d.). Assessing Portfolios – Turning complex data into pass/fail (No. 183). Retrieved March 21, 2022.
Assessors were able to form a judgment based on the portfolio evidence alone. Although they reached the same overall judgments, they differed in the way they processed the evidence and in the reasoning behind their judgments. Differences sprung from assessors’ divergent assessment beliefs, performance theories, and inferences acting in concert. These findings support the notion that portfolios should be judged by multiple assessors who should thoroughly substantiate their judgments. Also, assessors should receive training that provides insight into factors influencing their own decision making process and group decisions.
A useful discussion that highlights the challenges of reliability and bias that intrude upon any thoughts we may have that assessment of anything as complex as a portfolio can be objective. It seems as if diverse assessor panels may be a more fair and appropriate method of evaluating portfolios, especially when the evidence collated in the portfolio is diverse.
Article
Driessen, E. W., Van Tartwijk, J., Overeem, K., Vermunt, J. D., & Van Der Vleuten, C. P. M. (2005). Conditions for successful reflective use of portfolios in undergraduate medical education. Medical Education, 39(12), 1230–1235.
The conditions for successful reflective use of portfolios that emerged from the interviews fell into 4 categories: coaching; portfolio structure and guidelines; relevant experiences and materials, and summative assessment. According to the mentors, working with a portfolio designed to meet these conditions will stimulate students’ reflective abilities.
We often implement portfolios (for learning) without adequately preparing students to make the best use of this learning task, assuming that reflection will simply occur, and thus lead to the kind of self-regulated learning that we know is important for professional development. However, simply asking students to collect and present evidence of learning may lead to a box-ticking exercise, or worse, massive collections of irrelevant information that must nonetheless be evaluated. The 4 design categories that emerged from this study may help clinical educators who’d like to use portfolios increase the chances that it will be a fruitful exercise.
For another good paper on the potential for poor outcomes with portfolios, see: Driessen, E., Van Tartwijk, J., Van Der Vleuten, C., & Wass, V. (2007). Portfolios in medical education: Why do they meet with mixed success? A systematic review. Medical Education, 41(12), 1224–1233.
Resource
Baume, D. (2003). Supporting Portfolio Development. Advance HE.
This guide explores some of the many ways in which lecturers can productively use portfolios in courses of higher education at all levels and offers guidance on how lecturers can support students to develop and use a portfolio. Much of this can also be applied or adapted to help staff to produce portfolios for their initial and continuing professional development.
A 27-page report that includes an overview of the uses of portfolios, their forms, content, and ideas around student support. The section on technology is obviously out of date but the rest of the report contains a wide range of information that remains relevant for anyone interested in implementing portfolios as part of a broader assessment programme in higher education.
Community
In Beta Unconference 2022
Planning for the 2022 In Beta Unconference on 8th & 9th September has started! We’re really excited to be trying a hybrid model which is going to take the best bits of our previous in person and online Unconferences and add some new ideas in a genuinely innovative hybrid model.
We’re really keen to engage the In Beta community from the start in the development and coordination of the unconference. We would love to hear from you if you:
- Would like to be involved in the planning and coordination of the unconference programme and supporting the delivery either online or in a local face to face event
- Have ideas about what we should focus on in our guided reflection podcast stream this year and /or would like to lead on developing a podcast