Open Thinkering


Tag: assessment

Levelling up?

I’ve spent 13 years now interested in Open Badges and, more recently, Verifiable Credentials.* When you explain something over and over again you get better at explaining it. You also start to notice patterns. This post is about one of those patterns.

* Happily, v3.0 of the Open Badges specification uses the Verifiable Credentials data model. Find out more.

Define something worth learning, build a curriculum and scheme of work, then design some learning activities. Create an assessment based on the learning activities, and then issue credentials based on the outcome of the activities.
Image CC BY-NC Visual Thinkery

In broad brushstrokes, credentials are awarded in a similar way within academic systems. Define something worth learning, build a curriculum and scheme of work, then design some learning activities. Create an assessment based on the learning activities, and then issue credentials based on the outcome of the activities.

We’re so used to this that we forget that this is very far removed from how the world actually works. Learning outside of the classroom is messy, episodic, and relational. So how do we go about capturing this?

A common mental model I’ve seen is using gold, silver, and bronze as ‘tiers’ within a badging system. However, without a background in assessment design, these tiers often become even more arbitrary than those in formal education. There’s often a huge ask even to get on the bottom rung of the ladder. Why? I’d just give people badges for turning up. They’re free! You can issue as many as you like.

At Mozilla, where I also served as Web Literacy Lead, we aimed to link the Web Literacy Map to badges, and initially considered levels. However, we quickly realised that doing this globally in a decentralised way is essentially impossible. Instead, mapping badges to skills in specific areas made much more sense. Context matters: what might be ‘advanced’ in one place could be ‘beginner’ elsewhere.

Instead, mapping badges related to skills in a particular area made much more sense. Context does, after all, matter: what might be seen as ‘advanced’ somewhere might be seen as ‘beginner’ elsewhere, and vice-versa. Badges for levels are all well and good, but those levels need to describe something worthwhile.

After leaving Mozilla, I spent all my consultancy time with City & Guilds , collaborating extensively with Bryan Mathers (who created the images in this post). Even as an awarding body, it took City & Guilds staff a while to grasp all the possibilities badges offered.

Image CC BY-NC Visual Thinkery

Bryan created this super-simple taxonomy from our conversations as a conversation starter with City & Guilds staff, helping them realise that recognition in the form of participation in something, or membership of a thing, was just as legitimate as reaching a defined standard or demonstrating excellence. Badges help us tell a story about the learning journey we’ve been on.

For me, this has been one of the main takeaways from my own learning journey with Open Badges so far: when you’ve got enough verified ways of showing what you’ve done, levels don’t matter that much. We’re all different, so recognising and celebrating that is, to me, more important than expecting everyone to fit into pre-defined boxes.

So if you’re designing assessments based on academic research for something that’s high stakes then, by all means, design a rigorous system. For everything else, treat it like a product: figure out how your users (the people to be badged) want to be recognised, design around that, and iterate.

If you’re curious in how to go beyond the ‘microcredential’ approach to digital credentials, you might be interested in the free WAO email-based course Reframing Recognition. You’re also welcome to join us as part of the Open Recognition is for Everybody community.

TB872: Concept map to help with my EMA

Note: this is a post reflecting on one of the modules of my MSc in Systems Thinking in Practice. You can see all of the related posts in this category

A concept map based on the structure of TB872's End of Module Assessment questions.

It’s 15 years since I spent days creating a concept map for my Ed.D. thesis. Thankfully, the End of Module Assessment (EMA) for this MSc module is a mere 4,000 words, meaning it’s only taken me a few hours to create this one using Whimsical.

The requirements for the EMA are outlined in a previous post. All I’ve got to do now is write it. It’s such an interesting topic that I need to remind myself that I’ve given myself until next Friday to get it written. I’m moving house the week after, and I want this to be done.

In related news, although I’d originally planned not to do the other compulsory introductory module for this MSc (TB871) until 2025, I’ve changed my mind. Never one to shirk from a challenge, I’ll be starting that one on May 1st — a couple of weeks after finishing this assignment 😅

Elevator pitch on Open Badges for SQA Expert Assessment Group

Update: Martin Hamilton from Jisc kindly recorded my elevator pitch. You can watch it here.

Tomorrow, I’m in London to take part in the Scottish Qualifications Authority‘s Expert Assessment Group. The SQA have been forward-thinking about Open Badges over the last few years, so I’m delighted to have been asked to attend.

There’s five people been asked to give input in the morning from a ‘future of assessment’ point of view, and five in the afternoon on ways technology might be able to help enable that future. I’ve got a very short amount of time, so I’ve boiled it down to the slides below.

(Note: go fullscreen by clicking the arrows in the black bar at the bottom)

Backup locations: Slideshare / Internet Archive

The flow for my pitch starts with a tweet I saw earlier today from the influential Paul Graham. He links to an article in The New York Times which talks about skills-based hiring, but which completely disregards digital credentialing. From there, I discuss Michael Feldstein’s recent post about badging gaining huge traction in very specific areas. And then I launch into a pretty familiar flow using Bryan Mathers‘ excellent visuals.

There’s loads more I want to say about how version 2.0 of the Open Badges specification allows for really interesting dynamic badges that ‘grow’ over time. Kerri Lemoie and Lucas Blair recently wrote about this from a technical point of view, and I presented my thoughts last week at the University of Dundee, including this slide:

Dynamic badging

Perhaps I’ll get a chance to discuss these new developments if my pitch is selected to be discussed further. I’d bring up blockchain technologies and their potential uses in credentialing, but I’ve got to catch a train back home in the evening…

Photo by John-Mark Kuznietsov on Unsplash