Open Thinkering

Menu

Tag: Technology

The role of endorsement in Open Badges and Open Recognition

Lately, I’ve been rethinking the importance of endorsements in the Open Badges ecosystem. This is due to changes in version 3.0, which now links Open Badges with a model called Verifiable Credentials. These changes have made me revisit some of my earlier ideas, especially as I’ve been doing more work on Open Recognition.

Some context

After leaving Mozilla in 2015, I worked as a consultant for City & Guilds, focusing on badges and digital qualifications. Originally coming from a background in formal education, I found it really interesting to see how organisations made sure their assessment systems were credible.

Validity, Reliability, and Viability combining to produce Credibility

Image CC BY-ND Bryan Mathers

To explain the above diagram:

  • Validity refers to the extent to which an assessment accurately measures what it is intended to measure. In other words, a valid assessment is one that successfully captures the skills, knowledge, or attributes it claims to evaluate.
  • Reliability is concerned with the consistency and stability of assessment scores over time and across different conditions. A reliable assessment will yield similar results when administered multiple times under similar conditions.
  • Viability refers to the practicality and feasibility of the assessment system. This includes considerations like cost, time, resources, and the technological infrastructure needed to administer the assessment.

These three elements combine to produce Credibility in the assessment system, by which we mean that stakeholders such as educational institutions, learners, and employers have confidence in the results it produces.

While it’s good that established awarding bodies like City & Guilds are using badges, my main interest is in challenging the existing system. That’s why it’s crucial for more people to understand the concepts that these organisations use.

What is endorsement?

When we’re unsure about the importance of something or whether to pay it attention, we often look for signs to help us decide. As William James said, “Our faith is faith in some one else’s faith, and in the greatest matters this is most the case.” So, if someone or an organisation you trust vouches for another person or topic, you’re likely to see it in a positive light.

This trust is often formally known as ‘endorsement,’ and it’s a feature of the Open Badges standard that’s often overlooked and underused.

In a 2016 book chapter called ‘The Role of Endorsement in Open Badges Ecosystems,’ Deb Everhart, Anne Derryberry, Erin Knight, and Sunny Lee highlighted how crucial endorsement is. It’s not just about supporting badge pathways but also about building networks of trust. These networks are key to the practice of Open Recognition.

Endorsement encourages the development of trust networks and connections among stakeholders in communities such as education, government, standards bodies, employers, and industry associations. Badge endorsers make their values known by analyzing the quality of specific badges, including how the badge is defined, the competencies it represents, its standards alignments, the process of assessing badge earners, and the qualifications of the badge issuer to structure and evaluate the learning achievement represented by the badge. With endorsement, badge earners are better able to understand which badges carry the most value for their goals. Badge issuers benefit from external validation of their badges. Educators, employers, and other consumers who evaluate learners’ achievements can better determine which badges are most appropriate in their contexts.

At the moment, the majority of badge platforms use v2 of the Open Badge standard. Endorsement isn’t a mandatory field when setting up the badge’s metadata. So, what’s the benefit of using it?

Taking City & Guilds as an example, they endorse the RSA’s City of Learning Badging Standard. In other words, they put their reputation behind it after checking that the RSA’s approach is valid, reliable, and viable.

Just like anyone can follow a curriculum, anyone can align their badge system with the Engage / Participate / Demonstrate / Lead approach. But this doesn’t guarantee their badges will get endorsed. To get an endorsement, there needs to be a relationship between the one giving the badge and the one endorsing it.

In theory, individual badges (called ‘assertions’) can get their own endorsements, separate from the general type of badge (known as ‘badge class’). But this is rare in practice. It’s usually the case that an organisation endorses a ‘Leadership’ badge as a whole, rather than endorsing a specific person’s ‘Leadership’ badge. However, this is getting easier, as some platforms are now set up to handle mass endorsement requests.

Changes to endorsement with v3.0 of the standard

Earlier this year, I published a post highlighting key updates to the Open Badges standard as it transitions to version 3.0. One significant change I didn’t mention at the time is how endorsements are managed.

The following example is taken from the endorsements section of the specification. I’ve rewritten it as the original is quite confusingly worded:

Ralph received a badge from the hospital where he works. This badge lists the skills needed for his role. He asks his workmates to vouch for these skills. The badge platform helps by sending this request to his peers, letting them review his skills, and then giving out endorsement badges. These new badges link back to Ralph’s original badge, name his colleagues as the ones who endorsed him, and show that Ralph is the one who received the endorsement.

A note from the editor below this example clarifies that, unlike previous versions of the standard, v3.0 requires giving out extra credentials for endorsements. This means endorsements are usually added after a badge has already been given. In previous versions, endorsement details could be part of the original badge, included in a reissued badge, or kept separately on the badge platform.

The change is possible because Open Badge v3.0 and Verifiable Credentials do not require a badge image. Although I saw this as some kind of sacrilege when I first heard of it, I’ve come to appreciate the merits of such an approach. It makes a lot sense for endorsements.

How endorsements enable Open Recognition

Another part of the v3.0 specification mentions self-assertion, which means an individual issuing a badge or credential to themself. We’ve talked about this ever since the early days of badges, but it was seen as either dangerous or frivolous by those with a vested interest in the status quo.

I’ve again slightly reworded the example found in the self-assertion section of specification for ease of understanding:

Stacy has made a mobile app that shows off her skills in coding, design, and product management. She sets up an account on a badge platform and designs a badge that lists these skills. Using her digital wallet app, she links to the badge platform and gives herself this badge. The badge includes screenshots and a link to her mobile app as proof. Stacy uses this badge and similar ones as items in her verified portfolio.

The editor’s note explains that in older versions of the specification, it was possible for individuals to create badges for yourself and colleagues. However, the details about who issued the badge were designed to be used by organisations. Now, v3.0, these details can refer to either an organisation or a person, with both the issuer and the recipient profiles having similar optional details. This makes means that an organisation can also be listed as the recipient of a badge.

Person giving a badge to someone else. A cloud of 'credentialing' surrounds them, with a wider cloud of 'recognition' around that.

Image CC BY-ND Visual Thinkery for WAO

This is huge for the purposes of Open Recognition:

Open Recognition is the awareness and appreciation of talents, skills and aspirations in ways that go beyond credentialing. This includes recognising the rights of individuals, communities, and territories to apply their own labels and definitions. Their frameworks may be emergent and/or implicit.

It means the following workflow is possible (and entirely legitimate):

  1. An individual self-issues a badge for something that makes sense to them, in their own language, and in their own context/community
  2. They ask for this badge to be endorsed by other members of the community
  3. The individual uses their badge, endorsed multiple times, to gain greater recognition in their community/sector/field

It’s worth noting that this doesn’t require traditional awarding bodies to validate or acknowledge their experience. This is key. While it’s possible for these bodies to do so, the groundbreaking aspect of Open Badges has always been to democratise the issuing of credentials, allowing anyone to issue badges to anyone, for anything.

Conclusion

Person standing in concentric circles entitled 'Self-issued', 'Issued', 'Verified', and 'Endorsed'

Image CC BY-ND Bryan Mathers

The above image came out of a conversation I had with Bryan back in 2017. At the time, endorsement was still a very organisation-centric way of creating credibility within the Open Badges ecosystem.

What I’m delighted to see is that finally the early revolutionary promise of badges is being recognised at a technical level. I can’t wait to see how individuals and organisations start using endorsement with v3.0 of the standard, and I’m excited to help them explore ways of doing so!

Handing over ownership of exercise.cafe

Running track

A year ago, I realised that competing with other people via Strava wasn’t good for my physical or mental health. I wondered about setting up a friendly Fediverse instance for exercise-related updates and chat. This turned into exercise.cafe, running on Pixelfed.

A year later, I’ve posted screenshots of most of my runs there, as well as swims and gym sessions. Other people have done likewise about the different activities they’re into. One person in particular (@ryancoordinator) has been using it every day.

It was only yesterday that I was once again describing how I subscribe to the SOFA principle of starting things but not necessarily keeping them going. So I’m delighted that, at a time when I was thinking of shutting it down, Ryan has volunteered to take over the ownership and running of exercise.cafe. We’re currently in the midst of handing things over, but I’m really pleased that it will keep going.

Ryan has experience of running all kinds of platforms and events, so I think that the site is in good hands. If you’re interested in joining exercise.cafe, registrations remain open!

Using AI to help solve Bloom’s Two Sigma Problem

Three curved lines showing performance. There are two standard deviations (i.e. two sigma) between Conventional Learning and 1:1 Tutoring.

Imagine we’re all surfers. The ocean we’re in is the educational system, and we’re all trying to ride the wave of knowledge to the shore of understanding. Some of us have master surfers as guides – personal tutors who are right there with us, helping us manoeuvre the currents and ride high on the knowledge wave. They know our strengths, they know our fears, and they ensure we don’t wipe out. These fortunate few reach the shore faster, more smoothly and often with a lot more fun.

Then there are the rest of us. We’re in a giant surf class. There’s one instructor and dozens of us learners. The instructor is doing their best, but they can’t give us all the personalised attention we need. Some of us catch the wave, some of us don’t. This is Bloom’s Two Sigma Problem.

Brought to the fore by educational psychologist Benjamin Bloom in the 1980s, the Two Sigma Problem highlights a gap in education. Personal tutoring can propel students’ performance by two standard deviations – like moving from the middle of a class right to the top 2%. The problem is, we can’t give everyone a personal tutor. It’s just not feasible. So, the question is, how do we give each student the benefits of one-on-one instruction, at scale?


Enter Artificial Intelligence (AI) and, in particular, Large Language Models (LLMs) such as ChatGPT. I’ve been experimenting with using ChatGPT as a tutor for my son during the revision period for his exams. It’s great at coming up with questions, marking them, and suggesting how to improve. This kind of feedback is absolutely crucial to learning. It’s also great at exploring the world and allowing curiosity to take you in new directions.

So, if we revisit the Two Sigma Problem based on what’s possible with LLMs, it looks like there’s a possible solution with multiple advantages:

  1. Personalisation: Like a master surfer guiding us through the waves, AI offers individualised instruction. It can adapt to each learner’s pace, skill level, and areas of interest. It’s like your own personal Mr. Miyagi, providing the right lesson at the right time. Wax on, wax off.
  2. 24/7 Availability: With AI, it’s always high tide. The learning doesn’t stop when the school bell rings. Whether it’s the middle of the day or the middle of the night, your AI tutor is there to help, guide, and explain.
  3. Scalability: One-to-one tutoring might not be feasible, but AI makes one-to-one-to-many a reality. An AI tutor doesn’t get exhausted or overbooked. It can help an unlimited number of students at once, ensuring everyone gets the ride of their lives on the knowledge wave.
  4. Feedback and Assessment: Picture a surf instructor who can instantly replay your wipeouts, showing you exactly what went wrong and how to fix it. That’s what AI can do. It provides immediate feedback, helping learners understand and correct their mistakes right away.
  5. Enhanced Resources: LLMs are like a treasure trove of knowledge. Trained on a vast array of educational content, they’re like having the British Library at your fingertips, ready to generate explanations, examples, and answers on a multitude of topics.
  6. Removing Bias: AI doesn’t care about your background, your accent, or the colour of your board shorts. When designed and trained properly, it treats all learners equally, providing a level playing field.

No technology is a silver bullet. As an educator, I know that while curiosity and feedback is really important, there’s nothing like another human providing emotional input — including motivation. AI is here to support, not replace, our human guides.

Even though it’s early days, we’re already seeing some really interesting developments in the application of LLMs in education. I’m no fan of Microsoft, but I will acknowledge that a feature they have in development called ‘passage generation’ looks interesting. This tool reviews data to create personalised reading passages based on the words or phonics rules a student finds most challenging. Educators can customise the passage, selecting suggested practice words and generating options, then publish the passage as a new reading assignment. I find this kind of thing really useful in Duolingo for learning Spanish. Context matters.

As a former teacher, I know how important prioritisation can be for the limited amount of time you have with each student. And as a parent, I’m a big believer in the power of deliberate practice for getting better at all kinds of things. Freeing up teachers to be more like coaches than instructors has been the dream ever since someone came up with the pithy phrase “guide on the side, not sage on the stage”.


One of the main concerns I think a lot people have with AI in general is that it will “steal our jobs”. I’d point out here that the main problem here isn’t AI, it’s capitalism. Any tool or system be used for good or for ill. If you’re not sure how we can approach this post-scarcity world, I’d recommend reading Fully Automated Luxury Communism by Aaron Bastani. Of course, regulation is and should be an issue, too.

The main issue I see with this is centralised LLMs run by companies running opaque models and beholden to shareholders. That’s why I envisage educational institutions running local LLMs, or at least within a network that only connects to the internet when it needs to. Just as Google Desktop used to allow you to search through your local machine and the web, I can imagine us all having an AI assistant that has full context, while preserving our privacy.


So the way to approach any new tool or service is to ask critical questions such as “who benefits?” but also to fully explore what’s possible with all of this. I’m hugely hopeful that AI won’t lead us into a sci-fi dystopia, but rather help to even out the playing field when it comes to human learning and flourishing.

What do you think? I’d love to hear in the comments!


Image remixed from an original on the SkillUp blog. Text written with the help of ChatGPT (it’s particularly good at coming up with metaphors, I’ve found!)

css.php