Notes on ACE’s ‘Quality Dimensions for Connected Credentials’

I’m not in the habit of doing this, and I’m not one of the authors, but this is an important, useful report related to Open Badges which many people may never get around to reading. The report can be downloaded for free from the Lumina Foundation’s website, and I’ve also included a link to a backup at the Internet Archive in case that site is ever down:

---

  Download: Quality Dimensions for Connected Credentials (backup)

---

Context

The first thing to say is that this report speaks into a specific context: Higher Education in the USA. Despite this, I think there’s much to glean for international audiences, and those outside of the university system. There are some areas in which I disagree with the authors (e.g. the definition of a ‘credential’, around the role of endorsement, and the need to prop up the existing system) but I’ll save that for another time. Overall, the report is excellent.

What follows is lengthy quotations from the paper, divided into the sections the authors themselves use. I’ve tried to avoid any sections specifically tied to the American education system. I’ve added my own where necessary by means of introduction and connection to help you understand the flow of the paper.

Introduction

The authors identify problems around the granularisation of credentials, as well as the disintermediation of educational institutions:

[T]he diversity of credentials is not always meeting the needs of students, educational institutions, and employers, and unfortunately the proliferation of credentials is causing confusion. There is a lack of shared understanding about what makes credentials valuable, how that value varies across different types of credentials for different stakeholders, what constitutes quality, and how credentials are connected to each other and to opportunities for the people who have earned them.

It’s worth noting that the work leading to the paper was funded by the Bill & Melinda Gates Foundation:

It provides context for higher education decision makers by describing the problems caused by fractured credentialing systems, articulating quality dimensions that help to address these problems, and visualizing how institutions can improve their credentials to increase their value while meeting the
needs of diverse stakeholders.

Competency-based education is all the rage in the US at the moment, although not everyone agrees with the approach:

This paper is closely related to another resource from the ACE Center for Education Attainment and Innovation, Communicating the Value of Competencies (Everhart, Bushway, and Schejbal 2016), which focuses on how to improve communication of the value of competencies among educational institutions, students, and employers. These papers are related because connected credentials are premised on two foundational concepts: that the competencies a credential represents should be clearly defined, and that these competencies can carry independent value, including the possibility of individual competencies
having currency value as very granular credentials. The competencies paper dovetails with this paper in that understanding and improving the value of connected credentials is directly applicable to communicating the value of competencies (and vice versa). Therefore, improving the value of competencies
is a targeted set of approaches in the broader context of improving the value of credentials. Definitions, concepts, and the dimensions of quality are shared across these two papers, with different focuses.

There are frustrations in the current landscape shared by earners, issuers, and consumers of credentials:

Today, stakeholders experience numerous critical problems:

• Students do not always have reliable ways to compare credentials with regard to what they include, their market value, their transferability, their relationship to other credentials, and other important factors.
• Educational institutions need well-defined information about the value of their credentials for employment, career advancement, civic engagement, and other desired outcomes in order to attract students
  and guide them to successful credential attainment.
• Employers have difficulty understanding the competencies potential employees may or may not have mastered through the credentials they have earned.

There are, however, ways to deal with these issues:

Many organizations are already contributing to initiatives to support connected
credentials—for example:

• Defining common language to profile the types and levels of knowledge and skills credentials represent, enabling explicit description of the relationship between one credential and other credentials
• Using clearly defined descriptors to characterize credentials with regard to market value, transfer value, assessment rigor, third-party approval status, and more, empowering institutions to publicize the
  characteristics of their credentials
• Providing students with clear milestones based on modular components of credentials and relationships among credentials, helping them to understand and document their progress over time along career pathways

The best way to connect credentials is not in some top-down ways, but through more grassroots approaches:

A top-down or “authoritative requirements” approach is not desirable, and in fact, probably would not work, given the diversity of credentialing. Approaches to quality connected credentials more
appropriately emerge from within communities of practice. The framing premise of this paper is that higher education leaders and decision makers are well-positioned to contribute to national initiatives already underway and improve their own credentials in ways that are appropriate for their institutions and communities

What Are Credentials?

The authors use the following definition of a ‘credential’:

“A documented award by a responsible and authorized body that has determined that an individual has achieved specific learning outcomes relative to a given standard. Credential in this context is an umbrella term that includes degrees, diplomas, licenses, certificates, badges, and professional/industry certifications” (Lumina Foundation 2015a, 11).

They also supply a (long) definition what they mean by ‘badges’:

Badges use digital technologies to represent learning achievements; however, not all digital badges are open badges, in that not all badges use open standards that support interoperability and connections among systems and contexts. In this paper, “badge” refers to “open badges” and therefore includes technical and conceptual frameworks for openness, transparency, and interoperability (for more context on open badges, see Derryberry, Everhart, and Knight 2016). “Badges signify accomplishments such as completion of a project, mastery of a skill, or marks of experience” (Casilli and Knight 2012, 1) and can be created and awarded by institutions, organizations, groups, or individuals. Badges are flexible with regard to how issuers create them, define their use, and develop their criteria (which are publicly
viewable, embedded in the badge, and verifiable). Therefore badges can be used in numerous ways to meet a community’s needs, to represent granular competencies as well as deeply linked, rich experiences and complex learning. Badges are being used in conjunction with and/or as modular components of traditional credentials such as degrees. In some cases, especially when badges link to evidence, they are being used as representations of credentials. Badges can expire or be revoked, making them useful for credentials that are not continuously valid. Given their flexibility, badges bridge traditional, accredited credentials, professional and industry-recognized credentials, and nontraditional, experimental credentials.

Importantly, the authors explain that their approach is separate from the (controversial) approach of ‘competency based education’:

Note that a focus on the value of competencies is not the same as “competency-based education,” that is, the restructuring of academic programs to focus on mastery of competencies rather than focusing on time. References to competencies in this paper are relevant for knowledge and skills in all types of credentials and academic programs.

The authors also give a definition of ‘connected credentials’ which seems to be a Lumina Foundation shorthand for an emergent taxonomy of badges:

In the context of this paper, “connected credentials” refers broadly to multiple aspects of connectedness, including connections and relationships among credentials, connections to purpose and value in multiple contexts, and connections to opportunities for credential earners.

Collective Impact for Connecting Credentials

There are many different types of credentials, serving different purposes, and issued by different kinds of organisations. This is confusing for people:

In the face of… many variations, stakeholders struggle to make sense of how credentials are related to each other. This is a particularly poignant struggle for those who seek to earn credentials, since they have few guides or coordinated information to help them make decisions and appropriate investments.

The authors outline the particular problems in the US, which I think are more widely applicable:

The Connecting Credentials Initiative’s Making the Case paper provides a clear overview of the situation, outlining the contextual factors that put pressure on our credentialing ecosystem (Lumina Foundation 2015b, 1–4):

• The diverse range of students pursuing postsecondary education, including approximately 85 percent post-traditional students (Soares 2013, 6)
• The mismatch between what employers need and job seekers’ capabilities
• Lack of clear credential pathways to help students understand and reach their goals
• The proliferation of education and training providers, with most people using multiple providers
• Lack of transparency and consistency in quality assurance for credentials

The report goes on to reference many different aspects of the Connecting Credentials initiative, including their Beta Credentials Framework, which “uses competencies as common reference points to help users understand and compare the levels of knowledge and skills that underlie all credentials” — although this is, of course, focused on the American system.

Key Stakeholders

The authors introduce some useful nomenclature for describing the differing roles of stakeholders in the ecosystem:

For the purposes of these descriptions, the complexity of participants in credentialing ecosystems has been simplified to focus on four types of stakeholders:

• Credential earners
• Credential issuers
• Credential consumers
• Credential endorsers

Earners are “the people who attain credentials”. They have a number of problems around comparing credentials, signaling their competencies, credentials expiring, a lack of modularity and ‘stackability’ in credentials, ‘dead ends’, a lack of on-ramps, socio-economic issues, issues around transparency, and a lack of employer/industry understanding of the credentials they have earned.

Issuers are “organizations that award credentials to earners”. Their problems are around the ‘market value’ of their credentials, how credentials stack together, collaboration with other issuers, and the lack of common descriptors for popular credentials.

Consumers are “those who use credentials to make judgments and decisions about the qualifications and competencies of earners for specific purposes”. Their problems include understanding what the credential represents, and the burden of ensuring employees have the skills to go with the knowledge they earned while earning a credential.

Endorsers have “traditionally… been accrediting bodies or other independent third parties that vouch for the institution or organization and the quality and validity of its credentials. [They] are often also the ones who authorize issuers to award specific credentials. In the case of certifications and licenses, these authorizing entities can be licensure boards, state agencies, or industry organizations.” Endorsers’ problems are around determining the value of credentials, a lack of transparency and clear frameworks, and the paucity of information on which to base their endorsements.

Dimensions of Quality for Connected Credentials

This is the ‘meat’ of the report, in which the authors outline six dimensions for quality, ‘connected’ credentials:

1. Transparency
2. Modularity
3. Portability
4. Relevance
5. Validity
6. Equity

[The dimensions] are not all-encompassing, but they provide useful ways of discussing credentials and how they can be improved, both generally and in the analysis of specific credentials.

The dimensions overlap and also mutually reinforce each other. For example, modularity supports portability by making it easier to move credentials from one context to another; transparency supports relevance, by making it easier for consumers to understand what a credential includes and therefore how it is relevant for their purposes.

The authors outline what they mean by each of the six dimensions in bullet point format. To avoid making this long post even longer, I will simply quote the summary at the end of each dimension.

Transparency

Transparency supports connectedness by making credentials easier to understand and compare, facilitating the definition and implementation of relationships among credentials. Transparency also supports connections to opportunities by helping all stakeholders understand how credentials are valuable.

Modularity

Modularity supports connectedness by making credentials more componentized and less monolithic, leading to more connection points and possible relationships among credentials. Modularity also helps students understand the components within credentials and how they connect to each other and to larger goals such as socioeconomic mobility and lifelong learning.

Portability

Portability supports connectedness by making credentials more applicable in multiple contexts, connecting to multiple purposes and opportunities. Portability also facilitates connections among different types of credentials in different environments.

Relevance

Relevance supports connectedness by illuminating the applicability and purposes of credentials for specific stakeholders in their own contexts, thereby connecting to opportunities in those contexts. Relevance also connects and amplifies different types of value by helping stakeholders understand the
network of verification, documentation, evidence, and social interpretation supporting the credential.

Validity

Validity supports connectedness by illuminating the broad frameworks of meaning and value that connect the credential to opportunities. Validity provides a shared understanding and trust of how the credential is defined, including the evidence and quality assurance structures that are necessary to implement well-defined relationships among credentials.

Equity

Equity as a dimension of quality credentials helps people overcome their disadvantages and connect to opportunities. Equity provides a network of flexible access points and supports that connect students to credential attainment and the benefits of lifelong learning.

Describing the Current State of Credential Types

This section applies the six dimensions of quality as defined by the authors to different types of credentials, including badges. They find that badges pass all of the tests, with the only problems being (unsurprisingly, given the nature of the report) around the lack of a common language/framework for connecting them together.

Challenge Questions for Analyzing Credentials and Visualizing Potential Futures

The final section before the report’s conclusion is a series of questions under the six quality headings that issuers can ask of their credentials. For example:

Are the competencies (knowledge and specialized skills, personal skills, and social skills) represented by this credential clearly defined?

and:

What value does this credential carry for specific stakeholders that you identify as important? How do you know what these stakeholders value? Are they involved in your credential improvement
processes?

These are extremely useful questions for any issuer of credentials to consider.

Conclusion: Call to Action

The authors list various ways that those reading the report who are in US Higher Education can get involved. These include forming working groups, writing papers and book chapters, hosting workshops, and reading other papers by Connecting Credentials.

[A] willingness to ask and seek answers to these questions is an essential first step in breaking down the credentialing silos that sometimes impede student progress and cause our systems to be less effective and beneficial than they could be.

…

We encourage you to complete the arc of your journey: Identify your credentialing ecosystem stakeholders, articulate the problems they encounter when credentials are not connected, use the challenge questions to analyze and discuss the current state of your specific credentials with regard to the quality dimensions, and then establish a realistic plan and timeline for developing more valuable, robust, and connected credentials that reflect your new approach.

---

I’d recommend reading the report in full here and, more importantly, think about how you can apply the findings no matter what your context.

For more on Open Badges and how to get started with them, check out Open Badges 101, a free, open community course Bryan Mathers and I put together.