Open Thinkering

Menu

Tag: assessment

What did we learn during a ‘semester of learning’ on #openbadges over at P2PU.org?

Open Badges and Assessment

Seven weeks ago I proposed a ‘semester of learning’ about Mozilla’s Open Badges. This was originally going to be hosted on an installation of BuddyPress, but eventually resided at P2PU.org in a group called Open Badges and Assessment. It attracted a diverse mix of people, most of whom I’d never encountered before (I love it when that happens!) Many of use are continuing the conversation at a new HASTAC group.

In a similar way to a MOOC (Massive Online Open Course) the semester of learning was an informal affair where participants (of which there were 84 altogether) could be as active as they want. Again, as with MOOCs, many were content to listen in upon what others were talking about. Others played a more active role. I’ve archived the study group, but it will remain available indefinitely on P2PU.org for your perusal.

Things tailed off slightly towards the end, for two reasons. The first was that I was in the last couple of weeks of my thesis, so was spending all of my spare time on that. Secondly, the conversation moved from being in a niche area to being much more mainstream (via Twitter, etc.) with the launch of the DML Competition

As a taster of what went on in the semester of learning here’s some comments from the beginning and towards the end.

Week 1

There are key questions around ensuring quality for these badges to take hold. If they are to become something valuable on a CV for example then a prospective employer needs to be able to ascertain the level & rigour involved in the aquisition of the badge. (Dan Stucke)

I’m really impressed by the scope of this Mozilla project. I must admit, I signed up merely because I am interested in looking at ways for developing badges in a high school context, so to see this scale up in such a monumental way is pretty inspiring.

The potential for a new standard in qualifications that learners continue to build upon is very interesting. For example, my own degree and teaching qualifications are relatively old compared to everything I have learned since, and even though there is no formal recognition of my increased learning over the years, save a few references from previous employers, I’d say the undocumented skills I have now make me a far more qualified person than I appear to be on paper. I think the case studies from the open badge system framework draft make this point quite well. (Jackson Bates)

My main worry about the badges appraoch is that it will only be a kind of add-on to the normal educational modle. What I’m mainly interested in doing is entering into direct confrontation with the university as it currently exists. I want to fight with the university, offer an alternative to it, and fundamentally challange the values at work in the university. I’m worried that a badge just isn’t going to cut it, that it won’t be taken seriously enough or that it will only be taken seriously as an add-on to a “real” university education. (Thomas Gokey)

Week 6

Every time a new educational fad erupts it seems to be polarizing, which seems to hold true in the conversations surrounding the dml announcement.  Instead of talking about whether we agree or disagree with the movement a better topic would be, what can these badges do for education, specifically assessment?

I am excited to see what comes of the research grants for the badges.  Will we start giving badges instead of end of course assessments/exams?  Would that be a good thing?  How would it work?

Yes of course it would be messy, but what if students had to obtain specific badges to pass into the next grade or to receive a high school diploma?  Would it motivate students to complete their coursework or would it only further increase dropout rates?  At any rate it is obvious that we would have to get the buy in of students to pull this off effectively…. (AndiStrack)

In the twitter about the grants, people expressed concern that there would be a proliferation of badges of dubious value. Nobody can stop that from happening and it would not be desirable. Our organization plans to categorize and rank badges by difficulty. We think our website that lists the badges will get substantial traffic just as our lists of open textbooks have done. (Jacky Hood)

Conclusion

I found P2PU.org a fantastically easy way to setup a study group and would certainly do so again. I think that the semester of learning helped point people towards certain resources that they may not otherwise have seen and, perhaps more importantly, engage with other people they may not have come across. It was great to see, given some of the superficiality and shallow reading evident from those reacting in various backchannels during the announcement, that those who were part of the group were committed to going away to think and read.

What did we learn? Well, I think I can speak on behalf of us when I say that talking of ‘badges for lifelong learning’ sounds simple but actually contains a lot of nuance and hidden complexity around assessment. I’m very much looking forward to continuing the conversation both on Twitter (using the hashtags #openbadges and #dmlbadges) and within the new HASTAC group.  🙂

My Belbin results – Part 2

In My Belbin Results – Part 1 I outlined what the Belbin process is and listed the nine different characteristics that the process identifies for those who are part of a team. At the end of the blog post I asked people, whether they knew me solely online or also offline, to ascertain which three of the nine characteristics were most like me. Go and read that post (and especially the comments) before proceeding. 🙂

It was interesting that those who know me solely online seem to view me differently from those who know me offline as well. That showed up in my ‘official’ Belbin report as well – there was one external assessor who I’ve only ever talked to on Skype and over the phone.

So what were my results? In order:

Plant – Creative, imaginative, unorthodox. Solves difficult problems. Ignores incidentals. Too pre-occupied with own thoughts to communicate effectively.

Resource Investigator – Extrovert, enthusiastic, communicative. Explores opportunities. Develops contacts. Over-optimistic. Can lose interest once initial enthusiasm has passed.

Shaper – Challenging, dynamic, thrives on pressure. Has the drive and courage to overcome obstacles. Prone to provocation. Liable to offend others.

The above were agreed upon by all five of my observers, apart from one who came up with ‘Specialist’ instead of ‘Shaper’. As for me, I had down Plant and Shaper, but had ‘Monitor Evaluator’ down as number one. Perhaps I’m not so ‘serious minded’ after all… 😉

The pigeon-holing is interesting but some of the report was intriguing. There were six people who assessed me, if you include the self-assessment; here are the top words and phrases people selected to describe me (in order of most frequent):

  • innovative
  • impulsive
  • creative
  • imaginative
  • opportunistic
  • enterprising
  • provocative
  • encouraging of others
  • persistent
  • outspoken
  • technically skilful
  • clever
  • professionally dedicated

The word ‘aggressive’ also came out a couple of times, but right next to it was ‘calm & confident’ so I saw them as cancelling each other out. Talking to one of the people who assessed me, they explained the former as positive and being akin to ‘tenacious’. 🙂

There’s various other bits of feedback you get, including a ‘strengths’ and ‘possible weaknesses’ report, along with (hilariously) a ‘counselling report’. Here’s some choice excerpts:

Has innovative tendencies and needs to work in a mentally challenging environment. Requires work where he can use his outgoing nature… Needs to work in an environment which offers scope for personal expression.

Could have problems adapting to a supportive and subordinate role when necessary.

Needs to work within a loose framework. Will function best when given the freedom to roam.

Yours is essentially a pioneering profile. You are one of the few people equally read to develop new ideas on your own or in conjunction with others. Your best line of work is one in which you are required to explore possibilities and to take advantage of new opportunities. You have some features of the visionary. But take care you do not become isolated from others and resistant to the contributions they can make to the development of what is new.

For you above all others, it is best to establish the moment of exit. Do not outstay your welcome.

Your operating style is that of one who always seeks to be at the cutting edge of change. So remember that this is a hazardous spot to occupy. You will need to respect others of more traditional habits if you are to win respect yourself.

Does that seem a fair assessment? 😀

My Belbin results – Part 1

At the JISC infoNet quarterly planning meeting on Tuesday we got our Belbin feedback. For those who don’t know what that is (which would have included me until recently), go and read the Wikipedia article.

I’m not a huge fan of being pigeon-holed, but I found the results interesting nevertheless. I’ve only got a paper version of the results at the moment and, given it’s copyrighted material, I’m just going to share edited highlights. 🙂

There are nine defined roles with the Belbin process, the characteristics of which an individual is judged to exemplify to a greater or lesser extent. These are:

  1. Plant – Creative, imaginative, unorthodox. Solves difficult problems. Ignores incidentals. Too pre-occupied with own thoughts to communicate effectively.
  2. Resource Investigator – Extrovert, enthusiastic, communicative. Explores opportunities. Develops contacts. Over-optimistic. Can lose interest once initial enthusiasm has passed.
  3. Co-ordinator – Mature, confident. Clarifies goals. Brings other people together to promote team discussions. Can be seen as manipulative. Offloads personal work.
  4. Shaper – Challenging, dynamic, thrives on pressure. Has the drive and courage to overcome obstacles. Prone to provocation. Liable to offend others.
  5. Monitor Evaluator – Serious minded, strategic and discerning. Sees all options. Judges accurately. Can lack drive and ability to inspire others.
  6. Teamworker – Co-operative, mild, perceptive and diplomatic. Listens, builds, averts friction. Indecisive in crunch situations.
  7. Implementer – Disciplined, reliable, conservative in habits. A capacity for taking practical steps and actions. Somewhat inflexible. Slow to respond to new possibilities.
  8. Completer Finisher – Painstaking, conscientious, anxious. Searches out errors and omissions. Delivers on time. Inclined to worry unduly. Reluctant to let others into own job.
  9. Specialist – Single-minded, self-starting, dedicated. Provides knowledge and skills in rare supply. Contributes on only a limited front. Dwells on specialised personal interest.

For those who know me (either wholly through my work online or in person) I’d be interested in you participating in a little experiment:

If YOU had to choose three of these roles to describe me, which would you choose? Why?

(for a ‘Brucey bonus’ list some keywords you’d use to describe me)

I’ll share the keywords and roles my colleagues think fit me best in a forthcoming post. 😀

css.php